DETAILED ACTION
This communication is in response to the Arguments and Remarks filed on 2/17/2026. Claims 1-20 are pending and have been examined. Hence, this action is made FINAL.
Any previous objection/rejection not mentioned in this Office Action has been withdrawn by the examiner.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 13, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant claims the benefit of US Provisional Application No. 63/454,206, filed March 23, 2023. Claims 1-20 have been afforded the benefit of this filing date.
Response to Arguments
With regards to the rejections under 35 U.S.C. 101, the applicant asserts that if a claim includes an element that cannot be practically performed in the human mind, the claim is not directed to a mental process. Executing functions of machine learning models and training machine learning models such as those in Claim 1 cannot be practically performed in the human mind. Artificial intelligence models are complex and are, by their very nature, performed using electronic devices because the human mind cannot practically perform their functions.
Examiner respectfully disagrees, if the model is merely being used to apply a mental process via a computing device it is not enough to overcome the rejection such as translating one language to another. Furthermore, most of the method is a process of creating a training dataset that can be performed by the human mind. Then the multilingual LM gets trained by that data at the end which acts as intended use for the training data rather than something integrated throughout the claim language. There is no explicit claim language where a machine learning model is executing any functions, it is just having training data supplied to it. While artificial intelligence models are complex they are still treated as additional elements that need to be integrated within the system and the multilingual language model presented in the current claim language is merely receiving training data.
Applicant further points to an MPEP example in 2106.04(a)(1) and Example 39 in "Reminders on Evaluating subject matter eligibility of claims under 35 U.S.C. 101".
Examiner respectfully disagrees, the MPEP example shows a method where the training is integrated as the neural network is trained on the a first dataset and then based on the results of that training a second training set is made and used. The current claim language is creating a training dataset and then only applying it at the very end. In the “Reminders” examples specific training methods are presented for training a model in a specific way that would separate it from another implementation of the model. Merely applying a dataset is not a unique implementation as this is standard practice for training language models.
With regards to the rejections under 35 U.S.C. 102, the applicant asserts that the amendments to the claims are not taught by Li et al.
The argument for the limitations “wherein generating the multilingual training corpus comprises: performing a translation operation on a first training utterance in the first language from a training dataset to convert the first training utterance into a second training utterance in the second language; and performing a concatenation operation to add the second training utterance to the multilingual training corpus;” is considered moot as there has been new claim language added the claim has been reevaluated. It is now rejected under 35 U.S.C. 103 using a secondary reference of record (Hwang et al.).
Although the claim is now rejected under 35 U.S.C. 103, the limitation “wherein fine-tuning the multilingual language model comprises training the multilingual language model to recognize an utterance domain in the first language and the second language by simultaneously using the first training utterance and the second training utterance in the multilingual training corpus.” is still being taught by Li et al. This is because the reference associates the training examples with a domain and uses them in combination to train the language model (Paragraphs 30, 52, and 53). More details on this rejection can be found below
With regards to the rejections under 35 U.S.C. 103, the applicant asserts that these are no longer applicable due to the changes to the independent claims rejected under Li et al. These rejections still apply as the amended claim that overcame the Li et al. 35 U.S.C. 102 rejection has been rejected using Hwang et al. Thus, claims 1-6 and 8-13 are now rejected under Li et al. in view of Hwang et al. and claims 7 and 14-20 are rejected under Li et al. in view of Hwang et al. and Bohra et al. Details can be found below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1 and 8 recite generating, using at least one [processing device] of an [electronic device], a multilingual training corpus comprising labeled utterances in multiple languages including a first language and a second language, the multilingual training corpus comprising at least one utterance that has been translated into the multiple languages, wherein generating the multilingual training corpus comprises: performing a translation operation on a first training utterance in the first language from a training dataset to convert the first training utterance into a second training utterance in the second language; and performing a concatenation operation to add the second training utterance to the multilingual training corpus; and fine-tuning, using the at least one processing device, a [multilingual language model] using the multilingual training corpus, wherein fine-tuning the multilingual language model comprises training the multilingual language model to recognize an utterance domain in the first language and the second language by simultaneously using the first training utterance and the second training utterance in the multilingual training corpus.
The limitations in these claims, as drafted, are a process that, under broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. The human mind is capable of forming a collection or corpus of translated utterances with a detected domain. For example, someone can listen to someone speak in one language, write down the translation of the words in a different language along with the domain of the speech, and store multiple translations in a notebook. Humans are also capable of knowing multiple languages and thus translating things into multiple languages. This corpus can be used for fine-tuning related tasks such as teaching someone else a language, creating a reference guide for translations, or making the design decision to include their translations as training data for a language model. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claims recite the additional elements of a processor, an electronic device, and a language model. The processor is detailed in paragraph 31 of the specification and is described as a generic component. The electronic device is detailed in Fig. 1 and throughout paragraphs 29-42 of the specification and is described as capable of taking many general-purpose forms. The language model is detailed in paragraph 46 of the specification and can take the form of any transformer-based model such as BERT. Accordingly, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claims are not patent eligible.
As to claims 8 and 15 (with minor modification), method claims 1 and 15 and apparatus claim 8 are related as apparatus and the method of using same, with each claimed element's step corresponding to the claimed apparatus function. Accordingly claim 8 and 15 are similarly rejected under the same rationale as applied above with respect to apparatus claim 1. Additional rejection for claim 15 is detailed below, however the matching dependent claims share a rejection rationale.
Claim 15 recites receiving, using at least one processing device of an electronic device, an input utterance in a first language; applying, using the at least one processing device, a translation model to translate the input utterance into a second language; inputting, using the at least one processing device, the translated input utterance to a multilingual language model to predict a domain of the translated input utterance; and providing, using the at least one processing device, the predicted domain to a user; wherein the multilingual language model is trained using a multilingual training corpus comprising labeled utterances in multiple languages including the first language and the second language, the multilingual training corpus comprising at least one utterance that has been translated into the multiple languages; wherein the multilingual training corpus is generated by performing (i) a translation operation on a first training utterance in the first language from a training dataset to convert the first training utterance into a second training utterance in the second language and (ii) a concatenation operation to add the second training utterance to the multilingual training corpus; and wherein the multilingual language model is fine-tuned using the multilingual training corpus to recognize an utterance domain in the first language and the second language by simultaneously using the first training utterance and the second training utterance in the multilingual training corpus.
The limitations in this claim, as drafted, are a process that, under broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. The human mind is capable of receiving an utterance in order to translate it. A translation model in this case could be a reference book to translating a language or another person that is acting as a translator. A human is also capable of interpreting the domain of an utterance using their own prior knowledge. The human mind is capable of forming a collection or corpus of translated utterances with a detected domain. For example, someone can listen to someone speak in one language, write down the translation of the words in a different language along with the domain of the speech, and store multiple translations in a notebook. Humans are also capable of knowing multiple languages and thus translating things into multiple languages. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claim recites the additional elements of a processor, an electronic device, and a language model. The processor is detailed in paragraph 31 of the specification and is described as a generic component. The electronic device is detailed in Fig. 1 and throughout paragraphs 29-42 of the specification and is described as capable of taking many general-purpose forms. The language model is detailed in paragraph 46 of the specification and can take the form of any transformer-based model such as BERT. Accordingly, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim is not patent eligible.
Claims 2, 9, and 16 recites wherein generating the multilingual training corpus comprises: obtaining the first training utterance in the first language from the training dataset, the first training utterance having at least one domain label; delexicalizing the first training utterance into at least one slot and a remainder portion; translating the remainder portion into the second language; converting each of the at least one slot into the second language using locale-specific information; relexicalizing the at least one slot and the remainder portion into the second training utterance; and adding the second training utterance to the multilingual training corpus.
The limitations in these claims, as drafted, are a process that, under broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. A human can receive an utterance with a domain attached to it, for example, a labeled sentence on a piece of paper. A human could delexicalize it by separating parts that have a certain label. A human could translate the parts of the sentence into another language using different strategies such as using their own knowledge of the language vs following a reference guide. A human could then put the sentence back together and use it as training data. Where the training data could be them building their own translation reference guide or merely making the design decision to use that translation as training data. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claims do not recite any additional elements that were not present in the independent claims. Accordingly, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claims are not patent eligible.
Claims 3, 10, and 17 recites further comprising: repeating the obtaining, delexicalizing, translating, converting, relexicalizing, and adding for multiple training utterances and multiple second languages.
The limitations in these claims, as drafted, are a process that, under broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. A human is capable of learning many languages and repeating the previous process for all of them. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claims do not recite any additional elements that were not present in the independent claims. Accordingly, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claims are not patent eligible.
Claims 4, 11, and 18 recites wherein: the second language is associated with a specific locale; and the locale-specific information corresponds to the specific locale.
The limitations in these claims, as drafted, are a process that, under broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. A human can associate text with a locale if they are familiar enough with the language and how it differs in different regions. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claims do not recite any additional elements that were not present in the independent claims. Accordingly, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claims are not patent eligible.
Claims 5, 12, and 19 recites wherein the second training utterance has the same at least one domain label as the first training utterance.
The limitations in these claims, as drafted, are a process that, under broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. A human could translate an utterance and label the translation with domains that match the pre-translated labels. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claims do not recite any additional elements that were not present in the independent claims. Accordingly, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claims are not patent eligible.
Claims 6, 13, and 20 recites wherein the remainder portion is translated into the second language using an Internet-based language translation tool.
The limitations in these claims, as drafted, are a process that, under broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. A human is capable of using an internet-based translation tool or creating/using their own tool such as a reference guide to translating one language to another. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claims do not recite any additional elements that were not present in the independent claims. Accordingly, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claims are not patent eligible.
Claims 7 and 14 recites wherein the multilingual language model is configured to predict a domain of an input utterance that has been translated from the first language into the second language.
The limitations in these claims, as drafted, are a process that, under broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. A human is capable of predicting the domain of an utterance using prior knowledge and context clues. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claims do not recite any additional elements that were not present in the independent claims. Accordingly, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6 and 8-13 are rejected under 35 U.S.C. 103 as being unpatentable over US Patent Publication US 12236205 B2 (Li et al.) in view of US Patent Publication US 9613027 B2 (Hwang et al.).
Regarding Claims 1 and 8, Li et al. teaches A method comprising:
(An example method implemented in a data processing system for generating training data for a multilingual natural language processing model includes…) (Col. 2, Lines 36-38).
Alternatively, claim 8 recites An electronic device comprising: at least one processing device configured to:
(An example data processing system according to the disclosure may include a processor and a computer-readable medium storing executable instructions. The instructions when executed cause the processor to perform operations including…) (Col. 2, Lines 8-12).
generating, using at least one processing device of an electronic device, a multilingual training corpus comprising labeled utterances in multiple languages including a first language and a second language,
(The process 500 may include an operation 510 of obtaining a corpus 280 comprising a plurality of first content items and a plurality of second content items. The first content items comprise English-language textual content, and the plurality of second content items comprise translations of the first content items in one or more non-English target languages.) (Col. 12, Lines 9-15).
(The process 500 may include an operation 560 of generating first training data for fine tuning a multilingual NLP model by associating the first candidate label with the first content item.) (Col 13, Lines 8-11).
(The process 500 may include an operation 570 of generating second training data for fine tuning the multilingual NLP model by associating the first candidate label with a second content item of the plurality of second content items.) (Col 13, Lines 18-21).
Li et al. creates a multilingual training corpus with labeled utterances (textual content) and a first and second language. The labels connect the training data in each of the languages and it is all used in the corpus for training a model. This method can be visualized in Fig. 5.
the multilingual training corpus comprising at least one utterance that has been translated into the multiple languages;
(The first content items comprise English-language textual content, and the plurality of second content items comprise translations of the first content items in one or more non-English target languages.) (Col. 12, Lines 9-15).
(As discussed above, the parallel corpus 280 may include multiple parallel translations so that the English-language element has corresponding translations in multiple target languages.) (Col. 7, Lines 62-65).
Li et al. states how the method can be applied to multiple languages, thus meaning the first content item would be associated with multiple second content items in different languages.
and fine-tuning, using the at least one processing device, a multilingual language model using the multilingual training corpus.
(The process 500 may include an operation 580 of training a pretrained multilingual NLP model with the first training data and the second training data to fine tune the training of the NLP model with respect to English and a respective non-English target language associated with the second content item.) (Col 13, Lines 26-31).
The first and second training data, associated to each other by the labels, is used to train and fine tune a multilingual language model.
wherein fine-tuning the multilingual language model comprises training the multilingual language model to recognize an utterance domain in the first language and the second language by simultaneously using the first training utterance and the second training utterance in the multilingual training corpus.
(Otherwise, if a majority of the English-language NLP models 245 agreed on the context of the selected parallel corpora element 235 consisting of the English-language text, then the candidate labels associated with the selected parallel corpora element 235 is likely to be correct, and the voter logic unit 250 outputs a label associated with the English-language sentence or phrase 255 which may be used as training data for the multilingual NLP.) (Paragraph 30)
(The process 500 may include an operation 560 of generating first training data for fine tuning a multilingual NLP model by associating the first candidate label with the first content item. … The process 500 may include an operation 570 of generating second training data for fine tuning the multilingual NLP model by associating the first candidate label with a second content item of the plurality of second content items. … The process 500 may include an operation 580 of training a pretrained multilingual NLP model with the first training data and the second training data to fine tune the training of the NLP model with respect to English and a respective non-English target language associated with the second content item.) (Paragraphs 51-53)
The process creates candidate labels for words that associate them with an English sentence or phrase. The selected candidate label gets associated with first and second content items (English and translated words). Both pieces of training data are then used to train the language model.
Li et al. does not explicitly teach: wherein generating the multilingual training corpus comprises: performing a translation operation on a first training utterance in the first language from a training dataset to convert the first training utterance into a second training utterance in the second language; and performing a concatenation operation to add the second training utterance to the multilingual training corpus;
However, Hwang et al. teaches wherein generating the multilingual training corpus comprises: performing a translation operation on a first training utterance in the first language from a training dataset to convert the first training utterance into a second training utterance in the second language;
(Transitioning to operation 330, the carrier phrases are translated from the first language to the second language. The carrier phrases may be translated using machine translation or human translation, or some combination of machine translation and human translation.) (Col. 8, Lines 23-27).
Hwang et al. specifically teaches starting with a dataset in one language and using machine translation to convert it to a new language.
and performing a concatenation operation to add the second training utterance to the multilingual training corpus;
(Transitioning to operation 350, the training data that is created from the process can be used to train a model.) (Col. 8, Lines 40-41).
In Fig. 2 Path 1 shows the slot abstraction method of Fig. 3 where the machine translator first translates just the carrier phrase, the slot filler translates the slot. The resulting, relexicalized output can be seen in training data set (a) at the bottom of the figure.
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the multilingual language model as taught by Li et al. to translate a first language in dataset into a second language using to create training data as taught by Hwang et al. This would have been an obvious improvement as it reduces additional time and expense from expanding the system to new languages. (Hwang et al. Col. 1, Lines 8-15).
Regarding Claims 2 and 9, Li et al. in view of Hwang et al. teaches the method of claims 1 and 8.
Furthermore, Hwang et al. teaches wherein generating the multilingual training corpus comprises: obtaining the first training utterance in the first language from the training dataset,
(After a start operation, process 300 moves to operation 310, where training data in a first language is accessed. For example, the training data may include annotated sentences in English or some other language.) (Col. 8 Lines 5-8).
This method is visualized in Fig. 3 of Hwang et al. where the first step is obtaining a sentence in one language from the training dataset
the first training utterance having at least one domain label;
(The sentences in the training data may be manually annotated or automatically annotated. Flowing to operation 320, slot abstraction on the annotated sentences is performed. The slot labels, carrier phrase(s) and the slot(s) are determined for each annotated sentence.) (Col. 8, Lines 8-14).
The domain labels are present in the form of annotations done on the first utterance. Fig. 2 shows an example of these labels throughout the translation process.
delexicalizing the first training utterance into at least one slot and a remainder portion;
(Flowing to operation 320, slot abstraction on the annotated sentences is performed. The slot labels, carrier phrase(s) and the slot(s) are determined for each annotated sentence. A carrier phrase is a portion of the sentence that is not a slot label or a slot value. For example, in the sentence “albums out <music_release_date> this week <music_release_date>” the carrier phrase is “albums out.” The slot value is “this week” and the slot label is “music_release_date.”) (Col. 8, Lines 11-19).
The sentence is delexicalized into carrier phrases (remainder portions) and labeled slots.
translating the remainder portion into the second language;
(Transitioning to operation 330, the carrier phrases are translated from the first language to the second language. The carrier phrases may be translated using machine translation or human translation, or some combination of machine translation and human translation.) (Col. 8, Lines 23-27).
The carrier phrases are translated normally using any suitable machine translation.
converting each of the at least one slot into the second language using locale-specific information;
(Moving to operation 340, the tokens are replaced with entities. Generally, entities that are locale-dependent (e.g., album names, city names . . . ) are replaced with locale-dependent values. For example, the locale-dependent values may be obtained from content sources storing locale-dependent entities. Locale-dependent entities may also be determined using other methods (e.g., search, manual input, and the like).) (Col. 8, Lines 30-37).
The slots are replaced with a locale specific translation.
relexicalizing the at least one slot and the remainder portion into the second training utterance;
In Fig. 2 Path 1 shows the slot abstraction method of Fig. 3 where the machine translator first translates just the carrier phrase, the slot filler translates the slot. The resulting, relexicalized output can be seen in training data set (a) at the bottom of the figure.
and adding the second training utterance to the multilingual training corpus.
(Transitioning to operation 350, the training data that is created from the process can be used to train a model.) (Col. 8, Lines 40-41).
The translations are used as training data to train a translation language model
Regarding Claims 3 and 10, Li et al. in view of Hwang et al. teaches the method of claims 2 and 9.
Furthermore, Hwang et al. teaches repeating the obtaining, delexicalizing, translating, converting, relexicalizing, and adding for multiple training utterances and multiple second languages.
(After a start operation, process 300 moves to operation 310, where training data in a first language is accessed. For example, the training data may include annotated sentences in English or some other language. The sentences in the training data may be manually annotated or automatically annotated.) (Col.8, Lines 5-10).
Hwang et al. uses “sentences” instead of “utterances” and states how multiple sentences are used in order to build up the training corpus
Regarding Claims 4 and 11, Li et al. in view of Hwang et al. teaches the method of claims 2 and 9.
Furthermore, Hwang et al. teaches wherein: the second language is associated with a specific locale;
(Slot values may or may not be locale-dependent entities. Locale-dependent entities are slot values that are dependent upon the locale(s) of the second language. For example, city entities are different for a locale in China as compared to the city entities in America or Taiwan.) (Col. 3, Lines 20-25).
Hwang et al. uses associates the translation with a locale, for example, translating city names to specifically how they would be written in Chinese
and the locale-specific information corresponds to the specific locale.
(Generally, entities that are locale-dependent (e.g., album names, city names . . . ) are replaced with locale-dependent values. For example, the locale-dependent values may be obtained from content sources storing locale-dependent entities. Locale-dependent entities may also be determined using other methods (e.g., search, manual input, and the like).) (Col. 8, Lines 31-37).
Once again, the locale specific slot labels have corresponding information to the locale of the translation such as Chinese cities.
Regarding Claims 5 and 12, Li et al. in view of Hwang et al. teaches the method of claims 2 and 9.
Furthermore, Hwang et al. teaches wherein the second training utterance has the same at least one domain label as the first training utterance.
In Fig. 2 Path 1 follow the method shown in Fig. 1. It can be seen that the result training dataset (a) at the bottom maintains the labels as the input sentence at the top.
Regarding Claims 6 and 13, Li et al. in view of Hwang et al. teaches the method of claims 2 and 9.
Furthermore, Hwang et al. teaches wherein the remainder portion is translated into the second language using an Internet-based language translation tool.
(Machine translator 220 may use a general translator (e.g., a general translation service provided by Bing®, Google® . . . ), a domain specific translator, or some combination of a general translator and a domain specific translator.) (Col. 4, Lines 55-59).
Hwang et al. uses regular machine translation methods such as Google for translating the carrier phrases as can be seen in Fig. 2.
Claims 7 and 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over US Patent Publication US 12236205 B2 (Li et al.) in view of US Patent Publication US 9613027 B2 (Hwang et al.) and further in view US Patent Publication US 12393792 B2 (Bohra et al.).
Regarding Claims 7 and 14, Li et al. in view of Hwang et al. teaches the method of claims 1 and 8.
Li et al. in view of Hwang et al. does not explicitly teach: wherein the multilingual language model is configured to predict a domain of an input utterance that has been translated from the first language into the second language.
However, Bohra et al. teaches wherein the multilingual language model is configured to predict a domain of an input utterance that has been translated from the first language into the second language.
(An example method implemented in a data processing system for providing content recommendations based on a multilingual natural language processing model includes obtaining textual content in a first language from a first client device; segmenting the textual content into a plurality of first tokens; translating the plurality of first tokens to a second language using a first bilingual dictionary to create a plurality of second tokens; analyzing the second tokens to extract features information from the plurality of second tokens to generate a feature vector; providing the feature vector as an input to a first natural language processing model trained to analyze textual content in the second language, the natural language processing model being configured to output contextual information indicating one or more topics or subject matter of the first textual content; providing the contextual information obtained from the first natural language processing model as an input to a first machine learning model, the first machine learning model configured to analyze the contextual information and to output information identifying one or more content items predicted to be relevant to the contextual information; and providing the information identifying the one or more content items to the first client device.) (Col 2, Lines 14-36).
(The process 600 may include an operation 660 of providing the contextual information obtained from the first natural language processing model as an input to a first machine learning model. The first machine learning model may be configured to analyze the contextual information and to output information identifying one or more content items predicted to be relevant to the contextual information.) (Col. 13, Lines 51-57).
Bohra et al. teaches a multilingual language model that predicts a domain from a sentence that has been translated from one language to another
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the multilingual language model as taught by Li et al. in view of Hwang et al. to detect the domain of the translated sentence as taught by Bohra et al. This would have been an obvious improvement as domain detection is a commonly used tool in language processing and applications using it have a wider audience if it can create output in multiple languages. (Bohra et al. Col. 1, Lines 6-16).
Regarding Claim 15, Li et al. teaches A method comprising:
(An example method implemented in a data processing system for generating training data for a multilingual natural language processing model includes…) (Col. 2, Lines 36-38).
receiving, using at least one processing device of an electronic device, an input utterance in a first language;
(In the example shown in FIG. 1, the text analysis service 110 is implemented as a cloud-based service or set of services. The text analysis service 110 may be configured to receive a request to analyze textual content from the client device 105 and/or the application service 125. The text analysis service 110 may include one or more NLP models that are configured to analyze the textual input and provide an output based on the textual input based on a contextual analysis of the textual input.) (Col. 4, Lines 46-54).
Li et al. teaches a method which can begin by receiving a text input from the user’s device.
wherein the multilingual language model is trained using a multilingual training corpus comprising labeled utterances in multiple languages including the first language and the second language,
(The process 500 may include an operation 510 of obtaining a corpus 280 comprising a plurality of first content items and a plurality of second content items. The first content items comprise English-language textual content, and the plurality of second content items comprise translations of the first content items in one or more non-English target languages.) (Col. 12, Lines 9-15).
(The process 500 may include an operation 560 of generating first training data for fine tuning a multilingual NLP model by associating the first candidate label with the first content item.) (Col 13, Lines 8-11).
(The process 500 may include an operation 570 of generating second training data for fine tuning the multilingual NLP model by associating the first candidate label with a second content item of the plurality of second content items.) (Col 13, Lines 18-21).
Li et al. creates a multilingual training corpus with labeled utterances (textual content) and a first and second language. The labels connect the training data in each of the languages and it is all used in the corpus for training a model. This method can be visualized in Fig. 5.
the multilingual training corpus comprising at least one utterance that has been translated into the multiple languages;
(The first content items comprise English-language textual content, and the plurality of second content items comprise translations of the first content items in one or more non-English target languages.) (Col. 12, Lines 9-15).
(As discussed above, the parallel corpus 280 may include multiple parallel translations so that the English-language element has corresponding translations in multiple target languages.) (Col. 7, Lines 62-65).
Li et al. states how the method can be applied to multiple languages, thus meaning the first content item would be associated with multiple second content items in different languages.
and wherein the multilingual language model is fine-tuned using the multilingual training corpus to recognize an utterance domain in the first language and the second language by simultaneously using the first training utterance and the second training utterance in the multilingual training corpus.
(Otherwise, if a majority of the English-language NLP models 245 agreed on the context of the selected parallel corpora element 235 consisting of the English-language text, then the candidate labels associated with the selected parallel corpora element 235 is likely to be correct, and the voter logic unit 250 outputs a label associated with the English-language sentence or phrase 255 which may be used as training data for the multilingual NLP.) (Paragraph 30)
(The process 500 may include an operation 560 of generating first training data for fine tuning a multilingual NLP model by associating the first candidate label with the first content item. … The process 500 may include an operation 570 of generating second training data for fine tuning the multilingual NLP model by associating the first candidate label with a second content item of the plurality of second content items. … The process 500 may include an operation 580 of training a pretrained multilingual NLP model with the first training data and the second training data to fine tune the training of the NLP model with respect to English and a respective non-English target language associated with the second content item.) (Paragraphs 51-53)
The process creates candidate labels for words that associate them with an English sentence or phrase. The selected candidate label gets associated with first and second content items (English and translated words). Both pieces of training data are then used to train the language model.
Li et al. does not explicitly teach: applying, using the at least one processing device, a translation model to translate the input utterance into a second language; inputting, using the at least one processing device, the translated input utterance to a multilingual language model to predict a domain of the translated input utterance; and providing, using the at least one processing device, the predicted domain to a user; wherein the multilingual training corpus is generated by performing (i) a translation operation on a first training utterance in the first language from a training dataset to convert the first training utterance into a second training utterance in the second language and (ii) a concatenation operation to add the second training utterance to the multilingual training corpus;
However, Hwang et al. teaches wherein the multilingual training corpus is generated by performing (i) a translation operation on a first training utterance in the first language from a training dataset to convert the first training utterance into a second training utterance in the second language
(Transitioning to operation 330, the carrier phrases are translated from the first language to the second language. The carrier phrases may be translated using machine translation or human translation, or some combination of machine translation and human translation.) (Col. 8, Lines 23-27).
Hwang et al. specifically teaches starting with a dataset in one language and using machine translation to convert it to a new language.
and (ii) a concatenation operation to add the second training utterance to the multilingual training corpus;
(Transitioning to operation 350, the training data that is created from the process can be used to train a model.) (Col. 8, Lines 40-41).
In Fig. 2 Path 1 shows the slot abstraction method of Fig. 3 where the machine translator first translates just the carrier phrase, the slot filler translates the slot. The resulting, relexicalized output can be seen in training data set (a) at the bottom of the figure.
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the multilingual language model as taught by Li et al. to translate a first language in dataset into a second language using to create training data as taught by Hwang et al. This would have been an obvious improvement as it reduces additional time and expense from expanding the system to new languages. (Hwang et al. Col. 1, Lines 8-15).
Li et al. in view of Hwang et al. does not explicitly teach: applying, using the at least one processing device, a translation model to translate the input utterance into a second language; inputting, using the at least one processing device, the translated input utterance to a multilingual language model to predict a domain of the translated input utterance; and providing, using the at least one processing device, the predicted domain to a user;
However, Bohra et al. teaches applying, using the at least one processing device, a translation model to translate the input utterance into a second language;
(An example method implemented in a data processing system for providing content recommendations based on a multilingual natural language processing model includes obtaining textual content in a first language from a first client device; segmenting the textual content into a plurality of first tokens; translating the plurality of first tokens to a second language using a first bilingual dictionary to create a plurality of second tokens) (Col 2, Lines 14-36).
Bohra et al. takes a user input and translates using bilingual dictionary which acts as the translation model.
inputting, using the at least one processing device, the translated input utterance to a multilingual language model to predict a domain of the translated input utterance;
(analyzing the second tokens to extract features information from the plurality of second tokens to generate a feature vector; providing the feature vector as an input to a first natural language processing model trained to analyze textual content in the second language, the natural language processing model being configured to output contextual information indicating one or more topics or subject matter of the first textual content;) (Col 2, Lines 14-36).
The translated term is input into language model to detect contextual information which acts as a domain.
and providing, using the at least one processing device, the predicted domain to a user;
(providing the contextual information obtained from the first natural language processing model as an input to a first machine learning model, the first machine learning model configured to analyze the contextual information and to output information identifying one or more content items predicted to be relevant to the contextual information; and providing the information identifying the one or more content items to the first client device.) (Col 2, Lines 14-36).
(The process 600 may include an operation 660 of providing the contextual information obtained from the first natural language processing model as an input to a first machine learning model. The first machine learning model may be configured to analyze the contextual information and to output information identifying one or more content items predicted to be relevant to the contextual information.) (Col. 13, Lines 51-57).
The domain is output to the user in the form of content items relevant to the contextual information.
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the multilingual language model as taught by Li et al. in view of Hwang et al. to detect the domain of the translated sentence as taught by Bohra et al. This would have been an obvious improvement as domain detection is a commonly used tool in language processing and applications using it have a wider audience if it can create output in multiple languages. (Bohra et al. Col. 1, Lines 6-16).
Regarding Claim 16, Li et al. in view of Hwang et al. and Bohra et al. teaches the method of claim 15.
Furthermore, Hwang et al. teaches wherein generating the multilingual training corpus comprises: obtaining the first training utterance in the first language from the training dataset,
(After a start operation, process 300 moves to operation 310, where training data in a first language is accessed. For example, the training data may include annotated sentences in English or some other language.) (Col. 8 Lines 5-8).
This method is visualized in Fig. 3 of Hwang et al. where the first step is obtaining a sentence in one language from the training dataset
the first training utterance having at least one domain label;
(The sentences in the training data may be manually annotated or automatically annotated. Flowing to operation 320, slot abstraction on the annotated sentences is performed. The slot labels, carrier phrase(s) and the slot(s) are determined for each annotated sentence.) (Col. 8, Lines 8-14).
The domain labels are present in the form of annotations done on the first utterance. Fig. 2 shows an example of these labels throughout the translation process.
delexicalizing the first training utterance into at least one slot and a remainder portion;
(Flowing to operation 320, slot abstraction on the annotated sentences is performed. The slot labels, carrier phrase(s) and the slot(s) are determined for each annotated sentence. A carrier phrase is a portion of the sentence that is not a slot label or a slot value. For example, in the sentence “albums out <music_release_date> this week <music_release_date>” the carrier phrase is “albums out.” The slot value is “this week” and the slot label is “music_release_date.”) (Col. 8, Lines 11-19).
The sentence is delexicalized into carrier phrases (remainder portions) and labeled slots.
translating the remainder portion into the second language;
(Transitioning to operation 330, the carrier phrases are translated from the first language to the second language. The carrier phrases may be translated using machine translation or human translation, or some combination of machine translation and human translation.) (Col. 8, Lines 23-27).
The carrier phrases are translated normally using any suitable machine translation.
converting each of the at least one slot into the second language using locale-specific information;
(Moving to operation 340, the tokens are replaced with entities. Generally, entities that are locale-dependent (e.g., album names, city names . . . ) are replaced with locale-dependent values. For example, the locale-dependent values may be obtained from content sources storing locale-dependent entities. Locale-dependent entities may also be determined using other methods (e.g., search, manual input, and the like).) (Col. 8, Lines 30-37).
The slots are replaced with a locale specific translation.
relexicalizing the at least one slot and the remainder portion into the second training utterance;
In Fig. 2 Path 1 shows the slot abstraction method of Fig. 3 where the machine translator first translates just the carrier phrase, the slot filler translates the slot. The resulting, relexicalized output can be seen in training data set (a) at the bottom of the figure.
and adding the second training utterance to the multilingual training corpus.
(Transitioning to operation 350, the training data that is created from the process can be used to train a model.) (Col. 8, Lines 40-41).
The translations are used as training data to train a translation language model
Regarding Claim 17, Li et al. in view of Hwang et al. and Bohra et al. teaches the method of claim 16.
Furthermore, Hwang et al. teaches repeating the obtaining, delexicalizing, translating, converting, relexicalizing, and adding for multiple training utterances and multiple second languages.
(After a start operation, process 300 moves to operation 310, where training data in a first language is accessed. For example, the training data may include annotated sentences in English or some other language. The sentences in the training data may be manually annotated or automatically annotated.) (Col.8, Lines 5-10).
Hwang et al. uses “sentences” instead of “utterances” and states how multiple sentences are used in order to build up the training corpus
Regarding Claim 18, Li et al. in view of Hwang et al. and Borha et al. teaches the method of claim 16.
Furthermore, Hwang et al. teaches wherein: the second language is associated with a specific locale;
(Slot values may or may not be locale-dependent entities. Locale-dependent entities are slot values that are dependent upon the locale(s) of the second language. For example, city entities are different for a locale in China as compared to the city entities in America or Taiwan.) (Col. 3, Lines 20-25).
Hwang et al. uses associates the translation with a locale, for example, translating city names to specifically how they would be written in Chinese
and the locale-specific information corresponds to the specific locale.
(Generally, entities that are locale-dependent (e.g., album names, city names . . . ) are replaced with locale-dependent values. For example, the locale-dependent values may be obtained from content sources storing locale-dependent entities. Locale-dependent entities may also be determined using other methods (e.g., search, manual input, and the like).) (Col. 8, Lines 31-37).
Once again, the locale specific slot labels have corresponding information to the locale of the translation such as Chinese cities.
Regarding Claim 19, Li et al. in view of Hwang et al. and Bohra et al. teaches the method of claim 16.
Furthermore, Hwang et al. teaches wherein the second training utterance has the same at least one domain label as the first training utterance.
In Fig. 2 Path 1 follow the method shown in Fig. 1. It can be seen that the result training dataset (a) at the bottom maintains the labels as the input sentence at the top.
Regarding Claim 20, Li et al. in view of Hwang et al. and Bohra et al. teaches the method of claim 16.
Furthermore, Hwang et al. teaches wherein the remainder portion is translated into the second language using an Internet-based language translation tool.
(Machine translator 220 may use a general translator (e.g., a general translation service provided by Bing®, Google® . . . ), a domain specific translator, or some combination of a general translator and a domain specific translator.) (Col. 4, Lines 55-59).
Hwang et al. uses regular machine translation methods such as Google for translating the carrier phrases as can be seen in Fig. 2.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS DANIEL LOWEN whose telephone number is (571)272-5828. The examiner can normally be reached Mon-Fri 8:00am - 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Paras D Shah can be reached at (571) 270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICHOLAS D LOWEN/Examiner, Art Unit 2653
/Paras D Shah/Supervisory Patent Examiner, Art Unit 2653 06/24/2026
Prosecution Insights