DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Introduction
This office action is in response to communications field 05/22/2025. Claims 1-20 are pending and likewise have been examined.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 7-8 and 14 of U.S. Patent No. 11928440 in view of Kim et al. “Improving Unsupervised Word-by-Word Translation with Language Model and Denoising Autoencoder”, hereinafter Kim. See Table below for limitations which are claimed by U.S. Patent No. 11928440, and see below table for limitations taught by Kim.
Instant Application
U.S. Patent No. 11928440
Claim 1: A method comprising: training, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state;
See combination with Kim below
additionally training the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries;
See combination with Kim below
receiving a mixed language query comprising the first language and the second language;
Claim 1: receiving, at a computing device, an input; determining that the input comprises a multilingual query comprising at least a first source language and a second source language;
inputting the received mixed language query into the additionally trained machine learning model to generate an output in the first language;
translating, word for word, the multilingual query into the destination language to produce a monolingual query(See combination with Kim Below for additionally trained machine learning model specifically)
and outputting the generated output in the first language.
translating, word for word, the multilingual query into the destination language to produce a monolingual query
Claim 2: The method of claim 1 wherein the data from the first language is a mathematical representation of the first language.
See Note#1 below
Claim 3: The method of claim 2, wherein the mathematical representation is a language vector that represents relationships found between words in the first language.
See Note#1 below
Claim 4: The method of claim 1, wherein the data from the second language is a mathematical representation of the second language.
See Note#1 below
Claim 5: The method of claim 4, wherein the mathematical representation is a language vector that represents relationships found between words in the second language.
See Note#1 below
Claim 6: The method of claim 1, wherein the hidden state reflects structures of phrases in the first language.
See Note#1 below
Claim 7: The method of claim 1, wherein the additionally training comprises utilizing an encoder-decoder network.
See Note#1 below
Claim 8: The method of claim 7, wherein the encoder-decoder network is a bidirectional long short-term memory network.
See Note#1 below
Claim 9: The method of claim 1, wherein outputting the generated output in the first language comprises identifying the first language based, at least in part, on data stored in a user profile.
Claim 7: The method of claim 1, wherein identifying the destination language further comprises: accessing a user profile; and identifying, via the user profile, a user setting for a destination language.
Claim 10: The method of claim 1, wherein outputting the generated output in the first language comprises identifying the first language based, at least in part, on a language of an operating system associated with the output.
Claim 1: selecting, based on the identified language of the operating system, a destination language
Claim 11: A system comprising: input/output circuitry configured to:
Claim 8: A system for handling a multilingual query comprising two or more languages, the system comprising: a communications port; and control circuitry communicatively coupled to the communications port and configured to:
train, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state; additionally train the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries;
See combination with Kim below
receive a mixed language query comprising the first language and the second language;
Claim 8: receive an input via the communications port; determine that the input comprises a multilingual query comprising at least a first source language and a second source language;
and processing circuitry configured to: input the received mixed language query into the additionally trained machine learning model to generate an output in the first language;
translate, word for word, the multilingual query into the destination language to produce a monolingual query(See combination with Kim Below for additionally trained machine learning model specifically)
and output the generated output in the first language.
translate, word for word, the multilingual query into the destination language to produce a monolingual query
Claim 12: The system of claim 11 wherein the data from the first language is a mathematical representation of the first language.
See Note#1 below
Claim 13: The system of claim 12, wherein the mathematical representation is a language vector that represents relationships found between words in the first language.
See Note#1 below
Claim 14: The system of claim 11, wherein the data from the second language is a mathematical representation of the second language.
See Note#1 below
Claim 15: The system of claim 14, wherein the mathematical representation is a language vector that represents relationships found between words in the second language.
See Note#1 below
Claim 16: The system of claim 11, wherein the hidden state reflects structures of phrases in the first language.
See Note#1 below
Claim 17: The system of claim 11, wherein the additionally training comprises utilizing an encoder-decoder network.
See Note#1 below
Claim 18: The system of claim 17, wherein the encoder-decoder network is a bidirectional long short-term memory network.
See Note#1 below
Claim 19: The system of claim 11, wherein the processing circuitry configured to output the generated output in the first language is configured to identify the first language based, at least in part, on data stored in a user profile.
Claim 14: The system of claim 8, wherein the control circuitry configured to identify the destination language is further configured to: access a user profile; and identify, via the user profile, a user setting for a destination language.
Claim 20: The system of claim 11, wherein the processing circuitry configured to output the generated output in the first language is configured to identify the first language based, at least in part, on a language of an operating system associated with the output.
Claim 8: identify a language of an operating system running on the computing device; select, based on the identified language of the operating system, a destination language;
Regarding Claim 1:
U.S. Patent No. 11928440 claims the limitations shown in the table above, but does not claim a method comprising: training, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state; additionally training the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries.
In the same field of Machine Translation Kim teaches a method comprising: training, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state(Pg 863, 2 Cross-lingual Word embedding, Para 1, Ln 1-7, As a basic step for unsupervised MT, we learn a word translation model from monolingual corpora of each language. Pg 863, 2 Cross-lingual Word embedding, Para 2, Cross-lingual word embedding is a continuous representation of words whose vector space is shared across multiple languages. This enables distance calculation between word embeddings across languages, which is actually finding translation candidates. Pg 863, 2 Cross-lingual Word embedding, Para 3, Ln 1-12, We train cross-lingual word embedding in a fully unsupervised manner. Pg 863, 2 Cross-lingual Word embedding, Para 4, Ln 1-7, Once we have the cross-lingual mapping, we can transform the embedding of a given source word and find a target word with the closest embedding, i.e. nearest neighbor search. Pg 863, 2 Cross-lingual Word embedding, Para 5, Ln 1-10, Run a Procrustes problem solver with the dictionary from Step 3 to re-train the mapping);
additionally training the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries(Pg 863, 2 Cross-lingual Word embedding, Para 4, Ln 1-7, Once we have the cross-lingual mapping, we can transform the embedding of a given source word and find a target word with the closest embedding, i.e. nearest neighbor search. Pg 863, 2 Cross-lingual Word embedding, Para 5, Ln 1-10, Run a Procrustes problem solver with the dictionary from Step 3 to re-train the mapping. Pg 863, 2 Cross-lingual Word embedding, Para 3, Ln 1-12, We train cross-lingual word embedding in a fully unsupervised manner. Pg 863, 2 Cross-lingual Word embedding, Para 1, Ln 1-7, As a basic step for unsupervised MT, we learn a word translation model from monolingual corpora of each language).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify U.S. Patent No. 11928440 with the translation system of Kim, as it can improve translation performance(Pg 862, Abstract, Ln 1-13).
Regarding Claim 11:
U.S. Patent No. 11928440 claims the limitations shown in the table above, but does not claim train, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state; additionally train the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries.
In the same field of Machine Translation Kim teaches train, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state(Pg 863, 2 Cross-lingual Word embedding, Para 1, Ln 1-7, As a basic step for unsupervised MT, we learn a word translation model from monolingual corpora of each language. Pg 863, 2 Cross-lingual Word embedding, Para 2, Cross-lingual word embedding is a continuous representation of words whose vector space is shared across multiple languages. This enables distance calculation between word embeddings across languages, which is actually finding translation candidates. Pg 863, 2 Cross-lingual Word embedding, Para 3, Ln 1-12, We train cross-lingual word embedding in a fully unsupervised manner. Pg 863, 2 Cross-lingual Word embedding, Para 4, Ln 1-7, Once we have the cross-lingual mapping, we can transform the embedding of a given source word and find a target word with the closest embedding, i.e. nearest neighbor search. Pg 863, 2 Cross-lingual Word embedding, Para 5, Ln 1-10, Run a Procrustes problem solver with the dictionary from Step 3 to re-train the mapping);
additionally train the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries(Pg 863, 2 Cross-lingual Word embedding, Para 4, Ln 1-7, Once we have the cross-lingual mapping, we can transform the embedding of a given source word and find a target word with the closest embedding, i.e. nearest neighbor search. Pg 863, 2 Cross-lingual Word embedding, Para 5, Ln 1-10, Run a Procrustes problem solver with the dictionary from Step 3 to re-train the mapping. Pg 863, 2 Cross-lingual Word embedding, Para 3, Ln 1-12, We train cross-lingual word embedding in a fully unsupervised manner. Pg 863, 2 Cross-lingual Word embedding, Para 1, Ln 1-7, As a basic step for unsupervised MT, we learn a word translation model from monolingual corpora of each language).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify U.S. Patent No. 11928440 with the translation system of Kim, as it can improve translation performance(Pg 862, Abstract, Ln 1-13).
Examiner Note #1
Although the claims at issue are not identical, they are not patentably distinct from each other because simply changing the statutory category from computer program product and system, to a method and removing inherent and/or unnecessary limitations/step would be within the level of one of ordinary skill in the art. It is well settled that the omission of an element, e.g. “a computer-readable storage medium”, and its function is an obvious expedient if the remaining elements perform the same function as before. In re Karlson, 136 USPQ 184 (CCPA 1963). Also note Ex parte Rainu, 168 USPQ 375 (Bd. App. 1969). Omission of a reference element or step whose function is not needed would be obvious to one of ordinary skill in the art.
If applicant has any questions regarding this, please call the examiner.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 7, 9, 11, 17 and 19 of U.S. Patent No. 12333268 in view of Kim et al. “Improving Unsupervised Word-by-Word Translation with Language Model and Denoising Autoencoder”, hereinafter Kim. See Table below for limitations which are claimed by U.S. Patent No. 12333268, and see below table for limitations taught by Kim.
Instant Application
U.S. Patent No. 12333268
Claim 1: A method comprising: training, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state;
See combination with Kim below
additionally training the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries;
See combination with Kim below
receiving a mixed language query comprising the first language and the second language;
Claim 1: A method comprising: receiving, at a computing device, a multilingual query comprising at least a first source language and a second source language;
inputting the received mixed language query into the additionally trained machine learning model to generate an output in the first language;
translating, word for word, the words into the destination language to produce a monolingual query (See combination with Kim Below for additionally trained machine learning model specifically)
and outputting the generated output in the first language.
translating, word for word, the words into the destination language to produce a monolingual query
Claim 2: The method of claim 1 wherein the data from the first language is a mathematical representation of the first language.
See Note#1 above
Claim 3: The method of claim 2, wherein the mathematical representation is a language vector that represents relationships found between words in the first language.
See Note#1 above
Claim 4: The method of claim 1, wherein the data from the second language is a mathematical representation of the second language.
See Note#1 above
Claim 5: The method of claim 4, wherein the mathematical representation is a language vector that represents relationships found between words in the second language.
See Note#1 above
Claim 6: The method of claim 1, wherein the hidden state reflects structures of phrases in the first language.
See Note#1 above
Claim 7: The method of claim 1, wherein the additionally training comprises utilizing an encoder-decoder network.
See Note#1 above
Claim 8: The method of claim 7, wherein the encoder-decoder network is a bidirectional long short-term memory network.
See Note#1 above
Claim 9: The method of claim 1, wherein outputting the generated output in the first language comprises identifying the first language based, at least in part, on data stored in a user profile.
Claim 7: The method of claim 1, wherein selecting the destination language further comprises: accessing a user profile; and identifying, via the user profile, a user setting for the destination language.
Claim 10: The method of claim 1, wherein outputting the generated output in the first language comprises identifying the first language based, at least in part, on a language of an operating system associated with the output.
Claim 9: The method of claim 1, wherein selecting the language associated with the computing device further comprises selecting a language of an operating system running on the computing device.
Claim 11: A system comprising: input/output circuitry configured to:
Claim 11: A system comprising: input/output circuitry configured to:
train, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state; additionally train the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries;
See combination with Kim below
receive a mixed language query comprising the first language and the second language;
receive, at a computing device, a multilingual query comprising at least a first source language and a second source language;
and processing circuitry configured to: input the received mixed language query into the additionally trained machine learning model to generate an output in the first language;
translate, word for word, the words into the destination language to produce a monolingual query (See combination with Kim Below for additionally trained machine learning model specifically)
and output the generated output in the first language.
translate, word for word, the words into the destination language to produce a monolingual query
Claim 12: The system of claim 11 wherein the data from the first language is a mathematical representation of the first language.
See Note#1 above
Claim 13: The system of claim 12, wherein the mathematical representation is a language vector that represents relationships found between words in the first language.
See Note#1 above
Claim 14: The system of claim 11, wherein the data from the second language is a mathematical representation of the second language.
See Note#1 above
Claim 15: The system of claim 14, wherein the mathematical representation is a language vector that represents relationships found between words in the second language.
See Note#1 above
Claim 16: The system of claim 11, wherein the hidden state reflects structures of phrases in the first language.
See Note#1 above
Claim 17: The system of claim 11, wherein the additionally training comprises utilizing an encoder-decoder network.
See Note#1 above
Claim 18: The system of claim 17, wherein the encoder-decoder network is a bidirectional long short-term memory network.
See Note#1 above
Claim 19: The system of claim 11, wherein the processing circuitry configured to output the generated output in the first language is configured to identify the first language based, at least in part, on data stored in a user profile.
Claim 17: The system of claim 11, wherein the processing circuitry configured to select the destination language is further configured to: access a user profile; and identify, via the user profile, a user setting for the destination language.
Claim 20: The system of claim 11, wherein the processing circuitry configured to output the generated output in the first language is configured to identify the first language based, at least in part, on a language of an operating system associated with the output.
Claim 19: The system of claim 11, wherein the processing circuitry configured to select the language associated with the computing device is further configured to select a language of an operating system running on the computing device.
Regarding Claim 1:
U.S. Patent No. 12333268 claims the limitations shown in the table above, but does not claim a method comprising: training, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state; additionally training the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries.
In the same field of Machine Translation Kim teaches a method comprising: training, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state(Pg 863, 2 Cross-lingual Word embedding, Para 1, Ln 1-7, As a basic step for unsupervised MT, we learn a word translation model from monolingual corpora of each language. Pg 863, 2 Cross-lingual Word embedding, Para 2, Cross-lingual word embedding is a continuous representation of words whose vector space is shared across multiple languages. This enables distance calculation between word embeddings across languages, which is actually finding translation candidates. Pg 863, 2 Cross-lingual Word embedding, Para 3, Ln 1-12, We train cross-lingual word embedding in a fully unsupervised manner. Pg 863, 2 Cross-lingual Word embedding, Para 4, Ln 1-7, Once we have the cross-lingual mapping, we can transform the embedding of a given source word and find a target word with the closest embedding, i.e. nearest neighbor search. Pg 863, 2 Cross-lingual Word embedding, Para 5, Ln 1-10, Run a Procrustes problem solver with the dictionary from Step 3 to re-train the mapping);
additionally training the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries(Pg 863, 2 Cross-lingual Word embedding, Para 4, Ln 1-7, Once we have the cross-lingual mapping, we can transform the embedding of a given source word and find a target word with the closest embedding, i.e. nearest neighbor search. Pg 863, 2 Cross-lingual Word embedding, Para 5, Ln 1-10, Run a Procrustes problem solver with the dictionary from Step 3 to re-train the mapping. Pg 863, 2 Cross-lingual Word embedding, Para 3, Ln 1-12, We train cross-lingual word embedding in a fully unsupervised manner. Pg 863, 2 Cross-lingual Word embedding, Para 1, Ln 1-7, As a basic step for unsupervised MT, we learn a word translation model from monolingual corpora of each language).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify U.S. Patent No. 12333268 with the translation system of Kim, as it can improve translation performance(Pg 862, Abstract, Ln 1-13).
Regarding Claim 11:
U.S. Patent No. 12333268 claims the limitations shown in the table above, but does not claim train, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state; additionally train the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries.
In the same field of Machine Translation Kim teaches train, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state(Pg 863, 2 Cross-lingual Word embedding, Para 1, Ln 1-7, As a basic step for unsupervised MT, we learn a word translation model from monolingual corpora of each language. Pg 863, 2 Cross-lingual Word embedding, Para 2, Cross-lingual word embedding is a continuous representation of words whose vector space is shared across multiple languages. This enables distance calculation between word embeddings across languages, which is actually finding translation candidates. Pg 863, 2 Cross-lingual Word embedding, Para 3, Ln 1-12, We train cross-lingual word embedding in a fully unsupervised manner. Pg 863, 2 Cross-lingual Word embedding, Para 4, Ln 1-7, Once we have the cross-lingual mapping, we can transform the embedding of a given source word and find a target word with the closest embedding, i.e. nearest neighbor search. Pg 863, 2 Cross-lingual Word embedding, Para 5, Ln 1-10, Run a Procrustes problem solver with the dictionary from Step 3 to re-train the mapping);
additionally train the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries(Pg 863, 2 Cross-lingual Word embedding, Para 4, Ln 1-7, Once we have the cross-lingual mapping, we can transform the embedding of a given source word and find a target word with the closest embedding, i.e. nearest neighbor search. Pg 863, 2 Cross-lingual Word embedding, Para 5, Ln 1-10, Run a Procrustes problem solver with the dictionary from Step 3 to re-train the mapping. Pg 863, 2 Cross-lingual Word embedding, Para 3, Ln 1-12, We train cross-lingual word embedding in a fully unsupervised manner. Pg 863, 2 Cross-lingual Word embedding, Para 1, Ln 1-7, As a basic step for unsupervised MT, we learn a word translation model from monolingual corpora of each language).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify U.S. Patent No. 12333268 with the translation system of Kim, as it can improve translation performance(Pg 862, Abstract, Ln 1-13).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8 and 11-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nakayama et al. “Recognition and Translation of Code-switching Speech Utterances”, hereinafter Nakayama, and further in view of Kim et al. “Improving Unsupervised Word-by-Word Translation with Language Model and Denoising Autoencoder”, hereinafter Kim.
Regarding Claim 1:
Nakayama teaches a method comprising: training, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state(Pg 2, 2 CS2Mono NMT, Para 2, Ln 1-5, we trained NMT to learn CS to monolingual text translation on synthetic and natural data. Our NMT system is a standard attention-based encoder-decoder model. Pg 3, Fig 2, input, token embeddings);
receiving a mixed language query comprising the first language and the second language(Pg 2, 2 CS2Mono NMT, Para 2, Ln 1-5, we trained NMT to learn CS to monolingual text translation on synthetic and natural data. Our NMT system is a standard attention-based encoder-decoder model. Pg 4, B. Natural code-switching, Para 2, Ln 1-7, This corpus includes 0.9-K natural CS sentences for a training set and 0.1K for a test set among all the artificial CSs. Pg 4, Col 2, Para 4(second to last Para in A. Artificial code-switching), Ln 1-2, That corpus was divided 50-K sentences for a training set, 500 for a development set, and 500 for a test set);
inputting the received mixed language query into the additionally trained machine learning model to generate an output in the first language(Pg 2, 2 CS2Mono NMT, Para 2, Ln 1-5, we trained NMT to learn CS to monolingual text translation on synthetic and natural data. Our NMT system is a standard attention-based encoder-decoder model. Pg 3, Fig 2, Japanese-English -> English);
and outputting the generated output in the first language(Pg 2, 2 CS2Mono NMT, Para 2, Ln 1-5, we trained NMT to learn CS to monolingual text translation on synthetic and natural data. Our NMT system is a standard attention-based encoder-decoder model. Pg 3, Fig 2, Japanese-English -> English).
Nakayama does not teach additionally training the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries.
In the same field of Machine Translation, Kim teaches additionally training the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries(Pg 863, 2 Cross-lingual Word embedding, Para 1, Ln 1-7, As a basic step for unsupervised MT, we learn a word translation model from monolingual corpora of each language. Pg 863, 2 Cross-lingual Word embedding, Para 2, Cross-lingual word embedding is a continuous representation of words whose vector space is shared across multiple languages. This enables distance calculation between word embeddings across languages, which is actually finding translation candidates. Pg 863, 2 Cross-lingual Word embedding, Para 3, Ln 1-12, We train cross-lingual word embedding in a fully unsupervised manner. Pg 863, 2 Cross-lingual Word embedding, Para 4, Ln 1-7, Once we have the cross-lingual mapping, we can transform the embedding of a given source word and find a target word with the closest embedding, i.e. nearest neighbor search. Pg 863, 2 Cross-lingual Word embedding, Para 5, Ln 1-10, Run a Procrustes problem solver with the dictionary from Step 3 to re-train the mapping. “for processing mixed language queries” is taught by Nakayama, as shown above, and is also intended use);
It would have been obvious for one skilled in the art, at the effective time of filling, to modify Nakayama with the embeddings system of Kim, as it can improve translation performance(Pg 862, Abstract, Ln 1-13).
Regarding Claim 2:
The combination of Nakayama and Kim teaches the method of claim 1, and Nakayama teaches wherein the data from the first language is a mathematical representation of the first language(Pg 3, Fig 2, Input -> Token Embeddings).
Regarding Claim 3:
The combination of Nakayama and Kim teaches the method of claim 2, but does not explicitly teach wherein the mathematical representation is a language vector that represents relationships found between words in the first language.
In the same field of Machine Translation, Kim teaches wherein the mathematical representation is a language vector that represents relationships found between words in the first language(Pg 863, 2 Cross-lingual Word embedding, Para 2, Cross-lingual word embedding is a continuous representation of words whose vector space is shared across multiple languages. This enables distance calculation between word embeddings across languages).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify the combination of Nakayama and Kim with the embeddings system of Kim, as it can improve translation performance(Pg 862, Abstract, Ln 1-13).
Regarding Claim 4:
The combination of Nakayama and Kim teaches the method of claim 1, but does not teach wherein the data from the second language is a mathematical representation of the second language.
In the same field of Machine Translation, Kim teaches wherein the data from the second language is a mathematical representation of the second language(Pg 863, 2 Cross-lingual Word embedding, Para 2, Cross-lingual word embedding is a continuous representation of words whose vector space is shared across multiple languages. This enables distance calculation between word embeddings across languages. Nakayama also teaches embeddings for a second language, (Pg 3, Fig 2.), But Kim was used for rejecting the limitation that introduced the data from second language, hence Kim being used to teach above limitation).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify the combination of Nakayama and Kim with the embeddings system of Kim, as it can improve translation performance(Pg 862, Abstract, Ln 1-13).
Regarding Claim 5:
The combination of Nakayama and Kim teaches the method of claim 4, but does not teach wherein the mathematical representation is a language vector that represents relationships found between words in the second language.
In the same field of Machine Translation, Kim teaches wherein the mathematical representation is a language vector that represents relationships found between words in the second language(Pg 863, 2 Cross-lingual Word embedding, Para 2, Cross-lingual word embedding is a continuous representation of words whose vector space is shared across multiple languages. This enables distance calculation between word embeddings across languages).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify the combination of Nakayama and Kim with the embeddings system of Kim, as it can improve translation performance(Pg 862, Abstract, Ln 1-13).
Regarding Claim 6:
The combination of Nakayama and Kim teaches the method of claim 1, and Nakayama teaches wherein the hidden state reflects structures of phrases in the first language(Pg 3, Fig 2, Attention, BiLSTM. Pg 3, Col 1, Para 1, Ln 1-7, output goes through two stacked BiLSTM layers with 256 hidden units for each direction (512 hidden units in both directions). In the decoder, the characters were projected by two LSTM layers with 512 hidden units. The decoder used the attention module with scores calculated by a multilayer perceptron. While Kim was originally used to teach the hidden state in Claim 1, Nakayama also teaches hidden states).
Regarding Claim 7:
The combination of Nakayama and Kim teaches the method of claim 1, and Nakayama teaches wherein the additionally training comprises utilizing an encoder-decoder network(Pg 3, Fig 2, encoder, decoder. Pg 3, Col 1, Para 1, Ln 1-7, output goes through two stacked BiLSTM layers with 256 hidden units for each direction (512 hidden units in both directions). In the decoder, the characters were projected by two LSTM layers with 512 hidden units. The decoder used the attention module with scores calculated by a multilayer perceptron).
Regarding Claim 8:
The combination of Nakayama and Kim teaches the method of claim 7, and Nakayama teaches wherein the encoder-decoder network is a bidirectional long short-term memory network(Pg 3, Col 1, Para 1, Ln 1-7, output goes through two stacked BiLSTM layers with 256 hidden units for each direction (512 hidden units in both directions). In the decoder, the characters were projected by two LSTM layers with 512 hidden units. The decoder used the attention module with scores calculated by a multilayer perceptron).
Regarding Claim 11:
Nakayama teaches train, at a first computing device, a machine learning model network based on, at least in part, data from a first language to create a trained state(Pg 2, 2 CS2Mono NMT, Para 2, Ln 1-5, we trained NMT to learn CS to monolingual text translation on synthetic and natural data. Our NMT system is a standard attention-based encoder-decoder model. Pg 3, Fig 2, input, token embeddings);
receive a mixed language query comprising the first language and the second language(Pg 2, 2 CS2Mono NMT, Para 2, Ln 1-5, we trained NMT to learn CS to monolingual text translation on synthetic and natural data. Our NMT system is a standard attention-based encoder-decoder model. Pg 4, B. Natural code-switching, Para 2, Ln 1-7, This corpus includes 0.9-K natural CS sentences for a training set and 0.1K for a test set among all the artificial CSs. Pg 4, Col 2, Para 4(second to last Para in A. Artificial code-switching), Ln 1-2, That corpus was divided 50-K sentences for a training set, 500 for a development set, and 500 for a test set);
input the received mixed language query into the additionally trained machine learning model to generate an output in the first language(Pg 2, 2 CS2Mono NMT, Para 2, Ln 1-5, we trained NMT to learn CS to monolingual text translation on synthetic and natural data. Our NMT system is a standard attention-based encoder-decoder model. Pg 3, Fig 2, Japanese-English -> English);
and output the generated output in the first language(Pg 2, 2 CS2Mono NMT, Para 2, Ln 1-5, we trained NMT to learn CS to monolingual text translation on synthetic and natural data. Our NMT system is a standard attention-based encoder-decoder model. Pg 3, Fig 2, Japanese-English -> English).
Nakayama does not teach a system comprising: input/output circuitry configured to:…..additionally train the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries;….and processing circuitry configured to.
In the same field of Machine Translation, Kim teaches a system comprising: input/output circuitry configured to(Pg 866, Col 1, Para 1, Ln 1-5, The total training time of our method is only 1-2 days with a single GPU.):
additionally train the machine learning model network via unsupervised learning using data from a second language to create a hidden state for processing mixed language queries(Pg 863, 2 Cross-lingual Word embedding, Para 1, Ln 1-7, As a basic step for unsupervised MT, we learn a word translation model from monolingual corpora of each language. Pg 863, 2 Cross-lingual Word embedding, Para 2, Cross-lingual word embedding is a continuous representation of words whose vector space is shared across multiple languages. This enables distance calculation between word embeddings across languages, which is actually finding translation candidates. Pg 863, 2 Cross-lingual Word embedding, Para 3, Ln 1-12, We train cross-lingual word embedding in a fully unsupervised manner. Pg 863, 2 Cross-lingual Word embedding, Para 4, Ln 1-7, Once we have the cross-lingual mapping, we can transform the embedding of a given source word and find a target word with the closest embedding, i.e. nearest neighbor search. Pg 863, 2 Cross-lingual Word embedding, Para 5, Ln 1-10, Run a Procrustes problem solver with the dictionary from Step 3 to re-train the mapping. “for processing mixed language queries” is taught by Nakayama, as shown above, and is also intended use);
and processing circuitry configured to(Pg 866, Col 1, Para 1, Ln 1-5, The total training time of our method is only 1-2 days with a single GPU).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify Nakayama with the embeddings system and generic computer components of Kim, as it can improve translation performance(Pg 862, Abstract, Ln 1-13) and provides an environment for the system to be realized(Pg 866, Col 1, Para 1, Ln 1-5), respectively.
Regarding Claim 12:
Claim 12 contains similar limitations as Claim 2 and is therefore rejected for the same reasons.
Regarding Claim 13:
Claim 13 contains similar limitations as Claim 3 and is therefore rejected for the same reasons.
Regarding Claim 14:
Claim 14 contains similar limitations as Claim 4 and is therefore rejected for the same reasons.
Regarding Claim 15:
Claim 15 contains similar limitations as Claim 5 and is therefore rejected for the same reasons.
Regarding Claim 16:
Claim 16 contains similar limitations as Claim 6 and is therefore rejected for the same reasons.
Regarding Claim 17:
Claim 17 contains similar limitations as Claim 7 and is therefore rejected for the same reasons.
Regarding Claim 18:
Claim 18 contains similar limitations as Claim 8 and is therefore rejected for the same reasons.
Claim(s) 9 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Nakayama and Kim as applied to claim 1 above, and further in view of Needham (US 20180089172 A1).
Regarding Claim 9:
The combination of Nakayama and Kim teaches the method of claim 1, but does not teach wherein outputting the generated output in the first language comprises identifying the first language based, at least in part, on data stored in a user profile.
In the same field of Machine Translation, Needham teaches wherein outputting the generated output in the first language comprises identifying the first language based, at least in part, on data stored in a user profile(Para [0033], Ln 1-17, For example, each user may self-identify their language preferences and those pre-identified preferences may be stored (e.g. locally or on a cloud service) so that they can be retrieved as needed to guide the translator. Abstract, Ln 1-12, A mixed-language translator apparatus may include a language estimator to estimate two or more languages of at least two message fragments in an electronic representation of a mixed-language message including at least two languages, a language preference detector to determine a language preference of an intended recipient for the mixed-language message, and a translator communicatively coupled to language estimator and the language preference detector to translate the mixed-language message based).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify the combination of Nakayama and Kim, with the user preferences of Needham, as it improves user convenience and offers users additional functionality(Para [0033], Ln 1-17. Para [0043], Ln 1-16).
Regarding Claim 19:
Claim 19 contains similar limitations as Claim 9 and is therefore rejected for the same reasons.
Claim(s) 10 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Nakayama and Kim as applied to claim 1 above, and further in view of Lu (US 20160117315 A1).
Regarding Claim 10:
The combination of Nakayama and Kim teaches the method of claim 1, but does not teach wherein outputting the generated output in the first language comprises identifying the first language based, at least in part, on a language of an operating system associated with the output.
In the same field of Machine Translation, Lu teaches wherein outputting the generated output in the first language comprises identifying the first language based, at least in part, on a language of an operating system associated with the output(Para [0052], Ln 1-5, an Operating System (OS) language of the mobile terminal is obtained and the OS language is determined as the target language of the user of the mobile terminal. Abstract, Ln 1-8, the message corresponding to the first operation is translated into the target language and a translation message is obtained).
It would have been obvious for one skilled in the art at the effective time of filling, to modify the combination of Nakayama and Kim, with the target language identification of Lu, as it improves efficiency(Para [0051], Ln 1-6).
Regarding Claim 20:
Claim 20 contains similar limitations as Claim 10 and is therefore rejected for the same reasons.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Caskey et al. (US 20140012563 A1)
Translation of Text with mixed language transliterations.
Osipova (US 20140180670 A1)
Language identification for translation of mixed-language text.
Zhao et al. (US 20160350289 A1)
Contains translation of the second source language in text containing two source languages.
Farhan et al. (US 20190332677 A1)
Translation of sentences containing multiple source languages into a target language. Also contains unsupervised training of word embedding algorithm.
Matusov et al. (US 20200226327 A1)
Translation of speech input containing more than one source language into speech output of target language.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER G MARLOW whose telephone number is (571)272-4536. The examiner can normally be reached Monday - Thursday 10:00 am - 8:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richmond Dorvil can be reached at (571)272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEXANDER G MARLOW/ Assistant Examiner, Art Unit 2658
/RICHEMOND DORVIL/ Supervisory Patent Examiner, Art Unit 2658
