Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. CN202311578680, filed on November 23, 2023.
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites expanding vocabulary, filtering and retaining corpora meeting a quality standard, optimizing feature vectors, and training a translation model. Expanding vocabulary and determining whether corpora satisfy a quality standard are evaluations that, under their broadest reasonable interpretation, encompass mental processes. Optimizing feature vectors recites manipulation of numerical vector representations and therefore recites a mathematical concept.
The judicial exception is not integrated into a practical application. Although the claim recites performing operations using a translation model, the claim does not recite a specific technical mechanism for performing the optimization, filtering, or training. Instead, the claim merely applies the abstract idea using a generic computing environment and therefore does not improve computer functionality or another technology.
The additional element of training the translation model using the corpora, considered individually, amounts to well-understood, routine, and conventional activity previously known in the industry. In particular, the specification explains that there were “two commonly used types of translation models,” including specialized translation models and general large language models. The specification further explains that specialized translation models use an encoder-decoder architecture and “are trained with a large amount of open-source data” while general language models are “trained on datasets from various domains”, see paragraphs [0003-0004]. Thus, training a translation model using corpus of dataset information was acknowledged in the specification as part of the ordinary operation of commonly used translation models.
Claim 1 recites this training operation only at a high level of generality and does not recite a particular unconventional training architecture, loss function, parameter-update technique, or other specific technological mechanism by which the training is performed. Accordingly, the limitation amounts to no more than applying the identified judicial exception using well-understood, routine, and conventional translation-model training activity.
This conclusion is further supported by Pabolu et al. (US Patent 12,493,717 B2), which describes training machine-learning models using training datasets containing inputs and corresponding outputs. Whereby the model learns a function mapping the inputs to corresponding outputs. (Machine learning (ML) is an area of artificial intelligence (AI) where computers have the capability to learn without being explicitly programmed. There are different types of ML techniques including supervised learning techniques, unsupervised learning techniques, and others. In a supervised learning technique, an ML model is created and trained using training data, where the training data includes multiple training examples [col. 1, lines 23-30] As a part of the training, the model being trained learns a function that maps the inputs in the training data to their corresponding known outputs [col. 1, lines 33-35], the architecture of the input model 113 may be one of the architectures known to those skilled in the relevant art as being appropriate for the task [col. 13, lines 30-33]).
Claims 2-9 depend from claim 1, and they further recite additional details regarding optimizing feature vectors filtering corpora according to a quality standard, and training the translation model using the filtered corpora. These additional limitations do not integrate the judicial exception into a practical application and likewise do not provide significantly more than the judicial exception. Accordingly, claims 2-9 are directed to the same abstract idea as claim 1 and are not patent eligible under 35 USC §101.
Claims 10-20 contain similar limitations to claims 1-9 and thus are rejected for the same reasons.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. -
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Pabolu et al. (US Patent 12,493,717 B2)
Regarding claim 1, Pabolu discloses:
A method for training a translation model, comprising:
expanding vocabulary of the translation model with commonly used vocabulary and specialized vocabulary in advance (As a non-limiting example, the vocabulary selector 130 may select from 50,000 to 60,000 words for the target language and from 25,000 to 30,000 words for English. Such selection can provide a total amount of 70,000-80,000 vocabulary words for training of a base model [col. 11, lines 61-66]);
optimizing feature vectors in the vocabulary based on a large language model (To feed those tokens into the neural network, each token is converted into an embedding vector by an input embedding layer 403. Further, a positional encoding layer 404, e.g., a linear encoding layer, is provided and injects positional encoding into each embedding [col. 14, lines 18-23] For example, two additional layers are a constrained vocabulary embedding layer 432, e.g., an embedding layer with a desired vocabulary length input, and a constrained vocabulary linear layer 434 [col. 14 lines 65-67, col. 15 line 1], The base model training subsystem 156 uses the indexes of the base vocabulary 123 generated by the base vocabulary generation subsystem 122 and prunes all the remaining input embeddings that the mT5 model has been trained on [col. 15 lines 9-12]); -
-cleaning an open-source dataset using a data cleaning method to filter and retain corpora meeting a quality standard (The first filter 136 receives the article-summary pairs from the language detector 128 and filters the article-summary pairs based on one or more predetermined criteria [col. 17, lines 52-55]); and
training the translation model using the corpora (the training data generation subsystem 102 may receive, as an input, base input data from a large text corpus and perform the processing on the base input data that results in a generation of the base vocabulary 123 [col. 10, lines 26-29]);
Regarding claim 2, Pabolu discloses:
The method according to claim 1, wherein optimizing the feature vectors in the vocabulary based on a large language model comprises:
inputting a query into the large language model (A user may provide, through the UI subsystem 106, an input for identifying a target language to be processed by the natural language generation system 98 [col. 11, lines 15-16]), wherein the large language model processes the query and outputs specialized vocabulary for a designated domain (The data miner 127 can obtain or receive the enhanced input data, e.g., the article-summary pairs, and perform certain processing on the enhanced input data, to output da-ta in one or more languages, e.g., in a target language and English [col. 17, lines 27-31]); and
adjusting the feature vectors in the vocabulary (prunes all the remaining input embeddings that the mT5 model has been trained on. E.g., by modifying the mT5 model to include the constrained vocabulary embedding layer 432 and the constrained vocabulary linear layer 434, the mT5 model is constrained to the knowledge needed for the target language and English and all other languages are pruned from the model [col. 15, lines 11-17]) using the specialized vocabulary for the designated domain (For example, for producing texts in the medical domain, the base model can be generated starting with MedBERT instead of mT5. Similarly, for producing texts in the financial domain, the base model can be generated starting with FinBERT instead of mT5 [col 28, lines 47-51]);
Regarding claim 3, Pabolu discloses:
The method according to claim 1, wherein cleaning an open-source dataset (the base input data may be a web content, public news, etc., that can be obtained from a publicly available large dataset(s) [col. 19, lines 49-51]) using a data cleaning method to filter and retain corpora meeting a quality standard comprises at least one of the following:
cleaning the open-source dataset using a cross-lingual similarity learning (CL-SSL) method to filter corpora with a semantic similarity exceeding a threshold (For example, the MMR algorithm 142 considers the similarity of keywords/key phrases within the document, along with the similarity of already selected keywords and key phrases [col. 20, lines 45-48])
cleaning the open-source dataset using a language-agnostic sentence representation (LASER) method to filter corpora with a sentence embedding similarity exceeding a threshold (The keyword extracting model 141, e.g., LABSE, generates LAB SE embeddings for each input sentence. The keyword extractor 140 controls keyword extraction using LAB SE embeddings with a probability that varies based on the lengths of sentences of the obtained text [col. 19, line 66 and col. 20, lines 18-21] The MMR algorithm 142 is tasked with minimizing the redundancy and maximizing the diversity of results [col. 20, lines 43-45]);
cleaning the open-source dataset using a length-based method to filter corpora where a maximum-to-minimum sentence length ratio meets a threshold (For example, the given sentence is sampled by the number of words included in the sentence and the following processing is performed:
1. If the length of the sentence is less than 500 words, uniform probability distribution of choosing ngrams in the range of 1 to 6 along with a combination of choosing the number of such ngrams to be detected in the range of 1 to 3.
2. If length of the sentence is in between 500 to 2000 words, ngrams range is considered to be between 2 to 6 with a total number of such ngrams is between 5 to 9.
3. If the length of the sentence is between 2001 to 5000, ngrams range is considered to be between 2 to 6 with a total number of such ngrams between 10 to 17.
[col. 20, lines 22-39]);
cleaning the open-source dataset using a language identification (LID) method to filter corpora that match source and target languages (the language detector 128 may use a language detecting model, e.g., a classification model, which provides an identification of the languages. In some embodiments, the language detector 128 may associate a language identifier (ID) with the webpage or a portion of the content and sort the input data into datasets corresponding to certain languages, e.g., Spanish and English [col. 11, lines 27-34]);
cleaning the open-source dataset by removing duplicate data to filter corpora (For example, the first tokenizer 129 uses one or more neural network models to process the input data of one or more languages. In the process of tokenizing the input data, the first tokenizer 129 may remove language-specific stop words and maintain a counter of the most used vocabulary in both languages, e.g., Spanish and English [col. 11, lines 46-52] The MMR algorithm 142 is tasked with minimizing the redundancy and maximizing the diversity of results [col. 20, lines 43-45]); or
cleaning the open-source dataset using a translation filtering method to filter corpora that meet a translation quality metric (The first filter 136 receives the article-summary pairs from the language detector 128 and filters the article-summary pairs based on one or more predetermined criteria [col. 17, lines 51-55] In some embodiments, the second filter 172 may filter out one entity based keyword-text pairs, e.g., the keyword-text pairs having only one entity, to exclude the one entity based keyword-text pairs from further processing performed by the STMKNS triplet generation subsystem 168 [col. 12, lines 60-64]);
Regarding claim 4, Pabolu discloses:
The method according to claim 1, wherein training the translation model using the corpora comprises:
using the corpora (In some implementations, the base input data may be a web content, public news, etc., that can be obtained from a publicly available large dataset(s), e.g., Common Crawl and Leipzig corpora, e.g., first public data [col. 10, lines 49-52]) to train the translation model (the machine-learning system 500 may be a part of the natural language generation system 98 or may be in communication with the natural language generation system 98, to facilitate the training of the models [col. 31, lines 33-37]) with at least one specified method, the specified method comprising at least one of contrastive learning, supervised learning, reinforcement learning, semi-supervised learning, weakly supervised learning, or self-supervised learning (As used herein, a "machine learning model" or a "model" can refer to a software module configured to be run on one or more processors to provide a classification or numerical value of a property of one or more samples. An example type of model is supervised learning that can be used with embodiments of the present disclosure [col. 12, lines 47-52]);
Regarding claim 5, Pabolu discloses:
The method according to claim 1, further comprising: performing deduplication optimization of a translation result during the training process using a hybrid deduplication method (To further enhance the quality of keywords being extracted, the keyword extractor 140 applies a Maximal Marginal relevance (MMR) algorithm 142 to the extracted keywords. The MMR algorithm 142 is tasked with minimizing the redundancy and maximizing the diversity of results [col. 20, lines 40-45]) or a model-based training (may include a custom model training subsystem 134 configured to train the base model 148 using one or more enhanced training datasets 125 at multiple training sub-stages [col. 16, lines 44-47]) and prediction deduplication method (the custom model training subsystem 134 can train the base model 148 using the STMKNS triplets dataset 170, to generate the MKSNSGM 169, e.g., a third custom model that is a next sentence generation model, which can predict a next sentence that follows the given summary based on the given keywords [col. 23, lines 55-61]);
Regarding claim 6, Pabolu discloses:
The method according to claim 5, wherein the hybrid deduplication method comprises at least one (An example type of model is supervised learning that can be used with embodiments of the present disclosure. Example supervised learning models may include different approaches and algorithms [col. 12, lines 50-54] The examiner notes that while the algorithms below are not the same, one of ordinary skill in the art should note that the algorithms below function similarly with the same goal) of a repetition penalty algorithm, a contrastive search algorithm (To further enhance the quality of keywords being extracted, the keyword extractor 140 applies a Maximal Marginal relevance (MMR) algorithm 142 to the extracted keywords. The MMR algorithm 142 is tasked with minimizing the redundancy and maximizing the diversity of results [col. 20, lines 40-45] The examiner notes that the main algorithm used is the MMR algorithm, and functions very similarly to both a repetition penalty algorithm and a contrastive search algorithm), or a beam search algorithm (minimum message length (decision trees, decision graphs, etc.) …. nearest neighbor algorithm …. random forests [col. 12, lines 60-67] The examiner notes that the listed algorithms are heuristic in nature, similar to a beam search algorithm);
Regarding claim 7, Pabolu discloses:
The method according to claim 1, wherein expanded specialized vocabulary comprises at least one of proprietary brand vocabulary, geographic vocabulary, culturally-specific vocabulary, e-commerce-specific or English examination vocabulary (According to the described techniques, the generated data is semantically meaningful and structurally variant with texts in multiple domains of e-commerce, finance, medical, day-to-day life, etc., that helps the models to generalize on different language domains [col. 7, lines 31-35]);
Regarding claim 8, Pabolu discloses:
The method according to claim 1, further comprising:
obtaining source language text to be translated (In various embodiments, a computer-implemented method is provided that includes obtaining, from text corpus [col. 7, lines 41-42]);
inputting the source language text (Parameters in the models can be configured for generating hundreds of unique texts from just a single input [col. 7, lines 24-25]) into the translation model (the natural language generation system 98 may further include a user interface (UI) subsystem 106 for receiving a user input [col. 9, lines 5-7]) to obtain target language text after translation (perform certain processing on the enhanced input data, to output data in one or more languages, e.g., in a target language and English [col. 17, lines 29-31]);
Regarding claim 9, Pabolu discloses:
The method according to claim 8, wherein obtaining the source language text to be translated comprises:
obtaining product information as the source language text in a product search scenario on an e-commerce platform (The webpages are crawled from the whole web without restriction; they come in many different languages and the quality of the text varies greatly [col. 12, lines 61-63]), wherein the product information includes at least one of a title, a keyword, or details from a product description page (In some implementations, the base input data may be a web content, public news, etc., that can be obtained from a publicly available large dataset(s) [col. 12, lines 49-52]);
Regarding claim 10, in addition to the elements stated above regarding claim 1, Pabolu further discloses:
A non-transitory computer-readable storage medium (Storage subsystem 1218 includes tangible computer-readable storage media 1222 and a system memory 1210 [col. 48, lines 42-44]) configured with instructions executable by one or more processors to cause the one or more processors to perform operations (Processing unit Processing unit 1204, which can be implemented as one, which can be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller) [col. 48, lines 60-62]);
Claim 11 contains similar limitations to claim 2, and therefore is rejected for the same reasons.
Claim 12 contains similar limitations to claim 3, and therefore is rejected for the same reasons.
Claim 13 contains similar limitations to claim 4, and therefore is rejected for the same reasons.
Claim 14 contains similar limitations to claim 5, and therefore is rejected for the same reasons.
Claim 15 contains similar limitations to claim 6, and therefore is rejected for the same reasons.
Claim 16 contains similar limitations to claim 7, and therefore is rejected for the same reasons.
Claim 17 contains similar limitations to claim 8, and therefore is rejected for the same reasons.
Regarding claim 19, in addition to the elements stated above regarding claim 1, Pabolu further discloses:
An electronic device (Alternatively, or in addition, client computing devices may be any other electronic device [col. 39 lines 66-67 and col. 40 line 1]) comprising:
one or more processors (Processing unit Processing unit 1204, which can be implemented as one, which can be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller) [col. 48, lines 60-62]); and
one or more computer-readable memories (Storage subsystem 1218 includes tangible computer-readable storage media 1222 and a system memory 1210 [col. 48, lines 42-44]) coupled to the one or more processors and having instructions stored thereon that are executable by the one or more processors to perform one or more operations (System memory 1210 may store program instructions that are loadable and executable on processing unit 1204, as well as data generated during the execution of these programs [col. 50, lines 3-6]);
Claim 20 contains similar limitations to claim 2, and therefore is rejected for the same reasons.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Kyle Espinas whose telephone number is (571)270-0596. The examiner can normally be reached Monday Friday, 8 a.m. 5 p.m. ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Flanders can be reached at (571) 272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Kylenino Espinas/
Patent Examiner
Art Unit 2655
7/23/2026
/DOUGLAS GODBOLD/Primary Examiner, Art Unit 2655
Prosecution Insights