DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Response filed on 04/29/2026 has been correspondingly accepted and considered in the office action. Claims 1-5 are pending. Claims 1, 4 and 5 are independent and amended. Dependent Claims 2 and 3 are also amended.
The rejections to Claims 1-5 under 35 U.S.C. § 101 as being directed to abstract idea have been withdrawn in view of Applicant’s amendments to the claims and persuasive arguments.
Claims 1-5 stand rejected under 35 U.S.C. § 103. Applicant’s arguments with respect to Claims 1-5 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
In order to expedite prosecution, and as to the material from the Specifications that are not in the Claim and are argued by the Applicant, please note Wang et al. ("A joint model for question answering and question generation." arXiv preprint arXiv:1706.01450 (2017).), Ide et al. ("Multi-task learning of generation and classification for emotion-aware dialogue response generation." Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: student research workshop. 2021.), Lewis et al. ("Unsupervised question answering by cloze translation." Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019.), and Ghaeini et al. (US Pub 2020/0183963 A1). For at least the supra provided reasons, Applicant's arguments have been fully considered but they are not persuasive.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. ("A joint model for question answering and question generation." arXiv preprint arXiv:1706.01450 (2017).) in view of Ide et al. ("Multi-task learning of generation and classification for emotion-aware dialogue response generation." Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: student research workshop. 2021.) further in view of Lewis et al. ("Unsupervised question answering by cloze translation." Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019.).
Regarding Claim 1,
Wang discloses an information processing apparatus (Wang, Title, Abstract, "…We propose a generative machine comprehension model that learns jointly to ask and answer questions based on documents….") comprising: processing circuitry configured to: receive a text as an input (3. Model Description, "…the model takes a document (i.e., a word sequence) D and a condition sequence C as input ...");
generate a sentence using a trained machine learning model (Wang, 3. Model Description, "…outputs a target sequence Y{q,a} ..."; 5.3. Quantitative Evaluation, "…We choose mLSTM as the pretrained QA model and train it on SQuAD...For the pretrained language model, we train a single-layer LSTM language model on the combination of the text8 corpus, the Quora Question Pairs corpus, and the gold questions from SQuAD...");
the trained machine learning model is trained by:
generating training data including first and second data sets from a single data set (Wang, 5.2. Baseline Models, "…Joint-training (in model JointQA) is realized by feeding answer-generation and question-generation data to the model in an alternating fashion between mini-batches..."; 5.1. Dataset, "…We conduct our experiments on the SQuAD corpus"),
wherein: the first data set includes (i) a first text and (ii) a first sentence corresponding to a portion of the first text (Wang, 5.2. Baseline Models, "…We first establish two baselines without multi-task training. Specifically, model A-gen is trained only to generate an answer given a document and a question, i.e., as a conventional QA model..."); and
the second data set includes (i) a second sentence that is a question related to the first text and (ii) a second text that is the first text (Wang, 5.2. Baseline Models, "…Model Q-gen is trained only to generate questions from documents and answers...") from which an answer to the question related to the first text was removed,
the second data set being associated with a classification type that indicates the second sentence is a question to which an answer is not in the second text;
Wang discloses "a condition input," which indicates a sequence-to-sequence framework selects a mode between "a-gen" and "q-gen", but does not explicitly teaches the classification/prediction of the mode type as a trainable output.
However, Ide, in the analogous field of a neural sequence-to-sequence model generating natural-language output from a text input, discloses generate a sentence and a classification type using a trained machine learning model (Ide, Abstract, "…we propose a neural response generation model with multi-task learning of generation and classification… Our model based on BART (Lewis et al., 2020), a pre-trained transformer encoder decoder model, is trained to generate responses and recognize emotions simultaneously..."),
wherein: the generated classification type indicates whether the generated sentence is a question to which an answer is not in the received text or indicates the portion of the first text is extracted from the first text (Ide, Fig.1, 3.1 Overview, "…Our model learns response generation as a generation task and emotion recognition as a classification task...Our model is based on BART (Lewis et al.,2020). Its architecture is shown in Figure 1. The model has several output layers, or heads, for the tasks to be trained, which include an LM head for generating words in response generation and CLS heads for solving classification tasks. Given a sentence, the CLS head predicts its label"; i.e., it is construed that a shared encoder-decoder could be trained with additional classification heads to predict the category labels such as a question or answer/extraction jointly with the generated sentence instead of emotion as being taught by Ide);
processing the training data through the machine learning model to generate an output classification type and an output sentence (Ide, 3.1 Overview, "…Following the learning algorithm of MT-DNN (Liu et al., 2019), each task that the model learns is selected for each mini-batch. A different loss is calculated for each task, and the parameters are updated for each mini-batch...");
comparing the output classification type and the output sentence with a correct answer classification type and a correct answer generation sentence; and updating parameters of the machine learning model based on the comparison (Ide, see 3.2 Losses of Generation and Classification Tasks and 4.2 Training for training details); and
output the generated sentence and the classification type (Ide, Fig.1, 3.1 Overview, "…The model has several output layers, or heads, for the tasks to be trained, which include an LM head for generating words in response generation and CLS heads for solving classification tasks. Given a sentence, the CLS head predicts its label").
It would have been obvious to one of ordinary skill in the art to apply Ide's joint classification-and-generation training technique to Wang's joint model for question answering and question generation and substitute Wang's q-gen/a-gen mode indicator with a classification output (e.g., category labelling such as emotion) with a reasonable expectation of success to yield the predictable result of a sequence-to-sequence encoder-decoder model that generates sentences and predicts the classification (e.g., question generation or extraction).
Wang in view of Ide discloses a sequence-to-sequence model that receives input text, generate a sentence, and predicts a classification type, but does not explicitly teach how the second text is constructed as disclosed in limitations, "a second text that is the first text from which an answer to the question related to the first text was removed," and "the generated sentence is a question to which an answer is not in the received text."
Lewis, in the analogous field of neural question-answering model training, teaches a second text that is the first text from which an answer to the question related to the first text was removed (Lewis, Fig.1, 1 Introduction, "…method generates EQA training data in three steps. 1) We first sample a paragraph in a target domain—in our case, English Wikipedia. 2) We sample from a set of candidate answers within that context, using pretrained components (NER or noun chunkers) to identify such candidates...Given a candidate answer and context, we can extract “fill-the-blank” cloze questions 3) Finally, we convert cloze questions into natural questions using an un supervised cloze-to-natural question translator..."; see 2 Unsupervised Extractive QA; 3.1 Unsupervised QA Experiments, "…For the synthetic dataset training method, we consider two QA models: finetuning BERT (Devlin et al., 2018) and BiDAF + Self Attention (Clark and Gardner, 2017)...we extract cloze questions from both sentences and sub-clauses, and use the NMT models to estimate p(q|c,a)...").
Therefore, it would have been obvious to one of ordinary skill in the art, before effective filing date of the claimed invention, to have modified a neural sequence-to-sequence encoder-decoder model for sentence generation and classification of Wang in view of Ide with a method of answer-masking and cloze generation and translation of Lewis with a reasonable expectation that this would result in a neural joint question-answering and classification model producing the "question with the answer removed" as specified in the claim 1 by substituting masked-passage/question training data for the unmodified source document in question generating mode of Wang (Lewis, 2 Unsupervised Extractive QA).
Claim 4 is a method claim with limitations similar to the limitations of Claim 1 and is rejected under similar rationale. Rationale for combination is similar to that provided for Claim 1.
Claim 5 is a non-transitory computer-readable recording medium claim with limitations similar to the limitations of Claim 1 and is rejected under similar rationale. Rationale for combination is similar to that provided for Claim 1.
A person of ordinary skill in the field of a neural network question answering system would understand as common general knowledge that any published pre-trained question-answering algorithm is necessarily implemented as computer-executable instructions stored on a non-transitory computer-readable storage medium and executed on a computer system comprising at minimum a processor, memory, a GPU, and other computer components.
Claims 2 and 3 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Ide further in view of Lewis further in view of Ghaeini et al. (US Pub 2020/0183963 A1).
Regarding Claim 2,
Wang in view of Ide further in view of Lewis discloses information processing apparatus according to claim 1.
Wang in view of Ide further in view of Lewis does not explicitly teaches the limitation, "wherein the processing circuitry is further configured to acquire a viewpoint indicating a subject category, and receive the text and the viewpoint as inputs to generate the sentence and the classification type."
However, Ghaeini, in the analogous field of a neural question and answering systems, disclose wherein the processing circuitry is further configured to acquire a viewpoint indicating a subject category (Ghaeini, Fig.2, par [047], "…a dataset 202 or other input mechanism may provide a tuple 204 comprising a source paragraph, a focused fact, and a question type..."), and
receive the text and the viewpoint as inputs (Ghaeini, Fig.2, paras [031-35], "…a novel deep learning based question generation solution to generate relevant questions from textual content (e.g., a paragraph about specific topic( s) of interest)...may use a novel attention-based recurrent neural network (RNN) encoder decoder architecture with control gates for focused facts and question types..."; par [035], "…receive, as an input, a 3-tuple comprising a source text (e.g., a paragraph of information), a focused factual statement describing the topic of the question to be generated, and an indication of the question type...") to generate the sentence and the classification type.
Therefore, it would have been obvious to one of ordinary skill in the art, before effective filing date of the claimed invention, to have modified a neural joint question-answering and classification model with answer-masking and cloze generation of Wang in view of Ide further in view of Lewis with a fact-embedding module in a neural question-answering system of Ghaeini with a reasonable expectation of success to improve the performance of the QA system by guiding the question/answer generation with the "focused fact" input and subsequent fact embedding (Ghaeini, paras [002-005]).
Regarding Claim 3,
Wang in view of Ide further in view of Lewis further in view of Ghaeini discloses information processing apparatus according to claim 2, wherein the processing circuitry is further configured to add information of the viewpoint to output with the sentence and the classification type (Fig.3, par [051], "…the input into the question generator module 161 is a tuple including a paragraph 302 (i.e., free text), a focused-fact 304, and a question type 306..."; par [052], "…The paragraph 302 may be fed into a convolutional neural network (CNN) 308..."; paras [054-055], "…The embedded focused fact may be communicated to a squeezer module 312 that generates a vector representation from the sequence of word embeddings...The embedded focused fact 304 and the question type 306 may be fed into a concatenator module 314. The concatenator module 314 may be configured to concatenate the focused-fact 304 and the question type 306 and feed them into one or more layers of a bi-directional recurrent neural network (RNN) 310 implemented by the question generator 164...").
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Yuan et al. (US Pat 10,902738 B2) discloses a method, system, and storage device storing a computer program, for generating questions based on provided content, such as, for example, a document having words. The method comprises automatically estimating the probability of interesting phrases in the provided content, and generating a question in natural language based on the estimating (Yuan, Abstract).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JANGWOEN LEE whose telephone number is (703)756-5597. The examiner can normally be reached Monday-Friday 8:00 am - 5:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BHAVESH MEHTA can be reached at (571)272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JANGWOEN LEE/Examiner, Art Unit 2656
/BHAVESH M MEHTA/Supervisory Patent Examiner, Art Unit 2656
Prosecution Insights