ADAPTING A LANGUAGE MODEL FOR MULTIMODAL MULTI-TASK LEARNING

Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Status of Claims Applicant’s submission filed on 09.08.2026 has been entered. The status of claims is as follows: Claims 1-26 remain in the application. Claims 1, 9, 17, 25 and 26 are amended. Response to Arguments In reference to the rejection under 35 U.S.C 102: Applicant’s arguments, see Remarks pg. 1-3, filed 09.08.2026, with respect to the rejection(s) of claim(s) under 35 U.S.C 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Wang et al. (“Understanding ME? Multimodal Evaluation for Fine-grained Visual Commonsense”). In reference to the rejection under 35 U.S.C 103: Applicant asserts in Remarks pg. 3-4 that Wang does not cure the deficiencies of Zhou because Wang is not cited as teaching the limitation of applying question/ answer pairs to acquired domain-specific multimodal content to train a language model, where each question/ answer pair includes an inquiry regarding the domain-specific multimodal content and a corresponding solution. Applicant further argues that Wang does not qualify as prior art because Wang was published on October 23, 2023, whereas the present application claims priority to provisional applications filed on April 6, 2023. Accordingly, Applicant contends that Wang was not publicly accessible before the effective filling date and therefore cannot properly relied upon as prior art. Applicant’s arguments regarding Wang are not persuasive. As an initial matter, the Examiner acknowledges that the version of Wang previously attached to the Office Action was inadvertently the October 23, 2023 revised version. The Examiner intended to rely upon the earlier 2022 publication of Wang. The 2022 Wang publication predates the April 6, 2023 priority date asserted by Applicant. Examiner has also made a correction by attaching the 2022 Wang publication this time. Applicant’s substantive argument that Wang fails to disclose the disputed question/ answer pair limitation is also not persuasive. Wang expressly describes a multimodal evaluation (ME) pipeline that automatically generates question-answer pairs to test a model’s understanding of a visual scene, textual information and related knowledge. Wang further explains that, for an input image-question-answer pair, ME generates fine-grained sub-questions corresponding to three modalities: vision, text and background knowledge, and that each sub-question includes answer choices having a correct answer (i.e., the solution to the queries in question) (See Wang, Sections 3-4). Claim Rejections - 35 USC § 103 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. Claim(s) 1-6, 9-12-14, 17-22, 25-26 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al (US 2022/0130499 A1) in view of Wang et al. (“Understanding ME? Multimodal Evaluation for Fine-grained Visual Commonsense”). Regarding claim 1, Zhou explicitly discloses: A method for adapting a language model for understanding domain-specific multimodal content, comprising: acquiring domain-specific multimodal content for at least one content domain; and (Zhou, ¶[0038]: “The image embedder unit 102 can receive an image 110 from and extract domain-specific features describing objects in the image 110. The image embedder unit 102 can further employ a computer vision model to detect and label domain specific-objects in the image 110. The image embedder unit 102 can receive the image 110 as an input and predict a class for each object contained in the image 11”, ¶[0039]: “The text embedder unit 104 is operable to receive a query 112 in electronic format from a user computing device 114. The query 112 can be a question from a user requesting information about some aspect of the image, for example, "What is the most alarming part in this x-ray scan?" The text embedder unit 104 can apply a model that uses natural language processing (NLP) techniques to analyze to query 112 and determine a context of the query 112. The model can be, for example, a word embedding model.”) applying question/answer pairs to the acquired, domain-specific multimodal content for the at least one content domain to train the language model to learn tasks associated with the domain-specific multimodal content. (Zhou, ¶[0035]: “The multi-modal encoder 212 can transmit the image-text representation 500 to an answer decoder 502. The answer decoder 502 can receive a training query 112 and an image 110 as inputs and generate an answer prediction 504. The answer decoder 502 can be implemented by a neural network. The answer decoder 502 can further be in the form of a sequential generating model, such a long short-term memory (LTSM) or transformer decoder. An LSTM network is a form of a recurrent neural network (RNN) capable of learning order dependence in sequence prediction problem. A transformer decoder is another sequence learning neural network. The answer prediction 504 can be retrieved from a database 116 and provided in natural language. During the third training phase, a determination is made whether the answer prediction is correct or incorrect. The answer decoder 502 can be trained through supervised learning by matching an answer prediction 504 with an answer label 506 provided in a training set of answers. The answer decoder 502 can map a correct answer to an associated image-text representation 500.”) Zhou fails to disclose: wherein each question/answer pair comprises a data pair that describes an inquiry regarding the domain-specific multimodal content and a solution to the inquiry. However, Wang explicitly discloses: wherein each question/answer pair comprises a data pair that describes an inquiry regarding the domain-specific multimodal content and a solution to the inquiry. (Wang, Pg. 3, Section 3: “Given an input image-question(-answer) pair, ME first analyzes information from the pair. Then it would generate three follow-up fine-grained questions called sub-questions corresponding to three modalities: vision, text, and background knowledge. Following VCR’s format, each sub-question also has four answer choices with one correct answer”, Pg. 4, Section 4.3: “Given a triplet, we can ask questions about the subject, the object, or the predicate. For instance, in (boy, in front of, people), if asking about the object, we could use templates like "What is the [Subject] [Predicate]" (What is the boy in front of?). In this case, the basic answer would be [Object]( People) or the converted full SVO sentence of the triplet, [Subject][Predicate][Object](The boy is in front of people). When asking about the subject or the predicate, similar procedure applies”) The combination of Zhou and Wang are analogous art because they are in the same field of training time series data. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, having the teachings of Zhou and Wang before them, to modify the teachings of Zhou to include the teachings of Wang’s automatic selection and application of question/ answer pairs to the domain-specific multimodal content to ensure that the selected questions correspond to the particular objects, attributes, or relationships present in the content. Such a modification would improve the relevance and accuracy of the resulting question/ answer data, reduce the need for manual selection and avoid applying irrelevant questions, thereby increasing the efficiency of processing or annotating domain-specific multimodal content. Regarding claim 2, the combination of Zhou and Wang discloses all the limitations of claim 1 (as shown in the rejections above). Zhou in view of Wang further discloses: using the trained language model to answer questions directed to the domain-specific multimodal content for the at least one domain. (Zhou, ¶[0037]: “Turning now to FIG. 5, a visual question answer system 100 is generally shown in accordance with one or more embodiments of the present invention. The system 100 includes a text embedder unit 104 for receiving a natural language text, for example, a search query from a user, and semantically analyzing the text. The system 100 further includes an image embedder unit 102 for analyzing an image and extracting features from objects in the image. The system 100 further includes a multi-modal encoder unit 106 for combining multiple embedding vectors into a single vector. The system 100 further includes an answer unit 108 for generating an answer to the query 112 that includes search query in relation to an image 110.”) Regarding claim 3, the combination of Zhou and Wang discloses all the limitations of claim 1 (as shown in the rejections above). Zhou in view of Wang further discloses: using an in-context multimodal learning approach to train the language model to learn the tasks associated with the domain-specific multimodal content. (Zhou, ¶0023]: “Various techniques can be applied to derive a context of the text document 206. For example, the text embedder unit 104 can take a target word for the embedding being learned and attempt to predict the surrounding context words from it. In another embodiment of the present invention, the text embedder unit 104 analyzes the context of the words surrounding a masked target word and seeks to predict the target word based on the surrounding words. The text embedder unit 104 can be implemented through a neural network type architecture with input, hidden, and output layers. The text embedder unit 104 can be trained to semantically analyze text from a particular domain ( e.g., medical domain) by adjusting the weights and biases of the neural network. The text embedder unit 104 further uses the values of the weights and biases to generate a text embedding vector 208 to represent the semantic meaning of the text document 206.”) Regarding claim 4, the combination of Zhou and Wang discloses all the limitations of claim 1 (as shown in the rejections above). Zhou in view of Wang further discloses: wherein the domain-specific multimodal content is acquired from at least one of a storage device, a user input, or the language model. (Zhou, ¶[0037]: “The system 100 includes a text embedder unit 104 for receiving a natural language text, for example, a search query from a user, and semantically analyzing the text”) Regarding claim 5, the combination of Zhou and Wang discloses all the limitations of claim 1 (as shown in the rejections above). Zhou in view of Wang further discloses: wherein the question/answer pairs are automatically selected and applied to the acquired, domain-specific multimodal content based on a composition of the acquired, domain-specific multimodal content. (Wang, Pg. 4, Col. 2, Section “Selecting relevant sub-graphs”: “After obtaining the graph representation, we want to generate sub-questions relevant to the input image and VCR question of each sample u. Therefore, we filter each triplet in the multimodal graph by its relevance against the input image-question-answer pair… After ranking, we select the top-1 ranked triplet in every modality. If a triplet is selected in more than one modality, we replace the duplicate with the next following triplet in the same modality.”) [Examiner’s note: Under BRI, selecting content elements based on their visual/ textual relevance and converting them into applied question-answer pairs teaches selection based on the multimodal content’s composition] Regarding claim 6, the combination of Zhou and Wang discloses all the limitations of claim 1 (as shown in the rejections above). Zhou in view of Wang further discloses: wherein which question/answer pairs to apply to the acquired, domain-specific multimodal content are determined using a trained machine learning process. (Zhou, ¶[0022]: “the image embedder unit 102 employs a trained artificial neural network to execute the model”) Regarding claim 9, it is rejected under the same rationale with claim 1 (as shown in the rejections above), because they are analogous claims. Regarding claim 10, it is rejected under the same rationale with claim 2 (as shown in the rejections above), because they are analogous claims. Regarding claim 11, it is rejected under the same rationale with claim 3 (as shown in the rejections above), because they are analogous claims. Regarding claim 12, it is rejected under the same rationale with claim 4 (as shown in the rejections above), because they are analogous claims. Regarding claim 13, it is rejected under the same rationale with claim 5 (as shown in the rejections above), because they are analogous claims. Regarding claim 14, it is rejected under the same rationale with claim 6 (as shown in the rejections above), because they are analogous claims. Regarding claim 17, it is rejected under the same rationale with claim 1 (as shown in the rejections above), because they are analogous claims. Regarding claim 18, it is rejected under the same rationale with claim 2 (as shown in the rejections above), because they are analogous claims. Regarding claim 19, it is rejected under the same rationale with claim 3 (as shown in the rejections above), because they are analogous claims. Regarding claim 20, it is rejected under the same rationale with claim 4 (as shown in the rejections above), because they are analogous claims. Regarding claim 21, it is rejected under the same rationale with claim 5 (as shown in the rejections above), because they are analogous claims. Regarding claim 22, it is rejected under the same rationale with claim 6 (as shown in the rejections above), because they are analogous claims. Regarding claim 25, Zhou explicitly discloses: A computer-implemented method for training a language model for understanding domain-specific multimodal content, comprising: acquiring a set of domain-specific multimodal content data for at least one content domain; (Zhou, ¶[0038]: “The image embedder unit 102 can receive an image 110 from and extract domain-specific features describing objects in the image 110. The image embedder unit 102 can further employ a computer vision model to detect and label domain specific-objects in the image 110. The image embedder unit 102 can receive the image 110 as an input and predict a class for each object contained in the image 11”, ¶[0039]: “The text embedder unit 104 is operable to receive a query 112 in electronic format from a user computing device 114. The query 112 can be a question from a user requesting information about some aspect of the image, for example, "What is the most alarming part in this x-ray scan?" The text embedder unit 104 can apply a model that uses natural language processing (NLP) techniques to analyze to query 112 and determine a context of the query 112. The model can be, for example, a word embedding model.”) using a machine learning model, creating a set of question/answer pairs to apply to the domain-specific multimodal content data; (Zhou, ¶[0035]: “The multi-modal encoder 212 can transmit the image-text representation 500 to an answer decoder 502. The answer decoder 502 can receive a training query 112 and an image 110 as inputs and generate an answer prediction 504. The answer decoder 502 can be implemented by a neural network. The answer decoder 502 can further be in the form of a sequential generating model, such a long short-term memory (LTSM) or transformer decoder. An LSTM network is a form of a recurrent neural network (RNN) capable of learning order dependence in sequence prediction problem. A transformer decoder is another sequence learning neural network. The answer prediction 504 can be retrieved from a database 116 and provided in natural language. During the third training phase, a determination is made whether the answer prediction is correct or incorrect. The answer decoder 502 can be trained through supervised learning by matching an answer prediction 504 with an answer label 506 provided in a training set of answers. The answer decoder 502 can map a correct answer to an associated image-text representation 500.”) creating a training set comprising the acquired set of domain-specific multimodal content data and the created question/answer pairs; and (Zhou, ¶[0021]: “Referring to FIGS. 1, 2, 3, and 4, a multi-phase process of training a VQA system 100 is described. As seen in FIG. 1, a first phase of the training is illustrated. The first phase is executed on a first module 200, which includes an image embedder unit 102 and a text embedder unit 104. The image embedder unit 102 is operable to receive an image 202 as a training instance and employ a model to identify objects in the image 202. The image 202 can be, for example, an x-ray image, a CT scan, a photographic image, or other medically related image data.”, ¶[0023]: “The text embedder unit 104 can receive a text document 206 in electronic form as a training instance and derive a semantic meaning of the document. The text document 206 can be an electronic medical record, physician's notes, journal article or other textual document. The text document 206 describes at least a portion of the image 202. For example, the text document 206 can be a transcription of a physician's impressions of an x-ray, where the x-ray is the image 202”) training the language model using the training set by applying the question/answer pairs to the acquired, domain-specific multimodal content for the at least one content domain to train the language model to learn tasks associated with the domain-specific multimodal content. (Zhou, ¶[0035]: “The multi-modal encoder 212 can transmit the image-text representation 500 to an answer decoder 502. The answer decoder 502 can receive a training query 112 and an image 110 as inputs and generate an answer prediction 504. The answer decoder 502 can be implemented by a neural network. The answer decoder 502 can further be in the form of a sequential generating model, such a long short-term memory (LTSM) or transformer decoder. An LSTM network is a form of a recurrent neural network (RNN) capable of learning order dependence in sequence prediction problem. A transformer decoder is another sequence learning neural network. The answer prediction 504 can be retrieved from a database 116 and provided in natural language. During the third training phase, a determination is made whether the answer prediction is correct or incorrect. The answer decoder 502 can be trained through supervised learning by matching an answer prediction 504 with an answer label 506 provided in a training set of answers. The answer decoder 502 can map a correct answer to an associated image-text representation 500.”) Zhou fails to disclose: wherein each question/answer pair comprises a data pair that describes an inquiry regarding the domain-specific multimodal content and a solution to the inquiry. However, Wang explicitly discloses: wherein each question/answer pair comprises a data pair that describes an inquiry regarding the domain-specific multimodal content and a solution to the inquiry. (Wang, Pg. 3, Section 3: “Given an input image-question(-answer) pair, ME first analyzes information from the pair. Then it would generate three follow-up fine-grained questions called sub-questions corresponding to three modalities: vision, text, and background knowledge. Following VCR’s format, each sub-question also has four answer choices with one correct answer”, Pg. 4, Section 4.3: “Given a triplet, we can ask questions about the subject, the object, or the predicate. For instance, in (boy, in front of, people), if asking about the object, we could use templates like "What is the [Subject] [Predicate]" (What is the boy in front of?). In this case, the basic answer would be [Object]( People) or the converted full SVO sentence of the triplet, [Subject][Predicate][Object](The boy is in front of people). When asking about the subject or the predicate, similar procedure applies”) The combination of Zhou and Wang are analogous art because they are in the same field of training time series data. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, having the teachings of Zhou and Wang before them, to modify the teachings of Zhou to include the teachings of Wang’s automatic selection and application of question/ answer pairs to the domain-specific multimodal content to ensure that the selected questions correspond to the particular objects, attributes, or relationships present in the content. Such a modification would improve the relevance and accuracy of the resulting question/ answer data, reduce the need for manual selection and avoid applying irrelevant questions, thereby increasing the efficiency of processing or annotating domain-specific multimodal content. Regarding claim 26, Zhou explicitly discloses: A method for implementing a trained language model to answer inquiries directed to domain-specific multimodal content for the at least one domain, comprising: receiving an inquiry directed at the domain-specific multimodal content; and (Zhou, ¶[0039]: “The text embedder unit 104 is operable to receive a query 112 in electronic format from a user computing device 114. The query 112 can be a question from a user requesting information about some aspect of the image, for example, "What is the most alarming part in this x-ray scan?"”) providing a response to the inquiry using the trained language model, the language model having been trained by: (Zhou, ¶[0039]: “The text embedder unit 104 can apply a model that uses natural language processing (NLP) techniques to analyze to query 112 and determine a context of the query 112. The model can be, for example, a word embedding model”) acquiring a set of domain-specific multimodal content data for at least one content domain; (Zhou, ¶[0038]: “The image embedder unit 102 can receive an image 110 from and extract domain-specific features describing objects in the image 110. The image embedder unit 102 can further employ a computer vision model to detect and label domain specific-objects in the image 110. The image embedder unit 102 can receive the image 110 as an input and predict a class for each object contained in the image 11”, ¶[0039]: “The text embedder unit 104 is operable to receive a query 112 in electronic format from a user computing device 114. The query 112 can be a question from a user requesting information about some aspect of the image, for example, "What is the most alarming part in this x-ray scan?" The text embedder unit 104 can apply a model that uses natural language processing (NLP) techniques to analyze to query 112 and determine a context of the query 112. The model can be, for example, a word embedding model.”) using a machine learning model, creating a set of question/answer pairs to apply to the domain-specific multimodal content data; (Zhou, ¶[0035]: “The multi-modal encoder 212 can transmit the image-text representation 500 to an answer decoder 502. The answer decoder 502 can receive a training query 112 and an image 110 as inputs and generate an answer prediction 504. The answer decoder 502 can be implemented by a neural network. The answer decoder 502 can further be in the form of a sequential generating model, such a long short-term memory (LTSM) or transformer decoder. An LSTM network is a form of a recurrent neural network (RNN) capable of learning order dependence in sequence prediction problem. A transformer decoder is another sequence learning neural network. The answer prediction 504 can be retrieved from a database 116 and provided in natural language. During the third training phase, a determination is made whether the answer prediction is correct or incorrect. The answer decoder 502 can be trained through supervised learning by matching an answer prediction 504 with an answer label 506 provided in a training set of answers. The answer decoder 502 can map a correct answer to an associated image-text representation 500.”) creating a training set comprising the acquired set of domain-specific multimodal content data and the created question/answer pairs; and (Zhou, ¶[0021]: “Referring to FIGS. 1, 2, 3, and 4, a multi-phase process of training a VQA system 100 is described. As seen in FIG. 1, a first phase of the training is illustrated. The first phase is executed on a first module 200, which includes an image embedder unit 102 and a text embedder unit 104. The image embedder unit 102 is operable to receive an image 202 as a training instance and employ a model to identify objects in the image 202. The image 202 can be, for example, an x-ray image, a CT scan, a photographic image, or other medically related image data.”, ¶[0023]: “The text embedder unit 104 can receive a text document 206 in electronic form as a training instance and derive a semantic meaning of the document. The text document 206 can be an electronic medical record, physician's notes, journal article or other textual document. The text document 206 describes at least a portion of the image 202. For example, the text document 206 can be a transcription of a physician's impressions of an x-ray, where the x-ray is the image 202”) training the language model using the training set by applying the question/answer pairs to the acquired, domain-specific multimodal content for the at least one content domain to train the language model to learn tasks associated with the domain-specific multimodal content. (Zhou, ¶[0035]: “The multi-modal encoder 212 can transmit the image-text representation 500 to an answer decoder 502. The answer decoder 502 can receive a training query 112 and an image 110 as inputs and generate an answer prediction 504. The answer decoder 502 can be implemented by a neural network. The answer decoder 502 can further be in the form of a sequential generating model, such a long short-term memory (LTSM) or transformer decoder. An LSTM network is a form of a recurrent neural network (RNN) capable of learning order dependence in sequence prediction problem. A transformer decoder is another sequence learning neural network. The answer prediction 504 can be retrieved from a database 116 and provided in natural language. During the third training phase, a determination is made whether the answer prediction is correct or incorrect. The answer decoder 502 can be trained through supervised learning by matching an answer prediction 504 with an answer label 506 provided in a training set of answers. The answer decoder 502 can map a correct answer to an associated image-text representation 500.”) Zhou fails to disclose: wherein each question/answer pair comprises a data pair that describes an inquiry regarding the domain-specific multimodal content and a solution to the inquiry. However, Wang explicitly discloses: wherein each question/answer pair comprises a data pair that describes an inquiry regarding the domain-specific multimodal content and a solution to the inquiry. (Wang, Pg. 3, Section 3: “Given an input image-question(-answer) pair, ME first analyzes information from the pair. Then it would generate three follow-up fine-grained questions called sub-questions corresponding to three modalities: vision, text, and background knowledge. Following VCR’s format, each sub-question also has four answer choices with one correct answer”, Pg. 4, Section 4.3: “Given a triplet, we can ask questions about the subject, the object, or the predicate. For instance, in (boy, in front of, people), if asking about the object, we could use templates like "What is the [Subject] [Predicate]" (What is the boy in front of?). In this case, the basic answer would be [Object]( People) or the converted full SVO sentence of the triplet, [Subject][Predicate][Object](The boy is in front of people). When asking about the subject or the predicate, similar procedure applies”) The combination of Zhou and Wang are analogous art because they are in the same field of training time series data. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, having the teachings of Zhou and Wang before them, to modify the teachings of Zhou to include the teachings of Wang’s automatic selection and application of question/ answer pairs to the domain-specific multimodal content to ensure that the selected questions correspond to the particular objects, attributes, or relationships present in the content. Such a modification would improve the relevance and accuracy of the resulting question/ answer data, reduce the need for manual selection and avoid applying irrelevant questions, thereby increasing the efficiency of processing or annotating domain-specific multimodal content. Claim(s) 7-8, 15-16 and 23-24 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al (US 2022/0130499 A1) in view of Wang et al. (“Understanding ME? Multimodal Evaluation for Fine-grained Visual Commonsense”), and further in view of Sahu et al. (“Comprehension Based Question Answering using Bloom’s Taxonomy”) Regarding claim 7, the combination of Zhou and Wang discloses all the limitations of claim 1 (as shown in the rejections above). Zhou in view of Wang fails to disclose: wherein the question/answer pairs applied to the acquired, domain-specific multimodal content comprise varying levels of complexity However, Sahu explicitly discloses: wherein the question/answer pairs applied to the acquired, domain-specific multimodal content comprise varying levels of complexity. (Sahu, Pg. 21, Fig1: PNG media_image1.png 411 799 media_image1.png Greyscale ,Pg. 22, Col. 2, Section 3.3: “To test our idea of proximal context we consider the level L of task give by each dataset then allow only proximal clarifications of level L - 1. We label each question prefix with the level of Bloom’s Taxonomy that it falls into, and then force the model to choose from the set CL of clarifications of level L.”) The combination of Zhou, Wang and Sahu are analogous art because they are in the same field of training time series data. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, having the teachings of Zhou, Wang and Sahu before them, to modify the teachings of Zhou and Wang to include the teachings of Sahu organizing question/ answer pairs according to multiple complexity levels and a hierarchical taxonomy to the domain-specific multimodal content to provide questions suited to different levels of user knowledge and to systematically cover related domain concepts. This would improve the organization, relevance, and comprehensiveness of the question/ answer pairs while enabling progressive evaluation from simpler concepts to more complex concepts. Regarding claim 8, the combination of Zhou, Wang and Sahu discloses all the limitations of claim 7 (as shown in the rejections above). Zhou in view of Wang fails to disclose: wherein the question/answer pairs are applied to the acquired, domain-specific multimodal content as respective layers of at least one hierarchical taxonomy. However, Sahu explicitly discloses: wherein the question/answer pairs are applied to the acquired, domain-specific multimodal content as respective layers of at least one hierarchical taxonomy. (Sahu, Pg. 21, Fig1: PNG media_image1.png 411 799 media_image1.png Greyscale ) Regarding claim 15, it is rejected under the same rationale with claim 7 (as shown in the rejections above), because they are analogous claims. Regarding claim 16, it is rejected under the same rationale with claim 8 (as shown in the rejections above), because they are analogous claims. Regarding claim 23, it is rejected under the same rationale with claim 7 (as shown in the rejections above), because they are analogous claims. Regarding claim 24, it is rejected under the same rationale with claim 8 (as shown in the rejections above), because they are analogous claims. Conclusion Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMY TRAN whose telephone number is (571)270-0693. The examiner can normally be reached Monday - Friday 7:30 am - 5:00 pm EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /AMY TRAN/Examiner, Art Unit 2126 /DAVID YI/Supervisory Patent Examiner, Art Unit 2126