DETAILED ACTION
Status of Claims
This Office action is responsive to communications filed on 2025-10-22. Claim(s) 2, 4, 6, 10, 12, 14, 18, and 20 were cancelled. Claim(s) 1, 3, 5, 7-9, 11, 13, 15-17, and 19 is/are pending and are examined herein.
Claim(s) 1, 3, 5, 7-9, 11, 13, 15-17, and 19 is/are objected to.
Claim(s) 1, 3, 5, 7-9, 11, 13, 15-17, and 19 is/are rejected under 35 USC 112(b).
Claim(s) 1, 3, 5, 7-9, 11, 13, 15-17, and 19 is/are rejected under 35 USC 112(a).
Claim(s) 1, 3, 5, 7-9, 11, 13, 15-17, and 19 is/are rejected under 35 USC 101.
Claim(s) 1, 3, 5, 7-9, 11, 13, 15-17, and 19 is/are rejected under 35 USC 103.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Regarding objections for informalities, the applicant’s amendments do not adequately address all of the issues raised in the previous Office action and introduce new issues as described below. Issues in the pending claims are indicated below.
Regarding rejections under 35 USC 112, the applicant’s remarks have been fully considered.
Regarding the previously filed claims, the applicant “traverses the rejections” [remarks, pages 11-12] but provides no rationale in support of the traversals. Bare unsupported assertions of traversal fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the previously filed claims define a patentable invention without specifically pointing out the reasons therefor.
Regarding the amended claims, the amendments resolve the concerns raised in the previous Office action but also introduce new issues as described below.
Regarding rejections under 35 USC 101, the applicant’s remarks have been fully considered but they are unpersuasive.
The applicant argues that the “reading comprehension model’ of the claim “is a specialized machine learning model instead of a generic machine learning model” because it “takes the first input, question index, and the question mask as inputs” [remarks, page 14]. This argument is not persuasive: a machine learning model is not made specific just by indicating the type of input it receives. The examiner maintains that the “reading comprehension model” of the claim is generic and black-box because the claim recites nothing specific about the structure of the model beyond the type of input it receives and the type of output it produces.
The applicant argues that the claim “provide[s] a technical solution to a technical problem” because the use of a question mask allows it to “answer a plurality of questions while avoiding interference among the questions” [remarks, pages 14-15]. This argument is unpersuasive on at least two counts. First, MPEP 2106.05(a) also indicates that one of the requirements of the improvements analysis is that “the claim must be evaluated to ensure the claim itself reflects the disclosed improvement in technology”. Any purported improvement provided by the question mask of the claim is not clearly reflected by the broad language used in the pending claim to describe the question mask. Second, MPEP 2106.05(a) also indicates that “judicial exception alone cannot provide the improvement”. As explained in previous Office actions and below, the question mask as recited in the claim is an abstract idea: a human being could mentally or manually construct a question mask of the form described in the claims. Consequently, any purported improvement provided by the question mask do not meet the requirements of the improvements analysis.
The complete analysis, updated in view of the amended claims, is given below.
Regarding the rejections under 35 USC 103, the applicant’s remarks have been fully considered.
Regarding the previously filed claims, the applicant “traverses the rejections” [remarks, pages 15] but provides no rationale in support of the traversal. Bare unsupported assertions of traversal fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the previously filed claims define a patentable invention without specifically pointing out how the language of the previously filed claims patentably distinguishes them from the prior art made of record.
Regarding the amended claims, the applicant’s remarks are unpersuasive because the claim continues to include broadly recited claim language which does not adequately distinguish over the self-attention mask disclosed by Lu in view of Osugi and Dong. More precisely, the applicant appears to draw attention to the limitations regarding “the plurality of second elements corresponding to a plurality of characters to be masked” and “the plurality of characters to be masked correspond[ing] to other question(s) in the question set” [remarks, pages 19-20]. However, the examiner notes that the broadest reasonable interpretation of “correspond” encompasses any equivalence or similarity in character, quantity, quality, origin, structure, function, etc (cf. Wiktionary). The entries which are set to -∞ in Dong fall under the broadest reasonable interpretation of “corresponding” to a plurality of characters to be masked (since they occur in the same positions as characters to be masked), and the plurality of characters to be masked falls under the broadest reasonable interpretation of “correspond[ing]” to other question(s) in the question set (since, e.g., these characters occur in the same question set as the other questions). If the applicant believes the claimed question mask is substantively different from the mask disclosed by Lu in view of Osugi and Dong, the applicant is invited to narrow claim language to recite the differences precisely.
The complete prior art mapping, updated in view of the amended claims, is given below.
Claim Objections
Claim(s) 1, 3, 5, 7-9, 11, 13, 15-17, and 19 is/are objected to because of the following informalities:
Claims 1, 9, and 17 recite obtain a plurality of answers outputted by the interaction layer at the same time [emphasis added] but the underlined phrase lacks antecedent basis. Alternative language is advised. Dependent claims 3, 5, 7-8, 11, 13, 15-16, and 19 inherit the objection.
Claims 1, 9, and 17 recite the plurality of characters to be masked correspond to [emphasis added] but this should be “the plurality of characters to be masked corresponds to” for proper subject-verb agreement. Dependent claims 3, 5, 7-8, 11, 13, 15-16, and 19 inherit the objection.
Claims 7 and 15 recite the plurality of answers are generated [emphasis added] but this should be “the plurality of answers is generated” for proper subject-verb agreement. Dependent claims 8 and 16 inherit the objection.
Appropriate correction is required.
Claim Rejections - 35 USC 112(b)
The following is a quotation of 35 USC 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 USC 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 1, 3, 5, 7-9, 11, 13, 15-17, and 19 is/are rejected under 35 USC 112(b) or 35 USC 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 USC 112, the applicant), regards as the invention.
Claims 1, 9, and 17 were amended to recite wherein determining the question mask comprises: building a first data item … for each question in the question set: determining a plurality of second elements…; and determining the question mask by setting the plurality of second elements as a second value [emphasis added]. This appears to indicate that a single instance of “determining the question mask comprises” an instance of “building a first data item” as well as multiple repetitions of the steps of “determining a plurality of second elements” and “determining the question mask by setting the plurality of second elements as a second value” (one repetition “for each question in the question set”). However, this is indefinite because it is not clear to the examiner what it means for a single instance of “determining the question mask” to comprise multiple repetitions of the step of “determining the question mask”. The specification provides no clarification on this point since it does not appear to describe a repetition of steps “for each question in the question set” when determining the question mask (cf. 112(a) rejections). The claims are therefore indefinite, and dependent claims 3, 5, 7-8, 11, 13, 15-16, and 19 inherit the rejection. The examiner suggests removing the phrase “for each question in the question set” from the claim, and for the purpose of compact prosecution, the claim is interpreted accordingly herein.
Claims 1, 9, and 17 were amended to recite both acquiring… a first input comprising a first text and a question set as well as for each question in the question set… determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model [emphasis added]. This limitation is indefinite because it is not clear what it means to “make” a question in the question set when the question set is acquired as part of the input to the method. The specification provides no guidance on this point as it does not describe “making” questions (cf. 112(a) rejections). The claims are therefore indefinite, and dependent claims 3, 5, 7-8, 11, 13, 15-16, and 19 inherit the rejection. The examiner suggests removing the underlined phrase from the claim, and for the purpose of compact prosecution, the claim is interpreted accordingly herein.
Claims 1, 9, and 17 were amended to recite other question(s) in the question set [emphasis added]. This language is indefinite because it is unclear whether “other question(s)” is singular or plural. In other words, it is unclear whether the scope of the claim is intended to encompass one other question in the question set, more than one other questions in the question set, or both of these situations. For the purpose of compact prosecution, the claim is interpreted broadly as encompassing both of these situations. Alternative language which clarifies the intended scope of the claim is advised (e.g., “one or more other questions in the question set”).
Claim Rejections - 35 USC 112(a)
The following is a quotation of the first paragraph of 35 USC 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 USC 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim(s) 1, 3, 5, 7-9, 11, 13, 15-17, and 19 is/are rejected under 35 USC 112(a) or 35 USC 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 USC 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1, 9, and 17 were amended to recite wherein determining the question mask comprises: … for each question in the question set: determining a plurality of second elements…; and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model [emphasis added]. The underlined clauses are new matter for the following reasons. First, the limitation “for each question in the question set” appears to suggest that the following steps of “determining the plurality of second elements” and of “determining the question mask” are done repeatedly while building the question mask (one repetition for each question in the question set), since it is indicated that “determining the question mask comprises” these limitations, but a repetition of these steps for each question in the question set while building a single question mask is not described in the specification (cf. [specification, 0044-0051]). Second, the claim appears to indicate that that determining the question mask serves to “make the question”, but the specification does not describe the question mask being used to “make” questions (the questions appear to be provided as input to the reading comprehension system; they are not “made” by the system). The underlined limitations are therefore new matter and rejected for inadequate written description. The examiner suggests removing the underlined phrases from the claim. Dependent claims 3, 5, 7-8, 11, 13, 15-16, and 19 inherit the rejection.
Claim Rejections - 35 USC 101
35 USC 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1, 3, 5, 7-9, 11, 13, 15-17, and 19 is/are rejected under 35 USC 101 because the claimed invention(s) is/are directed to abstract ideas without significantly more.
Claim 1
Step 1. The claim and its dependents 3, 5, and 7-8 fall under the statutory category of methods. The step 2 analysis follows.
Step 2A Prong 1. The claim recites the following abstract ideas:
determining, [by the one or more computers,] a question index for indicating a position of the first separation identifier in the first input and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
obtain a first output… wherein the first output comprises a first text feature representation of the first text and a question set feature representation of the question set; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. The broadest reasonable interpretation of a feature representation encompasses any summary, and a human being can manually generate summaries of texts they read. See MPEP 2106.04(a)(2)(III).)
based on the question index, determining, [by the one or more computers,] from the question set feature representation, question feature representations associated with each question in the question set respectively; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
obtain a plurality of answers… wherein the plurality of answers corresponds to the plurality of questions, respectively, (This recites a mental process that can be performed in the human mind or by a human using pen and paper, since a human being can manually answer questions. See MPEP 2106.04(a)(2)(III).)
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model. (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
Step 2A Prong 2. The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
A computer-implemented method, comprising: … by one or more computers… by the one or more computers… by the one or more computers… (This recites generic computing components for performing an abstract idea. See MPEP 2106.05(f)(2).)
acquiring, [by one or more computers,] a first input comprising a first text and a question set associated with the first text, wherein the first input comprises a first separation identifier for separating a plurality of questions to be answered currently in the question set and a second separation identifier which is different from the first separation identifier and is configured to separate the first text from the question set; (This recites insignificant extra-solution activity. See MPEP 2106.05(g).)
inputting the first input, the question index, and the question mask into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model, wherein the reading comprehension model is a trained machine learning model (This recites a generic application of a machine learning model. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
[a trained machine learning model] trained based on a training dataset including a training text and a set of questions associated with the training text (This recites data of a particular type or source, merely linking an abstract idea to a particular field of use. See MPEP 2106.05(h).)
and is trained to process the set of questions simultaneously, and (This recites a general link between an abstract idea and a particular field of use or technological environment. See MPEP 2106.05(h).)
and inputting the first output into an interaction layer to [obtain a plurality of answers] outputted by the interaction layer at the same time, wherein the interaction layer comprises a neural network, (This recites a generic application of a neural network. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B. The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
A computer-implemented method, comprising: … by one or more computers… by one or more computers… by one or more computers… (This recites generic computing components for performing an abstract idea. See MPEP 2106.05(f)(2).)
acquiring, [by one or more computers,] a first input comprising a first text and a question set associated with the first text, wherein the first input comprises a first separation identifier for separating a plurality of questions to be answered currently in the question set and a second separation identifier which is different from the first separation identifier and is configured to separate the first text from the question set; (This insignificant extra-solution activity is well-understood, routine, conventional as it is mere data transfer. See MPEP 2106.05(d)(II), “Receiving or transmitting data over a network” and/or “Storing and retrieving information in memory”.)
inputting the first input, the question index, and the question mask into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model, wherein the reading comprehension model is a trained machine learning model (This recites a generic application of a machine learning model. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
[a trained machine learning model] trained based on a training dataset including a training text and a set of questions associated with the training text (This recites data of a particular type or source, merely linking an abstract idea to a particular field of use. See MPEP 2106.05(h).)
and is trained to process the set of questions simultaneously, and (This recites a general link between an abstract idea and a particular field of use or technological environment. See MPEP 2106.05(h).)
and inputting the first output into an interaction layer to [obtain a plurality of answers] outputted by the interaction layer at the same time, wherein the interaction layer comprises a neural network, (This recites a generic application of a neural network. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Claim 3
Step 2A Prong 1. The claim recites the following abstract ideas:
The abstract idea(s) in the claim(s) on which it depends.
[The method of claim 1, wherein determining the question index comprises:] building a question index vector associated with the question set, wherein the question index vector comprises first elements which correspond to characters in the question set and the first separation identifier and indicate positions of each character comprised in the question set and the first separation identifier; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
and determining the question index by setting each first element corresponding to an instance of the first separation identifier as a first value. (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
Step 2A Prong 2. The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the claim(s) on which it depends.
Step 2B. The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the claim(s) on which it depends.
Claim 5
Step 2A Prong 1. The claim recites the following abstract ideas:
The abstract idea(s) in the claim(s) on which it depends.
[The method of claim 1, wherein determining the plurality of second elements comprises:] determining a similarity among the plurality of questions; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
and determining the plurality of second elements based on the similarity. (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
Step 2A Prong 2. The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the claim(s) on which it depends.
Step 2B. The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the claim(s) on which it depends.
Claim 7
Step 2A Prong 1. The claim recites the following abstract ideas:
The abstract idea(s) in the claim(s) on which it depends.
[The method of claim 1, wherein the plurality of answers are generated by operations comprising:] based on the question feature representations, building a second data item represented in rows and columns, wherein one row in the second data item corresponds to the question feature representation associated with one question in the question set; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
determining a third data item represented in rows and columns by performing a first operation on the second data item and the first text feature representation, the third data item comprising a start identifier element and an end identifier element associated with the question set; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
and based on the start identifier element and the end identifier element, generating the plurality of answers by the first text. (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
Step 2A Prong 2. The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the claim(s) on which it depends.
Step 2B. The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the claim(s) on which it depends.
Claim 8
Step 2A Prong 1. The claim recites the following abstract ideas:
The abstract idea(s) in the claim(s) on which it depends.
[The method of claim 7, wherein the first operation comprises:] performing element multiplication of the second data item and the first text feature representation; (This recites a mathematical concept and a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(I, III).)
… to acquire the third data item. (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
Step 2A Prong 2. The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the claim(s) on which it depends.
and inputting a result of the element multiplication to the neural network [to acquire the third data item.] (This recites a generic application of a machine learning model. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B. The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the claim(s) on which it depends.
and inputting a result of the element multiplication to the neural network [to acquire the third data item.] (This recites a generic application of a machine learning model. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Claim 9
Step 1. The claim and its dependents 11, 13 and 15-16 fall under the statutory category of machines.
Step 2A Prong 1. The claim recites the following abstract ideas:
determining a question index for indicating a position of the first separation identifier in the first input and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
obtain a first output… wherein the first output comprises a first text feature representation of the first text and a question set feature representation of the question set; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. The broadest reasonable interpretation of a feature representation encompasses any summary, and a human being can manually generate summaries of texts they read. See MPEP 2106.04(a)(2)(III).)
based on the question index, determining, from the question set feature representation, question feature representations associated with each question in the question set respectively; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
obtain a plurality of answers… wherein the plurality of answers correspond to the plurality of questions, respectively, (This recites a mental process that can be performed in the human mind or by a human using pen and paper, since a human being can manually answer questions. See MPEP 2106.04(a)(2)(III).)
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model. (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
Step 2A Prong 2. The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
A computing device, comprising: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, (This recites generic computing elements for applying an abstract idea. See MPEP 2106.05(f)(2).)
the one or more programs comprising instructions for performing operations comprising: (This recites mere instructions to apply an abstract idea. See MPEP 2106.05(f).)
acquiring a first input comprising a first text and a question set associated with the first text, wherein the first input comprises a first separation identifier for separating a plurality of questions to be answered currently in the question set and a second separation identifier which is different from the first separation identifier and is configured to separate the first text from the question set; (This recites insignificant extra-solution activity. See MPEP 2106.05(g).)
inputting the first input, the question index, and the question mask into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model, wherein the reading comprehension model is a trained machine learning model (This recites a generic application of a machine learning model. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
[a trained machine learning model] trained based on a training dataset including a training text and a set of questions associated with the training text (This recites data of a particular type or source, merely linking an abstract idea to a particular field of use. See MPEP 2106.05(h).)
and is trained to process the set of questions simultaneously, and (This recites a general link between an abstract idea and a particular field of use or technological environment. See MPEP 2106.05(h).)
and inputting the first output into an interaction layer to [obtain a plurality of answers] outputted by the interaction layer at the same time, wherein the interaction layer comprises a neural network, (This recites a generic application of a neural network. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B. The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
A computing device, comprising: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, … by one or more computers (This recites generic computing elements for applying an abstract idea. See MPEP 2106.05(f)(2).)
the one or more programs comprising instructions for performing operations comprising: (This recites mere instructions to apply an abstract idea. See MPEP 2106.05(f).)
acquiring a first input comprising a first text and a question set associated with the first text, wherein the first input comprises a first separation identifier for separating a plurality of questions to be answered currently in the question set and a second separation identifier which is different from the first separation identifier and is configured to separate the first text from the question set; (This insignificant extra-solution activity is well-understood, routine, conventional as it is mere data transfer. See MPEP 2106.05(d)(II), “Receiving or transmitting data over a network” and/or “Storing and retrieving information in memory”.)
inputting the first input, the question index, and the question mask into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model, wherein the reading comprehension model is a trained machine learning model (This recites a generic application of a machine learning model. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
[a trained machine learning model] trained based on a training dataset including a training text and a set of questions associated with the training text (This recites data of a particular type or source, merely linking an abstract idea to a particular field of use. See MPEP 2106.05(h).)
and is trained to process the set of questions simultaneously, and (This recites a general link between an abstract idea and a particular field of use or technological environment. See MPEP 2106.05(h).)
and inputting the first output into an interaction layer to [obtain a plurality of answers] outputted by the interaction layer at the same time, wherein the interaction layer comprises a neural network, (This recites a generic application of a neural network. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Claims 11, 13, and 15-16 inherit limitations from claim 9 and then further recite limitations that are substantially similar to those of claims 3, 5, and 7-8, respectively, so they are rejected by the same rationale.
Claim 17
Step 1. The claim and its dependents 18-20 fall under the statutory category of machines.
Step 2A Prong 1. The claim recites the following abstract ideas:
determining a question index for indicating a position of the first separation identifier in the first input and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
obtain a first output… wherein the first output comprises a first text feature representation of the first text and a question set feature representation of the question set; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. The broadest reasonable interpretation of a feature representation encompasses any summary, and a human being can manually generate summaries of texts they read. See MPEP 2106.04(a)(2)(III).)
based on the question index, determining, from the question set feature representation, question feature representations associated with each question in the question set respectively; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
obtain a plurality of answers… wherein the plurality of answers correspond to the plurality of questions, respectively, (This recites a mental process that can be performed in the human mind or by a human using pen and paper, since a human being can manually answer questions. See MPEP 2106.04(a)(2)(III).)
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model. (This recites a mental process that can be performed in the human mind or by a human using pen and paper. See MPEP 2106.04(a)(2)(III).)
Step 2A Prong 2. The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
A non-transitory computer readable storage medium, (This recites generic computing elements for applying an abstract idea. See MPEP 2106.05(f)(2).)
storing one or more programs comprising instructions that, when executed by one or more processors of a computing device, cause the computing device to perform operations comprising: (This recites the insignificant extra-solution activity of data storage and describes the data being stored. See MPEP 2106.05(g)(3).)
acquiring a first input comprising a first text and a question set associated with the first text, wherein the first input comprises a first separation identifier for separating a plurality of questions to be answered currently in the question set and a second separation identifier which is different from the first separation identifier and is configured to separate the first text from the question set; (This recites insignificant extra-solution activity. See MPEP 2106.05(g).)
inputting the first input, the question index, and the question mask into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model, wherein the reading comprehension model is a trained machine learning model (This recites a generic application of a machine learning model. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
[a trained machine learning model] trained based on a training dataset including a training text and a set of questions associated with the training text (This recites data of a particular type or source, merely linking an abstract idea to a particular field of use. See MPEP 2106.05(h).)
and is trained to process the set of questions simultaneously, and (This recites a general link between an abstract idea and a particular field of use or technological environment. See MPEP 2106.05(h).)
and inputting the first output into an interaction layer to [obtain a plurality of answers] outputted by the interaction layer at the same time, wherein the interaction layer comprises a neural network, (This recites a generic application of a neural network. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Step 2B. The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
A non-transitory computer readable storage medium, (This recites generic computing elements for applying an abstract idea. See MPEP 2106.05(f)(2).)
storing one or more programs comprising instructions that, when executed by one or more processors of a computing device, cause the computing device to perform operations comprising: (This insignificant extra-solution activity is well-understood, routine, conventional as it is mere data storage. See MPEP 2106.05(d)(II), “Electronic recordkeeping” and/or “Storing and retrieving information in memory”.)
acquiring a first input comprising a first text and a question set associated with the first text, wherein the first input comprises a first separation identifier for separating a plurality of questions to be answered currently in the question set and a second separation identifier which is different from the first separation identifier and is configured to separate the first text from the question set; (This insignificant extra-solution activity is well-understood, routine, conventional as it is mere data transfer. See MPEP 2106.05(d)(II), “Receiving or transmitting data over a network” and/or “Storing and retrieving information in memory”.)
inputting the first input, the question index, and the question mask into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model, wherein the reading comprehension model is a trained machine learning model (This recites a generic application of a machine learning model. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
[a trained machine learning model] trained based on a training dataset including a training text and a set of questions associated with the training text (This recites data of a particular type or source, merely linking an abstract idea to a particular field of use. See MPEP 2106.05(h).)
and is trained to process the set of questions simultaneously, and (This recites a general link between an abstract idea and a particular field of use or technological environment. See MPEP 2106.05(h).)
and inputting the first output into an interaction layer to [obtain a plurality of answers] outputted by the interaction layer at the same time, wherein the interaction layer comprises a neural network, (This recites a generic application of a neural network. In other words, this recites merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea. See MPEP 2106.05(f).)
Claims 19 inherits limitations from claim 17 and then further recites limitations that are substantially similar to those of claims 3, so it is rejected by the same rationale.
Claim Rejections - 35 USC 103
The following is a quotation of 35 USC 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 7, 9, 15, and 17 is/are rejected under 35 USC 103 as being unpatentable over LU Lingyun et al. (CN111831805A, published 2020-10-27; hereafter, “Lu”) in view of Yasuhito OSUGI et al. (US20220229997A1, filed 2019-05-28; hereafter, “Osugi”) and Li DONG et al. (Unified Language Model Pre-training for Natural Language Understanding and Generation, published 2019-10-15; hereafter, “Dong”).
Claim 1
Lu discloses:
A computer-implemented method, comprising: ([Lu, 0123]: Lu discloses “a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the model creation method shown in the above method embodiment is implemented” [Lu, 0123].)
acquiring, by one or more computers, a first input comprising a first text and a question set associated with the first text, wherein the first input comprises [a first separation identifier for separating] a plurality of questions to be answered currently in the question set ([Lu, 0085-0087]: Lu discloses {Q_{group}, Text} for Q_{group} = [Q_1, …, Q_n] being input into a target prediction model [Lu, 0085-0087]. In other words, {Q_group, Text} is the “first input” of the claim, Text is the “first text” of the claim, Q_{group} is the “question set” of the claim, and Q_1, …, Q_n are the “plurality of questions to be answered currently in the question set” of the claim.)
and a second separation identifier [which is different from the first separation identifier] and is configured to separate the first text from the question set; ([Lu, 0070]: Lu discloses that the questions and text are “connected and separated by ‘[SEP]’” [Lu, 0070]. This token maps to the “second separation identifier” of the claim.)
obtain a first output… and inputting the first output into an interaction layer to obtain a plurality of answers outputted by the interaction layer at the same time, wherein the interaction layer comprises a neural network, and wherein the plurality of answers corresponds to the plurality of questions, respectively, ([Lu, 0070, 0088-0089]: Lu discloses that the target prediction model processes the input {Q_{group}, Text} = {[Q_1, …, Q_n], Text} to produce the output [A_1, …, A_n] [Lu, 0088-0089]. The model produces a “third feature set” of the input {Q_{group}, Text} which is input “into [a] fully connected layer and [a] SoftMax layer” [Lu, 0070] to obtain the final output [A_1, …, A_n]. The third feature set maps to the “first output” of the claim (this mapping is refined in the combination below). The fully connected and SoftMax layers map to the “interaction layer” and the “neural network” of the claim.)
While Lu, as noted above, discloses computing a “third feature set” from the input {Q_{group}, Text}, it does not distinctly disclose the feature representations described in the claim. More precisely, Lu does not distinctly disclose:
a first separation identifier for separating [a plurality of questions to be answered currently in the question set] which is different from the first separation identifier
determining, by the one or more computers, a question index for indicating a position of the first separation identifier in the first input,
and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input;
inputting the first input, the question index, and the question mask into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model,
wherein the reading comprehension model is a trained machine learning model trained based on a training dataset including a training text and a set of questions associated with the training text and is trained to process the set of questions simultaneously,
and wherein the first output comprises a first text feature representation of the first text and a question set feature representation of the question set;
based on the question index, determining, by the one or more computers, from the question set feature representation, question feature representations associated with each question in the question set respectively;
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text;
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model.
Osugi is in the field of natural language processing and discloses a system which takes a plurality of questions Q_i, Q_{i-1}, …, Q_{i-k} and a document P as input [Osugi, 0020]. In combination with Lu, the document P of Osugi corresponds to the Text in the input {Q_group, Text} of Lu, and the question group Q_group = [Q_1, …, Q_n] of Lu is used in place of Q_i, …, Q_{i-k}, A_{i-1}, …, A_{i-k} of Osugi. Moreover, Lu in view of Osugi discloses:
a first separation identifier for separating [a plurality of questions to be answered currently in the question set and a second separation identifier] which is different from the first separation identifier ([Osugi, 0044; Lu, 0070]: Osugi discloses using a “separator token [SEP]” to separate questions in the question set, which, in the combination, is the question group Q_{group} = [Q_1, …, Q_n] of Lu as noted above. The separator token [SEP] maps to the “first separation identifier” recited by the claim. The examiner notes that Osugi also discloses a “separator token [SEP]” to separate the text from the questions [Osugi, 0041] (cf. the “second separation identifier” of the claim and its mapping from [Lu, 0070] as described above), and different instances of the separator token [SEP] are used to separate the document from the questions [Osugi, 0041] and the questions from each other [Osugi, 0044].)
determining, by the one or more computers, a question index for indicating a position of the first separation identifier in the first input, ([Osugi, 0044]: As noted above, Lu in view of Osugi discloses a set of questions separated by separator tokens [SEP]. This set of questions separated by [SEP] maps to the “question index” recited by the claim: it itself indicates the positions of the delimiters between questions, because a string always indicates the positions of each of its constituent characters. The examiner notes that dependent claim 3 recites further details about the question index, and its analysis given below correspondingly gives an alternative mapping.)
inputting the first input, the question index, [and the question mask] into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model, ([Osugi, 0023]: Osugi discloses a “question encoding unit” and a “context encoding unit” [Osugi, 0023]. In the combination, the feature extraction components of the prediction model of Lu are replaced with the question and context encoding units of Osugi so that the output of the question and context encoding units corresponds to the “third feature set” of Lu and the “first output” of the claim. The question and context encoding units then map to the “reading comprehension model” of the claim. These mappings are elaborated in the next parentheticals.) wherein the reading comprehension model is a trained machine learning model ([Osugi, 0023]: Osugi discloses that the question and context encoding units are “implemented by one or more neural networks” [Osugi, 0023].) trained based on a training dataset including a training text and a set of questions associated with the training text ([Osugi, figure 2 and 0030-0033]: The model parameters of the question and context encoding units are trained by an update unit during learning [Osugi, figure 2 and 0030-0033]. Any of the documents used during training map to the “training text” of the claim and the questions associated to the training text map to the “set of questions associated with the training text” of the claim.) and is trained to process the set of questions simultaneously, ([Osugi, figure 2 and 0030-0033; Lu, 0087-0089]: Training the “reading comprehension model” as mapped above permits it to process the “set of questions” as mapped above. In the combination, the “set of questions” is processed “simultaneously” as described in Lu.)
and wherein the first output comprises a first text feature representation of the first text and a question set feature representation of the question set; ([Osugi, 0024-25]: The question and context encoding units of Osugi produce feature vectors u_j^t for varying j [Osugi, 0024] which are vectors are based on both the document P and on the questions Q_j for varying j. In other words, these vectors map to both the “first text feature representation” and to the “question set feature representation” as recited in the claim.)
based on the question index, determining, by the one or more computers, from the question set feature representation, question feature representations associated with each question in the question set respectively; ([Osugi, 0024-25]: The feature vectors u_j^t map to the “question feature representations associated with each question” Q_j of the claim. These feature vectors are based on the “question index” and on the “question set feature representation” as mapped above.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the machine reading comprehension system of Lu with parts of the question answering architecture disclosed by Osugi because the latter presents advantages such as allowing for answers in “both an extractive mode and a generative mode” [Osugi, 0007], thereby resulting in a more flexible and effective system.
Lu in view of Osugi does not disclose:
and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input; … and the question mask
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text;
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model.
Dong is also in the field of natural language processing and discloses a neural-network-based approach to natural language understanding tasks [Dong, Abstract]. Moreover, Lu in view of Osugi and Dong discloses:
and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input; … and the question mask ([Dong, section 2.3 and figure 1]: Dong discloses constructing a “self-attention mask” in order to mask/screen out portions of the input and thereby control the attention of the language model. In the combination, the self-attention masks are applied to the question set in order to screen portions of the question set.)
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text; ([Dong, section 2.3 and figure 1]: Dong discloses constructing a “self-attention mask” which is arranged in rows and columns, with the entries in each row corresponding to tokens in the input. The entries of this self-attention mask are the “second elements” recited by the claim, and each row of the self-attention mask is a “row of the first data item correspond[ing] to characters” as recited by the claim.)
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model. ([Dong, section 2.3]: Dong discloses setting values in the “self-attention matrix” to -∞ in order to block the model’s attention towards those tokens. In other words, a row of the self-attention mask maps to the “one group of second elements” of the claim, -∞ is the “second value” of the claim, and the entries which are set to -∞ map to the “plurality of second elements” of the claim. The examiner notes that the “plurality of second elements” fall under the broadest reasonable interpretation of “corresponding to a plurality of characters to be masked in the question set” (namely, characters which occur in positions where the mask is set to -∞), and that the “plurality of characters” thus determined in turn falls under the broadest reasonable interpretation of “correspond[ing] to other question(s) in the question set” (in, for example, the sense that these characters occur in the same question set as the “other question(s)” of the question set).)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the question answering system of Lu in view of Osugi with the attention masking method disclosed by Dong because the method taught by Dong “alleviat[es] the need of separately training and hosting multiple [language model]s” [Dong, page 2], so the combination would result in a more efficient system.
Claim 7
Lu in view of Osugi and Dong discloses the elements of the parent claim(s). It also discloses:
[The method of claim 1, wherein the plurality of answers are generated by operations comprising:] based on the question feature representations, building a second data item represented in rows and columns, wherein one row in the second data item corresponds to the question feature representation associated with one question in the question set; ([Osugi, 0024-25]: Osugi discloses producing feature vectors u_j^t associated with the questions, as noted under claim 6 above. The examiner notes that the “second data item” recited by the claim is simply a matrix whose jth row is the vector u_j^t.)
determining a third data item represented in rows and columns by performing a first operation on the second data item and the first text feature representation, the third data item comprising a start identifier element and an end identifier element associated with the question set; ([Osugi, 0068]: Osugi discloses the “extractive mode” of answer generation, in which “a pair of start and end positions of a range… in the document P”. These start and end positions map to the “start identifier element” and “end identifier element” recited by the claim. The examiner notes that the “third data item” recited by the claim is simply a matrix each row of which records the start and end positions for the answer for a question.)
and based on the start identifier element and the end identifier element, generating the plurality of answers by the first text. ([Osugi, 0068]: Osugi discloses an “extractive mode” of answer generation, in which “a pair of start and end positions of a range corresponding to the correct answer in the document P”. In other words, each answer is determined as the span between the identified start and end positions.)
The same motivation to combine applies.
Claim 9
Lu discloses:
A computing device, comprising: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs comprising instructions for performing operations comprising: ([Lu, 0113-0115]: Lu discloses the method disclosed therein may be implemented on “an electronic device, including a processor and a memory” [Lu, 0113] where the memory is “used for storing operation instructions” [Lu, 0114].)
acquiring a first input comprising a first text and a question set associated with the first text, wherein the first input comprises [a first separation identifier for separating] a plurality of questions to be answered currently in the question set ([Lu, 0085-0087]: Lu discloses {Q_{group}, Text} for Q_{group} = [Q_1, …, Q_n] being input into a target prediction model [Lu, 0085-0087]. In other words, {Q_group, Text} is the “first input” of the claim, Text is the “first text” of the claim, Q_{group} is the “question set” of the claim, and Q_1, …, Q_n are the “plurality of questions to be answered currently in the question set” of the claim.)
and a second separation identifier [which is different from the first separation identifier] and is configured to separate the first text from the question set; ([Lu, 0070]: Lu discloses that the questions and text are “connected and separated by ‘[SEP]’” [Lu, 0070]. This token maps to the “second separation identifier” of the claim.)
obtain a first output… and inputting the first output into an interaction layer to obtain a plurality of answers outputted by the interaction layer at the same time, wherein the interaction layer comprises a neural network, and wherein the plurality of answers corresponds to the plurality of questions, respectively, ([Lu, 0070, 0088-0089]: Lu discloses that the target prediction model processes the input {Q_{group}, Text} = {[Q_1, …, Q_n], Text} to produce the output [A_1, …, A_n] [Lu, 0088-0089]. The model produces a “third feature set” of the input {Q_{group}, Text} which is input “into [a] fully connected layer and [a] SoftMax layer” [Lu, 0070] to obtain the final output [A_1, …, A_n]. The third feature set maps to the “first output” of the claim (this mapping is refined in the combination below). The fully connected and SoftMax layers map to the “interaction layer” and the “neural network” of the claim.)
While Lu, as noted above, discloses computing a “third feature set” from the input {Q_{group}, Text}, it does not distinctly disclose the feature representations described in the claim. More precisely, Lu does not distinctly disclose:
a first separation identifier for separating [a plurality of questions to be answered currently in the question set] which is different from the first separation identifier
determining a question index for indicating a position of the first separation identifier in the first input,
and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input;
inputting the first input, the question index, and the question mask into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model,
wherein the reading comprehension model is a trained machine learning model trained based on a training dataset including a training text and a set of questions associated with the training text and is trained to process the set of questions simultaneously,
and wherein the first output comprises a first text feature representation of the first text and a question set feature representation of the question set;
based on the question index, determining from the question set feature representation, question feature representations associated with each question in the question set respectively;
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text;
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model.
Osugi is in the field of natural language processing and discloses a system which takes a plurality of questions Q_i, Q_{i-1}, …, Q_{i-k} and a document P as input [Osugi, 0020]. In combination with Lu, the document P of Osugi corresponds to the Text in the input {Q_group, Text} of Lu, and the question group Q_group = [Q_1, …, Q_n] of Lu is used in place of Q_i, …, Q_{i-k}, A_{i-1}, …, A_{i-k} of Osugi. Moreover, Lu in view of Osugi discloses:
a first separation identifier for separating [a plurality of questions to be answered currently in the question set and a second separation identifier] which is different from the first separation identifier ([Osugi, 0044; Lu, 0070]: Osugi discloses using a “separator token [SEP]” to separate questions in the question set, which, in the combination, is the question group Q_{group} = [Q_1, …, Q_n] of Lu as noted above. The separator token [SEP] maps to the “first separation identifier” recited by the claim. The examiner notes that Osugi also discloses a “separator token [SEP]” to separate the text from the questions [Osugi, 0041] (cf. the “second separation identifier” of the claim and its mapping from [Lu, 0070] as described above), and different instances of the separator token [SEP] are used to separate the document from the questions [Osugi, 0041] and the questions from each other [Osugi, 0044].)
determining a question index for indicating a position of the first separation identifier in the first input, ([Osugi, 0044]: As noted above, Lu in view of Osugi discloses a set of questions separated by separator tokens [SEP]. This set of questions separated by [SEP] maps to the “question index” recited by the claim: it itself indicates the positions of the delimiters between questions, because a string always indicates the positions of each of its constituent characters. The examiner notes that dependent claim 3 recites further details about the question index, and its analysis given below correspondingly gives an alternative mapping.)
inputting the first input, the question index, [and the question mask] into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model, ([Osugi, 0023]: Osugi discloses a “question encoding unit” and a “context encoding unit” [Osugi, 0023]. In the combination, the feature extraction components of the prediction model of Lu are replaced with the question and context encoding units of Osugi so that the output of the question and context encoding units corresponds to the “third feature set” of Lu and the “first output” of the claim. The question and context encoding units then map to the “reading comprehension model” of the claim. These mappings are elaborated in the next parentheticals.) wherein the reading comprehension model is a trained machine learning model ([Osugi, 0023]: Osugi discloses that the question and context encoding units are “implemented by one or more neural networks” [Osugi, 0023].) trained based on a training dataset including a training text and a set of questions associated with the training text ([Osugi, figure 2 and 0030-0033]: The model parameters of the question and context encoding units are trained by an update unit during learning [Osugi, figure 2 and 0030-0033]. Any of the documents used during training map to the “training text” of the claim and the questions associated to the training text map to the “set of questions associated with the training text” of the claim.) and is trained to process the set of questions simultaneously, ([Osugi, figure 2 and 0030-0033; Lu, 0087-0089]: Training the “reading comprehension model” as mapped above permits it to process the “set of questions” as mapped above. In the combination, the “set of questions” is processed “simultaneously” as described in Lu.)
and wherein the first output comprises a first text feature representation of the first text and a question set feature representation of the question set; ([Osugi, 0024-25]: The question and context encoding units of Osugi produce feature vectors u_j^t for varying j [Osugi, 0024] which are vectors are based on both the document P and on the questions Q_j for varying j. In other words, these vectors map to both the “first text feature representation” and to the “question set feature representation” as recited in the claim.)
based on the question index, determining from the question set feature representation, question feature representations associated with each question in the question set respectively; ([Osugi, 0024-25]: The feature vectors u_j^t map to the “question feature representations associated with each question” Q_j of the claim. These feature vectors are based on the “question index” and on the “question set feature representation” as mapped above.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the machine reading comprehension system of Lu with parts of the question answering architecture disclosed by Osugi because the latter presents advantages such as allowing for answers in “both an extractive mode and a generative mode” [Osugi, 0007], thereby resulting in a more flexible and effective system.
Lu in view of Osugi does not disclose:
and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input; … and the question mask
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text;
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model.
Dong is also in the field of natural language processing and discloses a neural-network-based approach to natural language understanding tasks [Dong, Abstract]. Moreover, Lu in view of Osugi and Dong discloses:
and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input; … and the question mask ([Dong, section 2.3 and figure 1]: Dong discloses constructing a “self-attention mask” in order to mask/screen out portions of the input and thereby control the attention of the language model. In the combination, the self-attention masks are applied to the question set in order to screen portions of the question set.)
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text; ([Dong, section 2.3 and figure 1]: Dong discloses constructing a “self-attention mask” which is arranged in rows and columns, with the entries in each row corresponding to tokens in the input. The entries of this self-attention mask are the “second elements” recited by the claim, and each row of the self-attention mask is a “row of the first data item correspond[ing] to characters” as recited by the claim.)
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model. ([Dong, section 2.3]: Dong discloses setting values in the “self-attention matrix” to -∞ in order to block the model’s attention towards those tokens. In other words, a row of the self-attention mask maps to the “one group of second elements” of the claim, -∞ is the “second value” of the claim, and the entries which are set to -∞ map to the “plurality of second elements” of the claim. The examiner notes that the “plurality of second elements” fall under the broadest reasonable interpretation of “corresponding to a plurality of characters to be masked in the question set” (namely, characters which occur in positions where the mask is set to -∞), and that the “plurality of characters” thus determined in turn falls under the broadest reasonable interpretation of “correspond[ing] to other question(s) in the question set” (in, for example, the sense that these characters occur in the same question set as the “other question(s)” of the question set).)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the question answering system of Lu in view of Osugi with the attention masking method disclosed by Dong because the method taught by Dong “alleviat[es] the need of separately training and hosting multiple [language model]s” [Dong, page 2], so the combination would result in a more efficient system.
Claim 15 inherit limitations from claim 9 and then further recite limitations that are substantially similar to those of claim 7, so it is rejected using the same rationale.
Claim 17
Lu discloses:
A non-transitory computer readable storage medium, storing one or more programs comprising instructions that, when executed by one or more processors of a computing device, cause the computing device to perform operations comprising: ([Lu, 0113-0115; 0120]: Lu discloses the method disclosed therein may be implemented on “an electronic device, including a processor and a memory” [Lu, 0113] where the memory is “used for storing operation instructions” [Lu, 0114]. It also discloses that the memory may be “a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device” [Lu, 0120].)
acquiring a first input comprising a first text and a question set associated with the first text, wherein the first input comprises [a first separation identifier for separating] a plurality of questions to be answered currently in the question set ([Lu, 0085-0087]: Lu discloses {Q_{group}, Text} for Q_{group} = [Q_1, …, Q_n] being input into a target prediction model [Lu, 0085-0087]. In other words, {Q_group, Text} is the “first input” of the claim, Text is the “first text” of the claim, Q_{group} is the “question set” of the claim, and Q_1, …, Q_n are the “plurality of questions to be answered currently in the question set” of the claim.)
and a second separation identifier [which is different from the first separation identifier] and is configured to separate the first text from the question set; ([Lu, 0070]: Lu discloses that the questions and text are “connected and separated by ‘[SEP]’” [Lu, 0070]. This token maps to the “second separation identifier” of the claim.)
obtain a first output… and inputting the first output into an interaction layer to obtain a plurality of answers outputted by the interaction layer at the same time, wherein the interaction layer comprises a neural network, and wherein the plurality of answers corresponds to the plurality of questions, respectively, ([Lu, 0070, 0088-0089]: Lu discloses that the target prediction model processes the input {Q_{group}, Text} = {[Q_1, …, Q_n], Text} to produce the output [A_1, …, A_n] [Lu, 0088-0089]. The model produces a “third feature set” of the input {Q_{group}, Text} which is input “into [a] fully connected layer and [a] SoftMax layer” [Lu, 0070] to obtain the final output [A_1, …, A_n]. The third feature set maps to the “first output” of the claim (this mapping is refined in the combination below). The fully connected and SoftMax layers map to the “interaction layer” and the “neural network” of the claim.)
While Lu, as noted above, discloses computing a “third feature set” from the input {Q_{group}, Text}, it does not distinctly disclose the feature representations described in the claim. More precisely, Lu does not distinctly disclose:
a first separation identifier for separating [a plurality of questions to be answered currently in the question set] which is different from the first separation identifier
determining a question index for indicating a position of the first separation identifier in the first input,
and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input;
inputting the first input, the question index, and the question mask into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model,
wherein the reading comprehension model is a trained machine learning model trained based on a training dataset including a training text and a set of questions associated with the training text and is trained to process the set of questions simultaneously,
and wherein the first output comprises a first text feature representation of the first text and a question set feature representation of the question set;
based on the question index, determining from the question set feature representation, question feature representations associated with each question in the question set respectively;
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text;
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model.
Osugi is in the field of natural language processing and discloses a system which takes a plurality of questions Q_i, Q_{i-1}, …, Q_{i-k} and a document P as input [Osugi, 0020]. In combination with Lu, the document P of Osugi corresponds to the Text in the input {Q_group, Text} of Lu, and the question group Q_group = [Q_1, …, Q_n] of Lu is used in place of Q_i, …, Q_{i-k}, A_{i-1}, …, A_{i-k} of Osugi. Moreover, Lu in view of Osugi discloses:
a first separation identifier for separating [a plurality of questions to be answered currently in the question set and a second separation identifier] which is different from the first separation identifier ([Osugi, 0044; Lu, 0070]: Osugi discloses using a “separator token [SEP]” to separate questions in the question set, which, in the combination, is the question group Q_{group} = [Q_1, …, Q_n] of Lu as noted above. The separator token [SEP] maps to the “first separation identifier” recited by the claim. The examiner notes that Osugi also discloses a “separator token [SEP]” to separate the text from the questions [Osugi, 0041] (cf. the “second separation identifier” of the claim and its mapping from [Lu, 0070] as described above), and different instances of the separator token [SEP] are used to separate the document from the questions [Osugi, 0041] and the questions from each other [Osugi, 0044].)
determining a question index for indicating a position of the first separation identifier in the first input, ([Osugi, 0044]: As noted above, Lu in view of Osugi discloses a set of questions separated by separator tokens [SEP]. This set of questions separated by [SEP] maps to the “question index” recited by the claim: it itself indicates the positions of the delimiters between questions, because a string always indicates the positions of each of its constituent characters. The examiner notes that dependent claim 3 recites further details about the question index, and its analysis given below correspondingly gives an alternative mapping.)
inputting the first input, the question index, [and the question mask] into a reading comprehension model to [obtain a first output] outputted by the reading comprehension model, ([Osugi, 0023]: Osugi discloses a “question encoding unit” and a “context encoding unit” [Osugi, 0023]. In the combination, the feature extraction components of the prediction model of Lu are replaced with the question and context encoding units of Osugi so that the output of the question and context encoding units corresponds to the “third feature set” of Lu and the “first output” of the claim. The question and context encoding units then map to the “reading comprehension model” of the claim. These mappings are elaborated in the next parentheticals.) wherein the reading comprehension model is a trained machine learning model ([Osugi, 0023]: Osugi discloses that the question and context encoding units are “implemented by one or more neural networks” [Osugi, 0023].) trained based on a training dataset including a training text and a set of questions associated with the training text ([Osugi, figure 2 and 0030-0033]: The model parameters of the question and context encoding units are trained by an update unit during learning [Osugi, figure 2 and 0030-0033]. Any of the documents used during training map to the “training text” of the claim and the questions associated to the training text map to the “set of questions associated with the training text” of the claim.) and is trained to process the set of questions simultaneously, ([Osugi, figure 2 and 0030-0033; Lu, 0087-0089]: Training the “reading comprehension model” as mapped above permits it to process the “set of questions” as mapped above. In the combination, the “set of questions” is processed “simultaneously” as described in Lu.)
and wherein the first output comprises a first text feature representation of the first text and a question set feature representation of the question set; ([Osugi, 0024-25]: The question and context encoding units of Osugi produce feature vectors u_j^t for varying j [Osugi, 0024] which are vectors are based on both the document P and on the questions Q_j for varying j. In other words, these vectors map to both the “first text feature representation” and to the “question set feature representation” as recited in the claim.)
based on the question index, determining from the question set feature representation, question feature representations associated with each question in the question set respectively; ([Osugi, 0024-25]: The feature vectors u_j^t map to the “question feature representations associated with each question” Q_j of the claim. These feature vectors are based on the “question index” and on the “question set feature representation” as mapped above.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the machine reading comprehension system of Lu with parts of the question answering architecture disclosed by Osugi because the latter presents advantages such as allowing for answers in “both an extractive mode and a generative mode” [Osugi, 0007], thereby resulting in a more flexible and effective system.
Lu in view of Osugi does not disclose:
and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input; … and the question mask
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text;
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model.
Dong is also in the field of natural language processing and discloses a neural-network-based approach to natural language understanding tasks [Dong, Abstract]. Moreover, Lu in view of Osugi and Dong discloses:
and a question mask for the question set, wherein the question mask is configured to screen the question set in the first input; … and the question mask ([Dong, section 2.3 and figure 1]: Dong discloses constructing a “self-attention mask” in order to mask/screen out portions of the input and thereby control the attention of the language model. In the combination, the self-attention masks are applied to the question set in order to screen portions of the question set.)
wherein determining the question mask comprises: building a first data item associated with the question set, the first data item comprising second elements arranged in rows and columns, wherein second elements in each row of the first data item correspond to characters in the question set, the first separation identifier, the second separation identifier, and the first text, and indicate positions of each character comprised in the question set, the first separation identifier, the second separation identifier, and the first text; ([Dong, section 2.3 and figure 1]: Dong discloses constructing a “self-attention mask” which is arranged in rows and columns, with the entries in each row corresponding to tokens in the input. The entries of this self-attention mask are the “second elements” recited by the claim, and each row of the self-attention mask is a “row of the first data item correspond[ing] to characters” as recited by the claim.)
for each question in the question set: determining a plurality of second elements in one group of second elements, the plurality of second elements corresponding to a plurality of characters to be masked in the question set, wherein the plurality of characters to be masked correspond to other question(s) in the question set; and determining the question mask by setting the plurality of second elements as a second value to make the question being processed by the reading comprehension model. ([Dong, section 2.3]: Dong discloses setting values in the “self-attention matrix” to -∞ in order to block the model’s attention towards those tokens. In other words, a row of the self-attention mask maps to the “one group of second elements” of the claim, -∞ is the “second value” of the claim, and the entries which are set to -∞ map to the “plurality of second elements” of the claim. The examiner notes that the “plurality of second elements” fall under the broadest reasonable interpretation of “corresponding to a plurality of characters to be masked in the question set” (namely, characters which occur in positions where the mask is set to -∞), and that the “plurality of characters” thus determined in turn falls under the broadest reasonable interpretation of “correspond[ing] to other question(s) in the question set” (in, for example, the sense that these characters occur in the same question set as the “other question(s)” of the question set).)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the question answering system of Lu in view of Osugi with the attention masking method disclosed by Dong because the method taught by Dong “alleviat[es] the need of separately training and hosting multiple [language model]s” [Dong, page 2], so the combination would result in a more efficient system.
Claim(s) 3, 11, and 19 is/are rejected under 35 USC 103 as being unpatentable over Lu in view of Osugi and Dong, further in view of Boris MIRKIN (Mathematical Classification and Clustering, “Section 4.1.1: Presentation of Subsets,” published 1996-08-31; hereafter, “Mirkin”).
Claim 3
Lu in view of Osugi and Dong discloses the elements of the parent claim(s). It does not explicitly disclose:
[The method of claim 1, wherein determining the question index comprises:] building a question index vector associated with the question set, wherein the question index vector comprises first elements which correspond to characters in the question set and the first separation identifier and indicate positions of each character comprised in the question set and the first separation identifier; and determining the question index by setting each first element corresponding to an instance of the first separation identifier as a first value.
However, Lu in view of Osugi, Dong, and Mirkin discloses:
[The method of claim 1, wherein determining the question index comprises:] building a question index vector associated with the question set, wherein the question index vector comprises first elements which correspond to characters in the question set and the first separation identifier and indicate positions of each character comprised in the question set and the first separation identifier; and determining the question index by setting each first element corresponding to an instance of the first separation identifier as a first value. ([Mirkin, page 170 section 4.1.1 first paragraph]: Mirkin discloses constructing a Boolean indicator vector corresponding to a given set and subset. The examiner notes that, in the combination, the given set and subset are, respectively, the question set (regarded as an ordered set of characters) and the delimiter between questions (regarded as a singleton set), so the resulting indicator vector has a bit corresponding to each character in the question set, the bit being set to 1 precisely when the corresponding character in the question set is the delimiter; in other words, the indicator vector thus constructed precisely indicates the position of the delimiters between questions. The entries of the indicator vector are the “first elements” recited by the claim, and the value 1 is the “first value” recited by the claim.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the question answering system of Lu in view of Osugi and Dong with indicator vectors as in Mirkin because an indicator vector with one entry for each element in an ordered set, with that entry being 1 if and only if the corresponding element of the ordered set is in a given subset, is one of the “major forms for presenting a subset” [Mirkin, page 170 section 4.1.1 first paragraph], and in the combination with Lu in view of Osugi and Dong, would record the positions of delimiters between questions in the question set, thereby allowing for quick identification of distinct questions within the question set.
Claims 11 and 19 inherit limitations from claim 9 and 17, respectively, and then further recite limitations that are substantially similar to those of claim 3, so they are rejected using the same rationale.
Claim(s) 5 and 13 is/are rejected under 35 USC 103 as being unpatentable over Lu in view of Osugi and Dong, further in view of Vivek TAWDE (US20070226207A1, published 2007-09-27; hereafter “Tawde”).
Claim 5
Lu in view of Osugi and Dong discloses the elements of the parent claim(s). It does not explicitly disclose:
[The method of claim 1, wherein determining the plurality of second elements comprises:] determining a similarity among the plurality of questions;
and determining the plurality of second elements based on the similarity.
Tawde is also in the field of natural language processing [Tawde, Abstract]. Moreover, Lu in view of Osugi, Dong, and Tawde discloses:
[The method of claim 1, wherein determining the plurality of second elements comprises:] determining a similarity among the plurality of questions; ([Tawde, 0006]: Tawde discloses computing “[a] similarity matrix representing the cosine similarity between texts”. The examiner notes that in the combination, the texts in consideration are the “plurality of questions” in the question set.)
and determining the plurality of second elements based on the similarity. ([Tawde, 0006]: Tawde discloses using the similarity matrix so that “[o]ne or more singleton tests that may not be similar to another text in the set may be excluded”. The examiner notes that the “plurality of second elements” recited by the claim are part of the question mask, as indicated in claim 4, and in the combination, the exclusion of texts disclosed by Tawde corresponds to masking questions within the question set.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the question answering system of Lu in view of Osugi and Dong with the similarity computation of Tawde because the method disclosed by Tawde “[a]dvantageously” and “flexibly use[s] metadata describing content to cluster items of aggregated content provided by multiple content feeds” [Tawde, 0008], so the combination would be a more effective and efficient system overall.
Claim 13 inherits limitations from claim 9 and then further recite limitations that are substantially similar to those of claim 5, so it is rejected using the same rationale.
Claim(s) 8 and 16 is/are rejected under 35 USC 103 as being unpatentable over Lu in view of Osugi and Dong, further in view of Danqi CHEN et al. (Reading Wikipedia to Answer Open-Domain Questions, published 2017-07, hereafter “Chen”).
Claim 8
Lu in view of Osugi and Dong discloses the elements of the parent claim(s). It does not explicitly disclose:
[The method of claim 7, wherein the first operation comprises:] performing element multiplication of the second data item and the first text feature representation;
and inputting a result of the element multiplication to the neural network to acquire the third data item.
Chen is also in the field of natural language processing, and it discloses a neural-network-based question-answering system [Chen, Abstract] which computes feature vector representations of the text and questions and which identifies answers to questions as spans of text between starting and ending indices [Chen, section 3.2]. Moreover, Lu in view of Osugi, Dong, and Chen discloses:
[The method of claim 7, wherein the first operation comprises:] performing element multiplication of the second data item and the first text feature representation; ([Chen, section 3.2 “Aligned question embedding”]: Chen discloses “dot products” between feature representations of the text E(p_i) and question E(q_j) as a part of computing the terms a_{i,j}. The computation of these dot products maps to the “element multiplication” recited by the claim. The examiner notes further that the “second data item” recited by the claim records feature representations of questions, as described in claim 7.)
and inputting a result of the element multiplication to the neural network to acquire the third data item. ([Chen, section 3.2]: Chen discloses passing the result of the dot product computation described above into a recurrent neural network to obtain feature representations of the text [section 3.2 “Paragraph encoding”] and then passing the result of that into “classifiers... for predicting the two ends of the span” [section 3.2 “Prediction”]. The examiner notes that the “third data item” recited by the claim records the starting and ending indices of the answer, as described in claim 7.)
Before the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to combine the question answering system of Lu in view of Osugi and Dong with techniques used in the question answering system of Chen because the question-answering system of Chen “outperforms the built-in Wikipedia search engine” and “reaches state-of-the art results on the very competitive SQuAD benchmark” [Chen, section 1], so the combination would result in a more effective system.
Claim 16 inherits limitations from claim 9 and then further recites limitations that are substantially similar to those of claim 8, so it is rejected using the same rationale.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Shishir AGRAWAL whose telephone number is +1 703-756-1183. The examiner can normally be reached Monday through Thursday, 08:30-14:30 Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey SHMATOV can be reached on +1 571-270-3428. The fax phone number for the organization where this application or proceeding is assigned is +1 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at +1 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call +1 800-786-9199 (IN USA OR CANADA) or +1 571-272-1000.
/S.A./Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights