DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see remarks, filed 04/07/2026, have been fully considered and are not persuasive:
Regarding the 101 rejection argument on pages 1-8:
Applicant argues regarding step 2A Prong one, “The amended claim recites feeding close-ended queries to "at least one machine learning-based conversational language model, trained to be used as an inference engine to evaluate text" and obtaining from that model "a plurality of inference values" generated by the model in response to said feeding. These steps cannot, practically or theoretically, be performed by the human mind. A machine learning-based language model operating as an inference engine - processing numerical embeddings, weight matrices, attention mechanisms, and probabilistic output distributions - is categorically beyond human mental capacity. The August 2025 Memo expressly acknowledges that where AI-implemented steps require computational processes that are not practically performable by a human mind, the mental process grouping does not apply. Accordingly, the amended claim, on its face, does not recite a mental process, and the rejection at Prong One should not be sustained.”
Examiner disagrees. The claim recites a mental process of “processing the textual content to generate a plurality of close ended queries; processing the plurality of inference values to generate at least one evaluation of the textual content; and using the at least one evaluation to generate at least one second prompt”, which is a mental processes- for example observation and evaluation of drawing conclusion from responses such as reading student work and infer student understanding or make questions as a fallow up, which can be done mentally or using a pen and paper.
The limitations of “"at least one machine learning-based conversational language model, trained to be used as an inference engine to evaluate text" and obtaining from that model "a plurality of inference values" generated by the model in response to said feeding”, are merely using a computer as a tool to perform the evaluation, Merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
Applicant argues regarding step 2A Prong two and step 2B, “The amended claim satisfies this standard. The recited language model is not a generic AI tool; it is "trained to be used as an inference engine to evaluate text." This training specificity defines a technically particular computational artifact - a model whose internal parameters have been optimized through a machine learning process for the specific task of text inference evaluation. This is not a generic computer or a generic AI; it is a specifically configured technical system performing a specific technical function, producing specific technical outputs (inference values) that are then further processed. The August 2025 Memo's guidance thus supports the conclusion that the amended claim integrates any abstract idea into a practical application.”
Examiner disagrees, the claim states, “feeding the plurality of close ended queries to at least one machine learning-based conversational language model, trained to be used as an inference engine to evaluate text”, therefore, there is no specificity on training, rather than just using an AI that has already been trained. Examiner suggest to add how the training is implanted in a specific way to show improvement and tie the improvement to the specific technology.
For at least the above reasons, 101 abstract idea rejection is maintained. Please see below updated rejection based on amendments.
Regarding applicant's arguments filed with respect to the prior art rejections have been fully considered but they are moot. Applicant has amended the claims to recite new combinations of limitations. Please see below for new grounds of rejection, necessitated by Amendment.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-23 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims’ subject matter eligibility will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019) (“2019 PEG”).
With respect to claim 1.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Is the claim to a process, machine, manufacture, or composition of matter? Yes—claim 1 recites a method, which is a process.
Step 2A, prong one: Does the claim recite an abstract idea, law of nature or natural phenomenon? Yes—the limitations identified below each, under its broadest reasonable interpretation, covers mental processes abstract idea grouping (concepts performed in the human mind (including an observation, evaluation, judgment, opinion)), see MPEP 2106.04(a)(2), subsection III and the 2019 PEG, but for the recitation of generic computer components:
“processing the textual content to generate a plurality of close ended queries; processing the plurality of inference values to generate at least one evaluation of the textual content; and using the at least one evaluation to generate at least one second prompt”: (Mental processes- concept of observation and evaluation of drawing conclusion from responses such as reading student work and infer student understanding or make questions as a fallow up).
Step 2A, prong two: Does the claim recite additional elements that integrate the judicial exception into a practical application? No—the judicial exception is not integrated into a practical application.
“receiving a textual content generated by group of users in response to an assignment;” , “obtaining from said conversational language model a plurality of inference values pertaining to the users”: involves the mere gathering of data, which is insignificant extra-solution activity. See MPEP § 2106.05(g).
“feeding the plurality of close ended queries to at least one machine learning-based conversational language model, trained to be used as an inference engine to evaluate text” and “said plurality of inference values are generated by said conversational language model in response to said feeding”: Merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f);
The generic computer components in these steps are recited at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No—there are no additional limitations beyond the mental processes identified above. The limitation treated above, are directed to the well-understood, routine, and conventional activity of storing and retrieving information in memory. See MPEP § 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). It also includes limitations that Merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). The additional element is insignificant application, which is similar to examples of activities that the courts have found to be insignificant extra-solution activity, in accordance with MPEP 2106.05(g), Insignificant Extra-Solution Activity. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 2.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “using the at least one second prompt for querying the group of users for an additional textual content”: This limitation merely specifies mental processes- concept of observation and evaluation of labeling datasets.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 3.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “wherein the at least one evaluation is based on relevance of the additional textual content to the at least one second prompt”: This limitation merely specifies mental processes- concept of observation and evaluation of labeling datasets and filtering the labeling between positive and negative sampling based on relevancy.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 4.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “wherein the at least one second prompt comprising a suggestion derived from the at least one evaluation”: This limitation merely specifies mental processes- concept of observation and evaluation of labeling datasets.
Step 2A Prong 2, Step 2B: The claim recites that “obtaining a set of the potential queries;” involves the mere gathering of data, which is insignificant extra-solution activity. See MPEP § 2106.05(g). This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 5.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “wherein the at least one evaluation is based on checking when a specified suggestion is present in the textual content”: This limitation merely specifies mental processes- concept of observation and evaluation of labeling datasets.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 6.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “wherein the at least one evaluation is counting how many users in the group of users included the specified suggestion in the textual content pertaining thereto”: This limitation merely specifies mental processes- concept of observation and evaluation of labeling datasets.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 7.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “wherein the suggestion is associated with an item pertaining to the assignment”: This limitation merely specifies mental processes- concept of observation and evaluation of labeling datasets.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 8.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “wherein the at least one evaluation is based on evaluating a coherence measure of the textual content”: This limitation merely specifies mental processes- concept of observation and evaluation of labeling datasets.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 9.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “wherein the evaluation comprising detection of suggestions not expected in response to the querying”: This limitation merely specifies mental processes- concept of observation and evaluation of labeling datasets.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 10.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “using the evaluation to partition a group of students according to a similarity measure”: This limitation merely specifies mental processes- concept of observation and evaluation of labeling datasets.
Step 2A Prong 2, Step 2B: This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claim 11.
Step 1: A method, as above.
Step 2A Prong 1: The claim recites that “generating a plurality of queries each from a combination of at least one part of the textual content pertaining to users from the group and one of the plurality of close ended questions”: This limitation merely specifies mental processes- concept of observation and evaluation of labeling datasets.
Step 2A Prong 2, Step 2B: The claim recites that “wherein processing the textual content comprising: accessing a storage to obtain a plurality of close ended questions.” involves the mere gathering of data, which is insignificant extra-solution activity. See MPEP § 2106.05(g). This judicial exception is not integrated into a practical application. Mere recitation of generic computer components neither integrates the judicial exception into a practical application nor provides an inventive concept.
Claims 12-22
Step 1: The claims recite a system; therefore, they fall into the statutory category of machines.
Step 2A Prong 1: The claims 12-22 recite the same mental processes as claims 1-11, respectively.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Claims 12-22 recite generic computer components, namely “system comprising a storage and at least one processing circuitry”. As before, the mere recitation that the method is to be performed on a generic computer amounts to a mere instruction to apply the exception on the computer. See MPEP § 2106.05(f). With that exception, the analysis mirrors that of claims 1-11, respectively.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The analysis, with the one exception noted above, mirrors that of claims 1-11, respectively.
Claim 23
Step 1: The claims recite a system; therefore, they fall into the statutory category of machines.
Step 2A Prong 1: The claim 23 recite the same mental processes as claim 1.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Claim 23 recite generic computer components, namely “one or more computer program products comprising… one or more processors of a computing system is to cause a computing system”. As before, the mere recitation that the method is to be performed on a generic computer amounts to a mere instruction to apply the exception on the computer. See MPEP § 2106.05(f). With that exception, the analysis mirrors that of claim 1.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The analysis, with the one exception noted above, mirrors that of claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4-7, 11-13, 15-18 and 22-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wong et al. (US 20180061264 A1) in view of Mark et al. (US 20230027078 A1).
Regarding claim 1.
Wong teaches a method for evaluating understanding in a textual content, comprising: receiving a textual content generated by group of users in response to an assignment (see ¶ 19, “The initial users provide answers to the open ended question as an initial set of textual responses at 32 in FIG. 2. After collecting some predetermined number of textual responses, the system automatically uses these textual responses to populate a set of possible textual responses and stores them in the storage 16 of FIG. 1.”, also see ¶ 14, “A self-learning survey contains a question that begins as an open-ended question and is sent out to participants. After an initial set of textual responses is collected”);
processing the textual content to generate a plurality of close ended queries (see ¶ 1, “automatically evolves open-ended questions into closed-ended questions ranked by numerical ratings.”, also see ¶ 15, “The survey should evolve into including closed-ended questions replacing the original open-ended question while some percentage of available respondents still have not answered the survey.”, also see ¶ 29, “the system must receive a first set of surveys used to gather potential textual responses, send out the potential textual responses for evaluation, receive the evaluations and then adjust the survey to change the original open-ended question into a closed-ended question…in order to provide some group of the potential respondents the ‘final’ version of the survey with the closed-ended question, this process has to occur in real-time, meaning while access to the survey is live and before the predetermined total number of responses has been received.”, also see ¶ 31, “The ‘top textual responses’ are those that have a high numerical rating, such as 3 or 4 in the previous example. These are then used to generate a new version of the survey with the open-ended question converted into close-ended questions as shown at 36 in FIG. 2.”);
processing see ¶ 20, “the system then uses a set of these numerical responses to generate an average or mean numerical rating for each textual response and a confidence value in the estimate of the average numerical rating of a textual response.”, also see ¶ 26, “the self-learning survey system can assign each textual response a rating adjusted by confidence.”, also ¶ 27, “if the answer evaluations of the ‘leisure time’ textual response of FIG. 5 is 1 “not insightful” with a confidence measure of 95%, that textual response can be ignored, as the system has a high confidence that the textual response is not insightful. FIG. 8 shows an example of an insightfulness rating distribution.”, also see ¶ 28, “The combination of the confidence measure and pruning of textual responses allows the self-learning survey to efficiently gain information per survey participant and to avoid spending time and attention on textual responses that are not promising.”, also see claim 1, “generate a confidence measure for each possible textual response based upon the participant numerical ratings; generating a ranked list using the confidence measure, with the processor”, also see ¶ 30, “The self-learning survey inherently contains a ranked list of the highest rated textual responses based on the confidence-adjusted score.”);
and using the at least one evaluation to generate at least one second prompt (see ¶ 31, “The ‘top textual responses’ are those that have a high numerical rating, such as 3 or 4 in the previous example. These are then used to generate a new version of the survey with the open-ended question converted into close-ended questions as shown at 36 in FIG. 2. These new versions of the survey are then sent out to the remaining users. The converted questions and numerical ratings are then stored in the storage 16 of FIG. 1.”, also see ¶ 29, “the system must receive a first set of surveys used to gather potential textual responses, send out the potential textual responses for evaluation, receive the evaluations and then adjust the survey to change the original open-ended question into a closed-ended question. During this period of time, the system may alternate between gathering new textual responses and adding those to the analysis.”).
Wong do not specifically teach feeding the plurality of close ended queries to at least one machine learning-based conversational language model, trained to be used as an inference engine to evaluate text; obtaining from said conversational language model a plurality of inference values pertaining to the users, said plurality of inference values are generated by said conversational language model in response to said feeding and processing the plurality of inference values to generate at least one evaluation of the textual content.
Marks teach feeding the plurality of close ended queries to at least one machine learning-based conversational language model, trained to be used as an inference engine to evaluate text (see ¶ 46, “the machine learning virtual agent model 140 that may be accessed, generated, and/or outputted by the model accessibility/development engine 130 may be capable of predicting and/or inferring responses based on user input.”, also see ¶ 47, “(BERT) for masked language model tasks and next sentence prediction tasks and the like, variations of BERT (i.e., ULMFiT, XLM UDify, MT-DNN, SpanBERT, RoBERTa, XLNet, ERNIE, KnowBERT, VideoBERT, ERNIE BERT-wwm, MobileBERT, TinyBERT, GPT, GPT-2, GPT-3, GPT-4 (and all subsequent iterations),”, also see ¶ 96, “S320, which includes generating a response inference, may function to provide the user communication or user stimulus as input to one or more pre-trained language models. In some embodiments, a chatbot generation service or the like may function as an intermediary between a client interface implementing a virtual dialogue agent (e.g., a subscriber system) and one or more remote or cloud-based systems implementing the one or more pre-trained language models.”, also see ¶ 97, “S320 may function to access one or more of the plurality of pre-trained language processing models by requesting or generating one or more API calls that include user stimulus data to APIs of one or more of the pre-trained language processing models for producing one or more inferred responses to the user stimulus.”);
obtaining from said conversational language model a plurality of inference values pertaining to the users, said plurality of inference values are generated by said conversational language model in response to said feeding (see ¶ 96, “S320, which includes generating a response inference, may function to provide the user communication or user stimulus as input to one or more pre-trained language models”, also see ¶ 98, “each of the distinct language processing models that may process the user stimulus or the user utterance differently and may use distinct embedding models to generate a distinct embedded query representation and may compute a predicted (e.g., embeddings) response inference that may vary from other language processing models. For instance, S320 may function to process the user stimulus or user utterance through a plurality of language processing models and each of the language processing models of the plurality of language processing models may be associated with a distinct embeddings models that generates a distinct user stimulus representation.”, also see ¶ 100, “S330 may function to evaluate each embedded response representation or a subset of the embedded response representations of the video response corpus with reference to each response inference computed by the plurality of pre-trained language processing models based on the user stimulus.”, also see ¶ 105, “the selected or optimal response (vector, item) to the user stimulus may be the embedded response representation that occurs with the highest frequency based on the results of the similarity analysis performed for multiple, distinct inferred responses of the plurality of distinct pre-trained language models. For instance, if three (3) distinct response inferences map to a first embedding value of a first response”, also see ¶ 106, “S330 may function to source three (3) response inferences or response predictions from three distinct pre-trained machine learning models based on a given user stimulus.”) and processing the plurality of inference values to generate at least one evaluation of the textual content (see ¶ 101, “S330 may function to compute a similarity metric between an inferred response and one or more embedded response representations of the video response corpus. In such embodiments, the computation of the similarity metric may include mathematically computing a distance value or a distance similarity metric between each embedded response representation of the video response corpus and each inferred response representation produced based on the user stimulus to thereby intelligently identify an optimal or most probable response based on the quantitative analysis of the mathematically computed distance”, also see ¶ 103, “S330 may function to mathematically compute, for each inference response to the user stimulus produced by the one or more the pre-trained language processing models, which embedded response representation of the video response corpus provides the most likely or most probable response”).
Both Wong and Marks pertain to the problem of textual responses analysis, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Wong and Marks to teach the above limitations. The motivation for doing so would be “Typically, a user interacting with a virtual assistant may pose a question, a message, or otherwise submit an input to the virtual assistant, to which, the virtual assistant may provide a response or perform some action as a result of the user input. Many of these virtual assistants may typically be portrayed as a user interface object, such as a chat window or may include an animated computer object that lack real human features and real human mannerisms. As a result, a significant class of users fail to fully engage with the virtual assistant and may continue to prefer interactions that involve a real human agent…a virtual assistant or automated interaction system personified with real human response may create a more personable and engaging conversation which may increase the likelihood of a user's satisfaction with its response and interactions and therefore, reducing interaction loads on often limited real human agents.” (see Mark ¶¶ 3-5).
Regarding claim 2.
Wong and Marks teaches the method of claim 1,
Wong further teaches further comprising using the at least one second prompt for querying the group of users for an additional textual content (see ¶ 31, “These are then used to generate a new version of the survey with the open-ended question converted into close-ended questions as shown at 36 in FIG. 2. These new versions of the survey are then sent out to the remaining users. The converted questions and numerical ratings are then stored in the storage 16 of FIG. 1.”).
Regarding claim 4.
Wong and Marks teaches the method of claim 2,
Wong further teaches wherein the at least one second prompt comprising a suggestion derived from the at least one evaluation (see ¶ 20, “FIG. 5, four possible ratings for the textual response are provided, with a fifth choice of 0 for ‘not relevant.’ The user selects one of five options as to the insightfulness of each of the suggested textual responses. The user's numerical ratings are received as values 0-4, with 0 being “not relevant”, 1 being ‘not insightful,’ and 4 being ‘very insightful.’”, also see ¶ 30, “if the administrator is satisfied with the insights gathered, he or she can eliminate the open-ended question to allow subsequent versions of the survey presented to focus on obtaining numerical ratings of the top textual responses.”).
Regarding claim 5.
Wong and Marks teaches the method of claim 1,
Wong further teaches wherein the at least one evaluation is based on checking when a specified suggestion is present in the textual content (see ¶ 20, “FIG. 5, four possible ratings for the textual response are provided, with a fifth choice of 0 for ‘not relevant.’ The user selects one of five options as to the insightfulness of each of the suggested textual responses. The user's numerical ratings are received as values 0-4, with 0 being “not relevant”, 1 being ‘not insightful,’ and 4 being ‘very insightful.’”, also see ¶ 30, “if the administrator is satisfied with the insights gathered, he or she can eliminate the open-ended question to allow subsequent versions of the survey presented to focus on obtaining numerical ratings of the top textual responses.”).
Regarding claim 6.
Wong and Marks teaches the method of claim 5,
Wong further teaches wherein the at least one evaluation is counting how many users in the group of users included the specified suggestion in the textual content pertaining thereto (see ¶ 16, “A closed-ended question is one in which the user selects from a fixed set of choices. An evaluation or rating, also referred to as a numerical rating, is a measure of the users' opinion as to the value or insight of an item. The item may be a title or description of a book, movie, or a textual response as defined above.”, also see ¶ 20, “FIG. 5, four possible ratings for the textual response are provided, with a fifth choice of 0 for ‘not relevant.’ The user selects one of five options as to the insightfulness of each of the suggested textual responses. The user's numerical ratings are received as values 0-4, with 0 being “not relevant”, 1 being ‘not insightful,’ and 4 being ‘very insightful.’”, also see ¶ 25, “textual response R2 is given a numerical rating score of 1 by two participants and 4 by two other participants, its SE will be greater than 0, specifically around 0.866 or low confidence, giving it a better chance to be selected for future versions of the survey by the sampling algorithm. Intuitively, confidence based sampling allows the self-learning survey to learn as much as it can about the true value of each textual response per participant evaluation. FIG. 6 shows the possible textual responses and their average numerical ratings.”).
Regarding claim 7.
Wong and Marks teaches the method of claim 5,
Wong further teaches wherein the suggestion is associated with an item pertaining to the assignment (see ¶ 31, “The ‘top textual responses’ are those that have a high numerical rating, such as 3 or 4 in the previous example. These are then used to generate a new version of the survey with the open-ended question converted into close-ended questions as shown at 36 in FIG. 2. These new versions of the survey are then sent out to the remaining users. The converted questions and numerical ratings are then stored in the storage 16 of FIG. 1.”).
Regarding claim 11.
Wong and Marks teaches the method of claim 1,
Wong further teaches wherein processing the textual content comprising: accessing a storage to obtain a plurality of close ended questions; and generating a plurality of queries each from a combination of at least one part of the textual content pertaining to users from the group and one of the plurality of close ended questions (see ¶ 19, “The initial users provide answers to the open ended question as an initial set of textual responses at 32 in FIG. 2. After collecting some predetermined number of textual responses, the system automatically uses these textual responses to populate a set of possible textual responses and stores them in the storage 16 of FIG. 1. In the next survey or set of surveys for subsequent participants who access the survey, a subset of the available textual responses is added to the surveys at 34.”).
Regarding Claim 12,
Claim 12 recites a system comprising a storage and at least one processing circuitry configured to perform the method recited in claim 1. Therefore the rejection of claim 1 above applies equally here. Wong also teaches the addition elements of claim 12 not recited in claim 1 comprising storage and at least one processing circuitry (see ¶ 17, “a computer such as 13 having a processor 26. The computer 13 has a connection, either through the network 18 or directly, to a database or other storage 16.”).
Regarding claims 13, 15-18 and 22,
claims 13, 15-18 and 22 are dependent to claim 12 and recites limitations that are similar to the limitations recited in claims 2, 4-7 and 11. Therefore, claims 13, 15-18 and 22 are rejected with the same rationale applied against claims 2, 4-7 and 11 above.
Regarding Claim 23,
Claim 23 recites a One or more computer program products comprising instructions to perform the method recited in claim 1. Therefore the rejection of claim 1 above applies equally here. Wong also teaches the addition elements of claim 23 not recited in claim 1 comprising one or more processors of a computing system (see ¶ 17, “a computer such as 13 having a processor 26. The computer 13 has a connection, either through the network 18 or directly, to a database or other storage 16.”).
Claim(s) 3, 8-10, 14 and 19-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wong et al. (US 20180061264 A1) in view of Mark et al. (US 20230027078 A1) in view of Attali et al. (US 20230080674 A1).
Regarding claim 3.
Wong and Marks teaches the method of claim 2,
Wong and Marks do not teach the limitation of claim 3.
Attali teaches wherein the at least one evaluation is based on relevance of the additional textual content to the at least one second prompt (see ¶ 165, “There are multiple methods of computing textual similarity, such as by generating word embeddings (vectors) and comparing two vectors using a defined metric (Euclidean distance or cosine distance, as non-limiting examples). The threshold value used to determine a situation of too great or too little similarity can be tuned based on internally evaluated metrics derived from generated answers and items, as well as by the performance of test takers on generated items and answers. Other approaches to determining textual similarity include evaluating semantic similarity, contextual evaluation, or another suitable approach.”, also see ¶¶ 39, “Filter the generated test items (the possible correct and/or incorrect responses) to remove categories of subject matter that are not appropriate or do not satisfy one or more desired metrics; [0040] For possible correct responses, non-limiting examples of such metrics include: [0041] similarity to the source text as estimated by vector similarities of the text encoded by a separate language model; [0042] similarity to individual sentences within the source text via the above mechanism; [0043] N-gram overlap with the source text; [0044] probability of the generated answer under the transformer-based language model; [0045] probability of being correct as estimated by a separately trained question answering model; [0046] length; or [0047] presence of overly rare words; [0048] For possible incorrect responses, non-limiting examples of such metrics include: [0049] similarity to the source text; [0050] similarity to the chosen correct answer; [0051] similarity to unchosen potential correct answers; [0052] similarity to other incorrect answers”).
Wong, Marks and Attali pertain to the problem of textual responses analysis, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Wong, Marks and Attali to teach the above limitations. The motivation for doing so would be “using a Transformer-based Language Model (for example, Generative Pretrained Transformer 3 or GPT-3) to automatically generate a set of content-controlled passages, and test items for each passage that can be used to evaluate a test taker's comprehension of the text. These test questions or items may be used to probe a test taker's comprehension of the text and their ability to synthesize and distill the information beyond simple vocabulary or standard questions (what, where, who, or why, as examples) that are generated using templates or conventional automated methods. Further, the test items are generated in a way that enables them to be automatically scored, thereby reducing, or eliminating the need for human resources as part of the test generation and scoring processes.” (see Attali ¶ 10).
Regarding claim 8.
Wong and Marks teaches the method of claim 1,
Wong and Marks do not teach the limitation of claim 8.
Attali teaches wherein the at least one evaluation is based on evaluating a coherence measure of the textual content (see ¶ 181, “a trained model that tries to evaluate the quality of a summary of a piece of text, and which could be used in the ranking equation. This could be used to help rank potential correct candidates based on their estimated “quality”.”).
Wong, Marks and Attali pertain to the problem of textual responses analysis, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Wong, Marks and Attali to teach the above limitations. The motivation for doing so would be “using a Transformer-based Language Model (for example, Generative Pretrained Transformer 3 or GPT-3) to automatically generate a set of content-controlled passages, and test items for each passage that can be used to evaluate a test taker's comprehension of the text. These test questions or items may be used to probe a test taker's comprehension of the text and their ability to synthesize and distill the information beyond simple vocabulary or standard questions (what, where, who, or why, as examples) that are generated using templates or conventional automated methods. Further, the test items are generated in a way that enables them to be automatically scored, thereby reducing, or eliminating the need for human resources as part of the test generation and scoring processes.” (see Attali ¶ 10).
Regarding claim 9.
Wong and Marks teaches the method of claim 1,
Wong and Marks do not teach the limitation of claim 9.
Attali teaches wherein the evaluation comprising detection of suggestions not expected in response to the querying (see ¶ 36, “Using the set of Alternative Passages generated or corresponding to each selected Source Passage, generate a set of possible incorrect responses to each of the multiple-choice questions referred to in the previous step using a Transformer-based Language Model by providing: [0037] The same Instructions and Examples used to generate the possible correct response(s) for the selected Source Passage; [0038] The Alternative Passage for which answers will be generated as the Conditioning; [0039] Filter the generated test items (the possible correct and/or incorrect responses) to remove categories of subject matter that are not appropriate or do not satisfy one or more desired metrics;”).
Wong, Marks and Attali pertain to the problem of textual responses analysis, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Wong, Marks and Attali to teach the above limitations. The motivation for doing so would be “using a Transformer-based Language Model (for example, Generative Pretrained Transformer 3 or GPT-3) to automatically generate a set of content-controlled passages, and test items for each passage that can be used to evaluate a test taker's comprehension of the text. These test questions or items may be used to probe a test taker's comprehension of the text and their ability to synthesize and distill the information beyond simple vocabulary or standard questions (what, where, who, or why, as examples) that are generated using templates or conventional automated methods. Further, the test items are generated in a way that enables them to be automatically scored, thereby reducing, or eliminating the need for human resources as part of the test generation and scoring processes.” (see Attali ¶ 10).
Regarding claim 10.
Wong and Marks teaches the method of claim 1,
Wong and Marks do not teach the limitation of claim 10.
Attali teaches further comprising using the evaluation to partition a group of students according to a similarity measure (see ¶ 29, “the self-learning survey can be configured to gather a total of 2000 responses, begin sampling textual responses for evaluation when the first 20 textual responses are received, and alternate between soliciting new open-ended textual responses every third participant. The link provided to the potential respondents stays ‘live’ until some predetermined total number of responses is reached. Within that time, the system must receive a first set of surveys used to gather potential textual responses, send out the potential textual responses for evaluation, receive the evaluations and then adjust the survey to change the original open-ended question into a closed-ended question.”).
Wong, Marks and Attali pertain to the problem of textual responses analysis, thus being analogous. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Wong, Marks and Attali to teach the above limitations. The motivation for doing so would be “using a Transformer-based Language Model (for example, Generative Pretrained Transformer 3 or GPT-3) to automatically generate a set of content-controlled passages, and test items for each passage that can be used to evaluate a test taker's comprehension of the text. These test questions or items may be used to probe a test taker's comprehension of the text and their ability to synthesize and distill the information beyond simple vocabulary or standard questions (what, where, who, or why, as examples) that are generated using templates or conventional automated methods. Further, the test items are generated in a way that enables them to be automatically scored, thereby reducing, or eliminating the need for human resources as part of the test generation and scoring processes.” (see Attali ¶ 10).
Regarding claims 14 and 19-21,
claims 14 and 19-21 are dependent to claim 12 and recites limitations that are similar to the limitations recited in claims 3 and 8-10. Therefore, claims 14 and 19-21 are rejected with the same rationale applied against claims 3 and 8-10 above.
Related prior arts:
Somasundaran et al. (US 9959776 B1) teaches he constructed response is processed to determine a third numerical measure indicative of a degree of awkward word usage in the constructed response. A model is applied to the first, second, and third numerical measures to determine a score for the constructed response indicative of the user's English language proficiency. The model includes first, second, and third variables with associated first, second, and third weighting factors, respectively.
Al-Rikabi et al. (US 12223080 B1) teaches a language model (LM) may be used to score and rank potential synthetic NLQs. A language model is a statistical model that assigns probabilities to words and sentences. Typically, the NLQ query service 110 may be trying to guess the next word w in a sentence given all previous words, often referred to as the “history.” The losses used in training a language model may be used to evaluate if some text make sense or not.
Reza et al. (US 20230237277 A1) teaches The relevant features 110 are then used for generating dynamic prompt templates 115. In some instances, a prompt template 115 is generated for each of the extracted relevant features 110. In other instances, a prompt template 115 is generated for each relevant feature (a subset of features) selected from the extracted relevant features 110.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IMAD M KASSIM whose telephone number is (571)272-2958. The examiner can normally be reached 10:30AM-5:30PM, M-F (E.S.T.).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J. Huntley can be reached at (303) 297 - 4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IMAD KASSIM/Primary Examiner, Art Unit 2129
Prosecution Insights