DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendments filed 02/24/2026 have been entered.
Claims 1, 4, and 8-10 remain pending within the application.
The amendments, in combination with arguments found persuasive, filed 02/24/2026 are sufficient to overcome the 101 rejections previously set forth in the Non-Final Office Action mailed 09/24/2025. The rejections have been withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, and 8-10 are rejected under 35 U.S.C. 103 as being unpatentable over Yeung et al. ("Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory"), hereafter Yeung, in view of Ding et al. ("Deep Neural Network Based Student Response Modeling With Uncertainty, Multimodality and Attention"), hereafter Ding, in further view of Tamano et al. (US 2023/0138245 A1), prior art made record of and not relied upon in the office action mailed 09/24/2025, hereafter Tamano.
Regarding claim 1, Yeung discloses:
A method of training a neural network model for calculating an … indicating accuracy of an expected score of a user on the basis of answering data of the user, the method comprising (Yeung, Fig. 2, and page 1, right column, paragraph 2, last 3 lines “predict the probability that a student will answer a question qt+1 correctly given the sequence Xt, i.e., P(at+1 = 1jqt+1;Xt)” teaches training the neural network model of Fig. 2 to calculate a probability indicating accuracy of answered questions of a student target user on the basis of answering data of the student),
obtaining a reference answering data set of a plurality of reference users, the reference answering data set including problem data solved by a reference user of the plurality of reference users and response data of the reference user to the problem data (Yeung, Fig. 2, and page 1, right column, paragraph 2, lines 6-10 “xt is usually represented as an ordered pair (qt; at) which constitutes a tag for the question qt being answered at time t and an answer label at indicating whether the question has been answered correctly.” Teaches xt qt as problem data solved by a reference user and xt at as response data of the reference user to the problem data),
obtaining expected score information of the reference user from the reference answering data set (Yeung, Fig. 2 and page 4, right column, paragraph 3, lines 1-5 “the DKVMN model can predict the probability that a student will answer qt correctly by the following process.” and page 4, left column, final paragraph, lines 1-3 “at time t, it first receives a KC qt, then predicts the probability of answering qt correctly…” teaches obtaining expected score information of the reference student from the reference dataset provided as input to Fig. 2),
obtaining actual score information of the reference user (Yeung, Fig. 2 and page 4, left column, final paragraph, lines 1-4 “at time t, it first receives a KC qt, then predicts the probability of answering qt correctly, and eventually updates the memory using the question-and-answer interaction (qt; at).” teaches obtaining a KC at, the answer label as the actual score information),
obtaining a training set on the basis of the reference answering data set, the expected score information, and the actual score information, the training set including label information that is defined as a difference between the expected score information and the actual score information (Yeung, Fig. 1, Fig. 2 and page 6, right column, paragraphs 3-4
PNG
media_image1.png
370
476
media_image1.png
Greyscale
teaches obtaining the training dataset for subsequent states of the NN based on initial training data and the output expected score information from a previous state, including cross entropy loss as label information that is defined as the difference between expected and actual score),
training a first neural network model for calculating an … accuracy of the expected score information of the reference user from the reference answering data set by using the training set including the label information (Yeung, Fig. 1, Fig. 2 and page 7, left column, paragraph 3, lines 1-4 “To report the model performance, we run the training and evaluation process for 5 times. The average and standard deviation of area under the ROC curve (AUC), the accuracy and the cross-entropy loss are reported.” Teaches training the model for calculating the accuracy of expected score information of the reference user from the reference answering data set using the training set including the label information),
obtaining target answering data of a target user, the target answering data including problem data previously solved by the target user and response data of the target user to the problem data (Yeung, Figure 2 and page 6, eight column, final paragraph, lines 1-2 “Of all the datasets, 30% of the sequences are held out as a test set and the remaining 70% are used as a training set... we use the combination that results in the smallest cross-entropy loss to retrain the model with the entire training set and evaluate the model performance on the test set.” Teaches obtaining a target answering data of a target user during model testing, where the target answering data includes problem data previously solved by the target user and response data of the target test user to the problem data),
obtaining an expected score of the target user calculated on the basis of the target answering data (Yeung, Figure 2, page 6, eight column, final paragraph, lines 1-2 “Of all the datasets, 30% of the sequences are held out as a test set and the remaining 70% are used as a training set... we use the combination that results in the smallest cross-entropy loss to retrain the model with the entire training set and evaluate the model performance on the test set.”, and Figure 4 teaches obtaining an expected score of the target user calculated on the basis of the target answering data during testing),
obtaining an … accuracy of the expected score of the target user using the first neural network model (Yeung, Figure 2 and Table 2 teaches obtaining an accuracy of the expected score of the target user using the first neural network model),
transmitting the … accuracy of the expected score of the target user … (Yeung, Figure 4 discloses transmitting the accuracy of the expected score of the target user to a visualized plot),
wherein the first neural network model includes an input layer for receiving the reference answering data set, an output layer for outputting a value related to … indicating the accuracy of the expected score information of the reference user, and a hidden layer having a plurality of nodes connecting the input layer to the output layer (Yeung, Figure 1 and figure 2 teaches the neural network model to includes an input layer for receiving the reference answering data set, an output layer for outputting an output value, and a hidden layer having a plurality of nodes connecting the input layer to the output layer).
wherein the training of the first neural network model comprises: inputting the reference answering data set to the input layer (Yeung, Figure 2 teaches inputting the reference answering data set to the input layer),
outputting the value … through the output layer (Yeung, Figure 2 teaches obtaining the output value through the output layer),
adjusting a weight of at least one node among the plurality of nodes based on (i) the value … and (ii) the label information defined as the difference between the expected score information of the reference user and the actual score information of the reference user (Yeung, Figure 1, Figure 2, and equation 15 teaches adjusting the weights of the nodes based on the output value and the label information through model update using equation 15).
wherein the label information is used to adjust the weight of the at least one node among the plurality of nodes of the first neural network model in the training of the first neural network model (Yeung, Fig. 1 and Fig. 2, page 4, equations (6)-(7) and paragraph below equation (7), lines 3-4 “These are parameterized by a weight matrix W and a bias vector b with appropriate dimensions.” and page 6, right column, paragraph 3, lines 3-7 “other model parameters W and b, … All of these model parameters are learned during the training process by minimizing the cross-entropy loss” teaches adjusting the weight of the nodes of the NN using label information in the training of the first neural network model).
Yeung teaches a method of training a neural network model for calculating an … indicating accuracy of an expected score of a user on the basis of answering data of the user, but does not disclose calculating an uncertainty index indicating accuracy.
Ding discloses:
a method of training a neural network model for calculating an uncertainty index indicating accuracy …(Ding, Fig. 4.2 and page 63, paragraph below figure 4.2 “The input to this model is one hot encoding of the correctness and the corresponding skill id. For each output vector p, containing the probabilities of getting different skills correct, there is a corresponding uncertainty vector u, which gives the uncertainty level for each predicted skill.” teaches a method of training a neural network model for calculating an uncertainty indicating accuracy),
Yeung and Ding are analogous art because they are from the same field of endeavor, response theory and machine learning models.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yeung to include a method of training a neural network model for calculating an uncertainty index indicating accuracy, based on the teachings of Ding. One of ordinary skill in the art would have been motivated to make this modification in order to provide much better explanation about the output while maintaining good performance, as suggested by Ding (page 62, paragraph 4, last 2 lines).
While Yeung teaches training a first neural network model for calculating … accuracy of the expected score information of the reference user from the reference answering data set by using the training set including the label information, they do not disclose calculating an uncertainty index indicating accuracy.
Ding discloses:
training a first neural network model for calculating an uncertainty index indicating accuracy of the expected score information … (Ding, Fig. 4.2, and page 68, paragraph 2, lines 4-5 “we have two output vectors decoded from the hidden state: the prediction vector and uncertainty vector. the model is trained end to end using the Adam optimizer.” Teaches training a neural network model for calculating an uncertainty index related to accuracy of expected score information).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yeung to include training a first neural network model for calculating an uncertainty index indicating accuracy of the expected score information, based on the teachings of Ding. One of ordinary skill in the art would have been motivated to make this modification in order to provide much better explanation about the output while maintaining good performance, as suggested by Ding (page 62, paragraph 4, last 2 lines).
While Yeung discloses obtaining an … accuracy of the expected score of the target user using the first neural network model, and transmitting … the accuracy of the expected score of the target user, they do not disclose uncertainty index related to accuracy and transmitting … to a user terminal of the target user.
Ding discloses:
obtaining an uncertainty index related to accuracy of the expected score of the target user …(Ding, Fig. 4.2, and page 68, paragraph 2, lines 4-5 “we have two output vectors decoded from the hidden state: the prediction vector and uncertainty vector. the model is trained end to end using the Adam optimizer.” And page 85, final paragraph, lines 3-4 “All models are cross validated using leave-one-participant-out cross-validation, where no user-specific data is used for training when that user is in the test fold.” Teaches obtaining an uncertainty index related to accuracy of the expected score of the target user during testing),
transmitting the uncertainty index related to the accuracy…(Ding, Figure 4.5 teaches the uncertainty index related to the accuracy to a visualized plot),
transmitting … to a user terminal of the target user (Ding, Figure 5.2 teaches transmitting self-assessments to a user terminal of the target user).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yeung to include obtaining an uncertainty index related to accuracy of the expected score of the target user, transmitting the uncertainty index related to the accuracy, and transmitting … to a user terminal of the target user, based on the teachings of Ding. One of ordinary skill in the art would have been motivated to make this modification in order to provide much better explanation about the output while maintaining good performance, as suggested by Ding (page 62, paragraph 4, last 2 lines).
While Yeung discloses wherein the first neural network model includes an input layer for receiving the reference answering data set, an output layer for outputting a value related to … indicating the accuracy of the expected score information of the reference user, and a hidden layer having a plurality of nodes connecting the input layer to the output layer they do not disclose the output value to be related to the uncertainty index.
Ding discloses:
…an output layer for outputting a value related to the uncertainty index indicating the accuracy …(Ding, Fig. 4.2 and paragraph below figure 4.2 discloses an output layer for outputting an output value related to the uncertainty index).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yeung to include an output layer for outputting a value related to the uncertainty index indicating the accuracy, based on the teachings of Ding. One of ordinary skill in the art would have been motivated to make this modification in order to provide much better explanation about the output while maintaining good performance, as suggested by Ding (page 62, paragraph 4, last 2 lines).
Yeung teaches outputting the value …. through the output layer, and adjusting a weight of at least one node among the plurality of nodes based on this output value, but does not disclose outputting the value related to the uncertainty index through the output layer and adjusting a weight … of nodes based on … value related to the uncertainty index.
Ding discloses:
outputting the value related to the uncertainty index through the output layer … adjusting a weight … of nodes based on … value related to the uncertainty index (Ding, Fig. 4.2, equations 4.12 - 4.14, paragraph below figure 4.2, and page 68, second paragraph, lines 4-6 “we have two output vectors decoded from the hidden state: the prediction vector and uncertainty vector. the model is trained end to end using the Adam optimizer. I use batch size 32, hidden state size 200 and the learning rate is set to be 0.1” discloses outputting value related to the uncertainty through the output layer, and adjusting, i.e. updating through end to end training using the update equations of aleatoric uncertainty, hidden state node weights based on the output uncertainty).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yeung to include outputting the value related to the uncertainty index through the output layer … adjusting a weight … of nodes based on … value related to the uncertainty index, based on the teachings of Ding. One of ordinary skill in the art would have been motivated to make this modification in order to provide much better explanation about the output while maintaining good performance, as suggested by Ding (page 62, paragraph 4, last 2 lines).
Yeung, in view of Ding, discloses transmitting the uncertainty index related to the accuracy of the expected score of the target user and transmitting to a user terminal of the target user, but does not explicitly disclose: transmitting … uncertainty index … to a user terminal.
Tamano discloses:
transmitting … uncertainty index … to a user terminal (Tamano, Fig 15 and ¶[0116] teaches transmitting uncertainty index to a user terminal).
Yeung, Ding, and Tamano are analogous art because they are from the same field of endeavor, response theory and learning models.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yeung, in view of Ding, to include transmitting … uncertainty index … to a user terminal, based on the teachings of Tamano. One of ordinary skill in the art would have been motivated to make this modification in order to improve the learner's sense of conviction, as suggested by Tamano (¶[0111]).
Regarding claim 4, Yeung, in view of Ding, in further view of Tamano, discloses the method of claim 1 (and thus the rejection of claim 1 is incorporated).
Yeung discloses the accuracy of the expected score information of the reference user (Table 2 column “Acc”), but does not disclose wherein the uncertainty index … is provided in a form of at least one of an error value between the expected score information of the reference user and the actual score information of the reference user, a reliability of the error value, and a probability value that the expected score information matches the actual score information.
Ding discloses:
wherein the uncertainty index … is provided in a form of at least one of an error value between the expected score information of the reference user and the actual score information of the reference user, a reliability of the error value, and a probability value that the expected score information matches the actual score information (Ding, Table 5.2, Figure 4.2, Figure 4.5, page 58, final paragraph, lines 10-11 “I provide a way of incorporating uncertainties by regularizing the cross entropy loss function explicitly” and page 74, final paragraph, lines 1-3 “Figure 4.5 illustrates this behavior by plotting the average uncertainties during the 50 exercises that a student undergoes. The average uncertainty for each exercise is plotted for the top performing students (denote as large values in the key).” Teaches an uncertainty index provided in the form of an error value between actual and expected score).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Yeung to include wherein the uncertainty index … is provided in a form of at least one of an error value between the expected score information of the reference user and the actual score information of the reference user, a reliability of the error value, and a probability value that the expected score information matches the actual score information, based on the teachings of Ding. One of ordinary skill in the art would have been motivated to make this modification in order to provide much better explanation about the output while maintaining good performance, as suggested by Ding (page 62, paragraph 4, last 2 lines).
Regarding claim 8, Yeung, in view of Ding, in further view of Tamano, discloses the method of claim 1 (and thus the rejection of claim 1 is incorporated). Yeung further discloses:
wherein the expected score of the target user is obtained through a second neural network model configured to receive the target answering data and output the expected score of the target user (Yeung, page 7, left column, second paragraph, last 4 lines “we use the combination that results in the smallest cross-entropy loss to retrain the model with the entire training set and evaluate the model performance on the test set.” , and page 4, left column, final paragraph, lines 1-4 “at time t, it first receives a KC qt, then predicts the probability of answering qt correctly, and eventually updates the memory using the question-and-answer interaction (qt; at).” teaches the expected score of the target test user is obtained through a retrained second neural network model during test phase as the second configured to receive the target answering data and output the expected score of the target user).
Claims 9 and 10 are substantially similar to claims 1 and 8 respectively, and thus rejected on the same basis as claims 1 and 8 respectively.
Response to Arguments
Applicant's arguments filed 02/24/2026 have been fully considered with regards to the 35 U.S.C. 101 rejection, and are found persuasive. The rejections have been withdrawn.
Applicant's arguments filed 02/24/2026 have been fully considered with regards to the 35 U.S.C. 102/103 rejection, but they are not persuasive.
The applicant asserts on page 15 of the remarks “However, the Examiner incorrectly characterizes the cross-entropy loss disclosed in Yeung as "label information." In neural network training, label information refers to ground- truth target data provided as supervision, such as an actual score, a correctness indicator, or a class label. By contrast, a loss function including cross-entropy loss is an optimization objective that is computed by comparing the model's prediction with the ground-truth label. Thus, the loss value is derived from the label and the prediction, is not itself a label, and does not constitute supervision data.”. The Examiner respectfully disagrees, as the BRI of label information can include any information pertaining to labels, and is not exclusive to a label. Furthermore, applicant’s claims also recite “label information that is defined as a difference between the expected score information and the actual score information”. This does not appear to refer to actual labels, but rather a difference between score information, which is taught by Yeung’s cross entropy loss. Thus, the claim limitation reciting “obtaining a training set on the basis of the reference answering data set, the expected score information, and the actual score information, the training set including label information that is defined as a difference between the expected score information and the actual score information” is taught by Yeung in page 6, right column, paragraphs 3-4, where the training process described teaches training sets that include cross entropy loss as the difference between expected and actual score.
The applicant discusses on page 17 of the remarks that Ding does not disclose adjusting weights of nodes based on uncertainty index and label information. However, this argument is directed to newly amended limitations that were not previously examined by the examiner. Therefore, applicant's arguments are rendered moot. The examiner refers to the rejection under 35 USC § 103 in the current office action for more details. The examiner also reminds applicant that one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUMAIRA ZAHIN MAUNI whose telephone number is (703)756-5654. The examiner can normally be reached Monday - Friday, 9 am - 5 pm (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MATT ELL can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.Z.M./Examiner, Art Unit 2141
/ANDREW L TANK/Primary Examiner, Art Unit 2141
Prosecution Insights