Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The office action is in response to arguments and amendments entered on January 16, 2025 for the patent application 17/476,443 originally filled on September 15, 2021. Claims 1-6 are amended. Claims 1-6 are pending. The first office action of October 16, 2024 is fully incorporated by reference into this final action.
Claim Rejections - 35 USC § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claim 1 is directed to a “a learning content recommendation apparatus” (i.e., a machine) and 6 is directed to a “an operation method of a learning content recommendation apparatus” (i.e., a process), hence the claims are directed to one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter). In other words, Step 1 of the subject-matter eligibility analysis is “Yes.”
However, the claims are drawn to an abstract idea of “determining a recommended question” in the form of “certain methods of organizing human activity,” in terms of managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions). Claim 1 requires the following limitations:
“a memory storing computer readable instructions; a processor configured to execute the instructions to: on the basis of user information including questions previously solved by a user and a responses of the user to the questions, calculate, for each of a plurality of candidate questions, predicted score information including a maximum predicted score, which is a predicted score obtained when the user correctly answers a corresponding candidate question of the plurality of candidate questions; and a minimum predicted score, which is a predicted score obtained when the user incorrectly answers the corresponding candidate questions
predict, for each of the plurality of candidate questions, correct answer rate information- which is a probability that the user correctly answers the corresponding candidate question of the plurality of candidate questions, on the basis of the user information by using an artificial neural network model related to one or more among a recursive artificial neural network (RNN), a long short-term memory (LSTM), a bidirectional LSTM, and a transformer structure-artificial neural network,
calculate a degree of learning, wherein the degree of learning is a probability that, after a first question, which has been previously solved incorrectly by the user, being learned by the user, the user solves a second question that is the same as or similar to the first question again and answers the second question correctly,
calculate, for each of the plurality of candidate questions, an expected score on the basis of the predicted score information, the correct answer rate information, and-a the degree of learning, and determine-a the one or more recommended questions among the plurality of candidate questions according to the expected score,
wherein the processor is further configured to execute the instructions to: calculate, for each of the plurality of candidate questions, a first expected score, in which the degree of learning is not reflected, on the basis of the predicted score information and the correct answer rate information, and calculate, for each of the plurality of candidate questions, a second expected score, in which the degree of learning is reflected, on the basis of the first expected score, the maximum predicted score, and the degree of learning.”
Claim 6 requires the following limitations:
“sampling, a plurality of candidate questions for determining a the one or more recommended questions;
on the basis of user information including a questions previously solved by a user and responses of the user to the questions, calculating, for each of a plurality of candidate questions, predicted score information including a maximum predicted score; which is a predicted score obtained when the user correctly answers a corresponding candidate question of the plurality of candidate questions, and a minimum predicted score; which is a predicted score obtained when the user incorrectly answers the corresponding candidate question;
predicting, for each of the plurality of candidate questions, correct answer rate information, which is a probability that the user correctly answers the corresponding candidate question of the plurality of candidate questions, on the basis of the user information by using an artificial neural network model related to one or more among a recursive artificial neural network (RNN), a long short-term memory (LSTM), a bidirectional LSTM, and a transformer structure-artificial neural network;
calculating a degree of learning, wherein the degree of learning is a probability that, after a first question, which has been previously solved incorrectly by the user, being learned by the user, the user solves a second question that is the same as or similar to the first question again and answers the second question correctly;
calculating, for each of the plurality of candidate questions, an expected score on the basis of the predicted score information, the correct answer rate information, and the degree of learning, and determining the one or more recommended questions among the plurality of candidate questions according to the expected score; and
transmitting the one or more recommended questions to a user terminal,
wherein the determining the one or more recommended questions includes: calculating, for each of the plurality of candidate questions, a first expected score, in which the degree of learning is not reflected, on the basis of the predicted score information and the correct answer rate information; and
calculating, for each of the plurality of candidate questions, a second expected score, in which the degree of learning is reflected, on the basis of the first expected score, the maximum predicted score, and the degree of learning."
These limitations simply describe a process of data gathering and manipulation, which is partially analogous to “collecting information, analyzing it, and displaying certain results of the collection analysis” (i.e., Electric Power Group, LLC, v. Alstom, 830 F.3d 1350, 119 U.S.P.Q.2d 1739 (Fed. Cir. 2016)). Hence, these limitations are akin to an abstract idea which has been identified among non-limiting examples to be an abstract idea. In other words, Step 2A, Prong 1 of the subject-matter eligibility analysis is “Yes.”
Furthermore, the Applicant’s claimed elements of “a memory” and “a processor” are merely claimed to generally link the use of a judicial exception (e.g., pre-solution activity of data gathering and post-solution activity of presenting data) to (1) a particular technological environment or (2) field of use, per MPEP §2106.05(h); and are applying the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, per MPEP §2106.05(f). In other words, the claimed “determining a recommended question” is not providing a practical application, thus Step 2A, Prong 2 of the subject-matter eligibility analysis is “No.”
Furthermore, the claimed “a memory” and “a processor” are described on page 16 lines 19-34 and page 17 line 1 as:
“In the above description, the embodiments of the present disclosure have been described with reference to FIGS. 1 to 6. Each of the user terminal 100 and the learning content recommendation apparatus 200 shown in FIGS. 1 to 3 may be a computing device including one or more processors.
In addition, elements forming the learning content recommendation apparatus 200 may be implemented in the form of modules. The module may refer to software or hardware, such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and may perform predetermined functions. However, the "modules" are not limited to meaning software or hardware. Each of the modules may be configured to be stored in a storage medium capable of being addressed and configured to execute one or more processors. For example, the modules may include elements such as software elements, object-oriented software elements, class elements, and task elements, processes, functions, attributes, procedures, subroutines, segments of a program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. Functions provided in elements and modules may be combined into fewer elements and modules or may be further divided into additional elements and modules.”
These claim elements are reasonably interpreted as generic hardware and provide no details of anything beyond its use as ubiquitous standard equipment. Therefore, Step 2B, of the subject-matter eligibility analysis is “No.”
Claims 2-5 are dependent from claim 1, and include all the limitations of the independent claims. Therefore, the dependent claims recite the same abstract idea. The limitation of the dependent claims fails to amount to significantly more than the judicial exception. For Example:
The limitations of claim 2 recite the additional limitations that the question information is stored in a generic database and that user information storage is used to provide user information to other modules. Per MPEP 2106.05(d)(II), storing and retrieving information are insignificant extra-solution activity and well-understood, routine, and conventional functions when claimed in a generic manner. Therefore, the limitations fail to provide any teaching that integrates the judicial exceptions into a practical application or amounts to significantly more than the judicial exception. For this reason, the analysis performed on the independent claims is also applicable on this claim.
The limitations of claims 4 and 5 clarify the types of data used when calculation the expected score. As such, these claims merely recite the type of data input and is therefore insignificant extra-solution activity. The limitations fail to provide any teaching that integrates the judicial exceptions into a practical application or amounts to significantly more than a judicial exception. For this reason, the analysis performed on the independent claims is also applicable on these claims.
Independent claims 1 and 6 do not provide a practical application and are insufficient to amount to significantly more than the judicial exception. Additionally, dependent claims 2-5 recite abstract idea without significantly more and are not drawn to eligible subject matter. Therefore, claims 1-6 are rejected under 35 U.S.C. § 101 as being directed to non-statutory subject-matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6 are rejected under 35 U.S.C. 103 as being unpatentable over Harlow et al. (Document ID US 20180357915 A1; 2018-12-13) in view of Noble et al. (WO 2019169338 A1; 2019-09-06).
Regarding Claim 1, Harlow et al. teaches:
A learning content recommendation apparatus for determining a one or more recommended questions by reflecting a learning effect of a user, the learning content recommendation apparatus comprising: a memory storing computer readable instructions; a processor configured to execute the instructions to (Para. [0017 and para. [0018], show that the system may comprise a processor and memory to preform various processes):
As depicted in FIG. 1, learning system 100 includes a controller 102 communicatively coupled to user device 115. In some embodiments, controller 102 may include a processor 104 (e.g., one or more hardware processors). Generally, processor 104 may include one or more general purpose central processing units (CPUs). Additionally or alternately, processor 104 may include at least one processor that provides accelerated performance when evaluating neural network models. For example, processor 104 may include a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a tensor processing unit (TPU), a digital signal processor (DSP), a single-instruction multiple-data (SIMD) processor, and/or the like. Generally, such processors may accelerate various computing tasks associated with evaluating neural network models (e.g., training, prediction, preprocessing, and/or the like) by an order of magnitude or more in comparison to a general purpose CPU.
Controller 102 may further include a memory 106 (e.g., one or more non-transitory memories). Memory 106 may include various types of short-term and/or long-term storage modules including cache memory, static random access memory (SRAM), dynamic random access memory (DRAM), non-volatile memory (NVM), flash memory, solid state drives (SSD), hard disk drives (HDD), optical storage media, magnetic tape, and/or the like. In some embodiments, memory 106 may store instructions that are executable by processor 104 to cause processor 104 to perform operations corresponding to processes disclosed herein and described in more detail below.
on the basis of user information including questions previously solved by a user and a responses of the user to the questions, calculate, for each of a plurality of candidate questions, predicted score information (Para. [0031], the user’s responses to questions are used to update the system’s interpretation of the user’s proficiency)
User model 180 may further include one or more user parameters 220. In some embodiments, user parameters 220 may reflect high-level and/or multi-factored assessments of the progress, performance, and/or potential of user 110. In comparison to interaction history 210, user parameters 220 may provide a more holistic snapshot of the progress, capabilities, and personality of user 110, thereby improving the customization and/or predictive capabilities of learning system 100….
including a maximum predicted score (i.e., a score of 1, high tier, or highest possible rating), which is a predicted score obtained when the user correctly answers a corresponding candidate question of the plurality of candidate questions; and a minimum predicted score(i.e., a score of 0, low tier, or lowest possible rating) , which is a predicted score obtained when the user incorrectly answers the corresponding candidate questions (Para. [0031], shows that a user’s response is scored on some form of scale in order to derive a proficiency score)
…As depicted in FIG. 2, illustrative examples of user parameters 220 include measures of the capability of user 110, such as an overall aptitude of user 110, specific areas of strength and/or weakness (e.g., particular concepts and/or quiz types that user 110 excels at), the level of knowledge that user 110 has attained in a given subject domain, metacognition (e.g., the understanding user 110 has of his or her own thought processes), and/or the like. In some embodiments, user parameters 220 may be represented using numerical scores (e.g., a value between zero and one), tiers (e.g., “high,” “medium,” “low,” etc.), ratings (e.g., letter grades), categories, percentiles, and/or the like.
predict, for each of the plurality of candidate questions, correct answer rate information- which is a probability that the user correctly answers the corresponding candidate question of the plurality of candidate questions, on the basis of the user information (Para. [0036], the system uses the chance of whether or not the user will get the problem correct when selecting an activity);
As depicted in FIG. 2, predictive update module 230 includes a prediction module 234 and an update module 236. Prediction module 234 predicts activity 232 based on one or more of interaction history 210 and user parameters 220. Prediction module 234 may also receive context information 238, such as the prompt provided to user 110 (e.g., a multiple-choice question) in a scenario where activity 232 is prompted. Prediction module 234 may predict the occurrence of activity 232 (e.g., whether user 110 will log in at a particular time) or may make predictions associated with the performance of activity 232 (e.g., how user 110 will score on a particular assessment). For example, when activity 232 corresponds to a response to a multiple choice quiz question, prediction module 234 may predict which answer choice user 110 will select and/or whether or not user 110 will select the correct answer choice…
by using an artificial neural network model related to one or more among a recursive artificial neural network (RNN), a long short-term memory (LSTM), a bidirectional LSTM, and a transformer structure-artificial neural network (Para. [0036]),
…In some embodiments, prediction module 234 may predict activity 232 using a statistical model, a neural network model, a rule-based model, and/or the like
calculate a degree of learning, wherein the degree of learning is a probability that, after a first question, which has been previously solved incorrectly by the user, being learned by the user, the user solves a second question that is the same as or similar to the first question again and answers the second question correctly (Para. [0037], future question recommendation will change depending on the correlation between the user’s predicted performance on a question and the actual performance as the user’s parameters change accordingly),
Update module 236 compares the activity predicted by prediction module 234 with the actual activity 232 of user 110. When the predicted and actual activities match, update module 236 may determine that user parameters 220 accurately reflect the progress and/or capabilities of user 110 and therefore may not adjust or update user parameters 220. When the predicted and actual activities do not match, update module 236 may adjust the values of one or more of user parameters 220. To illustrate, if prediction module 234 predicts that user 110 will select the wrong answer choice to a multiple-choice quiz question but user 110 actually selects the correct answer choice, update module 236 may determine that user parameters 220 underestimate the overall aptitude, diligence, etc. of user 110 and therefore may increase the values assigned to one or more of user parameters 220. Conversely, if prediction module 234 predicts that user 110 will select the correct answer choice but user 110 actually selects the wrong answer choice, update module 236 may determine that user parameters 220 overestimate the overall aptitude, diligence, etc. of user 110 and therefore may decrease the values assigned to one or more of user parameters 220. But if prediction module 234 predicts that user 110 will select the correct answer choice and user 110 does in fact select the correct answer choice (and/or vice versa), update module 236 may leave user parameters 220 unchanged.
calculate, for each of the plurality of candidate questions, an expected score on the basis of the predicted score information, the correct answer rate information, and the degree of learning (Para [0037], the prediction module considers if the user will get the question correct, predict the grade, and will update the user’s performance information based on the response),
Update module 236 compares the activity predicted by prediction module 234 with the actual activity 232 of user 110. When the predicted and actual activities match, update module 236 may determine that user parameters 220 accurately reflect the progress and/or capabilities of user 110 and therefore may not adjust or update user parameters 220. When the predicted and actual activities do not match, update module 236 may adjust the values of one or more of user parameters 220. To illustrate, if prediction module 234 predicts that user 110 will select the wrong answer choice to a multiple-choice quiz question but user 110 actually selects the correct answer choice, update module 236 may determine that user parameters 220 underestimate the overall aptitude, diligence, etc. of user 110 and therefore may increase the values assigned to one or more of user parameters 220…
and determine the one or more recommended questions among the plurality of candidate questions according to the expected score (Para [0015], the user’s performance information is used to schedule a series of learning interactions, which includes things such as questions),
To facilitate knowledge acquisition by user 110, learning system 100 may provide a series of learning interactions to user 110. The learning interactions may be intended to introduce user 110 to new knowledge items 125, to reinforce previously presented knowledge items 125, to assess the progress of user 110, to provide feedback to user 110, and/or the like. In some embodiments, sets of learning interactions may be grouped and/or arranged to form courses, modules, sub-modules, learning sessions (e.g., sets of learning interactions intended to be performed in a single sitting), and/or the like. In some embodiments, learning system 100 may schedule the learning interactions to achieve a desired outcome, such as long-term retention of knowledge items 125, peak short-term retention of knowledge items 125 (e.g., in preparation for a particular event or deadline, such as an exam or performance), and/or the like. In this regard, user 110 may be regarded as having a memory state that generally corresponds to the set of knowledge items 125 that user 110 knows at a point in time and how well user 110 knows the information (e.g., how quickly or precisely user 110 is able to recall the information, and for how long user 110 is likely to remember the information). In some embodiments, learning system 100 may measure the memory state of user 110 in order to customize the schedule of learning interactions to user 110.
wherein the processor is further configured to execute the instructions to: calculate, for each of the plurality of candidate questions, a first expected score, in which the degree of learning is not reflected, on the basis of the predicted score information and the correct answer rate information (Para. [0036], the system considers if the user will get the problem right, i.e. correct answer rate information, and the score associated with the problem, i.e. predicted score information),
As depicted in FIG. 2, predictive update module 230 includes a prediction module 234 and an update module 236. Prediction module 234 predicts activity 232 based on one or more of interaction history 210 and user parameters 220. Prediction module 234 may also receive context information 238, such as the prompt provided to user 110 (e.g., a multiple-choice question) in a scenario where activity 232 is prompted. Prediction module 234 may predict the occurrence of activity 232 (e.g., whether user 110 will log in at a particular time) or may make predictions associated with the performance of activity 232 (e.g., how user 110 will score on a particular assessment). For example, when activity 232 corresponds to a response to a multiple choice quiz question, prediction module 234 may predict which answer choice user 110 will select and/or whether or not user 110 will select the correct answer choice…
and calculate, for each of the plurality of candidate questions, a second expected score, in which the degree of learning is reflected, on the basis of the first expected score, the maximum predicted score, and the degree of learning (Para [0037], the prediction module considers if the user will get the question correct, predict the grade, and will update the user’s performance information based on the response).
Update module 236 compares the activity predicted by prediction module 234 with the actual activity 232 of user 110. When the predicted and actual activities match, update module 236 may determine that user parameters 220 accurately reflect the progress and/or capabilities of user 110 and therefore may not adjust or update user parameters 220. When the predicted and actual activities do not match, update module 236 may adjust the values of one or more of user parameters 220. To illustrate, if prediction module 234 predicts that user 110 will select the wrong answer choice to a multiple-choice quiz question but user 110 actually selects the correct answer choice, update module 236 may determine that user parameters 220 underestimate the overall aptitude, diligence, etc. of user 110 and therefore may increase the values assigned to one or more of user parameters 220…
Harlow et al. does not explicitly teach:
The operations of claim 1 for each of the plurality of candidate questions
Noble et al teaches:
on the basis of user information including questions previously solved by a user and a responses of the user to the questions (Para. [0150], show that user information gathered includes information regarding the user’s past question responses),
In some embodiments in which the one or several end users are individuals, and specifically are students, the user profile database 301 can further include information relating to these students’ academic and/or educational history. This information can identify one or several courses of study that the student has initiated, completed, and/or partially completed, as well as grades received in those courses of study. In some embodiments, the student’s academic and/or educational history can further include information identifying student performance on one or several tests, quizzes, and/or assignments. In some embodiments, this information can be stored in a tier of memory that is not the fastest memory in the content delivery network 100. In some embodiments, this can comprise response information such as, for example, information identifying one or several questions or pieces of content and responses provided to the same. In some embodiments, this response information can be formed into one or several matrices“D” containing information for n users responding to p items, these one or several matrices D are also referred to herein as the matrix D, the D matrix, the user matrix, and/or the response matrix. Thus, the matrix D can have n x p dimensions, and in some embodiments, the matrix D can identify whether user responses to items were correct or incorrect. In some embodiments, for example, the matrix D can include an entry“1” for an item when a user response to that item is correct and can otherwise include and entry“0”.
calculate, for each of a plurality of candidate questions, predicted score information including a maximum predicted score (i.e., a score of 1, high tier, or highest possible rating), which is a predicted score obtained when the user correctly answers a corresponding candidate question of the plurality of candidate questions; and a minimum predicted score(i.e., a score of 0, low tier, or lowest possible rating) , which is a predicted score obtained when the user incorrectly answers the corresponding candidate questions (Para. [0017], shows that the score/weight of a potential task may be considered)
The prioritization database 311 can further include information relevant to the prioritization of one or several tasks and/or the prioritization database 311 can include information that can be used in determining the prioritization of one or several tasks. In some embodiments, this can include weight data which can identify a relative and/or absolute weight of a task. In some embodiments, for example, the weight data can identify the degree to which a task contributes to an outcome such as, for example, a score or a grade. In some embodiments, this weight data can specify the portion and/or percent of a grade of a class, section, course, or study that results from, and/or that is associated with the task.
predict, for each of the plurality of candidate questions, correct answer rate information which is a probability that the user correctly answers the corresponding candidate question of the plurality of candidate questions, on the basis of the user information (Para. [0195], shows that the system considers the chance of whether or not the user will get the problem, chosen from a number of problems, correct when selecting an activity);
The recommendation engine can identify one or several potential data packets for providing and/or one or several data packets for providing to the user based on, for example, one or several rules, models, predictions, or the like. The recommendation engine can use the skill level of the user to generate a prediction of the likelihood of one or several users providing a desired response to some or all of the potential data packets. In some embodiments, the recommendation engine can pair one or several data packets with selection criteria that may be used to determine which packet should be delivered to a user based on one or several received responses from that student-user. In some embodiments, one or several data packets can be eliminated from the pool of potential data packets if the prediction indicates either too high a likelihood of a desired response or too low a likelihood of a desired response.
calculate a degree of learning, wherein the degree of learning is a probability that, after a first question, which has been previously solved incorrectly by the user, being learned by the user, the user solves a second question that is the same as or similar to the first question again and answers the second question correctly (Para. [0152], shows that a degree of skill the user has acquired based on past performance of questions is gathered; Para. [0153], additionally shows that information that the student may have learned outside of the past question performance is gathered),
In some embodiments, the user profile data store 301 can further include information identifying one or several user skill levels. In some embodiments, these one or several user skill levels can identify a skill level determined based on past performance by the user interacting with the content delivery network 100, and in some embodiments, these one or several user skill levels can identify a predicted skill level determined based on past performance by the user interacting with the content delivery network 100 and one or several predictive models.
The user profile database 301 can further include information relating to one or several teachers and/or instructors who are responsible for organizing, presenting, and/or managing the presentation of information to the user. In some embodiments, user profile database 301 can include information identifying courses and/or subjects that have been taught by the teacher, data identifying courses and/or subjects currently taught by the teacher, and/or data identifying courses and/or subjects that will be taught by the teacher. In some embodiments, this can include information relating to one or several teaching styles of one or several teachers. In some embodiments, the user profile database 301 can further include information indicating past evaluations and/or evaluation reports received by the teacher. In some embodiments, the user profile database 301 can further include information relating to improvement suggestions received by the teacher, training received by the teacher, continuing education received by the teacher, and/or the like. In some embodiments, this information can be stored in a tier of memory that is not the fastest memory in the content delivery network 100.
calculate, for each of the plurality of candidate questions, an expected score on the basis of the predicted score information, the correct answer rate information, and the degree of learning and determine the one or more recommended questions among the plurality of candidate questions according to the expected score (Para. [0017], shows that the score/weight of a potential task may be considered when selecting an activity; Para. [0195], shows that the system considers the chance of whether or not the user will get the problem, chosen from a number of problems, correct when selecting an activity; Para. [0313], shows that the information gathered in the user profile is utilized when selecting content),
The prioritization database 311 can further include information relevant to the prioritization of one or several tasks and/or the prioritization database 311 can include information that can be used in determining the prioritization of one or several tasks. In some embodiments, this can include weight data which can identify a relative and/or absolute weight of a task. In some embodiments, for example, the weight data can identify the degree to which a task contributes to an outcome such as, for example, a score or a grade. In some embodiments, this weight data can specify the portion and/or percent of a grade of a class, section, course, or study that results from, and/or that is associated with the task.
The recommendation engine can identify one or several potential data packets for providing and/or one or several data packets for providing to the user based on, for example, one or several rules, models, predictions, or the like. The recommendation engine can use the skill level of the user to generate a prediction of the likelihood of one or several users providing a desired response to some or all of the potential data packets. In some embodiments, the recommendation engine can pair one or several data packets with selection criteria that may be used to determine which packet should be delivered to a user based on one or several received responses from that student-user. In some embodiments, one or several data packets can be eliminated from the pool of potential data packets if the prediction indicates either too high a likelihood of a desired response or too low a likelihood of a desired response.
After the user profile has been received and/or retrieved, the process 740 can proceed to block 749, wherein next content is selected and/or provided. In some embodiments, the next content can be selected and/or provided based on one or several attributes, such as one or several difficulty levels, of potential next content and one or several attributes, such as a user skill level, of the user requesting the next content. The next content can be selected by the recommendation engine 686, and can be provided to the student via the user device 106.
wherein the processor is further configured to execute the instructions to: calculate, for each of the plurality of candidate questions, a first expected score, in which the degree of learning is not reflected, on the basis of the predicted score information and the correct answer rate information (Para. [0017], shows that the score/weight of a potential task may be considered when selecting an activity; Para. [0195], shows that the system considers the chance of whether or not the user will get the problem, chosen from a number of problems, correct when selecting an activity),
The prioritization database 311 can further include information relevant to the prioritization of one or several tasks and/or the prioritization database 311 can include information that can be used in determining the prioritization of one or several tasks. In some embodiments, this can include weight data which can identify a relative and/or absolute weight of a task. In some embodiments, for example, the weight data can identify the degree to which a task contributes to an outcome such as, for example, a score or a grade. In some embodiments, this weight data can specify the portion and/or percent of a grade of a class, section, course, or study that results from, and/or that is associated with the task.
The recommendation engine can identify one or several potential data packets for providing and/or one or several data packets for providing to the user based on, for example, one or several rules, models, predictions, or the like. The recommendation engine can use the skill level of the user to generate a prediction of the likelihood of one or several users providing a desired response to some or all of the potential data packets. In some embodiments, the recommendation engine can pair one or several data packets with selection criteria that may be used to determine which packet should be delivered to a user based on one or several received responses from that student-user. In some embodiments, one or several data packets can be eliminated from the pool of potential data packets if the prediction indicates either too high a likelihood of a desired response or too low a likelihood of a desired response.
and calculate, for each of the plurality of candidate questions, a second expected score, in which the degree of learning is reflected, on the basis of the first expected score, the maximum predicted score, and the degree of learning (Para. [0017], shows that the score/weight of a potential task may be considered when selecting an activity; Para. [0195], shows that the system considers the chance of whether or not the user will get the problem, chosen from a number of problems, correct when selecting an activity; Para. [0313], shows that the information gathered in the user profile is utilized when selecting content),
The prioritization database 311 can further include information relevant to the prioritization of one or several tasks and/or the prioritization database 311 can include information that can be used in determining the prioritization of one or several tasks. In some embodiments, this can include weight data which can identify a relative and/or absolute weight of a task. In some embodiments, for example, the weight data can identify the degree to which a task contributes to an outcome such as, for example, a score or a grade. In some embodiments, this weight data can specify the portion and/or percent of a grade of a class, section, course, or study that results from, and/or that is associated with the task.
The recommendation engine can identify one or several potential data packets for providing and/or one or several data packets for providing to the user based on, for example, one or several rules, models, predictions, or the like. The recommendation engine can use the skill level of the user to generate a prediction of the likelihood of one or several users providing a desired response to some or all of the potential data packets. In some embodiments, the recommendation engine can pair one or several data packets with selection criteria that may be used to determine which packet should be delivered to a user based on one or several received responses from that student-user. In some embodiments, one or several data packets can be eliminated from the pool of potential data packets if the prediction indicates either too high a likelihood of a desired response or too low a likelihood of a desired response.
After the user profile has been received and/or retrieved, the process 740 can proceed to block 749, wherein next content is selected and/or provided. In some embodiments, the next content can be selected and/or provided based on one or several attributes, such as one or several difficulty levels, of potential next content and one or several attributes, such as a user skill level, of the user requesting the next content. The next content can be selected by the recommendation engine 686, and can be provided to the student via the user device 106.
It would be obvious, before the effective filing date of the claimed invention, for someone of ordinary skill in the art to apply the use of known techniques of Noble et al., regarding operation for each of the plurality of questions, to the similar device in Harlow et al., a system for scheduling learning interaction, to yield the predicable result of outputting higher quality learning interactions. Someone of ordinary skill in the art would be motivated to apply the known techniques of Noble et al. to the device of Harlow et al. as assessment of each individual question for user suitability would allow for higher quality learning interactions.
Regarding Claim 2, Harlow et al. teaches:
The learning content recommendation apparatus of claim 1, wherein the processor is further configured to execute the instructions to: receive question information from a question database (Para. [0020], shows that the various learning assets may be derived from a database) and
In some embodiments, learning system 100 may include an asset generator module 140. As depicted in FIG. 1, asset generator module 140 extracts knowledge items 125 from knowledge sources 120 and generates learning assets 150 based on knowledge items 125. In general, learning assets 150 correspond to sets of knowledge items 125 selected based on a predetermined relationship. For example, a given learning asset 150 may include knowledge items 125 related to a particular topic or sub-topic. To illustrate, a learning asset 150 corresponding to a historical person may include a set of knowledge items 125 that identify the person's name, date of birth, historical significance, and/or the like. In some embodiments, learning assets 150 may be structured according to predefined templates that include a set of slots (e.g., name, date of birth, historical significance, etc.) with values that are populated using knowledge items 125. Illustrative examples of predefined templates may include association templates, vocabulary templates, passage templates, image and/or video region templates, sequence templates, and/or pattern templates, and/or the like. Examples of predefined templates are further described in U.S. patent application Ser. No. 15/836,631. In some embodiments, learning assets 150 may further include metadata that assists learning system 100 in using learning assets 150 effectively. For example, the metadata may include sequencing information, priority information, information identifying relationships among learning assets 150, and/or the like. It is to be understood that template-based learning assets as described above are merely illustrative, and that learning assets 150 may additionally or alternately correspond to various types of collections that facilitate storing and retrieving knowledge items 125 (e.g., databases, file systems, etc.).
store response information according to question solving of the user (Para. [0035], shows that user response information is recorded).
To maintain user parameters 220, user model 180 may include a predictive update module 230 that updates user parameters 220 based on an activity 232 of user 110. Activity 232 may take the form of (but is not limited to) a response by user 110 to a prompt provided by learning system 100, such as a responsive learning interaction, a survey, an assessment, a simulation, interactive multimedia, and/or the like. Additionally or alternately, activity 232 may include patterns of engagement with learning system 100 (which may or may not be prompted), such as login patterns, responses to notifications or lack thereof, the duration spent on various tasks or screens, and/or the like. In some embodiments, activity 232 may include activities independent of learning system 100, such as an externally administered exam, real-world jobs or tasks, and/or the like.
Harlow et al. does not explicitly teach:
sample the plurality of candidate questions for determining the one or more recommended questions
Noble et al. teaches:
sample the plurality of candidate questions for determining the one or more recommended questions (Para. [0437] and para. [0438], show that multiple candidate questions are sampled before determining which will be the recommended questions).
After attributes relevant to mastery concept, and been determined, the process 1040 proceeds to block 1051, wherein items associated with the identified attributes are determined, and wherein the difficulty of those items is determined. In some embodiments, the difficulty of these items can be determined based on metadata associated with the items, which metadata can be stored in the database server 104 and specifically in the content library database 303. After the item difficulty has been determined, the process 1040 proceeds to block 1052, wherein the user skill level is determined. In some embodiments, the user skill level can be determined based on information contained in the student metadata retrieved in block 1044, and in some embodiments, the user skill level can be contained in the student metadata retrieved in block 1044. After the user skill level has been determined, the process 1040 proceeds to block 1053, wherein items having a difficulty level corresponding to the student skill level are identified. In some embodiments, this can include identifying items that have a difficulty level closely corresponding to the skill level of the student.
After these items corresponding to the student skill level have been identified, the process 1040 proceeds to block 1054 wherein the content item having the greatest mastery contribution is identified and selected. In some embodiments, this can include identifying the content item that has a difficulty level adequately matching the student skill level, and that contains the most attributes associated with the concept that the student is currently trying to master and/or the concept at which the student is currently located in the domain graph. After the content item with the greatest mastery contribution is selected, the process 1040 proceeds to block 1055, wherein the content item is provided to the student.
Regarding Claim 4, Harlow et al. teaches:
The learning content recommendation apparatus of claim 1, wherein the processor is further configured to calculate the first expected score on the basis of the predicted score information and the correct answer rate information by using a first algorithm (Para. [0031], the user’s responses to questions are used to update the system’s interpretation of the user’s proficiency).
User model 180 may further include one or more user parameters 220. In some embodiments, user parameters 220 may reflect high-level and/or multi-factored assessments of the progress, performance, and/or potential of user 110. In comparison to interaction history 210, user parameters 220 may provide a more holistic snapshot of the progress, capabilities, and personality of user 110, thereby improving the customization and/or predictive capabilities of learning system 100….
Regarding Claim 5, Harlow et al. teaches:
The learning content recommendation apparatus of claim 1, wherein the processor is further configured to calculate the second expected score on the basis of the predicted score information, the correct answer rate information, and the degree of learning by using a second algorithm (Para. [0036], the system uses the chance of whether or not the user will get the problem correct when selecting an activity).
As depicted in FIG. 2, predictive update module 230 includes a prediction module 234 and an update module 236. Prediction module 234 predicts activity 232 based on one or more of interaction history 210 and user parameters 220. Prediction module 234 may also receive context information 238, such as the prompt provided to user 110 (e.g., a multiple-choice question) in a scenario where activity 232 is prompted. Prediction module 234 may predict the occurrence of activity 232 (e.g., whether user 110 will log in at a particular time) or may make predictions associated with the performance of activity 232 (e.g., how user 110 will score on a particular assessment). For example, when activity 232 corresponds to a response to a multiple choice quiz question, prediction module 234 may predict which answer choice user 110 will select and/or whether or not user 110 will select the correct answer choice…
Regarding Claim 6, Harlow et al. teaches:
An operation method of a learning content recommendation apparatus for determining one or more recommended questions by reflecting a learning effect of a user, the operation method performed by the learning content recommendation apparatus and comprising: transmitting the one or more recommended questions to a user terminal (Para [0016]),
User 110 may access learning system 100 via a user device 115. User device 115 may correspond to a local terminal of learning system 100 and/or a remote node for