DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 – 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Step 1: Is the claimed invention a statutory category of invention?
Claims 1 and 18 are directed to a method for automatic grading of mathematical problems (Step 1, Yes).
Step 2A, Prong 1: Does the claim recite an abstract idea?
The limitation of steps: Re claim 1 … converting a student's hand-written answer containing mathematical expressions to a digital file format; executing a first software program to convert the mathematical expressions of the digital file format to a digital formula; executing a second software program to check the correctness of the mathematical expressions of the digital formula, the second software configured to create a model with noise to check the correctness of derivations between the mathematical expressions; wherein, in the event of an incorrect derivation, the software is configured to apply a machine learning model or state estimation technique to determine mistakes between derivations and to suggest correct coefficients and other values within the mathematical expressions. Re claim 18 a student solving a mathematical problem by generating a plurality of mathematical expressions in a digital formula; executing a software program to check the correctness of the plurality of mathematical expressions of the digital formula, the software program configured to: create a dataset using random numbers; input a random number from the dataset for each variable within a first mathematical expression of the plurality of mathematical expressions; generate a first array of output values for the first mathematical expression; input the random numbers from the dataset into each subsequent mathematical expression of the plurality of expressions to generate a respective array of output values for each subsequent mathematical expression; and compare the first array of output values to the array of output values for each subsequent mathematical expression to determination if each derivation was done correctly; wherein, in the event of an incorrect derivation, the software is configured to apply a machine learning model or state estimation technique to determine mistakes between derivations and to suggest correct coefficients and other values within the mathematical expressions and as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind of a teacher. Claims 1 and 18 fail to adequately dislcose any computer structures for implementing the software elements. The mere nominal recitation of first and second software programs and software program performing these steps does not take the claim limitation outside of the mental processes grouping. Thus, the claim recites a mental process (Step 2A, Prong 1: yes).
Step 2A, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
Per the 2019 Revised Patent Subject Matter Eligibility Guidance, if a claim as a whole integrates the recited judicial exception into a practical application of that exception, a claim is not "directed to" a judicial exception. Alternatively, a claim that does not integrate a recited judicial exception into a practical application is directed to the exception. Evaluating whether a claim integrates an abstract idea into a practical application is performed by a) identifying whether there are any additional elements recited in the claim beyond the abstract idea, and b) evaluating those additional elements individual and in combination to determine whether they integrate the abstract idea into a practical application, using one or more of the considerations laid out by the Supreme Court and the Federal Circuit. Exemplary considerations indicative that an additional element (or combination of elements) may have or has not been integrated into a practical application are set forth in the 2019 PEG.
With respect to the instant claims, claims 1 and 18 do not require any statutory product or any computer structures. Even in combination, the recited additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits, such as an improvement to a computing system, on practicing the abstract idea (STEP 2A, Prong 2: NO).
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Claims 1 and 18 do not require any statutory product or any computer structures set forth above for Step 2A, Prong 2. The claimed methods can be readily performed by the mind of a teacher. Dependent claims 2 – 17 and 19 – 20 inherit the deficiencies of their respective parent claims through their dependencies and do not recite additional limitations sufficient to direct the claims to more than the claimed abstract idea, and are thus rejected for the same reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 8, 11-15 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Labine et al. (US 2011/0318724 A1) in view of Fuka (US 2018/0096619 A1).
Re claims 1, 18:
Labine teaches 1. A method for automatic grading of mathematical problems (Labine, Abstract), comprising:
converting a student's hand-written answer containing mathematical expressions to a digital file format (Labines, fig. 15A – 15B; [0065]);
executing a first software program to convert the mathematical expressions of the digital file format to a digital formula (Labine, fig. 15A – 15B; [0047], “converted to a LaTex solution expression”);
executing a second software program to check the correctness of the mathematical expressions of the digital formula (Labine, [0014], “grading the response based on the result of the comparison”), the second software configured to create a model with noise to check the correctness of derivations between the mathematical expressions (Labine, [0063], “Variables in the response and solution expressions are identified and a set of random values are generated for each variable (step 1418) … The response expression and the solution expression are evaluated using the same random values for each variable (step 1420), and the results of the evaluation are compared (step 1422). If the results are within a suitable tolerance for all of the random variables substituted”; noise – random variable);
wherein, in the event of an incorrect derivation, the software is configured to apply a machine learning model or state estimation technique to determine mistakes (Labine, fig. 11, 1210 - “Student response Incorrect”; fig. 7; [0057], “If the syntax trees do not match, then the participant's response is graded as incorrect (step 1210), and no score for the response is added to the participant's grade”; [0055]; [0057], “Formal Equivalence grading algorithm carries out a verification step to determine if the comparison results signify that the solution syntax tree and the response syntax tree comprise the same nodes and thus match (step 1216). If the syntax trees match, then the response is graded as correct (step 1218), and a score for the response is added to the participant's grade. If the syntax trees do not match, then the participant's response is graded as incorrect (step 1210)”; the “formal equivalence grading algorithm” -state estimation technique; fig. 11).
Labine teaches 18. A method for automatic grading of mathematical problems (Labine, Abstract), comprising:
a student solving a mathematical problem by generating a plurality of mathematical expressions in a digital formula (Labines, fig. 15A – 15B; [0065]);
executing a software program to check the correctness of the plurality of mathematical expressions of the digital formula (Labine, fig. 15A – 15B; [0047], “converted to a LaTex solution expression”), the software program configured to:
create a dataset using random numbers (Labine, fig. 14, “Identify variables and generate random values for each variable”, “Evaluate students and teachers expression with randomly generated values”; [0063]; figs. 12A – 12J show an arrays of questions/mathematical expressions);
input a random number from the dataset for each variable within a first mathematical expression of the plurality of mathematical expressions (Labine, figs. 12A – 12J show plurality of questions/mathematical expressions);
generate a first array of output values for the first mathematical expression (Labine, figs 12A – 12J; fig. 14; [0061], “The LaTeX response and solution expressions are each parsed into a tree-shaped hierarchy of operators and operands based generally on the order of operations”);
input the random numbers from the dataset into each subsequent mathematical expression of the plurality of expressions to generate a respective array of output values for each subsequent mathematical expression (Labine, fig. 14; figs. 12A – 12J); and
compare the first array of output values to the array of output values for each subsequent mathematical expression to determination if each derivation was done correctly (Labine, [0063], “Variables in the response and solution expressions are identified and a set of random values are generated for each variable (step 1418) … The response expression and the solution expression are evaluated using the same random values for each variable (step 1420), and the results of the evaluation are compared (step 1422). If the results are within a suitable tolerance for all of the random variables substituted”; noise – random variable);
wherein, in the event of an incorrect derivation, the software is configured to apply a machine learning model or state estimation technique to determine mistakes between derivations (Labine, fig. 11, 1210 - “Student response Incorrect”; fig. 7; [0057], “If the syntax trees do not match, then the participant's response is graded as incorrect (step 1210), and no score for the response is added to the participant's grade”; [0055]; [0057], “Formal Equivalence grading algorithm carries out a verification step to determine if the comparison results signify that the solution syntax tree and the response syntax tree comprise the same nodes and thus match (step 1216). If the syntax trees match, then the response is graded as correct (step 1218), and a score for the response is added to the participant's grade. If the syntax trees do not match, then the participant's response is graded as incorrect (step 1210)”; the “formal equivalence grading algorithm” -state estimation technique; fig. 11).
Labine does not explicitly disclose wherein, in the event of an incorrect derivation, the software is configured to apply a machine learning model or state estimation technique to determine mistakes between derivations and to suggest correct coefficients and other values within the mathematical expressions.
Fuka (US 2018/0096619 A1) teaches expert systems and methods that can intelligently assist users such as students in answering questions and progressing in their individual learning paths and can intelligently assist users such as question authors and instructors in creating assessment questions (Fuka, Abstract). Fuka teaches wherein, in the event of an incorrect derivation, the software is configured to apply a machine learning model or state estimation technique to determine mistakes between derivations (Fuka, [0029]; [0154], “an ESS module may implement a technique like fuzzy logic to investigate probable mistakes they made in answering the question, and give hints that express the uncertainty, for example, “You probably forgot to multiply terms before adding terms.”; ”[0159], “using a set of rules or a similar data structure to represent frequent student math or logic errors, the ESS module could suggest that one possible incorrect answer”) and to suggest correct coefficients and other values within the mathematical expressions (Fuka, fig. 11A – 11E, “Show Me A Hint”, i.e., fig. 11D, suggestions - “One of your ‘X’ coefficients is incorrect”; [0149] – [0151]).
Therefore, in view of Fuka, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method described in Labine, by providing hints to math solution steps as taught by Fuka, since hints may be dynamically generated relative to the context of the problem as well as the particular student's work in trying to solve the problem and/or the incorrect answer(s) from the student (Fuka, [0176]).
Re claim 2:
2. The method of claim 1, wherein converting the student's hand-written answer comprises using optical character recognition that electronically converts images of typed, handwritten, and printed text into machine-encoded text (Labine, [0043]; [0067]; [0046]).
Re claim 8:
8. The method of claim 3, wherein the random numbers are between 0 and 1 (Labine, [0063]; fig. 6A, “Yes or No”, “True or False”; Fuka, [0155]).
Re claims 11 – 13:
11. The method of claim 3, further comprising more than two mathematical expressions and comparing output values for each mathematical expression to determine the correctness of each mathematical expression (Labine, figs. 5 – 9; figs. 15A – 15B).
12. The method of claim 11, wherein each mathematical expression over two is an intermediate step of the student's hand-written answer (Labines, fig. 15A – 15B; [0065]).
13. The method of claim 12, further comprising awarding partial credit based upon each correct derivation (Labine, [0057], “a partial score may be added to the participant's grade depending on the number of nodes in the response syntax tree that exist in the solution syntax tree”).
Re claims 14 – 15:
14. The method of claim 1, wherein the mathematical expressions are linear coefficients (Labine, fig. 6B, 716; fig. 7).
15. The method of claim 1, wherein the mathematical expressions are nonlinear equations (Labine, fig. 6B, 716; fig. 7).
Claims 3 – 5 are rejected under 35 U.S.C. 103 as being unpatentable over Labine and Fuka as applied to claim 1 above, and further in view of Timmermans (US 9,294,456 B1)
Re claims 3 - 5:
Labine teaches 3. The method of claim 1, further comprising:
the model with noise comprising a dataset using random numbers as one or more variables within a first mathematical expression and generating a first array of output values for the first mathematical expression (Labine, [0063], “Variables in the response and solution expressions are identified and a set of random values are generated for each variable (step 1418) … The response expression and the solution expression are evaluated using the same random values for each variable (step 1420), and the results of the evaluation are compared (step 1422). If the results are within a suitable tolerance for all of the random variables substituted”; noise – random variable);
Labine does not explicitly disclose
inputting the random numbers into a second mathematical expression to generate a second array of output values for the second mathematical expression; and
comparing the first array of output values to the second array of output values.
Timmermans (US 9,294,456 B1) teaches a method include Generation of the questions can include using a random order in which the questions are selected. Furthermore, a difficulty level of the question can be used in the question generation (Timmermans, col. 5, lines 1 – 25). Timmermans teaches the model with noise comprising a dataset using random numbers as one or more variables within a first mathematical expression and generating a first array of output values for the first mathematical expression (Timmermans, col. 5, lines 1 – 25);
inputting the random numbers into a second mathematical expression to generate a second array of output values for the second mathematical expression; and comparing the first array of output values to the second array of output values (Timmermans, fig. 7A – 8B; col. 10, lines 24 – 63; col. 3, lines 12 – 55, “Once the exact answer is found, it can be used to set the upper and lower bounds”; upper bound values – first expression; lower bound values – second expression). Timmermans further teaches 4. The method of claim 3, wherein if a variation between the first array of output values and the second array of output values is less than a precision value, then the derivation between the first and second mathematical expressions is deemed correct (Timmermans, fig. 7A – 8B; col. 10, lines 24 – 63; col. 3, lines 12 – 55, “Once the exact answer is found, it can be used to set the upper and lower bounds”; upper bound values – first expression; lower bound values – second expression; answer between upper and lower boundary is correct; or ; answer outside of upper and lower boundary is incorrect). Timmermans further teaches 5. The method of claim 3, wherein if a variation between the first array of output values and the second array of output values is greater than a precision value, then the derivation between the first and second mathematical expressions is deemed incorrect (Timmermans, fig. 7A – 8B; col. 10, lines 24 – 63; col. 3, lines 12 – 55, “Once the exact answer is found, it can be used to set the upper and lower bounds”; upper bound values – first expression; lower bound values – second expression; answer between upper and lower boundary is correct; or ; answer outside of upper and lower boundary is incorrect).
Therefore, in view of Timmermans, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method described in Labine, by providing the first and second expression for a question as taught by Timmermans, since a question can have a range of acceptable answers, wherein the range includes an upper bound and a lower bound. The question can also have an exact answer, which is typically at or near the center of the range (Timmermans, col. 3, lines 12 – 55).
Claims 9 – 10 and 19 – 20 are rejected under 35 U.S.C. 103 as being unpatentable over Labine and Fuka as applied to claims 3 and 18 above, and further in view of Glickman et al. (US 9305059 B1).
Re claims 9 – 10, 19 – 20:
Labine does not explicitly disclose generating a sample size. Glickman teaches a method for dynamically selecting questions to be presented in a survey (Glickman, Abstract). Glickman teaches 9. The method of claim 3, further comprising generating a sample size for each variable within the first and second mathematical expressions. 10. The method of claim 9, wherein the sample size is 10,000. 19. The method of claim 18, further comprising generating a sample size for each variable within the respective mathematical expressions. 20. The method of claim 19, wherein the sample size is greater than 1,000 (Glickman, col. 7, line 15 – 16; col. 2, lines 65 - 67). Therefore, in view of Glickman, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method in Labine, by providing sampling size of questions as taught by Glickman, since it was known in the art to generate question based on a predetermine sample question database.
Claims 16 – 17 are rejected under 35 U.S.C. 103 as being unpatentable over Labine and Fuka as applied to claims 1 and 16 above, and further in view of Haik et al. (US 2020/0133182 A1).
Re claims 16 – 17:
Labine does not explicitly disclose comparing students’ hand-written answers. Haik teaches a monitoring device includes circuitry to compare a printed output with a reference representing a target output and to determine potential defects in the printed output based on the comparison (Haik, Abstract). Haik teaches 16. The method of claim 1, further comprising comparing the student's hand-written answer to other students' hand-written answers to authenticate authorship. 17. The method of claim 16, wherein the hand-written answers are compared using a Siamese Neural Network (Haik, [0090]). Therefore, in view of Haik, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the method described in Labine, by comparing handwriting using SNN as taught by Haik, since siamese networks may be suitable for analyzing two handwriting samples to determine whether or not they were written by the same person (Haik, [0090]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACK YIP whose telephone number is (571)270-5048. The examiner can normally be reached Monday thru Friday; 9:00 AM - 5:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, XUAN THAI can be reached at (571) 272-7147. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACK YIP/Primary Examiner, Art Unit 3715