DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/09/2026 has been entered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
SECOND NON-FINAL OFFICE ACTION
Responsive to the Amendment/Remarks filed 09/03/2026, filed in response to the Non-Final Office Action mailed 06/03/2026, the following is a Second Non-Final Office Action on the merits. This action is made non-final because it introduces new grounds of rejection under 35 U.S.C. § 112(b) and 35 U.S.C. § 103 that are not necessitated by any amendment to the claims. See MPEP § 706.07(a).
Status of Claims
Claims 1–15 are pending. Claims 1–14 stand rejected and have been examined on the merits. Claim 15 remains withdrawn from consideration pursuant to 37 C.F.R. § 1.142(b) as directed to a non-elected invention (Group II). No claims have been amended in the Remarks. This action is in reply to the papers filed on 09/03/2026 (effective filing date 10/23/2019).
Information Disclosure Statement
The information disclosure statement(s) submitted: 03/04/2024, has/have been considered by the Examiner and made of record in the application file.
Amendment
The present Office Action is based upon the original patent application filed on 03/04/2024 as modified by the amendment filed on 09/03/2026.
Election by Original Presentation — Maintained
Applicant maintains the traversal of the restriction requirement between Group I (claims 1–14) and Group II (claim 15), as set forth at pages 6–7 of the response filed 03/09/2026, and incorporated by reference into the Remarks. Applicant argues that (i) separate utility has not been shown, and (ii) no serious search or examination burden has been shown and further argues that the prior Office Action carried the requirement forward solely by reference to the Final Rejection dated 12/08/2025 without addressing either point.
The Examiner has considered the traversal and maintains the restriction requirement for the following reasons.
Separate utility: Group I is directed to training a model, calculating uncertain rank counts and estimated error rates across successive model builds, determining whether those measures satisfy stability conditions, and transmitting an indication that a stopping point has been reached. The transmitted indication of Group I has utility independent of any subsequent elusion test — for example, to trigger a downstream workflow such as closing a review queue or reallocating reviewer resources, without any elusion test ever being conducted. Group II, by contrast, recites additional and different steps not required by Group I, including receiving a corpus of electronic documents, executing the trained model to assign a relevance rank to each document of the corpus, and tracking and periodically updating the model as documents are coded, and is directed to displaying an indication “for conducting an elusion test” — a specific downstream validation procedure neither recited nor required by Group I. Each group accordingly has utility apart from the other.
Search and examination burden: Group I is properly searched in art directed to computing and cross-build comparison of a trained model's own error and uncertainty statistics. Group II additionally requires a search of art directed to corpus-wide relevance ranking and display-based elusion-test workflows, which is not required to examine Group I. Because Group II's distinguishing limitations raise a materially different technical question from Group I's, searching and examining both together would impose a serious search and examination burden within the meaning of MPEP § 806.05(d).
For these reasons, the restriction requirement is maintained and claim 15 remains withdrawn.
Double Patenting — Maintained, Held in Abeyance
Claims 1–14 remain rejected under the judicially created doctrine of obviousness-type double patenting as unpatentable over claims 1–20 of U.S. Patent No. 11,921,568, of which this application is a continuation. Applicant has indicated its intent to file a terminal disclaimer if the claims are found otherwise allowable. Because this rejection turns solely on common ownership and overlapping claim scope rather than on patentability over the prior art, the Examiner agrees to hold it in abeyance pending resolution of the rejections below. A terminal disclaimer under 37 C.F.R. § 1.321(c) will be required before the application may be passed to issue.
Terminal Disclaimer
The terminal disclaimer filed on xxx disclaiming the terminal portion of any patent granted on this application which would extend beyond the expiration date of US Pat. No. xxxx has been reviewed and has been placed in the file.
Examiner acknowledges Applicant’s filed Terminal Disclaimer to prior art patent McCauley et al. US Pat. No. 5,930,775. A terminal disclaimer may be filed to overcome or obviate a nonstatutory double patenting rejection (37 CFR 1.321; MPEP 706.02; 1490).
Double Patenting - Withdrawn
The double patenting rejection is withdrawn per the filed terminal disclaimer noted above.
Reasons For Allowance
Prior-Art Rejection withdrawn
Claims xxx are allowed. The closest prior art (See PTO-892, Notice of References Cited) does not teach the claimed:
The closest prior-art (xxx) teach the features as disclosed in Non-final Rejection (xxxx), however, these cited references do not teach and the prior-art does not teach at least the following:
Claim Rejections - 35 USC §101 - Withdrawn
Per Applicant’s amendments and arguments and considering new guidance in the MPEP, the rejections are withdrawn. Specifically, in Applicant’s Remarks (dated 03/14/2017, pgs. 8-11), Applicant traverses the 35 USC §101 rejections arguing that the amended claims recite new limitations that are not abstract, amount to significantly more, are directed to a practical application, etc… For example, Applicant argues….
In support of their arguments, Applicant cites to the following recent Fed. Cir. court cases (i.e., Alice Corp. v. CLS Bank Int’l, SRI Int’l, Inc. v. Cisco Systems, Inc., Ultramercial, Inc. v. Hulu, LLC, Berkheimer, Core Wireless, McRO, Enfish, Bascom, DDR, etc…).
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-14 are rejected on the ground of anticipatory-nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 11,921,568.
18/595,261 – Claim 1. (Currently Amended) A computer-implemented method for identifying a stopping point of a machine learning-assisted review process, comprising:
US 11,921,568 – Claim 1. A computer-implemented method for identifying a stopping point of an active learning process, comprising:
18/595,261 – Claim 1. calculating, using a machine learning model, a first uncertain rank count and a second uncertain rank count; and
US 11,921,568 – Claim 1. calculating a first estimated error rate, a second estimated error rate, a first uncertain rank count and a second uncertain rank count; and
18/595,261 – Claim 1. based on a first estimated error rate, a second estimated error rate, the first uncertain rank count, the second uncertain rank count, and a target error rate, displaying, in a display of a computing device, an indication that the stopping point has been reached.
US 11,921,568 – Claim 1. based on the first estimated error rate, the second estimated error rate, the first uncertain rank count, the second uncertain rank count, and a target error rate, displaying, in a display of a computing device, an indication that the stopping point has been reached.
18/595,261 – Claim 2. (Currently Amended) The computer-implemented method of claim 1 further comprising calculating the first estimated error rate, wherein calculating the first estimated error rate includes receiving a coverage review indication from a user.
US 11,921,568 – Claim 2. The computer-implemented method of claim 1, wherein calculating the first estimated error rate includes receiving a coverage review indication from a user.
18/595,261 – Claim 3. (Currently Amended) The computer-implemented method of claim 1 further comprising calculating the first estimated error rate, wherein calculating the first estimated error rate includes determining whether a user has coded a minimum number of documents.
US 11,921,568 – Claim 3. The computer-implemented method of claim 1, wherein calculating the first estimated error rate includes determining whether a user has coded a minimum number of documents.
18/595,261 – Claim 4. (Currently Amended) The computer-implemented method of claim 1 further comprising calculating the first estimated error rate, wherein calculating the first estimated error rate includes determining whether a user has coded a minimum number of document groups.
US 11,921,568 – Claim 4. The computer-implemented method of claim 1, wherein calculating the first estimated error rate includes determining whether a user has coded a minimum number of document groups.
18/595,261 – Claim 5. (Original) The computer-implemented method of claim 1, wherein the target error rate is a configurable constant.
US 11,921,568 – Claim 5. The computer-implemented method of claim 1, wherein the target error rate is a configurable constant.
18/595,261 – Claim 6. (Original) The computer-implemented method of claim 1, wherein calculating the first uncertain rank count and the second uncertain rank count includes comparing uncertain rank counts across a configurable number of previous builds.
US 11,921,568 – Claim 6. The computer-implemented method of claim 1, wherein calculating the first uncertain rank count and the second uncertain rank count includes comparing uncertain rank counts across a configurable number of previous builds.
18/595,261 – Claim 7. (Original) The computer-implemented method of claim 1, wherein displaying, in the display of the computing device, the indication that the stopping point has been reached includes generating a message indicating that the stopping point has been reached; and transmitting the message via one or both of (i) a push message, and (ii) an email message.
US 11,921,568 – Claim 7. The computer-implemented method of claim 1, wherein displaying, in the display of the computing device, the indication that the stopping point has been reached includes generating a message indicating that the stopping point has been reached; and transmitting the message via one or both of (i) a push message, and (ii) an email message.
18/595,261 – Claim 8. (Currently Amended) A computing system for determining a stopping point of a machine learning process, comprising: one or more processors; and a memory storing instructions that, when executed, cause the computing system to: calculate, using a machine learning model, a first uncertain rank count and a second uncertain rank count; and based on a first estimated error rate, a second estimated error rate, the first uncertain rank count, the second uncertain rank count, and a target error rate, display, in a display of a computing device, an indication that the stopping point has been reached.
US 11,921,568 – Claim 8. A computing system for determining a stopping point of an active learning process, comprising: one or more processors; and a memory storing instructions that, when executed, cause the computing system to: calculate a first estimated error rate, a second estimated error rate, a first uncertain rank count and a second uncertain rank count; and based on the first estimated error rate, the second estimated error rate, the first uncertain rank count, the second uncertain rank count, and a target error rate, display, in a display of a computing device, an indication that the stopping point has been reached.
18/595,261 – Claim 9. (Original) The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: receive a coverage review indication from a user.
US 11,921,568 – Claim 9. The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: receive a coverage review indication from a user.
18/595,261 – Claim 10. (Original) The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: determine whether a user has coded a minimum number of documents.
US 11,921,568 – Claim 10. The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: determine whether a user has coded a minimum number of documents.
18/595,261 – Claim 11. (Original) The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: determine whether a user has coded a minimum number of document groups.
US 11,921,568 – Claim 11. The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: determine whether a user has coded a minimum number of document groups.
18/595,261 – Claim 12. (Original) The computing system of claim 8, wherein the target error rate is a configurable constant.
US 11,921,568 – Claim 12. The computing system of claim 8, wherein the target error rate is a configurable constant.
18/595,261 – Claim 13. (Original) The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: compare uncertain rank counts across a configurable number of previous builds.
US 11,921,568 – Claim 13. The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: compare uncertain rank counts across a configurable number of previous builds.
18/595,261 – Claim 14. (Original) The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: generate a message indicating that the stopping point has been reached; and transmit the message via one or both of (i) a push message, and (ii) an email message.
US 11,921,568 – Claim 14. The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: generate a message indicating that the stopping point has been reached; and transmit the message via one or both of (i) a push message, and (ii) an email message.
The remaining independent claims contain feature like that of claim 1 and are rejected accordingly. The dependent claims are further rejected for their dependency upon a rejected independent base claim.
Claim Rejections — 35 U.S.C. § 101 (Maintained)
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1–14 remain rejected under 35 U.S.C. § 101 because the claimed invention is directed to a judicial exception (an abstract idea) without significantly more. Applicant's arguments have been fully considered but are not persuasive, for the reasons below. This rejection applies the two-part framework of Alice Corp. v. CLS Bank International, 573 U.S. 208 (2014), and Mayo Collaborative Services v. Prometheus Laboratories, Inc., 566 U.S. 66 (2012), as set out in MPEP § 2106, including the current guidance at MPEP § 2106.04(d) as revised by the Advance Notice of Change to the MPEP in Light of Ex Parte Desjardins (Dec. 5, 2025).
Step 1 — Statutory Category
Claims 1–7 are directed to a process, and claims 8–14 are directed to a machine (a computing system comprising one or more processors and a memory), each a statutory category under 35 U.S.C. § 101. The analysis proceeds to Step 2A for each claim.
Claim 1 – Step 2A, Prong One — Does the Claim Recite a Judicial Exception?
The Examiner has considered Applicant's argument that the “determining” limitation does not fall within the mental processes grouping because it operates on quantities — classification predictions and per-build counts generated by a trained machine learning model — that the human mind is not equipped to produce. This argument is persuasive as far as it goes, and the Examiner withdraws reliance on the mental processes grouping as an independent basis for the Prong One finding.
Withdrawal of the mental-processes ground does not change the outcome, because Prong One remains independently satisfied under the mathematical concepts grouping. Applicant argues that “the mathematical concept itself is not recited in the claim” because the specific formula of Specification paragraph [0053] does not appear in claim 1. This is not persuasive. MPEP § 2106.04(a)(2) excludes a limitation from the mathematical concepts grouping only where the claim may be performed without ever carrying out the mathematical operation — not merely because the claim omits a specific equation. Claim 1 requires “deriving … a first estimated error rate … and a second estimated error rate,” which cannot be performed without a numerical comparison of the model's classification predictions against document coding values, and requires “determining whether” that rate has “remained below a target error rate” and whether counts “are not increasing,” which are themselves numerical comparisons. That the claim does not recite the specific fractional formula of paragraph [0053] does not remove these limitations from the mathematical concepts grouping. See Parker v. Flook, 437 U.S. 584, 594–95 (1978) (claim ineligible where its point of novelty was an algorithm for updating an alarm-limit value, notwithstanding that the claim recited a process rather than a bare formula); SAP America, Inc. v. InvestPic, LLC, 898 F.3d 1161, 1163, 1167 (Fed. Cir. 2018) (series of steps for selecting, analyzing, and manipulating data using statistical methods held abstract even though no single claimed step recited a formula). Claim 1 accordingly recites a judicial exception under the mathematical concepts grouping, and Step 2A, Prong One is satisfied on that basis.
Claim 1 – Step 2A, Prong Two — Practical Application
The Examiner has considered Applicant's extensive argument that claim 1 integrates the abstract idea into a practical application under the two-part inquiry of the Advance Notice of Change to the MPEP in Light of Ex parte Desjardins (Dec. 5, 2025): (i) whether the Specification discloses sufficient detail that a person of ordinary skill in the art (“POSITA”) would recognize the claimed invention as an improvement, and (ii) whether the claim recites the components or steps that provide that improvement. The Examiner does not dispute that the Specification identifies a genuine technical problem: repeatedly re-running an elusion test at the wrong time consumes computational and storage resources through repeated sampling, computation, and reprocessing of, in some instances, a thousand or more documents. Specification ¶¶ [0006-0008; 0056]. The dispositive question is whether claim 1 recites the components or steps that provide a solution comparable to the claims held eligible in Ex parte Desjardins.
In Desjardins, the claims recited a specific training methodology — computing an approximation of a posterior distribution over parameter values, determining importance measures for each parameter relative to a first task, and training on a second task using those importance measures as a penalty term protecting the first task's performance — that itself changed how the model was trained, allowing a single model instance to replace what conventional approaches required as multiple stored models. The Appeals Review Panel credited “reduced storage requirements,” “preserved sequential task performance,” and “reduced system complexity” because those benefits flowed directly from the claimed training mechanism. The claimed improvement was, in the Panel's words, “an improvement to how the machine learning model itself operates” — and that improvement was recited, not merely described and left unclaimed.
Claim 1 does not recite a comparable mechanism. Its “training” step requires only “training, by a computing device and based on one or more coding decisions, a machine learning model to predict relevance ranks for documents in a corpus of electronic documents” — a conventional supervised-training step described in the Specification in generic terms. Specification ¶¶ [0086-0089] (“adjust weights of a machine learning model such as an artificial neural network,” “iteratively training the network using labeled training samples,” reducing loss to converge to “learned” values). No penalty term, no parameter-protection mechanism, and no other departure from conventional training is recited or required. What claim 1 recites beyond training is a sequence of measurements taken of that conventionally trained model's outputs — an uncertain rank count and an estimated error rate, each compared across builds against configurable thresholds — followed by transmission of the result. Unlike Desjardins, where the claimed steps changed the training process and thereby reduced the number of stored models, claim 1's steps do not change how the model is trained, structured, or what it stores; they observe and report on a conventionally trained model after the fact.
The computational savings the Specification identifies — avoiding the storage, retrieval, and reprocessing cost of an unnecessary elusion-test cycle, Specification ¶ [0056] is a consequence of making a correctly timed decision to stop reviewing, not a consequence of any change to the model, its training, or its architecture. This places the claimed advance closer to Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354 (Fed. Cir. 2016) (claims to gathering, analyzing, and presenting information about a power grid held ineligible because “the advance they purport to make is a process of gathering and analyzing information … and merely presenting the results”), and SAP America, 898 F.3d at 1167 (“[E]ven if a process of collecting and analyzing information is limited to particular content … that limitation does not make the collection and analysis other than abstract.”), than to Desjardins. Limiting the measurement to a “trained machine learning model” narrows the field of use to machine-learning-monitored review processes; it does not change the character of the advance from “make a better-timed decision using calculated statistics” to “improve how the model itself operates.”
As to the individual additional-element characterizations Applicant disputes: the Examiner agrees that the training step is not incidental or tangential — it supplies the input to every later step — and withdraws its characterization as insignificant extra-solution activity under MPEP § 2106.05(g). This does not change the result, because a significant-but-conventional step does not by itself integrate an abstract idea into a practical application. The Examiner also agrees that “transmitting” is not “displaying” and withdraws reliance on display-specific authority; on reconsideration, transmitting an indication that a stopping point has been reached nonetheless remains insignificant post-solution activity, because it does no more than communicate the outcome of the preceding determination without any further recited processing of that outcome. See Electric Power Group, 830 F.3d at 1354; TLI Communications LLC v. AV Automotive, L.L.C., 823 F.3d 607, 613–14 (Fed. Cir. 2016) (transmission of data is well-understood extra-solution activity). The Examiner further agrees the corpus of electronic documents is not a mere field-of-use label, since it is the source of the data operated on, and withdraws that characterization; this likewise does not change the result, because identifying the source of the data operated on by an abstract calculation does not itself integrate the calculation into a practical application — it confines the abstract idea to a field of use. See Affinity Labs of Tex., LLC v. DIRECTV, LLC, 838 F.3d 1253, 1258–59 (Fed. Cir. 2016).
For the foregoing reasons, claim 1 does not integrate the recited judicial exception into a practical application, and the rejection is maintained under Step 2A, Prong Two.
Claim 1 – Step 2B — Inventive Concept
Applicant argues that the additional elements have not been evaluated in combination. The Examiner has reconsidered the additional elements of claim 1 as an ordered combination: a computing device trains a conventional machine learning model on conventional supervised-training inputs; the trained model is queried, in conventional fashion, for per-document classification outputs from which the recited counts and rates are computed; those computed quantities are compared, using ordinary numerical comparison, against thresholds and against their own prior-build values; and the result is transmitted using conventional data transmission. This is the ordinary combination of (i) applying a conventionally trained model to generate outputs, (ii) performing mathematical/statistical calculations on those outputs, and (iii) communicating the result, each performed in its conventional order for its conventional purpose. No technical interaction among the training, calculating, deriving, determining, and transmitting steps beyond this ordinary sequential data flow has been identified. See BSG Tech LLC v. Buyseasons, Inc., 899 F.3d 1281, 1290–91 (Fed. Cir. 2018). The rejection is maintained under Step 2B.
Claim 8
Claim 8 recites the system counterpart of claim 1: a computing system comprising one or more processors and a memory storing instructions that, when executed, cause the system to train, calculate, derive, determine, and transmit in language corresponding limitation-for-limitation to claim 1's method steps. The generic “one or more processors” and “a memory” are conventional computer components that do not add significantly more, and Applicant's arguments as to claim 8 track its arguments as to claim 1 without additional distinction. For the same reasons set forth above as to claim 1 — Step 1 (statutory machine), Prong One (mathematical concepts grouping), Prong Two (no practical-application integration; Desjardins distinguished), and Step 2B (ordered combination of conventional steps) — claim 8 remains rejected under 35 U.S.C. § 101.
Dependent Claims
Claims 2 and 9 add “receiving a coverage review indication from a user.” This limitation does not recite an additional judicial exception; it adds a user-supplied toggle input. At Prong Two, the Specification describes this indication as “a server computer collecting a user indication (e.g., from a client device) wherein the indication toggles a coverage review flag.” Specification ¶ [0051]. Receiving a flag that changes which documents are subsequently measured does not change how the model is trained or how the recited rate or count is calculated; it is, at most, insignificant extra-solution activity in the nature of a data-gathering input selection. See MPEP § 2106.05(g). At Step 2B, a conventional user-toggled input adds nothing beyond the ordinary combination already addressed as to claim 1. Claims 2 and 9 remain rejected.
Claims 3 and 10 add determining “whether a user has coded a minimum number of documents,” and claims 4 and 11 add the corresponding determination as to “a minimum number of document groups.” Each is a condition precedent — a numerical comparison of a count to a threshold — gating when the already-abstract error-rate calculation is performed. A gating condition on when an abstract calculation is carried out does not change what is calculated or how, and is not, by itself, a technical means of implementing the calculation. At Step 2B, these gating comparisons add only further conventional numerical comparison to the ordered combination already addressed. Claims 3, 4, 10, and 11 remain rejected.
Claims 5 and 12 add that “the target error rate is a configurable constant.” A user-adjustable numerical parameter remains a numerical parameter; configurability of a threshold used in an abstract comparison is not a technical means of implementation and does not integrate the abstract idea into a practical application. See In re Katz Interactive Call Processing Patent Litig., 639 F.3d 1303, 1316 (Fed. Cir. 2011) (generic customizability does not confer eligibility). Claims 5 and 12 remain rejected.
Claims 6 and 13 add that calculating the uncertain rank counts “includes comparing uncertain rank counts across a configurable number of previous builds.” This is part of the same mathematical/statistical comparison already addressed as to claim 1 — comparing a computed quantity across a window of iterations — merely specifying that the window size is itself configurable. Configurability of a comparison window does not add a technical means of implementation. Claims 6 and 13 remain rejected.
Claims 7 and 14 add “generating a message indicating that the stopping point has been reached; and transmitting the message via one or both of (i) a push message, and (ii) an email message.” Generating a message that restates the result of the abstract determination, and transmitting it over one of two conventional, generic communication channels, remains insignificant post-solution activity: it communicates the outcome without any further recited processing of, or reliance on, that outcome. See Electric Power Group, 830 F.3d at 1354; TLI Communications, 823 F.3d at 611 (specifying a generic communication channel for transmitting data does not integrate an abstract idea into a practical application). Selecting between two well-known, generic notification channels (push or email) does not supply a technical means of solving the computational-waste problem the Specification identifies. Claims 7 and 14 remain rejected.
Conclusion — § 101
For the foregoing reasons, claims 1–14 remain rejected under 35 U.S.C. § 101. To advance prosecution, Applicant may consider amending the claims to recite a specific, non-conventional training methodology or model architecture that itself produces the disclosed improvement (analogous to the penalty-term training mechanism in Desjardins), rather than a conventionally trained model whose outputs are subsequently monitored.
Claim Rejections — 35 U.S.C. § 112(b) – Indefiniteness (New Ground)
The following is a quotation of 35 U.S.C. §112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1–14 are rejected under 35 U.S.C. § 112(b) as indefinite for failing to particularly point out and distinctly claim the subject matter that the inventor regards as the invention.
Claim 1 recites determining “whether the first estimated error rate and the second estimated error rate have each remained below a target error rate for at least a predetermined number of sequential model builds, and whether the first uncertain rank count and the second uncertain rank count are not increasing across a predetermined number of previous builds.” Claim 1 uses two different phrases — “a predetermined number of sequential model builds” and “a predetermined number of previous builds” — each introduced with the indefinite article “a,” apparently to describe what the Specification discloses as a single window of builds over which both conditions are evaluated together. See Specification ¶ [0051] (describing both conditions evaluated together across, e.g., “three successive builds”). It is unclear whether the two phrases refer to the same number and window of builds, to independently configurable windows that may differ in size, or to windows measured from different reference points. Claim 8 recites the corresponding limitation and is rejected for the same reason. A POSITA reading claim 1 in light of the Specification would not be able to determine, with reasonable certainty, whether the two recited windows must coincide or may differ. See Nautilus, Inc. v. Biosig Instruments, Inc., 572 U.S. 898, 901 (2014). Claims 2–7 and 9–14 are rejected by virtue of their dependency from claim 1 or claim 8.
To overcome this rejection, Applicant may amend claim 1 (and claim 8) to recite a single “predetermined number of model builds” against which both conditions are evaluated, consistent with Specification paragraph [0051], or, if the two windows are intended to differ, may recite each with sufficient structure to make clear that they are separately and independently configured.
Claim Rejections — 35 U.S.C. § 103 (New Ground)
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 2, 3, 6, 8, 9, 10, 13 are rejected under 35 U.S.C. 103 as being unpatentable over: Johnson et al., US 2017/0116519 A1 (“Johnson”); in view of Naslund et al., US 2012/0278266 A1 (“Naslund”).
Claims 1–14 are rejected under 35 U.S.C. § 103 as being unpatentable over Johnson et al., US 2017/0116519 A1 (“Johnson”) as a primary reference, in view of one or more of Cormack et al., US 2016/0371262 A1 (“Cormack”); Naslund et al., US 2012/0278266 A1 (“Naslund”); and Sattler et al., US 2006/0010025 A1 (“Sattler”), as set forth claim-by-claim below.
Prior Art of Record
Johnson (US 2017/0116519 A1), published April 27, 2017, first-named inventor Jeffrey A. Johnson, assigned to ControlDocs.com, Inc.: discloses apparatuses and methods for batch-mode active learning in technology-assisted review (TAR) of documents, in which a classifier is iteratively trained on reviewer-coded documents to score and rank documents for relevance, and in which a generalized stopping-criteria technique computes a per-iteration stability score — in one embodiment, a Cohen's Kappa agreement value comparing predicted labels across successive iterations — and signals to the user once that score meets or exceeds a specified threshold for a number of consecutive iterations that the process has stabilized and further training is unnecessary.
Cormack (US 2016/0371262 A1), published December 22, 2016, first-named inventor/applicant Gordon V. Cormack: discloses a Scalable Continuous Active Learning (S-CAL) approach to TAR in which a classifier is iteratively retrained on a growing training set drawn in successive batches, the prevalence of relevant documents is estimated from the results of successive iterations, and a classification threshold is calculated — in certain disclosed embodiments, using a targeted level of recall — once a stopping criterion is reached.
Naslund (US 2012/0278266 A1), published November 1, 2012, first-named inventor Jeffrey David Naslund, assigned to Kroll Ontrack, Inc.: discloses a document review system in which an artificial-intelligence model scores documents, and in which some documents are pulled based on that ranking while other, randomly-selected documents are interspersed into the set of documents served to a reviewer, without differentiating the randomly-selected documents from the ranked ones, for quality-assurance purposes; and further discloses withholding reliance on the model's determinations until a minimum number of coded documents has been reached (e.g., “three or more documents from each class in the class distinction”).
Sattler (US 2006/0010025 A1), published January 12, 2006, first-named inventor Juergen Sattler, assigned to SAP Aktiengesellschaft: discloses a workflow management system in which a workflow engine, responsive to a notification that a current task has been completed, automatically identifies the next task owner and causes an e-mail server to generate and transmit an e-mail notification to that owner.
18/595,261 – Claim 1. (Previously Presented) Johnson teaches A computer-implemented method for identifying a stopping point of a machine learning-assisted review process (Johnson ¶0007 “determination that the stopping criteria has not been met, repeat the perform operation, the third construct operation, and the determine operation … based on the determination that the stopping criteria has been met”), comprising: training, by a computing device and based on one or more coding decisions, a machine learning model to predict relevance ranks for documents in a corpus of electronic documents (Johnson, Abstract; ¶0030 “need for technology-assisted review (TAR) and the development of “predictive coding” software. In a traditional linear review, an attorney who is an expert in the subject matter trains a group of contract attorneys or junior associates so that they can churn through the documents for the weeks or months that it may take to complete the review … objective of predictive coding is to design a machine-learning based system that labels documents as relevant or non-relevant to a specific issue or issues, and hence, minimizes the review-cost and time by maximizing the focus on the relevant documents…”; ¶¶0037–0038 (active learning builds a learning model, updated iteratively as labeled documents are received); ¶0106 (“At each iteration of a learning process, the current classifier is used to generate a score for every document profile in the entire collection.”)); calculating, using the trained machine learning model, a first uncertain rank count for a first model build and a second uncertain rank count for a second model build (Johnson ¶0107 (“the number of matching predicted labels between each iteration is counted and adjusted for the number of matching labels that would be expected to match … by chance”). Examiner's claim interpretation: under BRI, a per-iteration count reflecting the instability/uncertainty of the model's rank predictions between builds reads on the recited “uncertain rank count.”); deriving, based upon classification predictions of the trained machine learning model and document coding values, a first estimated error rate for a first set of recently coded documents and a second estimated error rate for a second set of recently coded documents (Johnson ¶¶0106–0107 (per-iteration comparison of classification predictions) in view of Johnson's recall computation “over the held-back test dataset using the final model.” Examiner's claim interpretation: a rate derived by comparing the trained model's predictions against actual/held-out coding values reads on the recited “estimated error rate.”; ¶0042 “when the max confidence, min-error, overall uncertainty, and combination of these three reaches a certain threshold [references 23, 24 and 25]; (4) when the entropy of each selected sample or error on prediction is less than a threshold…”); determining, by the computing device, whether the first estimated error rate and the second estimated error rate have each remained below a target error rate for at least a predetermined number of sequential model builds, and whether the first uncertain rank count and the second uncertain rank count are not increasing across a predetermined number of previous builds (Johnson ¶0108 (“At each iteration the stability score is checked to see whether it meets or exceeds a certain specified threshold. Once the threshold has been met or exceeded, the system signals …”); ¶0059 (“the training is stopped when the K reaches >0.991 for several consecutive iterations”).); and transmitting an indication that the stopping point has been reached (Johnson ¶0108 (“the system signals to the user that it has stabilized and further iterations of the learning process are unnecessary”).).
Johnson may not expressly disclose the “machine learning model to predict relevance ranks for documents” features, however, Naslund teaches (Naslund ¶0040 “artificial intelligence techniques are used to train a predictive model based on the labels provided for each document. A wide range of artificial intelligence techniques can be used, particularly ones based on supervised learning (learning from labeled units of data). In one example, the model is a logistic regression model whose output is an estimate of the probability a document belongs to each of the possible classes in a class distinction (e.g., whether or not a document is responsive), and which is trained by finding the model coefficients that approximately maximize the likelihood of observing the class labels given the input features of the labeled training documents”). Johnson and Naslund are both directed to the identical technical problem of selecting which documents, at each iteration of an active-learning document-review process, to serve to a human reviewer. It would have been obvious to a person of ordinary skill in the art, at the time of the invention, to incorporate Naslund's machine learning features into Johnson’s technology-assisted review (TAR) of documents application to improve the user experience by implementing well known features and tools useful in document review.
18/595,261 – Claim 8. (Previously Presented) Johnson teaches A computing system for determining a stopping point of a machine learning process, comprising: one or more processors (Johnson ¶¶ 0007; 0009; 0116; claim 1); and a memory storing instructions that, when executed, cause the computing system to (Johnson ¶¶ 0007; 0009; 0116; claim 1): train, based on one or more coding decisions, a machine learning model to predict relevance ranks for documents in a corpus of electronic documents (Johnson, Abstract; ¶¶0037–0038 (active learning builds a learning model, updated iteratively as labeled documents are received); ¶0106 (“At each iteration of a learning process, the current classifier is used to generate a score for every document profile in the entire collection.”)); calculate, using the trained machine learning model, a first uncertain rank count for a first model build and a second uncertain rank count for a second model build (Johnson ¶0107 (“the number of matching predicted labels between each iteration is counted and adjusted for the number of matching labels that would be expected to match … by chance”). Examiner's claim interpretation: under BRI, a per-iteration count reflecting the instability/uncertainty of the model's rank predictions between builds reads on the recited “uncertain rank count.”); derive, based upon classification predictions of the trained machine learning model and document coding values, a first estimated error rate for a first set of recently coded documents and a second estimated error rate for a second set of recently coded documents (Johnson ¶¶0106–0107 (per-iteration comparison of classification predictions) in view of Johnson's recall computation “over the held-back test dataset using the final model.” Examiner's claim interpretation: a rate derived by comparing the trained model's predictions against actual/held-out coding values reads on the recited “estimated error rate.”); determine whether the first estimated error rate and the second estimated error rate have each remained below a target error rate for at least a predetermined number of sequential model builds, and whether the first uncertain rank count and the second uncertain rank count are not increasing across a predetermined number of previous builds (Johnson ¶0108 (“At each iteration the stability score is checked to see whether it meets or exceeds a certain specified threshold. Once the threshold has been met or exceeded, the system signals …”); ¶0059 (“the training is stopped when the K reaches >0.991 for several consecutive iterations”).); and transmit an indication that the stopping point has been reached (Johnson ¶0108 (“the system signals to the user that it has stabilized and further iterations of the learning process are unnecessary”).).
Johnson may not expressly disclose the “machine learning model to predict relevance ranks for documents” features, however, Naslund teaches (Naslund ¶0040 “artificial intelligence techniques are used to train a predictive model based on the labels provided for each document. A wide range of artificial intelligence techniques can be used, particularly ones based on supervised learning (learning from labeled units of data). In one example, the model is a logistic regression model whose output is an estimate of the probability a document belongs to each of the possible classes in a class distinction (e.g., whether or not a document is responsive), and which is trained by finding the model coefficients that approximately maximize the likelihood of observing the class labels given the input features of the labeled training documents”). Johnson and Naslund are both directed to the identical technical problem of selecting which documents, at each iteration of an active-learning document-review process, to serve to a human reviewer. It would have been obvious to a person of ordinary skill in the art, at the time of the invention, to incorporate Naslund's machine learning features into Johnson’s technology-assisted review (TAR) of documents application to improve the user experience by implementing well known features and tools useful in document review.
The remaining features/limitations of Claim 8, have similar features/limitations as of Claim 1, therefore those features/limitations and the claims are REJECTED under the same rationale as Claim 1.
18/595,261 – Claim 2. (Previously Presented) Johnson further teaches The computer-implemented method of claim 1, wherein calculating the first estimated error rate includes receiving a coverage review indication from a user (Johnson ¶0005 technology-assisted review (TAR); ¶0042 error).
Johnson may not expressly disclose the “coverage review indication” features, however, Naslund teaches (Naslund ¶0052 (“… some documents can be pulled based on a ranking[.] Other documents, such as randomly selected documents, can be interspersed into the set of documents pulled based on ranking …”). Examiner's claim interpretation: a signal that switches the served document set from a purely ranked composition to this alternate, quality-control-oriented composition reads on the recited “coverage review indication.”). Johnson and Naslund are both directed to the identical technical problem of selecting which documents, at each iteration of an active-learning document-review process, to serve to a human reviewer. It would have been obvious to a person of ordinary skill in the art, at the time of the invention, to incorporate Naslund's alternate, quality-control-oriented document-selection mode — in which the served set is drawn to include interspersed, non-purely-ranked documents rather than only the highest-scoring documents — as a user-selectable input into Johnson's iterative active-learning apparatus, because doing so is the application of a known review-mode-selection technique (Naslund) to a known active-learning document-review apparatus (Johnson) ready for the improvement, yielding the predictable result of a system offering the reviewer a choice between Johnson's standard ranked/uncertainty-based queue and Naslund's alternate coverage/quality-control-oriented queue. See MPEP § 2143(G).
18/595,261 – Claim 9. (Original) The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: receive a coverage review indication from a user ().
Claim 9, has similar limitations as of Claim 2, therefore it is REJECTED under the same rationale as Claim 2.
18/595,261 – Claim 3. (Previously Presented) Johnson further teaches The computer-implemented method of claim 1, wherein calculating the first estimated error rate includes determining whether a user has coded a minimum number of documents (Johnson ¶¶0007-0010 labeling a batch of documents (claim interpretation: labeling documents interpreted as coded or coding documents); ¶0031 “maximize the discovered relevant documents as measured by recall while minimizing the human labeling efforts as measured by the number of documents which are labeled by the attorneys”; ¶0068 “yield curve shows the relationship between recall and the minimum fraction of documents that must be reviewed to achieve that recall value”).
Johnson may not expressly disclose the “minimum number of documents” features, however, Naslund teaches (Naslund ¶0055 (“One rule for making this determination is to start using the model when three or more documents from each class in the class distinction were used in training.”).). It would have been obvious to a person of ordinary skill in the art to combine Naslund's minimum-document-count gating principle with Johnson's stopping-criteria calculation, because Naslund teaches that a model-derived determination should not be trusted until a minimum number of coded documents has been reached, Johnson's stopping-criteria calculation is itself a model-derived determination of the same general character, and applying Naslund's known gating technique to Johnson's known stopping-criteria apparatus would have yielded no more than the predictable result of gating Johnson's calculation on a minimum coded-document count. See MPEP § 2143(A), (G).
18/595,261 – Claim 10. (Original) The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: determine whether a user has coded a minimum number of documents ().
Claim 10, has similar limitations as of Claim 3, therefore it is REJECTED under the same rationale as Claim 3.
18/595,261 – Claim 6. (Original) Johnson further teaches The computer-implemented method of claim 1, wherein calculating the first uncertain rank count and the second uncertain rank count includes comparing uncertain rank counts across a configurable number of previous builds (Johnson ¶0108 (stability score evaluated against the threshold “for several consecutive iterations” before signaling).). Johnson ¶ 0108 discloses evaluating its stability score against the specified threshold “for several consecutive iterations” before signaling that the process has stabilized, i.e., a comparison of the relevant per-iteration metric across a settable number of consecutive builds.
18/595,261 – Claim 13. (Original) The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: compare uncertain rank counts across a configurable number of previous builds ().
Claim 13, has similar limitations as of Claim 6, therefore it is REJECTED under the same rationale as Claim 6.
Claims 4, 5, 11, 12 are rejected under 35 U.S.C. 103 as being unpatentable over: Johnson et al., US 2017/0116519 A1 (“Johnson”); in view of Naslund et al., US 2012/0278266 A1 (“Naslund”); in further view of Cormack et al., US 2016/0371262 A1 (“Cormack”).
18/595,261 – Claim 4. (Previously Presented) Johnson further teaches The computer-implemented method of claim 1, wherein calculating the first estimated error rate includes determining whether a user has coded a minimum number of document groups (Johnson ¶¶0007-0010 labeling a batch of documents (claim interpretation: labeling documents interpreted as coded or coding documents); ¶0031 “maximize the discovered relevant documents as measured by recall while minimizing the human labeling efforts as measured by the number of documents which are labeled by the attorneys”; ¶0068 “yield curve shows the relationship between recall and the minimum fraction of documents that must be reviewed to achieve that recall value”).
Johnson may not expressly disclose the “document group” features, however, Cormack and Naslund teach (Cormack, claim 1 (iterative selection of “a first batch size documents,” “a second batch size documents,” etc.) in view of Naslund ¶0055 (minimum-count gating principle). Examiner's claim interpretation: BRI reads Cormack's “batch” as a “group,” and extends Naslund's minimum-document gating principle to a minimum-batch/group count.). It would have been obvious to a person of ordinary skill in the art to extend the minimum-count gating rationale set forth immediately above from a minimum count of documents to a minimum count of document batches or groups, in further view of Cormack's disclosure that its active-learning process proceeds through discrete, iteratively-selected batches (“a first batch size,” “a second batch size”). Gating a model-derived calculation on a minimum number of completed batches, rather than on a raw document count, addresses the identical underlying concern — ensuring sufficient data has accumulated before the calculation is trusted — using a batch-oriented processing structure that Cormack already discloses as inherent to iterative active-learning review. This is a predictable variant of the combination discussed as to claims 3 and 10. See MPEP § 2143(A).
18/595,261 – Claim 11. (Original) The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: determine whether a user has coded a minimum number of document groups ().
Claim 11, has similar limitations as of Claim 4, therefore it is REJECTED under the same rationale as Claim 4.
18/595,261 – Claim 5. (Original) Johnson further teaches The computer-implemented method of claim 1, wherein the target error rate is a configurable constant (Johnson, Table #3, Table #4, Table#9 (stopping-criteria function accepts “t, threshold at which to stop” as an explicit input parameter) in view of Cormack, claim 11 (“wherein the threshold is calculated using a targeted level of recall”); ¶0021.). Johnson discloses its stopping-criteria function as accepting “t, threshold at which to stop” as an explicit input parameter, i.e., a settable rather than hard-coded value, but does not expressly state that this threshold is calibrated to a targeted performance metric. Cormack, in the same field of TAR stopping-criteria design, discloses that “the threshold is calculated using a targeted level of recall.” It would have been obvious to a person of ordinary skill in the art to configure Johnson's already-parameterized stopping threshold using Cormack's target-recall calibration technique, because both references address the identical technical problem of determining when to stop an iterative active-learning review, Johnson's own stopping-criteria function already accepts the threshold as a settable input, and applying Cormack's known threshold-calibration technique to Johnson's known, similarly-purposed stopping apparatus would have yielded no more than the predictable result of a configurable, performance-targeted stopping threshold. See MPEP § 2143(A), (G).
18/595,261 – Claim 12. (Original) The computing system of claim 8, wherein the target error rate is a configurable constant ().
Claim 12, has similar limitations as of Claim 5, therefore it is REJECTED under the same rationale as Claim 5.
Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over: Johnson et al., US 2017/0116519 A1 (“Johnson”); in view of Naslund et al., US 2012/0278266 A1 (“Naslund”); in further view of Sattler et al., US 2006/0010025 A1 (“Sattler”).
18/595,261 – Claim 7. (Previously Presented) Johnson further teaches The computer-implemented method of claim 1, wherein transmitting the indication that the stopping point has been reached includes: generating a message indicating that the stopping point has been reached; and transmitting the message via one or both of (i) a push message, and (ii) an email message (Johnson ¶0108 (message/signal generation upon the stopping condition being met) in view of (Sattler, ¶0013 (workflow engine that, upon completion of a task, automatically generates an e-mail notification and calls the e-mail service to transmit it to the next task owner) and claim 1 (workflow engine that, responsive to completion of a task, “cause[s] the email server to notify the next task owner” by e-mail).). Johnson and Sattler are both directed to notifying a user or workflow participant that a computational or process condition has been satisfied. It would have been obvious to a person of ordinary skill in the art to implement Johnson's stopping-point signal using Sattler's known e-mail-notification mechanism, because doing so is the application of a known communication technique (automatically generating and transmitting an e-mail notification upon satisfaction of a process condition, as Sattler discloses) to a known apparatus with a comparable need (Johnson's stopping-point signaling system), yielding the predictable result of transmitting Johnson's stopping-point message by e-mail. Because claim 7 (and claim 14) recite transmission via “one or both” of a push message and an e-mail message, Sattler's teaching of e-mail transmission alone is sufficient to satisfy this limitation under its broadest reasonable interpretation. See MPEP § 2143(G).
18/595,261 – Claim 14. (Original) The computing system of claim 8, the memory including further instructions that when executed, cause the computing system to: generate a message indicating that the stopping point has been reached; and transmit the message via one or both of (i) a push message, and (ii) an email message ().
Claim 14, has similar limitations as of Claim 7, therefore it is REJECTED under the same rationale as Claim 7.
Response to Arguments
Applicant's arguments, filed 09/03/2026, have been fully considered.
A. Election by Original Presentation
Applicant maintains the traversal of the restriction requirement between Group I and Group II, arguing that separate utility and a serious search or examination burden have not been shown, and that the prior Office Action carried the requirement forward by reference to the Final Rejection dated 12/08/2025 without addressing either point.
This argument is not persuasive. The rejection above sets forth a reasoned analysis on both separate utility (Group I's transmitted indication has utility independent of any elusion test) and search burden (Group I and Group II require materially different searches), directly answering the two errors Applicant identified in the prior Office Action's cross-reference-only treatment. For the reasons set forth above, the restriction requirement is maintained, and claim 15 remains withdrawn.
B. Double Patenting
Applicant states its intent to file a terminal disclaimer and requests that the double patenting rejection be held in abeyance until allowable subject matter is identified.
As set forth above, the Examiner agrees to hold the rejection in abeyance pending resolution of the rejections below, consistent with Applicant's request. Applicant is reminded that a terminal disclaimer under 37 C.F.R. § 1.321(c) will still be required before the application may be passed to issue.
C. Claim Rejections — 35 U.S.C. § 101
I. Step 2A, Prong One. Applicant argues that (i) claim 1's determining limitation does not fall within the mental processes grouping because the human mind cannot generate the recited classification predictions and per-build counts, and (ii) the mathematical concepts grouping does not apply because the specific formula of Specification paragraph [0053] is not recited in the claim.
The Examiner agrees with Applicant's first point and, as set forth in detail above (Claim 1 — Step 2A, Prong One), withdraws reliance on the mental processes grouping as an independent basis for the rejection. The Examiner does not agree with the second point. MPEP § 2106.04(a)(2) excludes a limitation from the mathematical concepts grouping only where the claim can be performed without ever carrying out the calculation — not merely because the claim omits the specific formula used to perform it. Because claim 1 cannot be performed without deriving a numerical error rate and comparing it, and the recited counts, to thresholds, Prong One remains independently satisfied on the mathematical concepts grouping. This argument is not persuasive.
II. Step 2A, Prong Two. Applicant argues at length, relying on Ex parte Desjardins and the revised two-part Prong Two inquiry, that claim 1 integrates the alleged abstract idea into a practical application because the Specification discloses a technical problem (computational waste from repeated review and elusion-test cycles) and a mechanism (measuring the trained model's own predictive accuracy across builds) that claim 1 recites. Applicant further argues that the Office Action mischaracterized the training step as data-gathering, the transmitting step as display-based post-solution activity, and the corpus as a field-of-use limitation.
This argument has been considered in full above (Claim 1 — Step 2A, Prong Two) and is not persuasive, notwithstanding that three of the individual element characterizations Applicant disputes are withdrawn on reconsideration: training is not incidental or tangential, transmitting is not displaying, and the corpus is not a mere field-of-use label. Withdrawing those three characterizations does not change the outcome. The rejection is maintained because claim 1, unlike the claims in Desjardins, does not recite any change to how the machine learning model is trained, structured, or what it stores — it recites a conventional training step followed by measurement of that conventionally trained model's outputs. The computational savings the Specification identifies flow from making a better-timed stopping decision, not from any improvement to the model or its training, which places the claimed advance closer to Electric Power Group, 830 F.3d at 1354, and SAP America, 898 F.3d at 1167, than to Desjardins. This argument is not persuasive.
III. Step 2B. Applicant argues that the Office Action evaluated the additional elements individually rather than as an ordered combination, and that no factual determination supported in writing establishes that the combination is well-understood, routine, and conventional.
As set forth above (Claim 1 — Step 2B), the Examiner has reconsidered the additional elements as an ordered combination — a conventionally trained model queried for outputs, those outputs subjected to ordinary numerical comparison against thresholds and prior-build values, and the result conventionally transmitted — and finds that this ordered combination reflects the sequential application of each step for its conventional purpose, without any further technical interaction among the steps beyond that ordinary data flow. See BSG Tech LLC v. Buyseasons, Inc., 899 F.3d 1281, 1290–91 (Fed. Cir. 2018). This argument is not persuasive.
IV. Conclusion (Applicant's Remarks). Applicant concludes that claims 1–14 are eligible for three independent reasons: that no judicial exception is recited, that the claims integrate any exception into a practical application, and that the additional elements amount to significantly more in combination.
Each of these three positions has been addressed and found not persuasive above, for the reasons set forth in the Claim Rejections — 35 U.S.C. § 101 section and summarized in this Response to Arguments. The rejection under 35 U.S.C. § 101 is maintained as to claims 1–14.
D. Prior Art and § 112(b) Indefiniteness
Applicant's Remarks do not address prior art under 35 U.S.C. § 103 or indefiniteness under 35 U.S.C. § 112(b), as neither ground was of record in the Office Action to which the Remarks respond. Both are newly introduced in this action. Applicant will have the opportunity to respond to these new grounds in a reply to this Office Action.
Conclusion
No claim is allowed. Claims 1–14 stand rejected as set forth above; claim 15 remains withdrawn. Applicant is reminded that a terminal disclaimer will be required to overcome the double patenting rejection held in abeyance.
Any inquiry concerning this communication should be directed to the Examiner identified below.
Conclusion
PERTINENT PRIOR ART – Patent Literature
The prior-art made of record and considered pertinent to applicant's disclosure.
See Information Disclosure Statement by Applicant (IDS) filed 03/04/2024.
Hickey et al. 2020/0410440 [0062 - the stopping criteria data is used to determine whether the machine learning has reached an acceptable level of error rate]
Torkkola et al. 2004/0252027 [0042 – machine learning… stopping criteria… error…]
Glyman et al. 2019/0005389 [0110 - This training process can continue until a stopping point is reached. This stopping point may depend on an error rate, a number of training iterations, or an elapsed time.]
PERTINENT PRIOR ART – Non-Patent Literature (NPL)
The NPL prior-art made of record and considered pertinent to applicant's disclosure.
See Information Disclosure Statement by Applicant (IDS) filed 03/04/2024.
A review of data mining applications in crime. By: Hassani, Hossein; Huang, Xu; Silva, Emmanuel S.; In: Statistical Analysis & Data Mining, Jun2016.
A Bayesian Failure Prediction Network Based on Text Sequence Mining and Clustering. By: Chang, Wenbing; Xu, Zhenzhong; You, Meng; In: Entropy, Dec2018.
THIS ACTION IS MADE FINAL
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW T. SITTNER whose telephone number is (571) 270-7137 and email: matthew.sittner@uspto.gov. The examiner can normally be reached on Monday-Friday, 8:00am - 5:00pm (Mountain Time Zone).
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sarah M. Monfeldt can be reached on (571) 270-1833.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW T SITTNER/
Primary Examiner, Art Unit 3629b
Prosecution Insights