DETAILED ACTION Th is action is in response to the claims filed 09/19/2023 for Application number 18/470,233. Claims 1-20 are currently pending. Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims FILLIN "Pluralize the word 'Claim' if necessary and then identify the claim(s) being rejected." 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Regarding claim 1 , Step 1 Analysis: Claim 1 is directed to a process, which falls within one of the four statutory categories. Step 2A Prong 1 Analysis: Claim 1 recites, in part, The limitations of: determining, [ by the arbitrariness prediction computer program ] , a number of dropout models for the trained machine learning model to generate can be considered to be an evaluation in the human mind determining, [ by the arbitrariness prediction computer program ] , an arbitrariness for the trained machine learning model based on the outputs. These limitations as drafted, are processes that, under broadest reasonable interpretation, covers performance of the limitation in the mind or with the aid of pen and paper which falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. Step 2A Prong 2 Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements - “ arbitrariness prediction computer program ” , “ a trained machine learning model ”, and “ creating, by the arbitrariness prediction computer program, the number of dropout models ” . Thus, th e s e element s in the claim are recited at a high level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component. Please see MPEP 2106.05(f). Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim further recites: receiving, by arbitrariness prediction computer program, a trained machine learning model, wherein the trained machine learning model comprises a plurality of nodes, and each node has a weight providing, by the arbitrariness prediction computer program, sample data to each of the dropout models; receiving, by the arbitrariness prediction computer program, an output from each of the dropout models; Th ese limitation s amount to mere data gathering and outputting steps and thus are insignificant extra-solution activit ies . Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim as a whole is directed to an abstract idea. Step 2B Analysis: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of utilizing an arbitrariness prediction computer program and a trained machine learning model to perform the steps of the claimed process amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Furthermore, the limitation s of : receiving, by arbitrariness prediction computer program, a trained machine learning model, wherein the trained machine learning model comprises a plurality of nodes, and each node has a weight providing, by the arbitrariness prediction computer program, sample data to each of the dropout models; receiving, by the arbitrariness prediction computer program, an output from each of the dropout models; are well-understood, routine, and conventional, as evidenced by MPEP §2106.05(d)(II)(I), “receiving or transmitting data over a network” . These limitations therefore remain insignificant extra-solution activity even upon reconsideration, and does not amount to significantly more. Even when considered in combination, these additional elements amount to mere instructions to apply the exception using generic computer components and insignificant extra-solution activity, which cannot provide an inventive concept. The claim is not patent eligible. Regarding claim 2 , the rejection of claim 1 is further incorporated, and further, the claim recites: wherein the trained machine learning model comprises a neural network. This limitation amounts to generally linking the judicial exception to a field of use or technological environment. Please see MPEP 2106.05(h). The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible. Regarding claim 3 , the rejection of claim 1 is further incorporated, and further, the claim recites: wherein the number of dropout models to generate is received as a parameter. This limitation amounts to generally linking the judicial exception to a field of use or technological environment. Please see MPEP 2106.05(h). The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible. Regarding claim 4 , the rejection of claim 1 is further incorporated, and further, the claim recites: wherein the step of creating the number of dropout models comprises: removing, by the arbitrariness prediction computer program, a number or percentage of the plurality of nodes from each of the dropout models. This limitation amounts to mere instructions to apply the judicial exception using a generic computer component. Please see MPEP 2106.05(f). The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible. Regarding claim 5 , the rejection of claim 4 is further incorporated, and further, the claim recites: wherein the number or percentage of the plurality of nodes are removed by setting the weights for the number or percentage of the plurality of nodes to zero. This limitation amounts to mere instructions to apply the judicial exception using a generic computer component. Please see MPEP 2106.05(f). The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible. Regarding claim 6 , the rejection of claim 4 is further incorporated, and further, the claim recites: wherein the plurality of nodes to remove from each of the dropout models are randomly selected . This limitation amounts to more specifics of the judicial exception identified in the rejection of claim 1 above. The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible. Regarding claim 7 , the rejection of claim 4 is further incorporated, and further, the claim recites: wherein the number or the percentage of nodes to remove is received as a parameter . This limitation amounts to mere data gathering thus is an insignificant extra-solution activity. The claim does not include any additional elements that amount to significantly more than the judicial exception. This limitation is just a nominal or tangential addition to the claim, and is also well-understood, routine and conventional as evidenced by MPEP §2106.05(d)(II)(I), “ receiving or transmitting data over a network”. This limitation therefore remains insignificant extra-solution activity even upon reconsideration, and does not amount to significantly more. Even when considered in combination, this additional element represents an insignificant extra-solution activity which cannot provide an inventive concept. The claim is not patent eligible. Regarding claim 8 , the rejection of claim 1 is further incorporated, and further, the claim recites: wherein the step of creating the number of dropout models comprises: multiplying, by the arbitrariness prediction computer program, the weights for the plurality of nodes with Gaussian noise having a unit mean and a variance . This claim recites additional mathematical steps in addition to the judicial exception identified in the rejection of claim 1, thus recites a judicial exception. The claim does not include any additional elements that amount to an integration of the judicial exceptions into a practical application, nor to significantly more than the judicial exceptions. The claim is not patent eligible. Regarding claim 9 , the rejection of claim 1 is further incorporated, and further, the claim recites: wherein the arbitrariness is a ratio of outputs of the dropout models that are the same over the number of dropout models . This limitation amounts to more specifics of the judicial exception identified in the rejection of claim 1 above. The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible. Regarding claim 10 , the rejection of claim 4 is further incorporated, and further, the claim recites: providing, by the arbitrariness prediction computer program, a second sample to the dropout models; and receiving, by the arbitrariness prediction computer program, second outputs from each of the dropout models for the second sample; wherein the arbitrariness is based on the outputs and the second outputs . Th ese limitation s amount to mere data gathering and outputting thus is an insignificant extra-solution activity. The claim does not include any additional elements that amount to significantly more than the judicial exception. Th ese limitation s are just nominal or tangential addition s to the claim, and are also well-understood, routine and conventional as evidenced by MPEP §2106.05(d)(II)(I), “ receiving or transmitting data over a network”. Th ese limitation s therefore remain insignificant extra-solution activity even upon reconsideration, and does not amount to significantly more. Even when considered in combination, th e s e additional element s represent an insignificant extra-solution activity which cannot provide an inventive concept. The claim is not patent eligible. Claim 1 1 recites features similar to claim 1 and is rejected for at least the same reasons therein. Claim 1 1 additionally requires analysis for “ A non-transitory computer readable storage medium, including instructions stored thereon, which when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising …” however this is an additional element that amounts to mere instructions to apply the judicial exception using a generic computer component. Please see MPEP 2106.05(f). Regarding Claims 12-20 , it recites features similar to claim s 2-10 and are rejected for at least the same reasons therein. Claim Rejections - 35 USC § 103 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis ( i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness . Claim s FILLIN "Insert the claim numbers which are under rejection." \d "[ 1 ]" 1-7, 9-17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over FILLIN "Insert the prior art relied upon." \d "[ 2 ]" Hsu et al. ("Rashomon Capacity: A Metric for Predictive Multiplicity in Probabilistic Classification", hereinafter "Hsu") and further in view of Lemay et al. ("Improving the repeatability of deep learning models with Monte Carlo dropout", hereinafter "Lemay") . Regarding claim 1 , Hsu teaches A method for efficient test-time estimation of predictive multiplicity (Abstract) , comprising: receiving, by arbitrariness prediction computer program, a trained machine learning model, wherein the trained machine learning model comprises a plurality of nodes, and each node has a weight (“For the classifiers, we adopt feed-forward neural networks for the first three datasets, and a convolutional neural network VGG16 (Simonyan and Zisserman, 2014) for CIFAR-10. All numbers reported are evaluated on the test set. For more information on the datasets, neural network architectures, and training details, see Appendix C.” [pg. 9, § 4. Empirical Study , ¶2; note: plurality of nodes and nodes having weights are inherent given the use of neural networks ] ) ; determining, by the arbitrariness prediction computer program, a number of [ dropout ] models for the trained machine learning model to generate (“Remarkably, even when the Rashomon set has infinite cardinality, the value of Rashomon Capacity for a sample can be recovered by considering only a small number of models in the Rashomon set.” [pg. 8, § 3.3, ¶2]) ; creating, by the arbitrariness prediction computer program, the number of [ dropout ] models (“We propose a procedure for resolving predictive multiplicity in probabilistic classifiers. Even though the Rashomon set may span a large (potentially uncountable) number of models, we show that the score variation for a sample is fully captured by a small, discrete subset of models in the Rashomon set.” [pg. 3, 3 rd bullet point]) ; providing, by the arbitrariness prediction computer program, sample data to each of the [ dropout ] models (“Thus, when c is small, the predictions produced by the competing classifiers can be communicated to a stakeholder, empowering them to decide how to resolve conflicting scores. Note that the c models that capture predictive multiplicity can be different across samples.” [pg. 3, ¶2]) ; receiving, by the arbitrariness prediction computer program, an output from each of the [ dropout ] models (“Next, we formally define Rashomon Capacity in terms of the KL-divergence between the output scores of classifiers in the Rashomon set. We then use Rashomon Capacity to define a predictive multiplicity metric for individual samples in a dataset (similar to ambiguity in (2)).” [pg. 6, 3, ¶1]) ; and determining, by the arbitrariness prediction computer program, an arbitrariness for the trained machine learning model based on the outputs. (“Predictive multiplicity captures the potential individual-level harm introduced by an arbitrary choice of a single model in the Rashomon set.” [pg. 2, ¶1]) However Hsu fails to explicitly disclose that the models are specifically dropout models, Lemay teaches that the models are dropout models (“ The repeatability of each model was assessed on all available images of the same patient during the same visit. MC dropout models were associated with increased repeatability and accuracy for all models and tasks excluding regression models (Table 1 and Fig. 2). ” [pg. 2, §Results, ¶1] ) It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Hsu’s teachings by implementing the monte carlo dropout method of Lemay. One would have been motivated to make this modification as Lemay’s repeatability is similar to Hsu’s predictive multiplicity method as they both assess variability of model predictions th us implementing Lemay’s method would provide more consistent classification performance leading to less variability. [pg. 2, left col, ¶2, Lemay] Regarding claim 2 , Hsu/Lemay teaches The method of claim 1, Hsu teaches wherein the trained machine learning model comprises a neural network. (“For the classifiers, we adopt feed-forward neural networks for the first three datasets” [pg. 9, §4. Empirical Study, ¶2]) Regarding claim 3 , Hsu/Lemay teaches The method of claim 1, Hsu teaches wherein the number of dropout models to generate is received as a parameter. (“We describe one simple method for navigating the Rashomon set next and, later in the section, we also consider sampling models in the Rashomon set via random initialization of parameters prior to training” [pg. 9, §4.1, ¶1 ; note: Lemay teaches “dropout” models as cited above in the rejection of claim 1 thus when combined with Hsu would teach the limitation as claimed. ]) Same motivation to combine the teachings of Hsu/Lemay as claim 1. Regarding claim 4 , Hsu/Lemay teaches The method of claim 1, Lemay teaches wherein the step of creating the number of dropout models comprises: removing, by the arbitrariness prediction computer program, a number or percentage of the plurality of nodes from each of the dropout models (“Models with dropout were trained using spatial dropout with a dropout rate of 0.1 for cervical images and DMIST, and 0.2 for knee osteoarthritis and ROP. The dropout rates were determined based on preliminary explorations to optimize the model’s classification performance and values from the literature” [pg. 8, §Classification model training, ¶2]) Same motivation to combine the teachings of Hsu/Lemay as claim 1. Regarding claim 5 , Hsu/Lemay teaches The method of claim 4, Lemay teaches wherein the number or percentage of the plurality of nodes are removed by setting the weights for the number or percentage of the plurality of nodes to zero. (“Channels are independently and randomly zeroed for each dropout layer and forward pass, following the dropout rate from a Bernoulli distribution.” [pg. 8, §Classification model training, ¶2 ; See also the definition of dropout provided in Lemay: “ Dropout is the process of randomly removing units from a neural network during training to regularize learning and avoid overfitting ” pg. 1, Introduction, ¶2] ]) Same motivation to combine the teachings of Hsu/Lemay as claim 1. Regarding claim 6 , Hsu/Lemay teaches The method of claim 4, Lemay teaches wherein the plurality of nodes to remove from each of the dropout models are randomly selected. (“Channels are independently and randomly zeroed for each dropout layer and forward pass, following the dropout rate from a Bernoulli distribution.” [pg. 8, §Classification model training, ¶2 ]) Same motivation to combine the teachings of Hsu/Lemay as claim 1. Regarding claim 7 , Hsu/Lemay teaches The method of claim 4, Lemay teaches wherein the number or the percentage of nodes to remove is received as a parameter. (“Models with dropout were trained using spatial dropout with a dropout rate of 0.1 for cervical images and DMIST, and 0.2 for knee osteoarthritis and ROP. The dropout rates were determined based on preliminary explorations to optimize the model’s classification performance and values from the literature… This section enumerates the training parameters associated with the different datasets.” [pg. 8-9, §Classification model training, ¶2-3; See also Table 2 “Training Parameters: Dropout rate”) Same motivation to combine the teachings of Hsu/Lemay as claim 1. Regarding claim 9 , Hsu/Lemay teaches The method of claim 1, Hsu teaches wherein the arbitrariness is a ratio of outputs of the dropout models that are the same over the number of dropout models. (“For predictive multiplicity in classification problems, Semenova et al. (2019, Defn . 12) further proposed pattern Rashomon ratio for binary classification, which is the ratio of the count of all possible binary predicted classes given by the functions in the Rashomon set to that given by the functions in the hypothesis space.” [pg. 4, §2.1, ¶3 ; note: Lemay teaches “dropout” models as cited above in the rejection of claim 1 thus when combined with Hsu would teach the limitation as claimed. ]) Same motivation to combine the teachings of Hsu/Lemay as claim 1. Regarding claim 10 , Hsu/Lemay teaches The method of claim 1, Hsu teaches further comprising: providing, by the arbitrariness prediction computer program, a second sample to the dropout models (“We evaluate two methods11, random sampling with different weight initialization seeds and AWP (8), to obtain 100 models from the Rashomon set, and report the Rashomon Capacity in Fig. 3. In particular, we show the mean of the largest 1% and 5% Rashomon Capacity, and the cumulative distribution of the Rashomon Capacity across the samples.” [pg. 10, ¶2; across the samples implies “a second sample”]) ; and receiving, by the arbitrariness prediction computer program, second outputs from each of the dropout models for the second sample; wherein the arbitrariness is based on the outputs and the second outputs. (“we obtain models with output predictions pk by approximately solving the following optimization problem which maximizes the output score forall class k = [c]:” [pg. 10, top para]) Claim 1 1 recites features similar to claim 1 and is rejected for at least the same reasons therein. Claim 1 1 additionally requires A non-transitory computer readable storage medium, including instructions stored thereon, which when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising (Hsu, pg. 11, §5 “This is not only the computational bottleneck for estimating Rashomon Capacity, but also for other metrics including the Rashomon ratio and ambiguity/discrepancy.” implies use of computers/memory) Regarding claims 1 2-17 and 19-20 , they are substantially similar to claim s 2-7 and 9-10 respectively, and are rejected in the same manner, the same art, and reasoning applying. Claim s FILLIN "Insert the claim numbers which are under rejection." \d "[ 1 ]" 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over FILLIN "Insert the prior art relied upon." \d "[ 2 ]" Hsu in view of Lemay and further in view of Gal et al. ("Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning, hereinafter "Gal") . Regarding claim 8 , Hsu/Lemay teaches The method of claim 1, however fails to explicitly teach wherein the step of creating the number of dropout models comprises: multiplying, by the arbitrariness prediction computer program, the weights for the plurality of nodes with Gaussian noise having a unit mean and a variance. Gal teaches wherein the step of creating the number of dropout models comprises: multiplying, by the arbitrariness prediction computer program, the weights for the plurality of nodes with Gaussian noise having a unit mean and a variance. (“We show that a neural network with arbitrary depth and non-linearities, with dropout applied before every weight layer, is mathematically equivalent to an approximation to the probabilistic deep Gaussian process” [pg. 2, §3, ¶1 ; See further “ To estimate the predictive mean and predictive uncertainty we simply collect the results of stochastic forward passes through the model. ” [pg. 4, right col, ¶2 ]) It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Hsu’s/Lemay’s teachings in order to use mean and variance to create dropout models as taught by Gal. One would have been motivated to make this modification as information can be used with existing NN models trained with dropout. [pg. 4, right col, ¶2, Gal] Regarding claim 1 8 , it is substantially similar to claim 8 respectively, and is rejected in the same manner, the same art, and reasoning applying. Conclusion The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. FILLIN "Enter the appropriate information" \* MERGEFORMAT Ghost et al. ("US 20210374500 A1") discloses reproducibility of deep learning classifiers using ensembles of models . Any inquiry concerning this communication or earlier communications from the examiner should be directed to FILLIN "Examiner name" \* MERGEFORMAT MICHAEL H HOANG whose telephone number is FILLIN "Phone number" \* MERGEFORMAT (571)272-8491 . The examiner can normally be reached FILLIN "Work Schedule?" \* MERGEFORMAT Mon-Fri 8:30AM-4:30PM . Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, FILLIN "SPE Name?" \* MERGEFORMAT Kakali Chaki can be reached at FILLIN "SPE Phone?" \* MERGEFORMAT (571) 272-3719 . The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /MICHAEL H HOANG/ PRIMARY EXAMINER, Art Unit 2122