DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the amendment filed on Apr. 21st, 2026. The amendments are linked to the original application filed on Feb. 11th, 2022.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on Apr. 21st, 2026 has been entered.
Response to Amendment
The Examiner thanks the applicant for the remarks, edits and arguments.
Regarding Claim Rejections – 35 U.S.C. 103
The Applicant argues that the combination of proposed arts Zhang and Bie fails to teach key elements of the independent claims. For example, the applicant states that Bie and Zhang fail to teach, “determining a robustness of the machine learning based medical analysis network for performing the medical analysis task based on the input medical data and the results of the medical analysis task using a machine learning based audit network, the machine learning based audit network receiving as input the input medical data and the results of the medical analysis task and generating as output the robustness of the machine learning based medical analysis network, wherein the machine learning based audit network is trained using training medical images with labels of ground truth results of the medical analysis task”. The Examiner has reviewed the amended claims and has found the Applicants argument persuasive. Therefore, the Examiner no longer relies on Zhang and Bie to teach this amendment in particular.
Next, the Applicant argues that Zhang does not teach robustness testing and that Bie fails to teach the robustness testing as claimed above. The Applicant states that the citations from Bie fails to properly teach an audit network and Bie fails to use the results of the initial model in determining robustness. The Examiner would like to clarify that Zhang does not teach robustness testing and instead is a machine learning model that is designed to execute a medical task. Zhang discloses a system that is able to find centerlines of veins using reinforcement learning. This model is used to perform a medical task and another network is used to evaluate the model. The Examiner uses Zhang in combination with another art, such as Bie, to teach two separate models, one that performs a machine learning task and another that audits the other model. This interpretation is based on the broadest reasonable interpretation of the independent claims by the Examiner. However, The Examiner has reviewed the arguments against Bie and the Examiner has found the Applicants argument persuasive. Therefore, the Examiner no longer relies on Bie to teach robustness testing.
Finally, the applicant argues that Bie fails to teach, “the machine learning based audit network is trained using training medical images with labels of ground truth results of the medical analysis task,” as claimed. The Examiner has reviewed this limitation and Bie and has found the Applicants arguments persuasive. Therefore, the Examiner no longer relies on Bie to teach this limitation.
The Examiner has reviewed the amendments and arguments and would like to clarify the interpretation of the claims. A stated earlier the claims are interpreted by the Examiner using the broadest reasonable interpretation. The claims recite a process which uses an initial model, the medical analysis network, which uses input data to produce an output. Further, the claims recite a process of using another, separate model, the “audit network”. The audit network will use the input and output of the medical analysis model and determine if the first model is robust. The examiner would like to note that Bie does discloses a process of taking a separate machine learning model, a regression model, and the input data to that model and use the outputs of the model to determine the robustness of the input model. Further, Bie disclosed a system which was also able to explain the results using XAI methods. However, the examiner would like to note that applicant’s argument that audit network in Bie fails to teach, “wherein the machine learning based audit network is trained using training medical images with labels of ground truth results of the medical analysis task;” is a valid argument and the Examiner has found the argument persuasive. Bie does fail to explicitly disclose a process that uses trained, labeled image data from the initial model to train its own audit model. As a result, the Examiner can no longer rely on Bie to teach the core elements of the claimed invention.
The Examiner has reviewed the submitted remarks, claims and specification. Further, the Examiner is required to still perform a complete and thorough search of the art to ensure the claims comply with 35 U.S.C. 103, with or without relying on Bie. After completing this search, the Examiner has found new subject matter (Tian et al; hereinafter “Tian”) which is able to teach the claimed subject matter. The new art, Tian, uses a machine learning testing framework which is able to evaluate pre-trained, and separate, neural networks for accuracy and robustness. The Examiner believes this art also uses input and training datasets to the models as well as the output of the models into a separate model called evalDNN which evaluates the model for accuracy, robustness and other evaluation metrics. With this new subject matter and the combination of the previous presented arts, Zhang, Daughton and Mitchell, the Examiner believes that one of ordinary skill in the art would reasonably be able to use the teachings from these articles to disclose the claimed subject matter. Therefore, the Examiner believes that the current claims are rejected under 35 USC 103, see 103 rejection below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 9, 10, 11, 15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Tian et al, (Tian et al, “EvalDNN: A Toolbox for Evaluating Deep Neural Network
Models”, 2020, hereinafter “Tian”) in view of Zhang et al, (Zhang et al, “Deep Reinforcement Learning for Vessel Centerline Tracing in Multi-modality 3D Volumes”, 2018, hereinafter “Zhang”)
Regarding claim 1, Tian discloses, “determining a robustness of the machine learning based [medical] analysis network for performing the [medical] analysis task based on the input [medical] data and the results of the [medical] analysis task using a machine learning based audit network,” (Design of the Tool, pp. 46; “The architecture of EvalDNN is shown in Figure 1. EvalDNN encompasses a set of unified interfaces to evaluate a given model. Such interfaces can also access a model’s output and intermediate status to extract necessary information for evaluation.” This article discloses a machine learning based analysis network or EvalDNN. As seen in figure 1, the system will intake a DNN, which is interpreted to be the machine learning based task that is performed. The Figure shows the model and a set of input data is feed into the EvalDNN so the system can evaluate the model. As seen in the figure the robustness of a model can be evaluated using this framework)
PNG
media_image1.png
273
755
media_image1.png
Greyscale
“the machine learning based audit network receiving as input the input [medical] data and the results of the [medical] analysis task and generating as output the robustness of the machine learning based [medical] analysis network,” (Figure 1: Overview of EvalDNN, pp. 46; This figure shows the overview of the system. The EvalDNN model will take in a neural network, which is interpreted to be the input medical data, as well as a dataset containing training information and inferences, and perform different evaluations on the model and the data. Then the model will generate an output report of the evaluation. One of the evaluation metrices is to evaluation the robustness of a model, see Table 1.)
PNG
media_image1.png
273
755
media_image1.png
Greyscale
“wherein the machine learning based audit network is trained using training [medical] images with labels of ground truth results of the [medical] analysis task; and” (Evaluation, pp. 47; “To evaluate robustness, we select three adversarial attack methods: FGSM [7], BIM [8], and DeepFool [11]. Then we randomly choose 1,000 images from the whole set and use the above three methods to perform attacks on these images. The IDs of selected images are published on our website. For each model, the misclassified images are not involved in adversarial attack.” This model will test the robustness of a model using adversarial attacks using different algorithms. These algorithms, such as the DeepFool, will take in labeled images and generate adversarial images to trick the system. The EvalDNN model will then test the input model using the generated data to check to robustness. The three attack methods use the labeled dataset provided, which contains labeled images and ground truth images.)
“outputting the determination of the robustness of the machine learning based medical analysis network.” (Design of the Tool, pp. 46; “Listing 1 shows a usage example of EvalDNN. To use EvalDNN, users only need to feed the target DNN model and dataset into EvalDNN (lines 3–4), and specify the target evaluation metric (lines 6–7). EvalDNN will then evaluate the given model and generate the corresponding report (line 9).” As seen in Fig. 1, the model will output a report on the input model. This includes the robustness of the model. See table 1 for robustness testing descriptions.)
Tian fails to explicitly disclose:
“A computer-implemented method comprising:”
“receiving input medical data;”
“receiving results of a medical analysis task performed based on the input medical data using a machine learning based medical analysis network;”
However, Zhang discloses, “A computer-implemented method comprising:” (Introduction, pp. 756; "In this paper, we address the vessel centerline tracing problem with an end-to-end trainable deep reinforcement learning (DRL) network. An artificial agent is learned to interact with surrounding environment and collect rewards from the interaction." This article teaches a method to determine the centerline of vessels using an artificial agent using a computer.)
“receiving input medical data;” (Method, pp. 757; "Given a 3D volumetric image I and the list of ground truth vessel centerline points
G
=
[
g
0
,
g
1
,
…
,
g
n
]
we aim to learn a navigation model for an agent to trace the centerline through an optimal trajectory
P
=
[
p
0
,
p
1
,
…
,
p
n
]
" This model used takes in 3D volumetric image data to perform a medical task.)
“receiving results of a medical analysis task performed based on the input medical data using a machine learning based medical analysis network;” (Method, pp. 759; "From the neural network we generate an action
a
0
which moves the current point to
p
1
. Then, the current state is updated as
s
1
=
I
p
1
and fed into the neural network to generate action again. We repeat this process until the path converges on oscillatory-like cycles. To further stabilize the tracing process, we also apply momentum on action-values from network output:
r
t
←
α
r
t
-
1
+
(
1
-
α
)
r
t
, where a is the momentum factor. The centerline tracing process stops if the agent moves out of the volume or if a cycle is formed, i.e., moving to a position already visited previously." This model will take in input data and trace a vascular centerline and return a result.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Tian and Zhang. Tian teaches an evaluation framework that is able to take, as input, trained machine learning models and evaluate them using machine learning methods and systems. Zhang teaches a machine learning model is able to perform medical tasks such as finding centerlines of veins using reinforcement learning. One of ordinary skill would have motivation to combine To combine a system that is has generates and trains a machine learning model to perform a medical task with another model that is able to take the pretrained model and an evaluation of the model to determine model robustness and accuracy, “We first selected four popular model zoos, namely, TensorFlow-Slim, TorchVision, GluonCV, and Keras. We then selected 79 models targeted for image classification from these model zoos, following the three steps below. We only focus on image classification tasks in the experiments because most models in the above model zoos are designed for image classification.” (Tian, Model Selection, pp. 47) and “In this paper, we address the vessel centerline tracing problem with an end-to-end trainable deep reinforcement learning (DRL) network. An artificial agent is learned to interact with surrounding environment and collect rewards from the interaction. We can not only generate the vesselness map by training a classifier, but also learn to trace the centerline by training the artificial agent.” (Zhang, Introduction, pp. 756).
PNG
media_image2.png
175
983
media_image2.png
Greyscale
Regarding claim 2, Tian discloses, “in response to determining that the machine learning based medical analysis network is not robust, determining that the machine learning based medical analysis network is not robust due to the input medical data being out-of-distribution with respect to training data on which the machine learning based medical analysis network was trained or due to an artifact in at least one of the input medical data or the results of the medical analysis task.” (Table 1: Embedded Metrices in EvalDNN, pp. 47; The model in this system is able to evaluate a neural network using different evaluation metrics. Example metrics and descriptions are listed on this table. This model is able to evaluate a neural network and is able to identify if the results of the model are accurate, trustworthy and that the model is properly interpreting the input dataset. In this model the robustness testing includes generating adversarial images to trick the system. This is interpreted to being artifacts and the model is able to test for adversarial attacks or artifacts.)
Regarding claim 9, Zhang discloses, “wherein the medical analysis task comprises at least one of segmentation, determining centerlines of vessels, or computing a fractional flow reserve (FFR).” (Method, pp. 757; "In this section we propose a deep reinforcement learning based method for vessel centerline tracing in 3D volumes. Given a 3D volumetric image I and the list of ground truth vessel centerline points
G
=
[
g
0
,
g
1
,
…
,
g
n
]
, we aim to learn a navigation model for an agent to trace the centerline through an optimal trajectory
P
=
[
p
0
,
p
1
,
…
,
p
n
]
." This article teaches a ML model that is designed to determine the centerline of vessels.
Regarding claim 10, Tian discloses, “means for determining a robustness of the machine learning based medical analysis network for performing the medical analysis task based on the input medical data and the results of the medical analysis task using a machine learning based audit network,” (Design of the Tool, pp. 46; “The architecture of EvalDNN is shown in Figure 1. EvalDNN encompasses a set of unified interfaces to evaluate a given model. Such interfaces can also access a model’s output and intermediate status to extract necessary information for evaluation.” This article discloses a machine learning based analysis network or EvalDNN. As seen in figure 1, the system will intake a DNN, which is interpreted to be the machine learning based task that is performed. The Figure shows the model and a set of input data is feed into the EvalDNN so the system can evaluate the model. As seen in the figure the robustness of a model can be evaluated using this framework)
PNG
media_image1.png
273
755
media_image1.png
Greyscale
“the machine learning based audit network receiving as input the input medical data and the results of the medical analysis task and generating as output the robustness of the machine learning based medical analysis network,” (Figure 1: Overview of EvalDNN, pp. 46; This figure shows the overview of the system. The EvalDNN model will take in a neural network, which is interpreted to be the input medical data, as well as a dataset containing training information and inferences, and perform different evaluations on the model and the data. Then the model will generate an output report of the evaluation. One of the evaluation metrices is to evaluation the robustness of a model, see Table 1.)
“wherein the machine learning based audit network is trained using training medical images with labels of ground truth results of the medical analysis task; and” (Evaluation, pp. 47; “To evaluate robustness, we select three adversarial attack methods: FGSM [7], BIM [8], and DeepFool [11]. Then we randomly choose 1,000 images from the whole set and use the above three methods to perform attacks on these images. The IDs of selected images are published on our website. For each model, the misclassified images are not involved in adversarial attack.” This model will test the robustness of a model using adversarial attacks using different algorithms. These algorithms, such as the DeepFool, will take in labeled images and generate adversarial images to trick the system. The EvalDNN model will then test the input model using the generated data to check to robustness. The three attack methods use the labeled dataset provided, which contains labeled images and ground truth images.)
“means for outputting the determination of the robustness of the machine learning based medical analysis network.” (Design of the Tool, pp. 46; “Listing 1 shows a usage example of EvalDNN. To use EvalDNN, users only need to feed the target DNN model and dataset into EvalDNN (lines 3–4), and specify the target evaluation metric (lines 6–7). EvalDNN will then evaluate the given model and generate the corresponding report (line 9).” As seen in Fig. 1, the model will output a report on the input model. This includes the robustness of the model. See table 1 for robustness testing descriptions.)
Tian fails to explicitly disclose:
“An apparatus comprising:”
“means for receiving input medical data;”
“means for receiving results of a medical analysis task performed based on the input medical data using a machine learning based medical analysis network;”
However, Zhang discloses, “An apparatus comprising:” (Network Architecture and Implementation, pp. 760; "The experiments was conducted on a server with one Nvidia Titan X GPU." This model was executed on a computer system or server.)
“means for receiving input medical data;” (Method, pp. 757; "Given a 3D volumetric image I and the list of ground truth vessel centerline points
G
=
[
g
0
,
g
1
,
…
,
g
n
]
we aim to learn a navigation model for an agent to trace the centerline through an optimal trajectory
P
=
[
p
0
,
p
1
,
…
,
p
n
]
" This model used takes in 3D volumetric image data to perform a medical task.)
“means for receiving results of a medical analysis task performed based on the input medical data using a machine learning based medical analysis network;” (Method, pp. 759; "From the neural network we generate an action
a
0
which moves the current point to
p
1
. Then, the current state is updated as
s
1
=
I
p
1
and fed into the neural network to generate action again. We repeat this process until the path converges on oscillatory-like cycles. To further stabilize the tracing process, we also apply momentum on action-values from network output:
r
t
←
α
r
t
-
1
+
(
1
-
α
)
r
t
, where a is the momentum factor. The centerline tracing process stops if the agent moves out of the volume or if a cycle is formed, i.e., moving to a position already visited previously." This model will take in input data and trace a vascular centerline and return a result.)
Regarding claim 11, Tian discloses, “means for determining that the machine learning based medical analysis network is not robust due to the input medical data being out-of-distribution with respect to training data on which the machine learning based medical analysis network was trained or due to an artifact in at least one of the input medical data or the results of the medical analysis task in response to determining that the machine learning based medical analysis network is not robust.” (Table 1: Embedded Metrices in EvalDNN, pp. 47; The model in this system is able to evaluate a neural network using different evaluation metrics. Example metrics and descriptions are listed on this table. This model is able to evaluate a neural network and is able to identify if the results of the model are accurate, trustworthy and that the model is properly interpreting the input dataset. In this model the robustness testing includes generating adversarial images to trick the system. This is interpreted to being artifacts and the model is able to test for adversarial attacks or artifacts.)
PNG
media_image2.png
175
983
media_image2.png
Greyscale
Regarding claim 15, Tian discloses, “determining a robustness of the machine learning based medical analysis network for performing the medical analysis task based on the input medical data and the results of the medical analysis task using a machine learning based audit network,” (Design of the Tool, pp. 46; “The architecture of EvalDNN is shown in Figure 1. EvalDNN encompasses a set of unified interfaces to evaluate a given model. Such interfaces can also access a model’s output and intermediate status to extract necessary information for evaluation.” This article discloses a machine learning based analysis network or EvalDNN. As seen in figure 1, the system will intake a DNN, which is interpreted to be the machine learning based task that is performed. The Figure shows the model and a set of input data is feed into the EvalDNN so the system can evaluate the model. As seen in the figure the robustness of a model can be evaluated using this framework)
PNG
media_image1.png
273
755
media_image1.png
Greyscale
“the machine learning based audit network receiving as input the input medical data and the results of the medical analysis task and generating as output the robustness of the machine learning based medical analysis network,” (Figure 1: Overview of EvalDNN, pp. 46; This figure shows the overview of the system. The EvalDNN model will take in a neural network, which is interpreted to be the input medical data, as well as a dataset containing training information and inferences, and perform different evaluations on the model and the data. Then the model will generate an output report of the evaluation. One of the evaluation metrices is to evaluation the robustness of a model, see Table 1.)
“wherein the machine learning based audit network is trained using training medical images with labels of ground truth results of the medical analysis task; and” (Evaluation, pp. 47; “To evaluate robustness, we select three adversarial attack methods: FGSM [7], BIM [8], and DeepFool [11]. Then we randomly choose 1,000 images from the whole set and use the above three methods to perform attacks on these images. The IDs of selected images are published on our website. For each model, the misclassified images are not involved in adversarial attack.” This model will test the robustness of a model using adversarial attacks using different algorithms. These algorithms, such as the DeepFool, will take in labeled images and generate adversarial images to trick the system. The EvalDNN model will then test the input model using the generated data to check to robustness. The three attack methods use the labeled dataset provided, which contains labeled images and ground truth images.)
“outputting the determination of the robustness of the machine learning based medical analysis network.” (Design of the Tool, pp. 46; “Listing 1 shows a usage example of EvalDNN. To use EvalDNN, users only need to feed the target DNN model and dataset into EvalDNN (lines 3–4), and specify the target evaluation metric (lines 6–7). EvalDNN will then evaluate the given model and generate the corresponding report (line 9).” As seen in Fig. 1, the model will output a report on the input model. This includes the robustness of the model. See table 1 for robustness testing descriptions.)
Tian fails to explicitly disclose:
“A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:”
“receiving input medical data;”
“receiving results of a medical analysis task performed based on the input medical data using a machine learning based medical analysis network;”
However, Zhang discloses, “A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:” (Network Architecture and Implementation, pp. 760; "The experiments was conducted on a server with one Nvidia Titan X GPU." This model was executed on a computer system or server which contain memory with stored computer instructions to execute the functions.)
“receiving input medical data;” (Method, pp. 757; "Given a 3D volumetric image I and the list of ground truth vessel centerline points
G
=
[
g
0
,
g
1
,
…
,
g
n
]
we aim to learn a navigation model for an agent to trace the centerline through an optimal trajectory
P
=
[
p
0
,
p
1
,
…
,
p
n
]
" This model used takes in 3D volumetric image data to perform a medical task.)
“receiving results of a medical analysis task performed based on the input medical data using a machine learning based medical analysis network;” (Method, pp. 759; "From the neural network we generate an action
a
0
which moves the current point to
p
1
. Then, the current state is updated as
s
1
=
I
p
1
and fed into the neural network to generate action again. We repeat this process until the path converges on oscillatory-like cycles. To further stabilize the tracing process, we also apply momentum on action-values from network output:
r
t
←
α
r
t
-
1
+
(
1
-
α
)
r
t
, where a is the momentum factor. The centerline tracing process stops if the agent moves out of the volume or if a cycle is formed, i.e., moving to a position already visited previously." This model will take in input data and trace a vascular centerline and return a result.)
Regarding claim 20, Zhang discloses, “wherein the medical analysis task comprises at least one of segmentation, determining centerlines of vessels, or computing a fractional flow reserve (FFR).” (Method, pp. 757; "In this section we propose a deep reinforcement learning based method for vessel centerline tracing in 3D volumes. Given a 3D volumetric image I and the list of ground truth vessel centerline points
G
=
[
g
0
,
g
1
,
…
,
g
n
]
, we aim to learn a navigation model for an agent to trace the centerline through an optimal trajectory
P
=
[
p
0
,
p
1
,
…
,
p
n
]
." This article teaches a ML model that is designed to determine the centerline of vessels.)
Claims 3, 4, 7, 12, 13, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Tian and Zhang in view of Daughton et al, (Daughton et al, “PREDICTION OF PROBABILITY DISTRIBUTION FUNCTION OF CLASSIFIERS”, US 11,901,076 B1, Filed 2021, hereinafter “Daughton”)
Regarding claim 3, Daughton discloses, “in response to determining that the machine learning based medical analysis network is not robust, retraining the machine learning based medical analysis network and the machine learning based audit network based on the input medical data.” (Detailed Description, pp. 22, Col. 11, Ln. 55-61; "In step 230, program 115 determines if the uncertainty level is above a threshold associated with an acceptable level of uncertainty. Accordingly, if the threshold is exceeded, then program 115 proceeds to retrain the models in step 230. Stated another way, there may be models generated that do not have a high enough confidence level for program 115 to make accurate evaluations of medical information 130." The method in this article is able to evaluate many different kinds of models which can perform different medical related predictions and tasks. This model also is able to determine when models need to be retrained. The citation above states that it will evaluate models and after a threshold is met the models will be retrained. This is similar to the limitation because it solves the same problem, after a particular threshold is met, the models, which includes different kinds of models, will be retrained.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Tian, Zhang, and Daughton. Tian teaches an evaluation framework that is able to take, as input, trained machine learning models and evaluate them using machine learning methods and systems. Zhang teaches a machine learning model is able to perform medical tasks such as finding centerlines of veins using reinforcement learning. Daughton teaches a method that is able to determine the accuracy of a machine learning model and determine when and how retraining would occur. One of ordinary skill would have motivation to combine a system that is able to evaluate the accuracy of machine learning models using a framework, which can be adjusted and other evaluation metrics can be added, with a system that performs a machine learning task, such as determining centerlines of veins using machine learning, with a system that, after determining an inaccurate machine learning model, is able to perform corrective actions such as alerting the user or requesting further information from a user, “During operation, the computer system may receive information (step 1010) corresponding to medical imaging and clinical data for a plurality of individuals. Then, the computer system may apply a pretrained predictive model (step 1020) to the information for at least a subset of the plurality of individuals. Moreover, the computer system may determine levels of uncertainty (step 1020) in results of the pretrained predictive model for at least the subset of the plurality of individuals. Next, the computer system may dynamically adapt (step 1030) a lower acceptable limit and an upper acceptable limit that define at least one threshold range based at least in part on the determined levels of uncertainty and a predefined target performance of the pretrained predictive model for the plurality of individuals." (Example 4, Daughton, Col. 17, pp. 25)
Regarding claim 4, Daughton discloses, “in response to determining that the machine learning based medical analysis network is not robust, presenting one or more alternate results of the medical analysis task from other machine learning based medical analysis networks.” (Additional Examples, Col. 20, Ln. 34-39; “Alternatively, for the second remedial action, if the image has an average angle above the lower bound of the uncertainty angle threshold (or, alternatively, within the threshold range), the classification density may be upgraded to the next density level in order to reduce false omission risk.” In this model the system is able to analyze the output of a model and in this example, if a level of uncertainty is above a threshold, a new classification result may be used as a different result than the previously uncertain result.)
Regarding claim 7, Daughton discloses, “in response to determining that the machine learning based medical analysis network is not robust, generating an alert to a user notifying the user that the machine learning based medical analysis network is not robust or requesting input from the user.” (Additional Examples, Col. 21, Ln. 42-50; “Alternatively, for the second remedial action, underestimating or overestimating cardiovascular risks can have serious consequences for the health of patients. Consequently, patients that have high uncertainty may be identified, and flagged because the model was not able to estimate the risk with good confidence. These patients may need to be re-evaluated for their risk using other tests, such as a stress test, CAC Agatson Scoring, etc.” This model is able to determine if the results of a model requires remedial actions based on the generated uncertainty score. In this example the system is able to flag results from a model and alert the patient and may request further information or testing from the user or patient.)
Regarding claim 12, Daughton discloses, “means for retraining the machine learning based medical analysis network and the machine learning based audit network based on the input medical data in response to determining that the machine learning based medical analysis network is not robust.” (Detailed Description, pp. 22, Col. 11, Ln. 55-61; "In step 230, program 115 determines if the uncertainty level is above a threshold associated with an acceptable leveI of uncertainty. Accordingly, if the threshold is exceeded, then program 115 proceeds to retrain the models in step 230. Stated another way, there may be models generated that do not have a high enough confidence level for program 115 to make accurate evaluations of medical information 130." The method in this article is able to evaluate many different kinds of models which can perform different medical related predictions and tasks. This model also is able to determine when models need to be retrained. The citation above states that it will evaluate models and after a threshold is met the models will be retrained. This is similar to the limitation because it solves the same problem, after a particular threshold is met, the models, which includes different kinds of models, will be retrained.)
Regarding claim 13, Daughton discloses, “means for presenting one or more alternate results of the medical analysis task from other machine learning based medical analysis networks in response to determining that the machine learning based medical analysis network is not robust.” (Additional Examples, Col. 20, Ln. 34-39; “Alternatively, for the second remedial action, if the image has an average angle above the lower bound of the uncertainty angle threshold (or, alternatively, within the threshold range), the classification density may be upgraded to the next density level in order to reduce false omission risk.” In this model the system is able to analyze the output of a model and in this example, if a level of uncertainty is above a threshold, a new classification result may be used as a different result than the previously uncertain result.)
Regarding claim 18, Daughton discloses, “in response to determining that the machine learning based medical analysis network is not robust, generating an alert to a user notifying the user that the machine learning based medical analysis network is not robust or requesting input from the user.” (Additional Examples, Col. 21, Ln. 42-50; “Alternatively, for the second remedial action, underestimating or overestimating cardiovascular risks can have serious consequences for the health of patients. Consequently, patients that have high uncertainty may be identified, and flagged because the model was not able to estimate the risk with good confidence. These patients may need to be re-evaluated for their risk using other tests, such as a stress test, CAC Agatson Scoring, etc.” This model is able to determine if the results of a model requires remedial actions based on the generated uncertainty score. In this example the system is able to flag results from a model and alert the patient and may request further information or testing from the user or patient.)
Claims 5, 14, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Tian and Zhang in view of Mitchell et al, (Mitchell et al, “Experience With a Learning Personal Assistant”, 1994, hereinafter “Mitchell”)
Regarding claim 5, Tian discloses, “wherein determining a robustness of the machine learning based [medical] analysis network for performing the [medical] analysis task based on the input [medical] data and the results of the [medical] analysis task using a machine learning based audit network comprises: determining the robustness of the machine learning based [medical] analysis network based on the final results of the [medical] analysis tasks.” (Design of the Tool, pp. 46; “The architecture of EvalDNN is shown in Figure 1. EvalDNN encompasses a set of unified interfaces to evaluate a given model. Such interfaces can also access a model’s output and intermediate status to extract necessary information for evaluation.” This article discloses a framework that evaluates different machine learning models. This system will take in a set of models from a model repository and analyze them for accuracy and robustness based in model output and input data to the model. See table 1 for robustness metrics and testing.)
Tian and Zhang fail to explicitly disclose:
“receiving user input editing the results of the medical analysis task to generate final results of the medical analysis task and”
However, Mitchell discloses, “receiving user input editing the results of the medical analysis task to generate final results of the medical analysis task and” (System Organization, pp. 3; "This suggestion is derived from a previously learned rule that matches the known features of this new meeting (i.e., those features for which the user has already been prompted, plus any features inferred from these). The user may accept this advice or override it by entering the desired value. In this figure, the user is overriding the advice, and instructing the system to allocate 30 minutes for this meeting. Whenever the user accepts or overrides CAP's advice, a training example is captured that is used for subsequent learning." The user can override the results of the output and enter their own final result. This override is taken in by the system and is used for subsequent training.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Tian, Zhang, and Mitchell. Tian teaches an evaluation framework that is able to take, as input, trained machine learning models and evaluate them using machine learning methods and systems. Zhang teaches a machine learning model is able to perform medical tasks such as finding centerlines of veins using reinforcement learning. Mitchell teaches a machine learning model whose results can be altered by the user to better fit their needs. One of ordinary skill would have motivation to combine a system that is able to evaluate machine learning models using a framework and produce a report of the evaluation, with a system that is able to alter the results of the input data to fit a more robust model, with a system that performs a specific machine learning task such as finding centerlines of veins using reinforcement learning, “While rules learned by CAP are useful for providing interactive advice to be approved or overridden by the user, they are not sufficiently accurate to support autonomous negotiation of all meetings by the agent on the user's behalf." (Mitchell, Conclusion and Prospects, pp. 11).
Regarding claim 14, Tian discloses, “wherein the means for determining a robustness of the machine learning based [medical] analysis network for performing the [medical] analysis task based on the input [medical] data and the results of the [medical] analysis task using a machine learning based audit network comprises: means for determining the robustness of the machine learning based [medical] analysis network based on the final results of the [medical] analysis tasks.” (Design of the Tool, pp. 46; “The architecture of EvalDNN is shown in Figure 1. EvalDNN encompasses a set of unified interfaces to evaluate a given model. Such interfaces can also access a model’s output and intermediate status to extract necessary information for evaluation.” This article discloses a framework that evaluates different machine learning models. This system will take in a set of models from a model repository and analyze them for accuracy and robustness based in model output and input data to the model. See table 1 for robustness metrics and testing.)
Tian and Zhang fail to explicitly disclose,
“means for receiving user input editing the results of the medical analysis task to generate final results of the medical analysis task and”
However, Mitchell discloses, “means for receiving user input editing the results of the medical analysis task to generate final results of the medical analysis task and” (System Organization, pp. 3; "This suggestion is derived from a previously learned rule that matches the known features of this new meeting (i.e., those features for which the user has already been prompted, plus any features inferred from these). The user may accept this advice or override it by entering the desired value. In this figure, the user is overriding the advice, and instructing the system to allocate 30 minutes for this meeting. Whenever the user accepts or overrides CAP's advice, a training example is captured that is used for subsequent learning." The user can override the results of the output and enter their own final result. This override is taken in by the system and is used for subsequent training.)
Regarding claim 17, Tian discloses, “wherein determining a robustness of the machine learning based [medical] analysis network for performing the [medical] analysis task based on the input [medical] data and the results of the [medical] analysis task using a machine learning based audit network comprises: determining the robustness of the machine learning based [medical] analysis network based on the final results of the [medical] analysis tasks.” (Design of the Tool, pp. 46; “The architecture of EvalDNN is shown in Figure 1. EvalDNN encompasses a set of unified interfaces to evaluate a given model. Such interfaces can also access a model’s output and intermediate status to extract necessary information for evaluation.” This article discloses a framework that evaluates different machine learning models. This system will take in a set of models from a model repository and analyze them for accuracy and robustness based in model output and input data to the model. See table 1 for robustness metrics and testing.)
Tian and Zhang fail to explicitly disclose,
“receiving user input editing the results of the medical analysis task to generate final results of the medical analysis task and”
However, Mitchell discloses, “receiving user input editing the results of the medical analysis task to generate final results of the medical analysis task and” (System Organization, pp. 3; "This suggestion is derived from a previously learned rule that matches the known features of this new meeting (i.e., those features for which the user has already been prompted, plus any features inferred from these). The user may accept this advice or override it by entering the desired value. In this figure, the user is overriding the advice, and instructing the system to allocate 30 minutes for this meeting. Whenever the user accepts or overrides CAP's advice, a training example is captured that is used for subsequent learning." The user can override the results of the output and enter their own final result. This override is taken in by the system and is used for subsequent training.)
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Tian and Zhang in view of Valiuddin et al, (Valiuddin et al, “OUT-OF-DISTRIBUTION DETECTION OF MELANOMA USING NORMALIZING FLOWS”, 2021, hereinafter “Valiuddin”)
Regarding claim 6, Valiuddin discloses, “wherein the machine learning based audit network is implemented using a normalizing flows model.” (Abstract, pp. 1; "While the generative abilities of NFs are typically explored, we focus on exploring the data distribution modelling for Out-of-Distribution (OOD) detection. Using one of the state-of-the-art NF models, GLOW, we attempt to detect OOD examples in the ISIC dataset." The model used in this article utilizes normalizing flow architectures to detect OOD in datasets.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Tian, Zhang and Valiuddin. Tian teaches an evaluation framework that is able to take, as input, trained machine learning models and evaluate them using machine learning methods and systems. Zhang teaches a machine learning model is able to perform medical tasks such as finding centerlines of veins using reinforcement learning. Valiuddin teaches a machine learning model that uses a normalizing flow model to ensure correctness of data used in machine learning models. One of ordinary skill would have motivation to combine the teachings of a system that is able to use an evaluation network, where the user can apply different evaluation metrics and tests to evaluate the accuracy of models, with a system that is able to test and evaluate the data input into the machine learning models to ensure it is correct and valid with a system that uses a machine learning model to perform a specific task such as finding the centerlines of veins, “We see many areas that can be researched to further understand which underlying mechanics enable Wavelet Flow to perform better than GLOW in BPD evaluation and how to improve its OOD detection capability. As mentioned by the authors of Wavelet Flow, Haar wavelets were used but other wavelets should be investigated as well. Also, a different NF architecture, for estimating the detail coefficients, that performs similar or better than GLOW in BPD evaluation can be implemented. If similar or better performance is found, it confirms our hypothesis that the improvements are not due to the choice of NF architecture but, in fact, as a result of the usages of wavelet transformations. If it results in worse results then it means that the decision of choosing a GLOW inspired NF was essential for the performance of the model.” (Valiuddin, Discussion, pp. 9).
Regarding claim 16, Valiuddin discloses, “wherein the machine learning based audit network is implemented using a normalizing flows model.” (Abstract, pp. 1; "While the generative abilities of NFs are typically explored, we focus on exploring the data distribution modelling for Out-of-Distribution (OOD) detection. Using one of the state-of-the-art NF models, GLOW, we attempt to detect OOD examples in the ISIC dataset." The model used in this article utilizes normalizing flow architectures to detect OOD in datasets.)
Claims 8 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Tian, Zhang, and Daughton in view of Mitchell.
Regarding claim 8, Mitchell discloses, “receiving the input from the user overriding the determination that the machine learning based medical analysis network is not robust or editing the results of the medical analysis task.” (System Organization, pp. 3; "This suggestion is derived from a previously learned rule that matches the known features of this new meeting (i.e., those features for which the user has already been prompted, plus any features inferred from these). The user may accept this advice or override it by entering the desired value. In this figure, the user is overriding the advice, and instructing the system to allocate 30 minutes for this meeting. Whenever the user accepts or overrides CA P's advice, a training example is captured that is used for subsequent learning." This system will allow the user to override a recommendation. This recommendation is a result of a ML model designed to be a personal assistant. The user will be allowed to edit the results.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Tian, Zhang, Daughton, and Mitchell. Tian teaches an evaluation framework that is able to take, as input, trained machine learning models and evaluate them using machine learning methods and systems. Zhang teaches a machine learning model is able to perform medical tasks such as finding centerlines of veins using reinforcement learning. Daughton teaches a method that is able to determine the accuracy of a machine learning model and determine when and how retraining would occur. Mitchell teaches a machine learning model whose results can be altered by the user to better fit their needs. One of ordinary skill would have motivation to combine the teachings of a machine learning framework that is able to evaluate a pretrained machine learning for accuracy and robustness, a system that is able to provide remedial actions based on a trained model determined to not be accurate or robust, a system that is able to take user input after an inconsistence in model output is detected and a model that is able to perform specific medical tasks such as determined the centerline of veins using reinforcement learning, “During operation, the computer system may receive information (step 1010) corresponding to medical imaging and clinical data for a plurality of individuals. Then, the computer system may apply a pretrained predictive model (step 1020) to the information for at least a subset of the plurality of individuals. Moreover, the computer system may determine levels of uncertainty (step 1020) in results of the pretrained predictive model for at least the subset of the plurality of individuals. Next, the computer system may dynamically adapt (step 1030) a lower acceptable limit and an upper acceptable limit that define at least one threshold range based at least in part on the determined levels of uncertainty and a predefined target performance of the pretrained predictive model for the plurality of individuals." (Example 4, Daughton, Col. 17, pp. 25) and “While rules learned by CAP are useful for providing interactive advice to be approved or overridden by the user, they are not sufficiently accurate to support autonomous negotiation of all meetings by the agent on the user's behalf." (Mitchell, Conclusion and Prospects, pp. 11).
Regarding claim 19, Mitchell discloses, “receiving the input from the user overriding the determination that the machine learning based medical analysis network is not robust or editing the results of the medical analysis task.” (System Organization, pp. 3; "This suggestion is derived from a previously learned rule that matches the known features of this new meeting (i.e., those features for which the user has already been prompted, plus any features inferred from these). The user may accept this advice or override it by entering the desired value. In this figure, the user is overriding the advice, and instructing the system to allocate 30 minutes for this meeting. Whenever the user accepts or overrides CA P's advice, a training example is captured that is used for subsequent learning." This system will allow the user to override a recommendation. This recommendation is a result of a ML model designed to be a personal assistant. The user will be allowed to edit the results.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL MICHAEL GALVIN-SIEBENALER whose telephone number is (571)272-1257. The examiner can normally be reached Monday - Friday 8AM to 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL M GALVIN-SIEBENALER/Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights