DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 1/28/2026 has been entered.
Applicant' s arguments, filed 1/28/2026, have been fully considered. The following rejections and/or objections are either reiterated or newly applied. They constitute the complete set presently being applied to the instant application.
Applicants have amended their claims, filed 1/28/2026, and therefore rejections newly made in the instant office action have been necessitated by amendment.
Claims 45-64 are the currently pending claims. Claims 58 and 59 have previously been withdrawn. Claims 45-46, 60, and 63 have been amended. Claims 45-57 and 60-64 are hereby under examination.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 45-48, 50-53, 56, and 60-64 are rejected under 35 U.S.C. 103 as being unpatentable over Yeung et al. (US 20180296281 A1), hereto referred as Yeung, and further in view of Boddington et al. (US 20210177522 A1), hereto referred as Boddington, and further in view of Huo et al. (Huo YR, Chan MV, Habib AR, Lui I, Ridley L. Pneumothorax rates in CT-Guided lung biopsies: a comprehensive systematic review and meta-analysis of risk factors. Br J Radiol. 2020 Apr 1;93(1108):20190866. doi: 10.1259/bjr.20190866. Epub 2020 Jan 3.), hereto referred as Huo, and further in view of Lu et al. (US 20180000446 A1), hereto referred as Lu, and further in view of He et al. (He, Changyan et al. “Enabling Technology for Safe Robot-Assisted Retinal Surgery: Early Warning for Unsafe Scleral Force.” Proceedings - IEEE International Conference on Robotics and Automation. IEEE, 2019. 3889–3894), hereto referred as He.
Regarding claim 45, Yeung teaches that a computer-implemented method of generating a data analysis algorithm for predicting and/or detecting a clinical condition related to insertion of a medical instrument toward a target in a body of a patient (Yeung, ¶[0200]: "The disclosed systems and methods can be used for automated steering control of robotic endoscopes", explains that Yeung discloses a computer-implemented method for automated navigation of a medical instrument in a patient; ¶[0036]: "In some embodiments, the output steering control signals of the training datasets are obtained from sets of empirical steering control instructions provided to the steering mechanism of a robotic colonoscope...The output steering control signals are for controlling one or more actuation units of a robotic colonoscope", explains that the data analysis and control relate to the insertion and actuation of a medical instrument toward a target within the body which is derived via “machine learning algorithms” (¶[0004])), the method comprises: collecting one or more datasets, at least one of the one or more datasets being related to an automated medical device configured to steer a medical instrument toward a target such that the medical instrument traverses a non-linear trajectory within a body of a patient and/or to operation thereof (Yeung, ¶[0006]: "Systems and methods are provided for automated steering control of a robotic endoscope...the control systems comprising: a) a first image sensor...and b) one or more processors...configured to generate a steering control output signal based on an analysis of data derived from the first input data stream using a machine learning architecture, wherein the steering control output signal adapts to changes in the data derived from the first input data stream in real time", explains that the system collects and analyzes data from sensors for automated medical device operation and steering; ¶[0097]: "Within the colon lumen, for instance, the environment 105 may comprise multiple flexural, looping or bending sections through which the steering control system allows the colonoscope to be maneuvered", teaches steering an instrument through non‑linear, complex trajectories in the body; ¶[0164]: "The input data may be obtained from a training process or stored in a database. In some cases, the sensor data and associated steering history data may be collected prior to obtaining the desired output data 811. In some cases, the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data", teaches collecting datasets related to operation of the automated device including sensor data and steering history; ¶[0084]: “The input data and the desired output data may be time-registered so that the correspondence between the input data and desired output data is known”, explains time-registration of procedural telemetry (sensor streams + steering signals) which the model uses as aligned inputs and outputs during data collection); creating a training set comprising a first data portion of the one or more datasets (Yeung, ¶[0034]: "The method comprises: a) providing two or more sets of input sensor data obtained from two or more different types of sensors; b) providing a plurality of training datasets, wherein a training dataset comprises at least a first set of known input sensor data for each of the two or more different types of sensors and a corresponding set of known output steering control signals; and c) analyzing the plurality of training datasets using the machine learning architecture...", explains that the system creates training sets from input sensor data for training machine learning models; ¶[0156]–[0157]: "A number of training datasets may be supplied to the neural network for training the parameters, e.g., weights, biases, and threshold values . . . FIG. 8A schematically illustrates exemplary training datasets supplied to a neural network 803 for learning the parameters of the network . . . the training datasets may comprise input data 813 and desired output data 811", teaches creating training datasets from collected data for model training); and validating the data analysis algorithm using a validation set, the validation set comprising a second data portion of the one or more datasets (Yeung, ¶[0034]: "...analyzing the plurality of training datasets using the machine learning architecture...", explains that multiple datasets are analyzed in model development;) ¶[0156]: "A number of training datasets may be supplied to the neural network for training the parameters, e.g., weights, biases, and threshold values...", explains use of multiple datasets for training and optimization; ¶[0157]: "...retraining and re-tuning the neural network parameters...can be performed iteratively as new input data becomes available, or as different sets of training data are provided to the model", explains use of separate sets of data at different stages of model development, which is consistent with dividing data into training and validation sets; ¶[0159]: "the model can be adjusted and re-optimized based on the performance metrics calculated from the results of the model’s predictions...", explains that model performance is assessed using results from a data set separate from that used for initial training. A person of ordinary skill in the art would understand that in the context of machine learning model development, it is standard practice to divide available data into training and validation sets—where the validation set is a second, independent data portion used specifically to evaluate model performance and prevent overfitting; ¶[0218]–[0221]: "Approximately 80% of the data generated from the video clips was used to train the ANN, while the remaining 20% was used to evaluate the performance of the system . . . The performance of the ANN‑based automated steering direction method was evaluated by comparing the separation distance, D . . . between the surgeon's indicated location for the lumen center and that determined by the ANN", teaches partitioning data into training and evaluation portions and validating model performance).
With respect to training the data analysis algorithm to output one or more of a prediction and a detection of a clinical condition related to insertion of the medical instrument toward the target in the body of the patient, using the training set, Yeung teaches training a machine learning model for controlling a robotic medical instrument using sensor data, but does not teach training the model to predict or detect a clinical condition caused by insertion (Yeung, ¶[0034], ¶[0156]–[0158], ¶[0219]–[0221]).
Boddington teaches that data science and machine/deep learning can be applied to intraoperative data to “calculate surgical decision risks, to predict a problem and provide guidance in real-time situations” and to produce “a surgical outcome prediction” using “multiple trained classifiers” presented in real-time (Boddington, ¶[0009], ¶[0069]; FIGs. 25–27). In context, Boddington’s disclosure demonstrates training models to output risk/prediction values (as opposed to control signals) during a procedure, using procedural data and subject information. Although Boddington is not limited to a particular device, it expressly teaches intra-operative risk and outcome prediction using procedural datasets and subject EHR information with multiple trained classifiers presented in real time, thereby supplying the precise prediction/detection output type that Yeung does not. However, Boddington does not specify that these risks are linked to device insertion.
Huo teaches that pneumothorax is the most frequent complication of CT-guided lung biopsy and provides probability statistics for its occurrence (Huo, Abstract, p. 2, Table 1), thus supplying the motivation to use the complication prediction systems described by Boddington for insertion-related clinical risks.
One skilled in the art would have found it obvious, in view of Huo’s clinical data and Boddington’s procedural prediction models, to adapt Yeung’s machine learning system so that it is trained to output a prediction or detection of a clinical condition (such as pneumothorax) related to insertion of a medical instrument.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yeung in view of Boddington and Huo to train the data analysis algorithm to output a prediction or detection of a clinical condition related to insertion of a medical instrument toward a target in a body of a patient. Yeung supplies the data pipeline and technical architecture, Boddington demonstrates that procedural data can routinely be used for risk prediction, and Huo provides the specific clinical motivation and probability data for insertion-related complications. Substituting an outcome-prediction output (probability/score) for Yeung’s control-signal output using Boddington’s intra-operative risk modeling would have been a predictable use of prior-art elements according to their established functions with a reasonable expectation of success. The benefit is real-time, automated detection of clinically significant risks to improve patient safety and outcomes.
With respect to the clinical condition comprising pneumothorax, breathing anomalies, internal bleeding, or any combinations thereof, Yeung teaches training and deploying a machine-learning architecture on procedural sensor datasets for automated steering, but does not teach that the model’s output pertains to specific clinical conditions such as pneumothorax, breathing anomalies, or internal bleeding (as shown above).
Boddington teaches that an intra-operative AI platform reads and interprets subject data to “calculate surgical decision risks” and to present “a surgical outcome prediction” with “known predictors and indicators of complications datasets” and “information from subject health records” among the enumerated inputs (Boddington, ¶[0104]-[0106]: “procedural medical image datasets”, “known predictors and indicators of complications datasets”, “subject HER information data” (where "HER" is understood to mean EHR, Electronic Health Records), “failure risk score datasets”, “information from subject health records… configured to include information that will potentially have an impact on the outcome of the procedure… provides the user with a prediction of optimal or suboptimal outcome and an associated Failure Risk Score”). However, it does not specify specific clinical conditions such as pneumothorax, breathing anomalies, or internal bleeding.
Huo further quantifies the clinical burden and actionable factors for insertion-related pneumothorax, reporting that “The overall pooled incidence for pneumothorax was 25.9% and chest drain insertion was 6.9%” (Huo, Abstract/Results), with institutional chest drain practice variability “(lowest 0.3% … vs highest 15%)” (Huo, Discussion). Huo enumerates specific, model-mappable risk factors including “fissure crossed,” “bulla crossed,” “multiple pleural punctures,” “emphysematous lungs,” lesion size and depth, and pleural contact (Huo, Abstract), and shows that “Multiple pleural punctures (>1) more than tripled the risk” (Huo, Results) and “Emphysema … increased pneumothorax incidence … [and] chest drain insertion” (Huo, Results). Huo also distinguishes modifiable vs non-modifiable determinants and highlights patient positioning as a modifiable factor (Huo, Introduction; Abstract). Detection protocols via plain film and/or CT are noted (Huo, Methods). These data concretely motivate configuring the model to output complication-specific predictions (e.g., pneumothorax) and to weight telemetry/features aligned with known risk factors during insertion.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung in view of Boddington and Huo to configure the trained model to output a prediction or detection for specific clinical conditions (including pneumothorax and, by extension, other intra-operative complications such as breathing anomalies or internal bleeding) during insertion. Yeung provides the procedural data pipeline and ML architecture; Boddington teaches intra-operative complication/outcome prediction using datasets that include known complication predictors and subject health-record information; and Huo provides clinical motivation and quantified incidence for pneumothorax in insertion procedures. The combination is feasible because Boddington’s risk-prediction modules operate on the same class of procedural and subject datasets that Yeung collects and structures, and obvious because targeting known insertion complications (with pneumothorax explicitly documented by Huo) is a predictable use of those models. The risk factors quantified by Huo (e.g., multiple pleural punctures, emphysema, lesion depth/pleural contact, patient positioning) correspond to procedural and subject attributes that can be encoded as model features or labels alongside Yeung’s time-registered device telemetry and Boddington’s EHR inputs, thereby making configuration of complication-specific outputs technically straightforward. The benefit is targeting these known complication-specific outputs (e.g., pneumothorax risk) and related condition signals (e.g., breathing anomalies, internal bleeding) with real-time probabilities yields improved intra-operative safety and workflow guidance.
With respect to wherein the one or more datasets comprise steering-phase operational data generated during execution of the non-linear trajectory, Yeung teaches collection of sensor data and associated steering history data, including time-registered input and output data generated during automated steering of a medical instrument through non-linear paths (Yeung, ¶[0164], ¶[0084], ¶[0097]). Yeung further expressly teaches that training datasets may include “steering history data” comprising information related to recent steering control actions, including steering vectors corresponding to motion of the distal end of the colonoscope during navigation (Yeung, ¶[0163]; FIG. 8B). Yeung also teaches that such sensor data and steering history data may be collected concurrently with obtaining desired output data during operation (Yeung, ¶[0164]). Because the colonoscope is maneuvered through “multiple flexural, looping or bending sections” (Yeung, ¶[0097]), these datasets inherently comprise steering-phase operational data generated during execution of a non-linear trajectory.
Lu teaches that during a target biopsy procedure, a needle is inserted into an anatomical region and ultrasound receivers sense the ultrasound plane as the needle is being inserted, and a controller tracks receiver positions and predicts the biopsy trajectory based on those sensed signals (Lu, ¶[0004], ¶[0009]). Lu further teaches that sensor data representative of sensing the ultrasound plane are generated as the needle is inserted into the anatomical region (Lu, ¶[0026]), and that trajectory prediction is performed based on image data and tracking data during the procedure (Lu, ¶[0028]). Accordingly, Lu explicitly teaches that operational data are generated during execution of the insertion trajectory.
Accordingly, Lu teaches operational data generated during execution of an insertion trajectory, including position and trajectory-related data derived from sensors during active insertion of the medical instrument. While Lu’s trajectory is substantially linear, the operative principle is that real-time operational data are collected during instrument movement through tissue and used to characterize the trajectory during execution. One of ordinary skill in the art would have recognized this as analogous to collecting steering-phase operational data during execution of a trajectory, and applicable to Yeung’s non-linear steering context.
Under the broadest reasonable interpretation, Yeung already teaches that the datasets comprise steering-phase operational data generated during execution of the non-linear trajectory (Yeung, ¶[0163]–¶[0164], ¶[0097]). Lu is cited to corroborate that, across interventional contexts, trajectory-characterizing sensor and image data are captured contemporaneously with active instrument motion and used to characterize the trajectory during execution. Thus, a person of ordinary skill in the art would have understood Yeung’s datasets to include during-execution trajectory-phase operational data, with Lu confirming this understanding as a well-established practice. The motivation is to improve the fidelity and relevance of the training data by including trajectory-execution features that more accurately reflect the instrument’s behavior in vivo, thereby enhancing the performance and reliability of the trained model. Lu’s specific contribution to the combination is not a new sensing modality, but rather the express confirmation that tracking and image data captured contemporaneously with active instrument motion are the operative inputs to trajectory characterization; when applied to Yeung’s non-linear steering context, this confirms that the datasets include data generated while the instrument is actively traversing its path, not merely pre-operative or retrospective data.
With respect to wherein the prediction and/or the detection is generated during insertion and steering of the medical instrument along the non-linear trajectory, Yeung teaches real-time automated steering of a medical instrument through non-linear paths in the body using time-registered sensor data and steering control outputs generated during the procedure (Yeung, ¶[0006], ¶[0084], ¶[0097], ¶[0164]). Boddington teaches intra-operative prediction of surgical outcomes and risks using trained models and procedural datasets, including real-time presentation of predictions (Boddington, ¶[0009], ¶[0069]). Accordingly, the modified Yeung teaches generating a prediction during an ongoing procedure while a medical instrument is being steered through a non-linear trajectory based on procedural data, but does not expressly tie the prediction to the insertion-phase steering behavior itself. In Yeung, the steering history data and time-registered sensor streams (¶[0163]–¶[0164]) encode trajectory execution characteristics, including motion vectors and spatial relationships during navigation, which constitute available predictive features for use with Boddington’s intra-operative risk modeling framework.
He teaches that during robot-assisted retinal surgery, a surgical tool is inserted into the eye and operational force data are collected during manipulation, and a trained neural network predicts imminent unsafe manipulation events based on the ongoing manipulation history, providing warning feedback during the procedure (He, Abstract; Section II-C; Section III-B). He further explains that the network uses a history of time-series force data collected during manipulation as input and outputs a prediction of future unsafe conditions during the ongoing task (He, Section II-C), and that the predictor operates in real time during active manipulation (He, Section III-B).
Accordingly, He teaches generating a prediction during insertion and steering of a medical instrument based on real-time operational data associated with the manipulation of the instrument.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung in view of He so that the prediction and/or detection is generated during insertion and steering of the medical instrument along the non-linear trajectory. The modified Yeung, in view of Boddington, already teaches generating intra-operative predictions based on procedural data collected during device operation. He teaches a specific implementation in which time-series operational data gathered during active instrument manipulation are used by a trained neural network to generate forward-looking predictions in real time. A person of ordinary skill in the art would have been motivated to apply He’s real-time, time-series prediction framework to the modified Yeung because both systems involve sensor-driven manipulation of instruments within a patient and rely on continuously acquired operational data streams. Incorporating He’s approach would enable the modified Yeung to produce predictions that are temporally synchronized with ongoing insertion and steering, rather than being delayed or retrospective. This modification would have been a predictable application of known machine learning techniques for real-time inference on streaming data, with a reasonable expectation of success given the compatibility of the data types and processing architectures. The benefit is the ability to provide immediate, actionable predictions during instrument manipulation, thereby improving intra-operative awareness, reducing the likelihood of adverse events, and enhancing procedural safety. He’s contribution to the combination is specifically the temporal inference framework, namely the use of a time-series neural network operating on continuously streamed intraoperative sensor data to generate forward-looking predictions in real time during active instrument manipulation, and not the specific prediction target, which is supplied by the modified Yeung in view of Boddington and Huo. The underlying architecture is agnostic to spatial scale and is applicable to both localized manipulation (e.g., retinal procedures) and larger-scale navigation (e.g., colonoscopy), as both involve temporally structured streaming operational data.
Regarding claim 46, the modified Yeung teaches that the training set and the validation set further comprise one or more data annotations (Yeung, FIG. 19; ¶[0034]: "The method comprises: a) providing two or more sets of input sensor data obtained from two or more different types of sensors; b) providing a plurality of training datasets, wherein a training dataset comprises at least a first set of known input sensor data for each of the two or more different types of sensors and a corresponding set of known output steering control signals...", explains that Yeung teaches both training and validation sets contain data annotations, specifically, the known output steering control signals, which are provided for each input sensor data set and serve as ground truth labels (i.e., annotations) for supervised machine learning, allowing comparison of model output to annotated values in both sets; additionally the figure depicts the surgeons annotations as compared to the systems output and the resulting error) and further comprises: calculating an error of output of the data analysis algorithm from the one or more data annotations (Yeung, ¶[0034]: "...analyzing the plurality of training datasets using the machine learning architecture to determine a set of weighting parameters...", explains that Yeung teaches calculating algorithm error by comparing the algorithm’s predicted outputs to the known annotated outputs (steering control signals), which is a necessary step in adjusting the algorithm’s internal parameters during learning; ¶[0220]: "The performance of the ANN-based automated steering direction method was evaluated by comparing the separation distance, Derr (in units of degrees), between the surgeon's indicated location for the lumen center and that determined by the ANN, as indicated in FIG. 19", explains that Yeung uses annotated data (surgeon's indication as ground truth) compared to model/algorithm prediction, and calculates the resulting error; FIG. 19: depicts the surgeons annotations as compared to the systems output and the resulting error between them; FIGS. 20-22: depict scatter plots of the resulting error); and optimizing the data analysis algorithm using the calculated error (Yeung, ¶[0034]: "...the weighting parameters are subsequently used by the machine learning architecture to provide a steering control signal for the automated colonoscope steering mechanism that is adaptive to changes in the two or more sets of input sensor data", explains that Yeung teaches optimization of the data analysis algorithm by minimizing the error between the predicted output and the annotated output (label), updating the algorithm 's parameters to improve performance in accordance with standard supervised learning practice; Yeung, ¶[0221]: "The results of the comparison between a surgeon's indicated location for the lumen center and that predicted by different combinations of image processing and ANN-based analysis are summarized in FIGS. 20-22... Current development work is focused on improving the combination of image processing algorithms utilized, e.g., by incorporation of optical flow methods, as well as on improving the design of the ANN model/algorithm and improving the quantitative methods used for validation of the process", explains that Yeung calculates error between the prediction and annotation, uses this for validation and further optimization of the ANN model/algorithm).
Regarding claim 47, the modified Yeung teaches that the one or more datasets further comprise one or more of: clinical procedure related dataset, patient related dataset and administrative related dataset (Yeung, ¶[0071]: "FIG. 15 shows a block diagram of an exemplary adaptive steering control system for a robotic endoscope that uses data collected by image sensors to generate a steering control signal, where the image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction", explains that Yeung teaches the collection and use of clinical procedure-related datasets, specifically image data acquired during the procedure and used as part of the system’s dataset for analysis and navigation; ¶[0164]: "The input data may be obtained from a training process or stored in a database. In some cases, the sensor data and associated steering history data may be collected prior to obtaining the desired output data 811. In some cases, the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data", explains that Yeung discloses collection and storage of sensor and steering data during operation of the endoscope, which constitutes a clinical procedure related dataset).
Regarding claim 48, the modified Yeung teaches that the automated medical device related dataset comprises parameters selected from: entry point, insertion angles, target position, target position updates, planned trajectory, trajectory updates, real-time positions of the medical instrument, number of checkpoints along the planned and/or updated trajectory, checkpoint locations, checkpoint locations updates, checkpoint errors, position of the automated medical device relative to the patient's body, steering steps timing, procedure time, steering phase time, procedure accuracy, target error, medical images, medical imaging parameters per scan, radiation dose per scan, total radiation dose in steering phase, total radiation dose procedure, errors indicated during the steering procedure, software logs, motion control traces, automated medical device registration logs, medical instrument detection logs, or any combination thereof (Yeung, ¶[0071], ¶[0164]: "FIG. 15 shows a block diagram of an exemplary adaptive steering control system for a robotic endoscope that uses data collected by image sensors to generate a steering control signal... The input data may be obtained from a training process or stored in a database. In some cases, the sensor data and associated steering history data may be collected prior to obtaining the desired output data 811. In some cases, the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data", explains that Yeung teaches collection and use of image data, navigation direction, steering control signals, sensor and steering history data, supporting parameters such as target position, planned trajectory, trajectory updates, real-time positions of the instrument, steering steps timing, procedure time, steering phase time, and motion control traces; ¶[0220]-[0221]: "The performance of the ANN-based automated steering direction method was evaluated by comparing the separation distance, Derr (in units of degrees), between the surgeon's indicated location for the lumen center and that determined by the ANN, as indicated in FIG. 19... The results of the comparison between a surgeon's indicated location for the lumen center and that predicted by different combinations of image processing and ANN-based analysis are summarized in FIGS. 20-22... Current development work is focused on... improving the design of the ANN model/algorithm l and improving the quantitative methods used for validation of the process", explains that Yeung teaches use of annotated data (surgeon's indication as ground truth) versus model/algorithm prediction, calculation of error, validation, and model/algorithm improvement, supporting parameters such as procedure accuracy, target error, checkpoint errors, and validation-related dataset entries).
Regarding claim 50, the modified Yeung teaches that one or more of the parameters of the one or more datasets is configured to be collected automatically (Yeung, ¶[0071]: "FIG. 15 shows a block diagram of an exemplary adaptive steering control system for a robotic endoscope that uses data collected by image sensors to generate a steering control signal, where the image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction", explains that Yeung teaches automatic collection of image sensor data and navigation/control parameters by the system during operation; ¶[0164]: "The input data may be obtained from a training process or stored in a database. In some cases, the sensor data and associated steering history data may be collected prior to obtaining the desired output data 811. In some cases, the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data", explains that Yeung teaches sensor data and steering history are collected automatically by the system as part of the procedure, without requiring manual entry).
Regarding claim 51, the modified Yeung teaches that the method further comprises performing one or more of: data cleaning, data pre-processing, data annotation and data augmentation, and extracting features from the one or more datasets (Yeung, ¶[0164]: "The input data may be obtained from a training process or stored in a database. In some cases, the sensor data and associated steering history data may be collected prior to obtaining the desired output data 811. In some cases, the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data. FIG. 8C shows an exemplary user interface 820 for a training process, in accordance with embodiments of the invention", explains that Yeung teaches collection, organization, and storage of data for use in machine learning, which requires data pre-processing, annotation, and extraction of features for training; ¶[0084]: "the image data may be pre-processed using one or more image processing algorithms prior to providing it as input to the machine learning algorithm", explains that Yeung expressly teaches data pre-processing, cleaning, and transformation for preparation of datasets; ¶[0007]: "the analysis comprises performing automated image processing and feature extraction ", explains that Yeung teaches extracting features from the collected datasets to use as inputs to the model/algorithm).
Regarding claim 52, the modified Yeung teaches that training the data analysis algorithm to output one or more first predictions relating to respective one or more first target variables (Yeung, ¶[0221]: "...results of the comparison between a surgeon's indicated location for the lumen center and that predicted by different combinations of image processing and ANN-based analysis...", explains that Yeung teaches each individual image processing method generates its own prediction of the lumen center (first predictions), supporting the training of multiple models/algorithms for respective target variables); the training of the data analysis algorithm to output at least one second prediction relating to a second target variable (Yeung, ¶[0221]: "The results of the comparison between a surgeon's indicated location for the lumen center and that predicted by different combinations of image processing and ANN-based analysis are summarized in FIGS. 20-22...", explains that Yeung teaches the ANN generates its own prediction of the lumen center (second prediction), with the possibility that this second prediction is informed by or combined with the outputs of the individual image processing methods); using the one or more first predictions (Yeung, ¶[0221]: "...a weighted average of the predicted X and Y positions for the lumen center and/or a weighted average of the confidence levels calculated for the results of the individual image processing methods could be used...", explains that Yeung teaches the outputs from the individual models/algorithms (first predictions) are used in combination, such as by weighted averaging, to inform or contribute to the ANN's prediction (second prediction)); calculating a prediction error of the at least one second prediction (Yeung, ¶[0220]: "...comparing the separation distance, Derr (in units of degrees), between the surgeon's indicated location for the lumen center and that determined by the ANN...", explains that Yeung calculates the prediction error as Derr, the difference between the ANN's (second) prediction and the surgeon's annotation (ground truth)); and optimizing the data analysis algorithm using the prediction error (Yeung, ¶[0221]: "...improving the design of the ANN model and improving the quantitative methods used for validation of the process...", explains that Yeung teaches using the error (Derr) calculated between the second prediction and annotation to validate and further optimize the ANN model).
Regarding claim 53, the modified Yeung teaches that the automated medical device is configured to allow real-time updating of a trajectory of the medical instrument and/or the automated medical device is configured to steer the medical instrument toward the target such that the medical instrument traverses the non-linear trajectory within the body of the patient (Yeung, ¶[0006]: "Systems and methods are provided for automated steering control of a robotic endoscope...the control systems comprising: a) a first image sensor...and b) one or more processors...configured to generate a steering control output signal based on an analysis of data derived from the first input data stream using a machine learning architecture, wherein the steering control output signal adapts to changes in the data derived from the first input data stream in real time", explains that the automated device adapts its steering signal in real time based on input data; ¶[0018]: "In some embodiments, the steering control output signal is used to control one or more actuators of the robotic endoscope...effect a movement of a steerable distal portion of the robotic endoscope...about one or more axes of rotation comprising a roll axis, a yaw axis, a pitch axis, or any combination thereof", explains that the system is capable of complex, multi-axis movement, which includes traversal of non-linear trajectories).
Regarding claim 56, the modified Yeung teaches that collecting one or more new datasets, at least one of the one or more new datasets being related to an automated medical device configured to steer the medical instrument toward the target in the body of the patient (Yeung, ¶[0164]: "the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data", showing collection of new datasets—including sensor data from a medical device—for use in analysis and control; ¶[0170]: "the sensor data may be used as input data directly. For instance, images captured by a camera that provide information about the position of the guiding portion relative to the colon wall may be supplied to the neural network as input data", showing collection of new datasets from a robotic device configured to steer a medical instrument in the body); pre-processing the one or more new datasets (Yeung, ¶[0084]: "the image data may be pre-processed using one or more image processing algorithms prior to providing it as input to the machine learning algorithm", expressly demonstrating data pre-processing; ¶[0170]: "data derived from pre-processing of sensor data, such as gradient maps, motion vectors, or locations of the lumen center extracted from images, may be supplied to the neural network as input data", explains that Yeung teaches supplying pre-processed data to the system); executing the data analysis algorithm using at least a portion of the one or more new datasets (Yeung, ¶[0201]: "a processor that is configured to generate a steering control output signal based on an analysis of the first image data stream using a machine learning architecture", explains that Yeung teaches execution of a data analysis algorithm on new datasets for real-time control; ¶[0206]: "the route may be automatically selected based on processing of the real-time image data stream", further demonstrating algorithmic processing of new datasets to generate outputs); extracting features from the one or more new datasets; (Yeung, ¶[0037]: "the control system comprises: a) one or more image sensors configured to capture two or more images of a colon lumen; and b) one or more processors that are individually or collectively configured to (1) perform automated feature extraction on the series of two or more images to determine a center position of the colon lumen", explains that Yeung teaches extracting features from new datasets using automated feature extraction); and wherein the one or more new datasets further comprise one or more of: clinical procedure related dataset, patient related dataset and administrative related dataset (Yeung, ¶[0071]: "FIG. 15 shows a block diagram of an exemplary adaptive steering control system for a robotic endoscope that uses data collected by image sensors to generate a steering control signal, where the image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction", explains that Yeung teaches the collection and use of clinical procedure related datasets, specifically image data acquired during the procedure and used as part of the system’s dataset for analysis and navigation; ¶[0164]: "The input data may be obtained from a training process or stored in a database. In some cases, the sensor data and associated steering history data may be collected prior to obtaining the desired output data 811. In some cases, the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data", explains that Yeung discloses collection and storage of sensor and steering data during operation of the endoscope, which constitutes a clinical procedure related dataset).
With respect to obtaining an output of the data analysis algorithm, the output being at least one of prediction and detection of a clinical condition related to the insertion of the medical instrument toward the target in the body of the patient, Yeung does not teach that the output of its data analysis algorithm is a prediction or detection of a clinical condition related to insertion. Boddington teaches trained outcome prediction modules and multiple classifiers applied to procedural data to output predictions of complications or problems during a procedure (Boddington, ¶[0009], ¶[0069]). Huo teaches that pneumothorax is a clinically significant and frequent complication of instrument insertion, and provides probabilities and clinical variables for predicting its occurrence (Huo, Abstract, Tables 2-4). One skilled in the art would have found it obvious, in view of Boddington’s procedural risk prediction and Huo’s clinical complication data, to adapt Yeung’s ML data analysis system so that the output is a prediction or detection of a clinical condition (such as pneumothorax) related to the insertion of a medical instrument. It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung in view of Boddington and Huo to obtain an output of the data analysis algorithm that is a prediction or detection of a clinical condition related to insertion. Yeung supplies the ML and data framework, Boddington teaches the procedural complication prediction output, and Huo provides clinical linkage and probability/statistics for the complication. The benefit is real-time, automated detection of clinically significant risks, supporting improved patient safety.
Regarding claim 60, Yeung teaches that the system comprises: a training module comprising: a memory configured to store one or more existing datasets, metadata, data annotations, a database of features extracted from the one or more existing datasets and/or one or more pre-trained models (Yeung, ¶[0071]: "FIG. 15 shows a block diagram of an exemplary adaptive steering control system for a robotic endoscope that uses data collected by image sensors to generate a steering control signal, where the image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction", demonstrating a module for collecting, storing, and processing data for model/algorithm training; ¶[0164]: "the input data may be obtained from a training process or stored in a database. In some cases, the sensor data and associated steering history data may be collected prior to obtaining the desired output data 811. In some cases, the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data", shows storing datasets (implicitly uses memory), annotations, and extracted features in a memory); one or more processors configured to: create a training set comprising a first data portion of the one or more existing datasets (Yeung, ¶[0037]: "one or more processors that are individually or collectively configured to (i) perform automated feature extraction ... (iii) determine a navigation direction that directs the robotic colonoscope towards the center position of the colon lumen using a machine learning architecture-based analysis", shows the processor is configured to extract features and train models; ¶[0164]: "the input data may be obtained from a training process or stored in a database", shows that Yeung teaches obtaining data for training sets from stored or newly collected data; ¶[0156]–[0157]: "A number of training datasets may be supplied to the neural network for training the parameters, e.g., weights, biases, and threshold values . . . FIG. 8A schematically illustrates exemplary training datasets supplied to a neural network 803 for learning the parameters of the network . . . the training datasets may comprise input data 813 and desired output data 811", teaches creating training datasets from collected data for model/algorithm training); train the data analysis algorithm using the training set (Yeung, ¶[0071]: "image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction", shows that Yeung teaches training a model/algorithm using the training data); and wherein at least one of the one or more existing datasets is related to the automated medical device configured to steer the medical instrument toward the target in the body of a patient and/or to operation thereof (Yeung, ¶[0200]: "The disclosed systems and methods can be used for automated steering control of robotic endoscopes", ¶[0004]: "The steering control system may receive input sensor data collected by the plurality of sensors and output a target direction or control signal to the actuation unit of the colonoscope", shows that Yeung teaches the datasets are related to an automated medical device used to steer an instrument to a target in the body; ¶[0084]: “The input data and the desired output data may be time-registered so that the correspondence between the input data and desired output data is known”, explains time-registration of procedural telemetry (sensor streams + steering signals) which the model/algorithm uses as aligned inputs and outputs during data collection).
With respect to the system for generating a data analysis algorithm for predicting and/or detecting a clinical condition related to insertion of a medical instrument toward a target in a body of a patient, Yeung teaches a system and algorithm for collecting, processing, and analyzing procedural data during instrument insertion (Yeung, ¶[0037], ¶[0071], ¶[0164]), but does not disclose training models to predict or detect clinical complications. Boddington teaches that a system may use multiple trained classifiers and outcome prediction modules on intraoperative/procedural data to calculate decision risks and output clinical outcome predictions (Boddington, ¶[0009], ¶[0069], ¶[0167]). Huo teaches that pneumothorax is a frequent, clinically significant complication of CT-guided instrument insertion and provides statistics and modifiable risk factors for its occurrence (Huo, Abstract, Tables 2–4). One skilled in the art would have found it prima facie obvious to modify Yeung in view of Boddington and Huo to generate and train a data analysis algorithm for predicting and/or detecting a clinical condition related to insertion of a medical instrument. Yeung supplies the technical ML platform, Boddington supplies procedural risk prediction models, and Huo supplies the clinical motivation, context, and probability data. The benefit is a real-time, automated system for clinical risk detection and procedural safety.
With respect to the clinical condition comprising pneumothorax, breathing anomalies, internal bleeding, or any combinations thereof, Yeung teaches training and deploying a machine-learning architecture on procedural sensor datasets for automated steering, but does not teach that the model’s output pertains to specific clinical conditions such as pneumothorax, breathing anomalies, or internal bleeding (as shown above).
Boddington teaches that an intra-operative AI platform reads and interprets subject data to “calculate surgical decision risks” and to present “a surgical outcome prediction” with “known predictors and indicators of complications datasets” and “information from subject health records” among the enumerated inputs (Boddington, ¶[0104]-[0106]: “procedural medical image datasets”, “known predictors and indicators of complications datasets”, “subject HER information data” (where "HER" is understood to mean EHR, Electronic Health Records), “failure risk score datasets”, “information from subject health records… configured to include information that will potentially have an impact on the outcome of the procedure… provides the user with a prediction of optimal or suboptimal outcome and an associated Failure Risk Score”). However, it does not specify specific clinical conditions such as pneumothorax, breathing anomalies, or internal bleeding.
Huo further quantifies the clinical burden and actionable factors for insertion-related pneumothorax, reporting that “The overall pooled incidence for pneumothorax was 25.9% and chest drain insertion was 6.9%” (Huo, Abstract/Results), with institutional chest drain practice variability “(lowest 0.3% … vs highest 15%)” (Huo, Discussion). Huo enumerates specific, model-mappable risk factors including “fissure crossed,” “bulla crossed,” “multiple pleural punctures,” “emphysematous lungs,” lesion size and depth, and pleural contact (Huo, Abstract), and shows that “Multiple pleural punctures (>1) more than tripled the risk” (Huo, Results) and “Emphysema … increased pneumothorax incidence … [and] chest drain insertion” (Huo, Results). Huo also distinguishes modifiable vs non-modifiable determinants and highlights patient positioning as a modifiable factor (Huo, Introduction; Abstract). Detection protocols via plain film and/or CT are noted (Huo, Methods). These data concretely motivate configuring the model/algorithm to output complication-specific predictions (e.g., pneumothorax) and to weight telemetry/features aligned with known risk factors during insertion.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the modified Yeung in view of Boddington and Huo to configure the trained model/algorithm to output a prediction or detection for specific clinical conditions (including pneumothorax and, by extension, other intra-operative complications such as breathing anomalies or internal bleeding) during insertion. Yeung provides the procedural data pipeline and ML architecture; Boddington teaches intra-operative complication/outcome prediction using datasets that include known complication predictors and subject health-record information; and Huo provides clinical motivation and quantified incidence for pneumothorax in insertion procedures. The combination is feasible because Boddington’s risk-prediction modules operate on the same class of procedural and subject datasets that Yeung collects and structures, and obvious because targeting known insertion complications (with pneumothorax explicitly documented by Huo) is a predictable use of those models. The risk factors quantified by Huo (e.g., multiple pleural punctures, emphysema, lesion depth/pleural contact, patient positioning) correspond to procedural and subject attributes that can be encoded as model/algorithm features or labels alongside Yeung’s time-registered device telemetry and Boddington’s EHR inputs, thereby making configuration of complication-specific outputs technically straightforward. The benefit is targeting these known complication-specific outputs (e.g., pneumothorax risk) and related condition signals (e.g., breathing anomalies, internal bleeding) with real-time probabilities yields improved intra-operative safety and workflow guidance.
Regarding claim 61, Yeung teaches that the one or more processors are further configured to one or more of: perform pre-processing on the one or more existing datasets, extract features from the one or more existing datasets, perform data augmentation and validate the data analysis model using a second data portion of the one or more existing datasets (Yeung, ¶[0037]: "one or more processors that are individually or collectively configured to (i) perform automated feature extraction ...", shows processors are configured for feature extraction; ¶[0071]: "FIG. 15 shows a block diagram of an exemplary adaptive steering control system for a robotic endoscope that uses data collected by image sensors to generate a steering control signal, where the image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction", explains that Yeung teaches pre-processing and feature extraction from existing datasets before training a model; ¶[0164]: "the input data may be obtained from a training process or stored in a database. In some cases, the sensor data and associated steering history data may be collected prior to obtaining the desired output data 811. In some cases, the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data", shows the ability to use different portions of datasets for validation as well as training; ¶[0210]: "The three-dimensional image map may comprise image data and augmented information overlaid onto the image data", explains that Yeung teaches augmented information for richer datasets and enhanced analysis, supporting dataset augmentation).
Regarding claim 62, Yeung teaches wherein the training module is located on a remote server, an “on premise” server or a computer associated with the automated medical device; and/or wherein the remote server is a cloud server (Yeung, ¶[0174]: "The processor may be a processing unit of a computer system. The processors or the computer system used for training the machine learning algorithm may or may not be the same processors or system used for implementing the steering control system. The computer system may enable a cloud computing approach. The computer system may allow training datasets to be shared and updated by one or more computer systems or across one or more machine learning-based robotic colonoscopy systems", explains that Yeung teaches cloud computing, sharing and updating training datasets across remote and local computer systems, and distributed server architectures; ¶[0175]: "The computer system can be operatively coupled to a computer network (“network”) with the aid of a communication interface. The network can include one or more computer servers, which can enable distributed computing, such as cloud computing. In some instances, the machine learning architecture is linked to, and makes use of, data and stored parameters that are stored in cloud-based database", explains that Yeung teaches distributed computing and explicit use of cloud servers, remote servers, and peer-to-peer or client-server architectures for machine learning).
Regarding claim 63, Yeung, Boddington, and Huo collectively teach that an inference module comprising: a memory configured to store at least one of: one or more new datasets, metadata and the data analysis algorithm (Yeung, ¶[0071]: "FIG. 15 shows a block diagram of an exemplary adaptive steering control system for a robotic endoscope that uses data collected by image sensors to generate a steering control signal, where the image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction", explains that Yeung teaches a system with a memory for storing new datasets, metadata, and analysis algorithms for real-time data processing and inference; ¶[0164]: "the input data may be obtained from a training process or stored in a database. In some cases, the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data", shows Yeung anticipates storing new datasets and associated metadata for analysis and inference); and one or more processors configured to: perform pre-processing on the one or more new datasets (Yeung, ¶[0165]: "The input data may be processed, organized, or annotated prior to use as training data. In some cases, the input data may be processed and/or organized to facilitate use as input for a machine learning model", explains that Yeung teaches pre-processing and organization of new datasets before analysis by a machine learning algorithm; ¶[0037]: "one or more processors that are individually or collectively configured to (i) perform automated feature extraction ...", shows processors are configured for feature extraction and pre-processing), and wherein at least one of the one or more new datasets is related to the automated medical device configured to steer a medical instrument toward a target in the body of the patient and/or to operation thereof (Yeung, ¶[0037]: "a control system for providing an adaptive steering control output signal for steering a robotic colonoscope... one or more processors that are individually or collectively configured to... perform automated feature extraction... and determine a navigation direction that directs the robotic colonoscope towards the center position... using a machine learning architecture-based analysis", explains Yeung teaches new datasets related to an automated medical device configured for steering a medical instrument toward a target).
With respect to obtaining an output of the data analysis algorithm, the output being at least one of the prediction and detection of the clinical condition related to the insertion of the medical instrument toward the target in the body of the patient, Yeung teaches a system comprising a memory and processors configured to store and pre-process new datasets, metadata, and data analysis algorithms, and to execute machine learning models for real-time inference using procedural data collected by an automated medical device (Yeung, ¶[0037], ¶[0071], ¶[0164], ¶[0165], see claim 60 above). However, Yeung does not explicitly teach that the output is a prediction or detection of a clinical condition. Boddington teaches predictive models and classifiers for outputting surgical risk and complication probabilities based on intraoperative/procedural data (Boddington, ¶[0009], ¶[0069], ¶[0167], FIGs. 25–27). Huo teaches clinical motivation, quantitative probabilities, and modifiable factors for complications such as pneumothorax (Huo, Abstract, Tables 2–4, see claim 60 above). One of ordinary skill in the art would have found it prima facie obvious before the effective filing date of the claimed invention to have modified Yeung in view of Boddington and Huo to configure the inference module and data analysis algorithm to provide prediction or detection of clinical conditions related to instrument insertion. The benefit is automated, real-time risk prediction and enhanced procedural safety.
Regarding claim 64, Yeung, Boddington, and Huo collectively teach that the one or more processors are further configured to one or more of: load one or more trained models per task, extract features from the one or more new datasets, execute a post-inference business logic and display the output of the data analysis algorithm to a user (Yeung, ¶[0037]: "one or more processors that are individually or collectively configured to (i) perform automated feature extraction on the series of two or more images to determine a center position of the colon lumen... (iii) determine a navigation direction that directs the robotic colonoscope towards the center position of the colon lumen using a machine learning architecture-based analysis...", explains Yeung teaches feature extraction and task-specific inference/model loading; ¶[0071]: "where the image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction", explains Yeung teaches use of trained models per task and automated output display; ¶[0210]: "The three-dimensional image map may comprise image data and augmented information overlaid onto the image data. The augmented information may comprise steering vector data for the robotic endoscope, an image of the robotic endoscope, the position (or center line) of the lumen center, and various other pieces of information... The three-dimensional image map may be a first person view or a perspective view of a reconstructed lumen interior and the robotic endoscope", explains Yeung teaches displaying output (processed data and features) to a user in a variety of ways).
Claims 49 and 54-55 is rejected under 35 U.S.C. 103 as being unpatentable over Yeung et al. (US 20180296281 A1), hereto referred as Yeung, and further in view of Boddington et al. (US 20210177522 A1), hereto referred as Boddington, and further in view of Huo et al. (Huo YR, Chan MV, Habib AR, Lui I, Ridley L. Pneumothorax rates in CT-Guided lung biopsies: a comprehensive systematic review and meta-analysis of risk factors. Br J Radiol. 2020 Apr 1;93(1108):20190866. doi: 10.1259/bjr.20190866. Epub 2020 Jan 3.), hereto referred as Huo, and further in view of Lu et al. (US 20180000446 A1), hereto referred as Lu, and further in view of He et al. (He, Changyan et al. “Enabling Technology for Safe Robot-Assisted Retinal Surgery: Early Warning for Unsafe Scleral Force.” Proceedings - IEEE International Conference on Robotics and Automation. IEEE, 2019. 3889–3894), hereto referred as He, and further in view of Amarasingham et al. (US 20130262357 A1), hereto referred as Amarasingham.
The modified Yeung teaches claim 45 as described above.
Regarding claim 49, the modified Yeung teaches that the clinical procedure related dataset comprises parameters selected from: medical procedure type, target organ, target size, target type, type of medical instrument, dimensions of the medical instrument, complications before, during and/or after the procedure, adverse events before, during and/or after the procedure, respiration signals of the patient, or any combination thereof (Yeung, ¶[0013]: "The system 100 can be used to automatically control a robotic endoscope during a procedure, e.g., a colonoscopy or esophagogastroduodenoscopy (EGD)", explains that Yeung teaches the medical procedure type and target organ; ¶[0071]: "...adaptive steering control system for a robotic endoscope that uses data collected by image sensors... where the image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction", explains that Yeung teaches use of image data to support identification of the target organ, target size, and navigation related to the type of procedure and instrument).
With respect to the patient related dataset comprising parameters selected from: age, gender, race, medical condition, medical history, vital signs before, after and/or during the procedure, body dimensions, pregnancy, smoking habits, demographic data, or any combination thereof, Yeung teaches clinical procedure related datasets, as shown above, but does not expressly recite specific demographic, physiologic, or medical history parameters. Amarasingham is directed to a clinical predictive and monitoring system that relies on patient-related datasets, including "gender," "vital signs," "medical history," and "daily weight readings" (Amarasingham, ¶[0014]-[0016]). It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the modified Yeung in view of Amarasingham to incorporate additional patient-related dataset fields such as demographic, physiologic, and medical history parameters. The benefit of integrating such data is to enhance clinical decision-making and improve patient outcomes through more robust patient data integration.
With respect to the administrative related dataset comprises parameters selected from: institution, physician, staff, system serial number, disposable components used in the procedure, software version, operating system version, configuration parameters, or any combination thereof, Yeung does not enumerate administrative dataset parameters. However, Yeung repeatedly discusses use of system inputs, recorded and collected sensor and procedure data, and stored information in the context of a learning system (see, e.g., ¶[0071], ¶[0164]), thereby implying that other relevant dataset fields, including patient or administrative data, could be added or integrated as needed for improved analytics or system control, as would be recognized by a person of ordinary skill in the art. Amarasingham describes a clinical predictive and monitoring system that explicitly incorporates administrative data fields, including physician notes and hospital usage, as well as configuration parameters (Amarasingham, ¶[0014], ¶[0015], ¶[0048]). It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the modified Yeung in view of Amarasingham to include administrative parameters such as physician, institution, and configuration details. The benefit would be improved traceability, compliance, and operational integration.
Note: analogous art position (Amarasingham): The reference is used only for the limited proposition that EHR/administrative fields are recognized predictors of clinical outcomes. That proposition is reasonably pertinent to the claimed problem and, thus, Amarasingham qualifies as analogous art. In any event, the combination remains proper without Amarasingham because Boddington already discloses inclusion of EHR/predictor datasets in intra-operative outcome prediction; Amarasingham is cumulative corroboration and does not alter the rejection’s theory.
Regarding claim 54, the combined Yeung, Boddington, Huo, and Amarasingham does not fully teach that training the data analysis algorithm comprises training the data analysis algorithm to estimate probability of occurrence of the clinical condition during insertion of the medical instrument toward the target in the body of the patient; and the training set comprises one or more target parameters relating to the clinical condition occurrence during one or more previous procedures for inserting a medical instrument toward a target in a body of a patient. Rather, Yeung teaches collecting and using training data from previous procedures to train and optimize its control algorithms (see, e.g., ¶[0086]), but it does not expressly teach that these parameters relate to clinical condition occurrence or that they are used to estimate the probability of such occurrences, as required by the claim. Boddington, in contrast, investigates surgical guidance and outcome analytics using artificial intelligence, predictive modeling, and datasets from prior surgical procedures to optimize and predict patient outcomes. Specifically, it provides detailed and explicit support for using datasets from previous procedures—including outcome and complication data—to predict and calculate the probability of specific adverse events or clinical conditions (Boddington, ¶[0166], ¶[0167], ¶[0168]). The model/algorithm in Boddington is trained on multiple datasets, including those capturing the occurrence of clinical outcomes, to produce probability estimates of adverse events and conditions. It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combined Yeung, Boddington, Huo, and Amarasingham to train the data analysis algorithm to estimate the probability of occurrence of a clinical condition during insertion of a medical instrument toward a target in a body of a patient, and to use training sets comprising target parameters relating to clinical condition occurrence during previous procedures, as such probability estimation and dataset integration are well-established in medical outcome prediction and analytics. The benefit of the combination is improved clinical risk assessment, enabling the system to predict, quantify, and intervene in the likelihood of adverse events, thereby enhancing patient safety and procedure planning.
Regarding claim 55, the combined Yeung, Boddington, Huo, and Amarasingham teaches that the training of the data analysis algorithm further comprises training one or more individual models and using one or more predictions generated by the one or more individual models as input for training the data analysis algorithm (Yeung, ¶[0221]: ...results of the comparison between a surgeon’s indicated location for the lumen center and that predicted by different combinations of image processing and ANN-based analysis..., explains that Yeung teaches each individual image processing method generates its own prediction, and the outputs from the individual models are used in combination as input for a higher-level ANN-based analysis; the error between the ANN prediction and annotation is used to validate and optimize the model/algorithm (also see claim 52 above)).
With respect to when the clinical condition is pneumothorax the one or more individual models comprise one or more of: a model for predicting a patient pose during an instrument insertion and steering procedure, a model for estimating pleural cavity volume, a model for estimating fissure crossing, a model for estimating bulla crossing, and a model for predicting respiration anomalies during an instrument insertion and steering procedure, Yeung does not teach using individual models for predicting patient pose, pleural cavity volume, fissure crossing, bulla crossing, or respiration anomalies related to pneumothorax. Boddington teaches the use of learned classifiers—including deep neural networks—to predict or guide anatomical pose during a procedure using intraoperative image data and real-time analysis (Boddington, ¶[0119]–[0120]). Huo teaches that pleural cavity volume, fissure crossing, bulla crossing, and respiration anomalies are each recognized, quantifiable risk factors for pneumothorax after instrument insertion and provides clinical datasets and probabilities for these variables (Huo, Abstract, Tables 2-4). One of ordinary skill in the art would have been motivated by Huo’s teaching to include these risk factors as inputs to or outputs of individual predictive models, and would have found it obvious to apply Boddington’s procedural risk prediction approach within Yeung’s multi-model ensemble framework to generate predictions for each listed variable. It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combined Yeung, Boddington, Huo, and Amarasingham in view of Boddington and Huo to use individual models for predicting patient pose, pleural cavity volume, fissure crossing, bulla crossing, or respiration anomalies during an instrument steering procedure as part of a pneumothorax risk prediction system. The combination is possible because Yeung provides the ensemble model structure, Boddington provides procedural pose and risk prediction models, and Huo provides the clinical data, variable definitions, and motivation. The benefit is improved real-time, patient-specific pneumothorax risk prediction and intraoperative decision-making.
With respect to "and/or wherein when the clinical condition is internal bleeding, the one or more individual models comprise one or more of: a model for estimating blood vessel locations, a model for predicting blood vessel movement due to breathing motion, a model for estimating locations of sensitive tissues, a model for predicting movement of sensitive tissues due to breathing motion and a model for predicting entrance of blood vessels and/or sensitive tissues into the trajectory of the medical instrument during an instrument steering procedure", the claim recites “and/or” between the pneumothorax and internal bleeding elements, only one of these alternatives must be anticipated or rendered obvious for the claim to be unpatentable. As shown above, the combination of Yeung, Irvin, and Boddington renders obvious at least one of the required individual models for pneumothorax (specifically, a model for predicting patient pose during an instrument steering procedure). Thus, anticipation or obviousness of the internal bleeding element is not required for this rejection.
Claim 57 is rejected under 35 U.S.C. 103 as being unpatentable over Yeung et al. (US 20180296281 A1), hereto referred as Yeung, and further in view of Boddington et al. (US 20210177522 A1), hereto referred as Boddington, and further in view of Huo et al. (Huo YR, Chan MV, Habib AR, Lui I, Ridley L. Pneumothorax rates in CT-Guided lung biopsies: a comprehensive systematic review and meta-analysis of risk factors. Br J Radiol. 2020 Apr 1;93(1108):20190866. doi: 10.1259/bjr.20190866. Epub 2020 Jan 3.), hereto referred as Huo, and further in view of Lu et al. (US 20180000446 A1), hereto referred as Lu, and further in view of He et al. (He, Changyan et al. “Enabling Technology for Safe Robot-Assisted Retinal Surgery: Early Warning for Unsafe Scleral Force.” Proceedings - IEEE International Conference on Robotics and Automation. IEEE, 2019. 3889–3894), hereto referred as He, and further in view of Taylor et al. (Taylor, Andrew G, Clinton Mielke, and John Mongan. “Automated Detection of Moderate and Large Pneumothorax on Frontal Chest X-Rays Using Deep Convolutional Neural Networks: A Retrospective Study.” PLoS medicine 15.11 (2018): e1002697–e1002697. Web.), hereto referred as Taylor.
The modified Yeung teaches claim 45 as described above.
Regarding claim 57, the modified Yeung teach that the clinical condition the data analysis algorithm is trained to provide the prediction and/or detection thereof is pneumothorax (As shown above in claim 45, Yeung teaches that the data analysis algorithm can be trained to provide prediction and/or detection of a clinical condition (pneumothorax), with support for training, validation, and dataset integration), and the one or more new datasets comprise one or more images of a region of interest (Yeung, ¶[0205]: "FIG. 13 shows an exemplary field-of-view comprising an object of interested 1301 and a tool for surgical operations. The object of interest may be detected or recognized using methods such as automated image feature extraction as described elsewhere herein. The distal end of the colonoscope, where the camera is attached, may be controlled by the provided control system such that the object 1301 maintains a stationary location within the field-of-view. In some cases, augmented information such as a contour of the detected object of interest may be overlaid onto the real-time images. The control system may be configured to compensate for undesired movements such as shaking and automatically track the object of interest as the colonoscope is advanced or during the surgical procedure such that the surgeon is not distracted by the unwanted movements", explains that Yeung anticipates collection and real-time use of images of a region of interest during a procedure for analysis and navigation; ¶[0071]: "FIG. 15 shows a block diagram of an exemplary adaptive steering control system for a robotic endoscope that uses data collected by image sensors to generate a steering control signal, where the image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction", explains that Yeung further supports acquisition and use of new image data during the procedure for navigation and control).
With respect to the output of the data analysis algorithm comprises a probability of pneumothorax occurrence, Yeung provides the machine learning platform and data acquisition framework, but does not teach probability outputs for clinical conditions. Boddington teaches the use of trained classifiers and outcome prediction modules to generate probability outputs for procedural complications (Boddington, ¶[0009], ¶[0069], ¶[0167]). Huo provides clinical context, statistical probabilities, and procedural risk factors for pneumothorax related to instrument insertion (Huo, Abstract, Tables 2–4). One skilled in the art would have found it obvious, in view of Boddington’s outcome probability models and Huo’s quantified complication statistics, to adapt Yeung’s system to output a probability of pneumothorax occurrence for clinical decision support. It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified The modified Yeung in view of Boddington and Huo to output a probability of pneumothorax occurrence from new imaging and procedural data. The combination is possible because Yeung provides data integration and ML capability, Boddington teaches risk probability calculation, and Huo provides clinical probabilities and modifiable factors for pneumothorax. The benefit is automated, quantitative clinical risk assessment for real-time procedural safety.
With respect to detecting one or more critical tissues in the one or more images; detecting pleural cavity volume, Yeung teaches a system and analytical framework that uses image data collected by sensors for real-time analysis, navigation, and feature extraction during interventional procedures (Yeung, ¶[0071]; ¶[0205]), but does not teach automated detection of critical tissues or pleural cavity volume. Taylor teaches deep learning-based automated detection of moderate and large pneumothorax, which requires identifying lung margins and pleural cavity (Taylor, Abstract). Huo teaches evaluating and modifying patient and tissue positioning, and selecting procedural parameters based on direct analysis of tissue and cavity anatomy and modifiable factors (Huo, Abstract, Tables 2–4). It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified The modified Yeung in view of Taylor and Huo to enable detection and assessment of critical tissues and pleural cavity volume. The combination is possible because Yeung provides the data analysis framework, Taylor provides automated tissue/cavity identification and analysis necessary for pneumothorax detection, and Huo provides clinical integration, modifiable factors, and actionable anatomical evaluation. The benefit is improved safety through precise tissue and cavity analysis in interventional procedures.
With respect to determining if the probability of pneumothorax occurrence is above a predetermined threshold, Yeung provides real-time data integration, but not probability thresholding for clinical alerting. Boddington teaches use of probability outputs and clinical thresholds for decision risk prediction and alerting (Boddington, ¶[0069], ¶[0167]). Taylor teaches clinical workflow for thresholding AI-based probability outputs and flagging high-risk images (Taylor, p. 2, 'Author Summary'; p. 5–6, 'Frontal image selection'). Huo provides quantitative, evidence-based risk prediction using clinical and procedural variables, enabling clinicians to assess pneumothorax risk prior to procedures and tailor management accordingly (Huo, Abstract, Tables 2–4; p. 1, 'Introduction', ¶3). It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yeung in view of Boddington, Taylor, and Huo to provide clinical probability thresholding for pneumothorax. The combination is possible because Yeung supplies the procedural data pipeline, Boddington and Taylor teach clinical thresholding and alerting, and Huo provides the risk stratification and evidence-based context. The benefit is a robust, integrated, and automated workflow for real-time risk estimation, threshold-based triage, and targeted pneumothorax prevention in image-guided interventions.
With respect to if the probability of pneumothorax occurrence is determined to be above the predetermined threshold, generating an alert, and providing a recommendation of one or more mitigating actions to reduce the probability of pneumothorax occurrence, Yeung teaches a system that collects and analyzes imaging and procedural data in real time, providing the framework for risk assessment and procedural guidance (Yeung, ¶[0071]; ¶[0205]), but does not teach generating an alert or recommending mitigating actions based on elevated pneumothorax risk. Taylor teaches automated clinical alerting when the probability of pneumothorax exceeds a predetermined threshold, ensuring that clinicians are notified of high-risk cases for prompt intervention (Taylor, Abstract). Huo teaches that when pneumothorax risk is identified—based on clinical, lesion, and procedural factors—specific, evidence-based procedural modifications such as adjusting patient position, using smaller caliber needles, and applying tract sealants are known to reduce the probability of pneumothorax, and recommends integrating these actions into procedural workflow and guidelines (Huo, Tables 2–4; Abstract). It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yeung in view of Taylor and Huo to provide a system that, when the assessed probability of pneumothorax occurrence exceeds a threshold, automatically generates a clinical alert and recommends actionable procedural modifications to reduce pneumothorax risk. The combination is possible because Yeung supplies the clinical data and analytic platform, Taylor provides threshold-based clinical alerting, and Huo supplies specific, evidence-based recommendations for risk mitigation in high-probability cases. The benefit is a fully integrated system for real-time risk monitoring, threshold-based alerting, and targeted procedural guidance, maximizing patient safety during interventional procedures.
Response to Arguments
Objections
Applicant's arguments filed 1/28/2026, pages 10-11, regarding the previous Objections of claims 46 and 63 have been fully considered and are persuasive. The previous Objections have been withdrawn.
35 U.S.C. §103
Applicant's arguments filed 1/28/2026, pages 11-16, regarding the previous 103 Rejections of claims 45-57 and 60-64 have been fully considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. That is, there are new grounds of rejection. Additionally, the arguments are not persuasive at least for the outlined below.
Applicant’s Argument: Yeung is limited to closed-loop steering control and does not teach or suggest generating predictions or detections of clinical conditions.
Examiner’s Response: The argument is not persuasive. Yeung teaches collecting time-registered sensor data and steering history data generated during automated navigation of a medical instrument (Yeung, ¶[0084], ¶[0097], ¶[0164]). Yeung further teaches that such datasets include “steering history data” comprising steering vectors corresponding to motion of the distal end during navigation (Yeung, ¶[0163]; FIG. 8B), and that these data are collected concurrently during operation (Yeung, ¶[0164]). The claims do not require that such data be originally collected for prediction purposes. Under the broadest reasonable interpretation, such procedural telemetry constitutes general-purpose operational data that can be used as input features to train a model that outputs predictions.
Applicant’s Argument: Boddington does not utilize device-specific or trajectory-related data and is not tied to instrument steering.
Examiner’s Response: The argument is not persuasive. Boddington teaches intra-operative prediction using procedural datasets, including predictors of complications and subject data (Boddington, ¶[0104]–[0106]). The modified Yeung supplies device-generated, time-registered operational data during steering. Lu further confirms that insertion-phase tracking and image data are captured contemporaneously with instrument motion and used to characterize trajectory during the procedure (Lu, ¶[0026], ¶[0028]). Accordingly, the art teaches using device-related procedural data as inputs to a predictive model.
Applicant’s Argument: The cited art does not teach steering-phase operational data generated during execution of a non-linear trajectory.
Examiner’s Response: The argument is not persuasive. Yeung teaches collecting time-registered sensor and steering data during non-linear navigation. Lu teaches that, during insertion, sensor data representative of sensing the ultrasound plane are generated as the needle is inserted and are used, with image and tracking data, to predict the trajectory during the procedure (Lu, ¶[0026], ¶[0028]). While Lu’s trajectory is substantially linear, the operative principle is contemporaneous data capture during instrument motion; applied to Yeung’s non-linear steering context, this confirms datasets include data generated while the instrument is actively traversing its path.
Applicant’s Argument: The cited art does not teach generating a prediction during insertion and steering of the medical instrument.
Examiner’s Response: The argument is not persuasive. The modified Yeung, in view of Boddington, teaches intra-operative prediction based on procedural data collected during device operation (Boddington, ¶[0009], ¶[0069]). He teaches a time-series neural network that uses a history of operational data collected during manipulation to output a forward-looking prediction in real time during the ongoing task (He, Section II-C; Section III-B). Thus, the art teaches generating predictions during insertion and steering of the instrument.
Applicant’s Argument: The references relate to different anatomical contexts and are not applicable.
Examiner’s Response: The argument is not persuasive. Each reference concerns insertion and manipulation of a medical instrument in a patient and collection of procedural/sensor data during operation. Yeung expressly indicates applicability to various robotic or endoscopic systems (Yeung, ¶[0083]). Applying known data-driven prediction techniques across analogous interventional contexts would have been a predictable use of prior art elements. Yeung further expressly discloses applicability across a wide range of endoscopic systems used in varied anatomical regions, including bronchoscopes for examination of the bronchus, nephroscopes, cystoscopes, and laparoscopes (Yeung, ¶[0094]), confirming that the disclosed data-driven architecture is not confined to colorectal anatomy.
Applicant’s Argument: The cited art does not teach trajectory-dependent prediction.
Examiner’s Response: The argument is not persuasive. Yeung provides time-registered steering and sensor data corresponding to instrument motion along non-linear trajectories. Lu teaches deriving trajectory information from contemporaneously captured insertion-phase sensor data (Lu, ¶[0026], ¶[0028]). He teaches using time-series manipulation data to predict unsafe conditions during operation in real time (He, Section II-C; Section III-B). Together, these teachings demonstrate that trajectory-related operational data can be used as predictive inputs, rendering the claimed subject matter obvious. In particular, Yeung’s steering history data and time-registered sensor streams encode trajectory execution characteristics, including motion vectors and spatial relationships during navigation, which a person of ordinary skill in the art would have recognized as suitable predictive features when applying Boddington’s intra-operative risk modeling framework.
Conclusion
Any inquiry concerning this communication or earlier communications from the
examiner should be directed to AARON MERRIAM whose telephone number is (703) 756-
5938. The examiner can normally be reached M-F 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing
using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is
encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Jason Sims can be reached on (571)272-4867. The fax phone number for the
organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained
from Patent Center. Unpublished application information in Patent Center is available to
registered users. To file and manage patent submissions in Patent Center, visit:
https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for
more information about Patent Center and https://www.uspto.gov/patents/docx for
information about filing in DOCX format. For additional questions, contact the Electronic
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO
Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON MERRIAM/Examiner, Art Unit 3791
/MATTHEW KREMER/Primary Examiner, Art Unit 3791
Prosecution Insights