DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant' s arguments, filed 7/24/2026, have been fully considered. The following rejections and/or objections are either reiterated or newly applied. They constitute the complete set presently being applied to the instant application.
Applicants have amended their claims, filed 7/24/2026, and therefore rejections newly made in the instant office action have been necessitated by amendment.
Claims 45-64 are the currently pending claims. Claims 58 and 59 have previously been withdrawn. Claims 45, 55, 56, 60, and 63 have been amended. Claims 45-57 and 60-64 are hereby under examination.
Claim Interpretation
Claims 45 and 60 require the steering-phase operational data to include both a total radiation dose in the steering phase and checkpoint errors of a plurality of checkpoints. The separate requirement that the prediction or detection be "based at least in part on the steering-phase operational data" does not require every item contained in the steering-phase operational data, or specifically both the total radiation dose and checkpoint errors, to be used in generating the prediction or detection. The specification describes the automated-medical-device-related dataset as including, among other operational information, checkpoint errors and total radiation dose in the steering phase (Instant Application, ¶[0022]). The specification defines checkpoint error as deviation of the actual checkpoint location from the planned checkpoint location (Instant Application, ¶[0274]). The specification further states that the disclosed algorithms predict clinical conditions and/or complications "based on at least some of the collected data and/or parameters derived therefrom" (Instant Application, ¶[0008]). Consistent with that framework, the pneumothorax embodiment identifies checkpoint number and position, checkpoint errors and corrections, trajectory updates, magnitude of lateral steering, procedural images, and other procedure-related information as inputs to the pneumothorax-prediction model, without identifying total steering-phase radiation dose as an input (Instant Application, ¶[0283]). That same input set is used to train the model (Instant Application, ¶[0285]: "input data 1502, such as input described in FIG. 14, is used to train the pneumothorax model 1504"). The pneumothorax probability may then be recalculated during steering upon the instrument reaching individual checkpoints (Instant Application, ¶[0286]). No embodiment in which a recited clinical condition is predicted during the insertion and steering procedure identifies the total radiation dose in the steering phase as an input to that prediction. The pneumothorax embodiment of FIGS. 14-16 and the internal bleeding embodiment of FIG. 17 (Instant Application, ¶[0288]-[0289]) each describe generating the prediction without it. A construction requiring both recited data types to be used in generating the prediction or detection would therefore exclude every disclosed embodiment of the claimed prediction, and is not the broadest reasonable interpretation consistent with the specification. Accordingly, the steering-phase operational data may contain both recited data types while the prediction or detection is based on only some of the steering-phase operational data.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 45-48, 50-53, 56, and 60-64 are rejected under 35 U.S.C. 103 as being unpatentable over Yeung et al. (US 2018/0296281 A1), hereinafter Yeung, and further in view of Ben-David et al., “Evaluation of a CT-Guided Robotic System for Precise Percutaneous Needle Insertion,” Journal of Vascular and Interventional Radiology (2018), pp. 1-7, doi:10.1016/j.jvir.2018.01.002, hereinafter Ben-David, and further in view of Konh et al. (US 2020/0060772 A1), hereinafter Konh, and further in view of Komaki et al., “Robotic CT-guided out-of-plane needle insertion: comparison of angle accuracy with manual insertion in phantom and measurement of distance accuracy in animals,” European Radiology, vol. 30, pp. 1342-1349 (2020), doi:10.1007/s00330-019-06477-1, hereinafter Komaki, and further in view of Boddington et al. (US 2021/0177522 A1), hereinafter Boddington, and further in view of Zhao et al., “Logistic regression analysis and a risk prediction model of pneumothorax after CT-guided needle biopsy,” Journal of Thoracic Disease, vol. 9, no. 11, pp. 4750-4757 (2017), doi:10.21037/jtd.2017.09.47, hereinafter Zhao.
Regarding claim 45, Yeung teaches a computer-implemented method of generating a data analysis algorithm, wherein training datasets comprising known input sensor data and corresponding known output steering-control signals are analyzed using a machine-learning architecture (Yeung, ¶[0034], ¶[0156]-[0157]).
Yeung further teaches collecting one or more datasets, at least one of the one or more datasets being related to an automated medical device configured to steer a medical instrument toward a target such that the medical instrument traverses a non-linear trajectory within a body of a patient and/or to operation thereof (Yeung, ¶[0006]: “Systems and methods are provided for automated steering control of a robotic endoscope”; ¶[0097]: “the environment 105 may comprise multiple flexural, looping or bending sections through which the steering control system allows the colonoscope to be maneuvered”; ¶[0200]: “The disclosed systems and methods can be used for automated steering control of robotic endoscopes”). Maneuvering the instrument through the flexural, looping, or bending sections constitutes traversal of a non-linear trajectory.
Yeung further teaches wherein the one or more datasets comprise steering-phase operational data generated during execution of the non-linear trajectory (Yeung, ¶[0163]; Fig. 8B: “steering history data 815” include information related to recent steering-control actions, including steering vectors corresponding to motion of the distal end of the colonoscope; ¶[0164]: sensor data and associated steering-history data may be collected concurrently with obtaining desired output data; ¶[0097]: the colonoscope is maneuvered through flexural, looping, or bending sections).
Yeung further teaches creating a training set comprising a first data portion of the one or more datasets (Yeung, ¶[0034], ¶[0156]-[0157], wherein a plurality of training datasets comprising input data and desired output data are supplied to the neural network for training its parameters).
Yeung further teaches validating the data analysis algorithm using a validation set, the validation set comprising a second data portion of the one or more datasets (Yeung, ¶[0218]-[0221], wherein approximately 80% of the data were used to train the ANN and the remaining 20% were used to evaluate performance).
Also, regarding claim 45, Yeung does not fully teach wherein the steering-phase operational data comprise a total radiation dose in a steering phase and checkpoint errors of a plurality of checkpoints along the non-linear trajectory. Rather, Yeung teaches steering-history operational data generated during execution of a non-linear trajectory as discussed above. However, Yeung does not teach positional errors determined at a plurality of checkpoints or a total steering-phase radiation dose.
Ben-David teaches a real-time CT-guided robotic system that positions, inserts, and steers a percutaneous medical tool according to a predefined trajectory. The insertion module advances the tool while the robot positioning unit simultaneously “steers the tool during insertion according to the preplanned trajectory, making intraoperative needle error corrections and trajectory updates as appropriate” (Ben-David, p. 2, “Robotic and Navigation System”). Predetermined checkpoints are selected for assessment and course correction, approximately 1-2 cm under the skin and every 1-5 cm thereafter, and “[a]fter needle advancement, a new scan was obtained for course correction planning” (Ben-David, pp. 2-3, “Biopsy Methodology”). Ben-David reports 4.6 ± 1.3 predetermined checkpoints, range 2-9 (Ben-David, p. 4, “Results”; Fig. 2). Ben-David further teaches that “the number of checkpoints chosen for any given procedure will represent a balance between the need to minimize time and radiation exposure from multiple scans and sufficient imaging to make necessary corrections” (Ben-David, p. 7, “Discussion”).
Konh teaches a steerable surgical device advancing along one or more segments of a transit path and identifying deviation of position relative to the transit path for use in generating an updated transit path (Konh, ¶[0012]-[0013]). Konh further teaches that the path may include “one or more curves, bends, and/or twists” (Konh, ¶[0071]), and that the path may be updated numerous times during insertion, for example every 5 mm of insertion depth, based on deviation of the actual needle-tip location relative to the transit path, including calculation of a “deviation error” (Konh, ¶[0077]). Konh's needle-position tracking program further “provides the real position of the needle inside the tissue at each stage of the insertion,” locates the needle tip, and “calculates its deviation from the planned path” (Konh, ¶[0092]).
Komaki teaches robotic CT-guided needle insertion in which needle orientation is checked by CT at two points during insertion, the middle of the tract and 1 cm behind the target, and corrected as necessary based on deviations between the ideal and actual needle angles (Komaki, p. 1345, Figs. 3-4). Komaki records radiation exposure during insertion, including dose-length product, together with the number of CT scans and needle adjustments. Table 4 reports a median of 4 CT scans during insertion, range 2-8, for the adjustment group and cumulative DLP of 998.3 ± 380.1 mGy·cm, range 553.8-2011.7, compared with 181.5 ± 10.7 mGy·cm, range 147.7-184.6, without adjustment. The adjustment group also achieved overall insertion accuracy of 2.5 ± 0.8 mm compared with 5.0 ± 1.7 mm without adjustment (Komaki, p. 1347, Table 4).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yeung in view of Ben-David, Konh, and Komaki to determine, at each of Ben-David's predetermined checkpoints, the positional deviation between the CT-derived actual needle-tip location and the planned checkpoint location on the preplanned trajectory using Konh's planned-versus-actual positional-deviation technique, to include those deviations in the steering-phase operational dataset as checkpoint errors, and to include Komaki's cumulative DLP during CT-guided insertion as the total radiation dose in the steering phase. The modification would have been technically feasible because Ben-David already obtains a CT image at each predetermined checkpoint for course-correction planning, thereby providing the actual needle-tip location needed for comparison with the planned checkpoint location. The modification would use Ben-David's CT-derived needle-tip location in place of Konh's ultrasound-derived location because Konh's relied-upon operation compares an identified actual position with the planned path, and the positional comparison is independent of the imaging modality used to obtain the actual position. Komaki's DLP is likewise generated from the CT imaging already performed during insertion and can be retained with the other device-operational data. A person of ordinary skill would have been motivated to determine and retain the positional error at each checkpoint because Ben-David obtains each checkpoint image for the express purpose of planning course correction, while Konh teaches that determining deviation of the actual needle-tip position from the planned path provides the information used to determine corrective steering. A person of ordinary skill would also have been motivated to retain cumulative DLP because Ben-David expressly identifies checkpoint selection as a balance between obtaining sufficient imaging to make necessary corrections and limiting radiation exposure, while Komaki quantifies both the improved insertion accuracy and increased radiation exposure associated with intraprocedural adjustment. Recording the two quantities would therefore permit quantitative evaluation of the corrective error present at the checkpoints together with the radiation cost of obtaining sufficient imaging to correct that error. The modification would have constituted use of known position-error and radiation measurements to evaluate and optimize the recognized correction-versus-radiation tradeoff in a known checkpoint-guided CT steering process, resulting in steering-phase operational data comprising checkpoint errors of a plurality of checkpoints along the non-linear trajectory and total radiation dose during the steering phase.
Also, regarding claim 45, the modified Yeung does not fully teach training the data analysis algorithm to output one or more of a prediction and a detection of a clinical condition related to insertion of the medical instrument toward the target in the body of the patient, using the training set, wherein the prediction and/or the detection is generated during insertion and steering of the medical instrument along the non-linear trajectory, based at least in part on the steering-phase operational data. Rather, the modified Yeung collects steering-phase operational data generated during insertion and steering, but Yeung's trained algorithm is directed to steering control rather than prediction or detection of a clinical condition.
Boddington teaches applying machine-learning and deep-learning techniques to “calculate surgical decision risks, to predict a problem and provide guidance in real-time situations” (Boddington, ¶[0069]). Boddington further teaches training outcome classifiers using datasets containing information that can affect a surgical outcome and providing an outcome prediction and associated risk score. Relevant datasets may be selected during the surgical event and supplied to trained classifiers that contribute to the final surgical outcome prediction (Boddington, ¶[0106]-[0107], ¶[0166]-[0167]).
Ben-David teaches that steering behavior during a percutaneous insertion can bear directly on a clinical complication. Specifically, Ben-David teaches that “in the lung, it is crucial to line up a path before piercing the pleura to minimize pneumothorax formation from potential tearing owing to excessive angulation when correcting the needle path” (Ben-David, p. 7, “Discussion”).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung in view of Boddington and Ben-David to train and execute a clinical outcome-prediction model in parallel with the steering-control model and to supply Yeung's contemporaneous steering vectors and corrective-angulation information to the outcome-prediction model. Yeung already represents recent steering-control actions as digital steering-history vectors, and Boddington's outcome classifiers accept procedure-specific digital datasets during the surgical event. Ben-David supplies the reason to use that information: excessive corrective angulation when correcting a needle path can tear the lung and increase pneumothorax risk. The magnitude and direction of corrective steering during the current insertion therefore are a risk-relevant portion of the steering-phase operational data. The modification would have been technically feasible because the risk model can consume the same digital steering data while executing alongside, without changing, Yeung's steering-control function. It would have constituted use of known procedure-specific risk information with Boddington's known real-time outcome-prediction technique for its established purpose, with a reasonable expectation that the prediction would respond to risk-producing steering events during the particular insertion. Yeung's disclosed training and validation methodology, including the approximately 80% training and 20% evaluation portions, would be applied to the clinical-condition model produced by the combination, such that the same data analysis algorithm is trained using the training set, validated using the validation set, and used to generate the clinical-condition prediction during insertion and steering.
Also, regarding claim 45, the modified Yeung does not expressly teach wherein the clinical condition comprises pneumothorax, breathing anomalies, internal bleeding, or any combinations thereof.
Zhao teaches a pneumothorax risk-prediction model specifically for CT-guided needle biopsy. Zhao evaluates patient, lesion, and biopsy-related risk factors, identifies the number of pleural punctures as an independent risk factor for pneumothorax, and creates a logistic-regression model using multiple risk factors to determine the predictive probability of pneumothorax. Zhao states that “[t]he ability to predict the probability of a pneumothorax would be quite valuable when performing a CT-guided needle biopsy” (Zhao, p. 4754; Table 2).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung in view of Zhao to configure the outcome-prediction model to predict pneumothorax during the CT-guided insertion procedure. The modification would have been technically feasible because Zhao's model operates on numerical patient and procedure-related variables that can be supplied to the already incorporated outcome-prediction architecture. A person of ordinary skill would have been motivated to select pneumothorax as the predicted clinical condition because Zhao identifies it as a recognized complication of CT-guided needle biopsy for which probability prediction is useful during performance of the procedure, while Ben-David's previously incorporated teaching establishes that the course of corrective steering itself can affect that risk. The modification would have constituted application of Zhao's known pneumothorax-prediction technique in the intraoperative prediction framework of the modified Yeung, thereby providing the recited pneumothorax alternative.
Regarding claim 46, the modified Yeung teaches that the training set and the validation set further comprise one or more data annotations because Yeung's supervised training data pair known input sensor data with corresponding known output steering-control signals, which serve as annotations for the known inputs (Yeung, ¶[0034], ¶[0156]-[0157]). Yeung applies an approximately 80% training portion and 20% evaluation portion to the same annotated dataset (Yeung, ¶[0219]). Yeung further teaches calculating an error of output of the data analysis algorithm from the one or more data annotations by calculating the separation distance, Derr, between the surgeon-indicated lumen center and the center determined by the ANN (Yeung, ¶[0220]; Fig. 19). Yeung further teaches optimizing the data analysis algorithm using the calculated error because Yeung reports network hyperparameters selected after optimization and identifies further improvement of the ANN model and quantitative validation methods based on the measured comparison (Yeung, ¶[0219]-[0221]). It would have been obvious to use Derr as the optimization objective because Derr directly measures the discrepancy between the annotated desired output and the model output, and minimizing that discrepancy is the ordinary use of the disclosed error metric to improve prediction accuracy.
Regarding claim 47, the modified Yeung teaches that the one or more datasets further comprise one or more of: clinical procedure related dataset, patient related dataset and administrative related dataset, through at least the clinical-procedure-related-dataset alternative (Yeung, ¶[0071], wherein image data acquired by the robotic endoscope during the medical procedure are processed and supplied to the ANN; ¶[0164], wherein sensor data and associated steering-history data may be collected concurrently during operation and stored in a database).
Regarding claim 48, the modified Yeung teaches that the automated medical device related dataset comprises parameters selected from: entry point, insertion angles, target position, target position updates, planned trajectory, trajectory updates, real-time positions of the medical instrument, number of checkpoints along the planned and/or updated trajectory, checkpoint locations, checkpoint locations updates, checkpoint errors, position of the automated medical device relative to the patient's body, steering steps timing, procedure time, steering phase time, procedure accuracy, target error, medical images, medical imaging parameters per scan, radiation dose per scan, total radiation dose in steering phase, total radiation dose procedure, errors indicated during the steering procedure, software logs, motion control traces, automated medical device registration logs, medical instrument detection logs, or any combination thereof, through at least the medical images alternative (Yeung, ¶[0071]: “FIG. 15 shows a block diagram of an exemplary adaptive steering control system for a robotic endoscope that uses data collected by image sensors to generate a steering control signal, where the image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction”, wherein the image data collected from the robotic endoscope constitutes medical images within the automated medical device related dataset).
Regarding claim 50, the modified Yeung teaches that one or more of the parameters of the one or more datasets is configured to be collected automatically (Yeung, ¶[0071]: “FIG. 15 shows a block diagram of an exemplary adaptive steering control system for a robotic endoscope that uses data collected by image sensors to generate a steering control signal, where the image data is processed using two or more image processing algorithms and subsequently used as input for an artificial neural network (ANN) that maps the input data to a navigation direction”, explains that Yeung teaches automatic collection of image sensor data and navigation/control parameters by the system during operation; ¶[0164]: “The input data may be obtained from a training process or stored in a database. In some cases, the sensor data and associated steering history data may be collected prior to obtaining the desired output data 811. In some cases, the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data”, explains that Yeung teaches sensor data and steering history are collected automatically by the system as part of the procedure, without requiring manual entry).
Regarding claim 51, the modified Yeung teaches that the method further comprises performing one or more of: data cleaning, data pre-processing, data annotation and data augmentation, and extracting features from the one or more datasets, through at least the data-pre-processing and feature-extraction alternatives (Yeung, ¶[0084], wherein image data may be pre-processed using image-processing algorithms before being supplied to the machine-learning algorithm; ¶[0007], wherein the analysis performs automated image processing and feature extraction).
Regarding claim 52, the modified Yeung does not fully teach wherein the training of the data analysis algorithm comprises: training the data analysis algorithm to output one or more first predictions relating to respective one or more first target variables; training the data analysis algorithm to output at least one second prediction relating to a second target variable; using the one or more first predictions; calculating a prediction error of the at least one second prediction, and optimizing the data analysis algorithm using the prediction error. Rather, Yeung teaches training an ANN using known desired outputs, optimizing network parameters, and calculating an error between an ANN output and a known desired output. In particular, Yeung teaches training the ANN to map image-derived input data to output values for navigation direction and identifies network hyperparameters following optimization of the network (Yeung, ¶[0219]). Yeung further evaluates the ANN output by calculating the separation distance between the surgeon-indicated location and the location determined by the ANN (Yeung, ¶[0220]) and describes further development directed to improving the ANN model and quantitative validation methods (Yeung, ¶[0221]). However, the modified Yeung does not teach training the clinical-condition data analysis algorithm established regarding claim 45 so that trained first predictions relating to respective first target variables are used in generating a second prediction relating to a second target variable, nor does Yeung expressly teach using its calculated prediction error to optimize that second-prediction model.
Boddington teaches an AI Engine having multiple trained classifiers. Boddington teaches that multiple CNN-based classifiers may be selected using particular datasets for solving well-defined tasks, and that when information from multiple datasets is relevant, the information is included in the AI Engine in the form of multiple trained classifiers, each having a weighted contribution to the final surgical outcome prediction. Boddington describes this as a multiple prediction model and explains that it uses uncorrelated or partially correlated predictors of a specific outcome and may weigh the outcome-prediction data according to relative criticality (Boddington, ¶[0106]-[0107]). Boddington further discloses separate classifier algorithms for classifying reduction or alignment procedures and implant-fixation procedures into respective discrete categories, together with a surgical outcome prediction based on calculated intra-operative surgical decision risks (Boddington, claim 45).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung in view of Boddington so that the clinical-condition data analysis algorithm established regarding claim 45 uses Boddington's multiple-prediction architecture, with trained component classifiers generating first predictions relating to respective procedure-related target variables and those first predictions being used, with their contributions weighted according to their relative importance, to generate the clinical-condition prediction as a second prediction relating to a second target variable. The component-classifier outputs and final prediction are digital model outputs and can be passed between software models without changing the operation of the automated medical device. A person of ordinary skill would have been motivated to use this architecture because Boddington teaches that combining uncorrelated or partially correlated predictors and weighting their contributions according to criticality permits multiple sources of predictive information to contribute to the final surgical outcome prediction (Boddington, ¶[0107]). The modification would have constituted application of Boddington's known multiple-classifier outcome-prediction technique to the clinical-condition outcome model already incorporated into the modified Yeung. Yeung further teaches training network parameters from known desired outputs, optimizing the network, and quantitatively determining prediction error by comparison with the known output (Yeung, ¶[0156]-[0157], ¶[0219]-[0220]). It would therefore have been obvious to calculate the error between the second prediction and its known second-target value and use that error to optimize the parameters of the second-prediction model. Doing so applies Yeung's disclosed training, error-evaluation, and network-optimization techniques to the final prediction in Boddington's multiple-prediction architecture, with a reasonable expectation of reducing the measured prediction error and improving the trained model.
Regarding claim 53, the modified Yeung further teaches wherein the automated medical device is configured to allow real-time updating of a trajectory of the medical instrument; and/or wherein the automated medical device is configured to steer the medical instrument toward the target such that the medical instrument traverses the non-linear trajectory within the body of the patient, through at least the second alternative. Yeung teaches automated steering control of a robotic endoscope and explains that “the environment 105 may comprise multiple flexural, looping or bending sections through which the steering control system allows the colonoscope to be maneuvered” (Yeung, ¶[0097]). Yeung further teaches that “[t]he disclosed systems and methods can be used for automated steering control of robotic endoscopes” (Yeung, ¶[0200]). Thus, as also discussed regarding claim 45, Yeung's automated steering system steers the medical instrument through the flexural, looping, or bending sections of the patient's body, thereby traversing a non-linear trajectory toward the target.
Regarding claim 56, the modified Yeung further teaches collecting one or more new datasets, at least one of the one or more new datasets being related to an automated medical device configured to steer a medical instrument toward a target in a body of a patient (Yeung, ¶[0164]: “the sensor data and associated steering history data may be collected concurrently with obtaining the desired output data”; ¶[0170], wherein sensor data concerning operation and position of the instrument may be supplied as input data).
The modified Yeung further teaches pre-processing the one or more new datasets (Yeung, ¶[0084]: “the image data may be pre-processed using one or more image processing algorithms prior to providing it as input to the machine learning algorithm”; ¶[0170], wherein data derived from preprocessing, including gradient maps, motion vectors, and extracted lumen-center locations, may be supplied to the neural network).
The modified Yeung further teaches executing the data analysis algorithm using at least a portion of the one or more new datasets (Yeung, ¶[0201], wherein a processor generates a steering-control output based on analysis of a first image-data stream using a machine-learning architecture; ¶[0170], wherein raw sensor data or data derived from preprocessing, including gradient maps, motion vectors, and extracted lumen-center locations, are supplied to the neural network as input data).
The modified Yeung further teaches extracting features from the one or more new datasets (Yeung, ¶[0037], wherein the processors perform automated feature extraction on a series of images to determine a center position of the colon lumen).
The modified Yeung further teaches wherein the one or more new datasets further comprise one or more of: clinical procedure related dataset, patient related dataset and administrative related dataset, through at least the clinical-procedure-related-dataset alternative (Yeung, ¶[0071], wherein image data collected during operation of the robotic endoscope are processed and supplied to an ANN for navigation; ¶[0164], wherein sensor data and associated steering-history data may be collected concurrently during operation).
Also, regarding claim 56, Yeung does not teach obtaining an output of the data analysis algorithm, the output being at least one of prediction and detection of a clinical condition related to the insertion of the medical instrument toward the target in the body of the patient, wherein the one or more new datasets comprise steering-phase operational data generated during execution of the non-linear trajectory, and the output of the data analysis algorithm is obtained based at least in part on the steering-phase operational data. Yeung teaches collecting sensor and steering-history data concurrently during operation and supplying such newly collected data directly or after preprocessing to a machine-learning algorithm, but Yeung's resulting machine-learning output is directed to steering rather than prediction or detection of the recited clinical condition (Yeung, ¶[0164], ¶[0170]). As established regarding claim 45, the modification of Yeung in view of Ben-David, Konh, Komaki, Boddington, and Zhao applies Yeung's machine-learning methodology to a clinical-condition prediction model and supplies steering-phase operational data generated during execution of the non-linear trajectory to that model during insertion. Accordingly, execution of that same modified data analysis algorithm using newly collected steering-phase operational data produces the clinical-condition prediction or detection based at least in part on those data. It would have been obvious to execute the already-established clinical-condition model on the newly collected steering-phase operational data because Yeung teaches supplying newly collected or preprocessed operational data to its machine-learning architecture and Boddington seeks real-time problem prediction during the ongoing procedure. The modification would make the output responsive to conditions of the current insertion and would have required only supplying compatible digital input data to the already-established model.
Regarding claim 60, Yeung teaches a system for generating a data analysis algorithm comprising a machine-learning architecture used with an automated robotic medical device (Yeung, ¶[0034], ¶[0037], ¶[0156]-[0157]).
Yeung further teaches a training module comprising a memory configured to store one or more existing datasets, metadata, data annotations, a database of features extracted from the one or more existing datasets and/or one or more pre-trained models, through at least the pre-trained-model branch of the final and/or limitation (Yeung, ¶[0180], wherein the storage unit stores training datasets and trained neural-network parameters; ¶[0164], wherein input data may be stored in a database; ¶[0169], wherein time stamps register data from different sensor streams and provide metadata for those data; ¶[0034], ¶[0156]-[0157], wherein stored training datasets pair known input sensor data with corresponding known output steering-control signals and thereby provide supervised data annotations; ¶[0180], wherein the stored trained parameters define the pre-trained neural network).
Yeung further teaches one or more processors configured to create a training set comprising a first data portion of the one or more existing datasets (Yeung, ¶[0034], ¶[0156]-[0157]).
Yeung further teaches wherein at least one of the one or more existing datasets is related to the automated medical device configured to steer the medical instrument toward the target in the body of a patient and/or to operation thereof (Yeung, ¶[0034], wherein the training datasets include sensor information and corresponding steering-control signals associated with the robotic colonoscope; ¶[0099], wherein the steering-control system receives input sensor data and provides a target direction or control signal to the actuation unit; ¶[0200]).
Yeung further teaches wherein the at least one dataset comprises steering-phase operational data generated during execution of the non-linear trajectory (Yeung, ¶[0163]; Fig. 8B, wherein steering-history data include recent steering-control actions and steering vectors corresponding to movement of the distal end; ¶[0164], wherein the steering-history data may be collected concurrently with desired output data; ¶[0097], wherein the instrument traverses flexural, looping, or bending sections). Maneuvering the instrument through the flexural, looping, or bending sections constitutes traversal of a non-linear trajectory.
Also, regarding claim 60, Yeung does not fully teach wherein the steering-phase operational data comprise a total radiation dose in a steering phase and checkpoint errors of a plurality of checkpoints along the non-linear trajectory. Rather, Yeung teaches steering-phase operational data generated during non-linear navigation as discussed above. However, Yeung does not teach positional errors determined at a plurality of checkpoints or a total steering-phase radiation dose.
Ben-David teaches predetermined checkpoints selected for assessment and course correction during insertion, positioned approximately 1-2 cm under the skin and every 1-5 cm thereafter, with a new CT scan obtained after needle advancement “for course correction planning” (Ben-David, pp. 2-3, “Biopsy Methodology”). Ben-David reports 4.6 ± 1.3 predetermined checkpoints, range 2-9 (Ben-David, p. 4, “Results”; Fig. 2), and explains that checkpoint selection represents “a balance between the need to minimize time and radiation exposure from multiple scans and sufficient imaging to make necessary corrections” (Ben-David, p. 7, “Discussion”).
Konh teaches a system having a processor configured to advance a steerable surgical device along a segment of a transit path, identify the position of the device tip, identify deviation of that position relative to the transit path, generate an updated transit path, and advance the device along a segment of the updated transit path (Konh, ¶[0012]-[0013]). Konh further teaches that the path may include curves, bends, or twists and may be updated numerous times during insertion, for example every 5 mm, based on deviation of the actual needle-tip location relative to the transit path, including calculation of a “deviation error” (Konh, ¶[0071], [0077]). Konh also teaches locating the actual needle tip during insertion and calculating its deviation from the planned path (Konh, ¶[0092]).
Komaki teaches CT-guided robotic insertion in which needle orientation is checked at two points during insertion and corrected as necessary based on deviations between the ideal and actual needle angles (Komaki, p. 1345, Figs. 3-4). Komaki records radiation exposure during insertion as dose-length product. Table 4 reports a median of 4 CT scans during insertion, range 2-8, and cumulative DLP of 998.3 ± 380.1 mGy·cm, range 553.8-2011.7, with adjustment, compared with 181.5 ± 10.7 mGy·cm, range 147.7-184.6, without adjustment. Overall insertion accuracy was 2.5 ± 0.8 mm with adjustment compared with 5.0 ± 1.7 mm without adjustment (Komaki, p. 1347, Table 4).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yeung in view of Ben-David, Konh, and Komaki to determine, at each of Ben-David's predetermined checkpoints, the positional deviation between the CT-derived actual needle-tip location and the planned checkpoint location on the preplanned trajectory using Konh's planned-versus-actual positional-deviation technique, to store those deviations in the existing device-related dataset as checkpoint errors, and to store Komaki's cumulative DLP during CT-guided insertion as the total radiation dose in the steering phase. The modification would have been technically feasible because Ben-David already obtains CT imaging at the checkpoints for course-correction planning, providing the actual needle-tip location required for comparison with the planned checkpoint location. The modification would use Ben-David's CT-derived tip location in place of Konh's ultrasound-derived tip location because the relied-upon positional-deviation calculation is independent of the modality used to identify the actual position. Komaki's cumulative DLP is generated by the same CT imaging performed during insertion and can likewise be retained in Yeung's device-related dataset. A person of ordinary skill would have been motivated to retain the positional errors because they quantify the deviation Ben-David's checkpoint procedure is intended to identify and correct, and Konh teaches using that deviation to determine steering necessary to maintain the planned path. A person of ordinary skill would also have been motivated to retain cumulative DLP because Ben-David expressly identifies checkpoint selection as a balance between sufficient corrective imaging and radiation exposure, while Komaki quantifies both the resulting accuracy improvement and radiation burden. Recording the error and dose values would therefore provide quantitative measures of both sides of the recognized checkpoint-selection tradeoff. The modification would have constituted use of known position-error and radiation measurements to evaluate and optimize a known checkpoint-guided CT steering process, resulting in the recited steering-phase operational data.
Also, regarding claim 60, the modified Yeung does not fully teach training the data analysis algorithm using the training set, to output one or more of a prediction and a detection of a clinical condition during steering of the medical instrument along a non-linear trajectory within the body of the patient, based at least in part on the steering phase operational data generated during execution of the non-linear trajectory. Rather, the modified Yeung includes the recited steering-phase operational data, but Yeung's trained algorithm is directed to steering control rather than prediction or detection of a clinical condition.
Boddington teaches applying machine-learning and deep-learning techniques to “calculate surgical decision risks, to predict a problem and provide guidance in real-time situations” (Boddington, ¶[0069]). Boddington further teaches training outcome classifiers from datasets containing information that can affect a surgical outcome and generating outcome predictions and associated risk scores. Information from relevant datasets may be selected during a surgical event and supplied to trained classifiers contributing to the final surgical outcome prediction (Boddington, ¶[0106]-[0107], ¶[0166]-[0167]).
Ben-David teaches that the robotic system steers the tool during insertion while making intraoperative needle-error corrections and trajectory updates (Ben-David, p. 2, “Robotic and Navigation System”). Ben-David further teaches that “in the lung, it is crucial to line up a path before piercing the pleura to minimize pneumothorax formation from potential tearing owing to excessive angulation when correcting the needle path” (Ben-David, p. 7, “Discussion”).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung in view of Boddington and Ben-David to train and execute a clinical outcome-prediction model in parallel with the steering-control model and to supply Yeung's contemporaneous steering vectors and corrective-angulation information to the outcome-prediction model. Yeung already represents recent steering-control actions as digital steering-history vectors, and Boddington's outcome classifiers accept procedure-specific digital datasets during the surgical event. Ben-David supplies the reason to use that information: excessive corrective angulation when correcting a needle path can tear the lung and increase pneumothorax risk. The magnitude and direction of corrective steering during the current insertion therefore are a risk-relevant portion of the steering-phase operational data. The modification would have been technically feasible because the risk model can consume the same digital steering data while executing alongside, without changing, Yeung's steering-control function. It would have constituted use of known procedure-specific risk information with Boddington's known real-time outcome-prediction technique for its established purpose, with a reasonable expectation that the prediction would respond to risk-producing steering events during the particular insertion. Yeung's disclosed processor-based training methodology and training-set architecture would be applied to that clinical-condition model, such that the algorithm trained using the recited training set outputs the clinical-condition prediction during steering.
Also, regarding claim 60, the modified Yeung does not expressly teach wherein the clinical condition comprises pneumothorax, breathing anomalies, internal bleeding, or any combinations thereof.
Zhao teaches a logistic-regression model specifically configured to determine the predictive probability of pneumothorax following CT-guided needle biopsy. Zhao uses multiple patient, lesion, and biopsy-related variables, including the number of pleural punctures, which is identified as an independent risk factor for pneumothorax, and states that “[t]he ability to predict the probability of a pneumothorax would be quite valuable when performing a CT-guided needle biopsy” (Zhao, p. 4754; Table 2).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung in view of Zhao to configure the outcome-prediction model to output a prediction of pneumothorax. The modification would have been technically feasible because Zhao's model uses numerical patient and procedure-related variables compatible with the already incorporated outcome-prediction architecture. A person of ordinary skill would have been motivated to select pneumothorax as the predicted clinical condition because Zhao identifies it as a recognized complication of CT-guided needle biopsy for which probability prediction is valuable during performance of the procedure, and Ben-David's previously incorporated teaching establishes that corrective steering behavior can affect that risk. The modification would have constituted application of Zhao's established pneumothorax-prediction technique in the intraoperative prediction framework already established in the modified Yeung, thereby providing the recited pneumothorax alternative.
Regarding claim 61, the modified Yeung teaches that the one or more processors are further configured to one or more of: perform pre-processing on the one or more existing datasets, extract features from the one or more existing datasets, perform data augmentation and validate the data analysis model using a second data portion of the one or more existing datasets, through at least the feature-extraction alternative (Yeung, ¶[0037], wherein one or more processors are configured to perform automated feature extraction on a series of images to determine a center position of the colon lumen).
Regarding claim 62, the modified Yeung teaches wherein the training module is located on a remote server, an "on premise" server or a computer associated with the automated medical device; and/or wherein the remote server is a cloud server, through at least the cloud-server alternative (Yeung, ¶[0174], wherein the computer system used for training may employ cloud computing and share or update training datasets across systems; ¶[0175], wherein networked computer servers enable distributed cloud computing and the machine-learning architecture uses data and stored parameters in a cloud-based database).
Regarding claim 63, the modified Yeung teaches an inference module comprising: a memory configured to store at least one of: one or more new datasets, metadata, and the data analysis algorithm, through at least the new-dataset alternative (Yeung, ¶[0164], wherein input data may be stored in a database), and one or more processors configured to perform pre-processing on the one or more new datasets (Yeung, ¶[0084], wherein image data may be pre-processed using one or more image-processing algorithms before being provided as input to the machine-learning algorithm; ¶[0170], wherein data derived from preprocessing, including gradient maps, motion vectors, and extracted lumen-center locations, may be supplied to the neural network as input data).
The modified Yeung further teaches wherein at least one of the one or more new datasets is related to the automated medical device configured to steer a medical instrument toward the target in the body of the patient and/or to operation thereof. Yeung teaches steering-history data comprising information related to recent steering-control actions, including steering vectors corresponding to motion of the distal end of the instrument, and teaches that the sensor data and associated steering-history data may be collected concurrently during operation (Yeung, ¶[0163]-[0164]).
Also, regarding claim 63, the modified Yeung does not fully teach obtaining an output of the data analysis algorithm, the output being at least one of the prediction and detection of the clinical condition related to the insertion of the medical instrument toward the target in the body of the patient, wherein said at least one of the one or more new datasets comprises steering-phase operational data generated during execution of the non-linear trajectory, and the output of the data analysis algorithm is obtained based at least in part on the steering phase operational data. Yeung teaches collecting sensor and steering-history data concurrently during operation (Yeung, ¶[0163]-[0164]) and supplying preprocessed procedural data to a neural network (Yeung, ¶[0170]). As established regarding claim 60, the modified Yeung provides steering-phase operational data generated during execution of the non-linear trajectory and configures the same data analysis algorithm to generate a prediction or detection of the clinical condition during steering based at least in part on those data. Boddington further teaches applying machine learning during an ongoing procedure to calculate surgical decision risks, predict a problem, and provide guidance in real-time situations (Boddington, ¶[0069]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further configured the inference module of the modified Yeung to execute the trained clinical-condition data analysis algorithm using newly collected steering-phase operational data generated during execution of the non-linear trajectory. Yeung already teaches collecting operational sensor and steering-history data during the procedure and providing such data as machine-learning input, so the newly collected steering-phase data are technically compatible with the established inference architecture. A person of ordinary skill would have been motivated to make the modification so that the clinical-risk prediction reflects the conditions of the ongoing insertion and can provide useful risk information while the procedure is still underway, consistent with Boddington's express objective of using machine learning to calculate surgical decision risks, predict problems, and provide guidance in real-time situations (Boddington, ¶[0069]). Applying the established inference technique to the clinical-condition model would have been the use of the same machine-learning architecture with the current instance of the type of procedural data on which the clinical-condition prediction is based. The resulting inference module would obtain the clinical-condition prediction or detection based at least in part on steering-phase operational data generated during execution of the current non-linear trajectory.
Regarding claim 64, Yeung further teaches that the one or more processors are further configured to one or more of: load one or more trained models per task, extract features from the one or more new datasets, execute a post-inference business logic, and display the output of the data analysis algorithm to a user, through at least the feature-extraction alternative (Yeung, ¶[0037], wherein one or more processors are configured to perform automated feature extraction on a series of images to determine a center position of the colon lumen).
Claims 49 and 54 are rejected under 35 U.S.C. 103 as being unpatentable over Yeung et al. (US 2018/0296281 A1), hereinafter Yeung, and further in view of Ben-David et al., “Evaluation of a CT-Guided Robotic System for Precise Percutaneous Needle Insertion,” Journal of Vascular and Interventional Radiology (2018), pp. 1-7, doi:10.1016/j.jvir.2018.01.002, hereinafter Ben-David, and further in view of Konh et al. (US 2020/0060772 A1), hereinafter Konh, and further in view of Komaki et al., “Robotic CT-guided out-of-plane needle insertion: comparison of angle accuracy with manual insertion in phantom and measurement of distance accuracy in animals,” European Radiology, vol. 30, pp. 1342-1349 (2020), doi:10.1007/s00330-019-06477-1, hereinafter Komaki, and further in view of Boddington et al. (US 2021/0177522 A1), hereinafter Boddington, and further in view of Zhao et al., “Logistic regression analysis and a risk prediction model of pneumothorax after CT-guided needle biopsy,” Journal of Thoracic Disease, vol. 9, no. 11, pp. 4750-4757 (2017), doi:10.21037/jtd.2017.09.47, hereinafter Zhao, and further in view of Amarasingham et al. (US 2013/0262357 A1), hereinafter Amarasingham.
The modified Yeung teaches claim 45 as described above.
Regarding claim 49, the modified Yeung teaches that the clinical procedure related dataset comprises parameters selected from: medical procedure type, target organ, target size, target type, type of medical instrument, dimensions of the medical instrument, complications before, during and/or after the procedure, adverse events before, during and/or after the procedure, respiration signals of the patient, or any combination thereof (Yeung, ¶[0013], wherein the robotic endoscope is used during identified procedure types including colonoscopy and esophagogastroduodenoscopy; ¶[0071], wherein procedure-acquired image data are processed for endoscopic navigation).
Also, regarding claim 49, the modified Yeung does not fully teach that the patient related dataset comprises parameters selected from: age, gender, race, medical condition, medical history, vital signs before, after and/or during the procedure, body dimensions, pregnancy, smoking habits, demographic data, or any combination thereof; and wherein the administrative related dataset comprises parameters selected from: institution, physician, staff, system serial number, disposable components used in the procedure, software version, operating system version, configuration parameters, or any combination thereof.
Amarasingham teaches a clinical predictive and monitoring system that receives and stores patient-related clinical and non-clinical data, including gender, medical history, vital signs, and demographic information (Amarasingham, ¶[0014]-[0016).
Amarasingham further teaches receiving user-preference and system-configuration data from clinician’s computing devices (Amarasingham, ¶[0022]). Amarasingham uses a system-configuration interface to receive configuration data for initiating or adjusting system operations, including adjustment of risk-variable thresholds and weights used by the predictive model.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung in view of Amarasingham to include, within the one or more datasets used by the clinical-condition prediction system, Amarasingham's patient-related data including gender, medical history, and vital signs, and administrative-related system-configuration data. The modification would have been technically feasible because the modified Yeung already collects and stores digital procedure-related data and Boddington and Zhao establish a clinical-prediction architecture operating on multiple digital patient and procedure-related variables, while Amarasingham likewise receives and stores digital clinical, non-clinical, and configuration information. A person of ordinary skill would have been motivated to incorporate Amarasingham's patient-related data because Amarasingham expressly teaches that supplementing predictive modeling with additional patient information provides a more complete representation of the patient's healthcare environment and produces more robust and accurate predictive modeling. A person of ordinary skill would further have been motivated to include Amarasingham's system-configuration information because Amarasingham expressly uses such configuration data to initiate or adjust operation of the predictive system, including adjustment of predictive-model risk thresholds and weights. The modification would have constituted use of Amarasingham's known patient and configuration datasets in the existing digital clinical-prediction framework for their established predictive and system-configuration purposes.
Regarding claim 54, the modified Yeung does not fully teach wherein the training of the data analysis algorithm comprises training the data analysis algorithm to estimate probability of occurrence of the clinical condition during insertion of the medical instrument toward the target in the body of the patient; and the training set comprises one or more target parameters relating to the clinical condition occurrence during one or more previous procedures for inserting a medical instrument toward a target in a body of a patient. Yeung trains its neural network using known desired outputs but does not use prior clinical-condition occurrence as the target output of a probability model (Yeung, ¶[0156]-[0157]).
Zhao teaches developing a pneumothorax risk-prediction model from 864 previous CT-guided needle-biopsy procedures for which pneumothorax occurrence was recorded, using occurrence of pneumothorax as the dependent target in logistic-regression analysis and using the regression coefficients to determine the predictive probability of pneumothorax (Zhao, pp. 4751-4752, 4754). Thus, Zhao expressly supplies target parameters relating to occurrence of the clinical condition during previous instrument-insertion procedures and trains a model to estimate probability of that occurrence. Boddington corroborates the supervised-training framework by teaching that medical-image classifiers are trained using datasets having associated known outcomes data and output an outcome prediction with a statistical likelihood or confidence level (Boddington, ¶[0102]-[0104]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further configured the pneumothorax model established regarding claim 45 in accordance with Zhao to use recorded pneumothorax occurrence from prior CT-guided insertion procedures as the training target and to output a probability of pneumothorax occurrence during the current insertion. Zhao teaches both the labeled prior-procedure outcomes and the probability calculation, while Boddington confirms that known surgical outcomes are suitable training targets for an intraoperative outcome-prediction architecture. A person of ordinary skill would have been motivated to make the modification because Zhao states that predicting pneumothorax probability is valuable when performing CT-guided needle biopsy, and a quantitative probability permits the operator to assess risk while the insertion is underway. The modification would have been technically compatible because the prior-procedure occurrence labels are used as supervised training targets, while the current steering-phase operational data are supplied as instance-specific inputs during inference. The same data analysis algorithm therefore remains trained to generate its clinical-condition prediction during insertion based at least in part on the steering-phase operational data established regarding claim 45.
Claim 55 is rejected under 35 U.S.C. 103 as being unpatentable over Yeung et al. (US 2018/0296281 A1), hereinafter Yeung, and further in view of Ben-David et al., “Evaluation of a CT-Guided Robotic System for Precise Percutaneous Needle Insertion,” Journal of Vascular and Interventional Radiology (2018), pp. 1-7, doi:10.1016/j.jvir.2018.01.002, hereinafter Ben-David, and further in view of Konh et al. (US 2020/0060772 A1), hereinafter Konh, and further in view of Komaki et al., “Robotic CT-guided out-of-plane needle insertion: comparison of angle accuracy with manual insertion in phantom and measurement of distance accuracy in animals,” European Radiology, vol. 30, pp. 1342-1349 (2020), doi:10.1007/s00330-019-06477-1, hereinafter Komaki, and further in view of Boddington et al. (US 2021/0177522 A1), hereinafter Boddington, and further in view of Zhao et al., “Logistic regression analysis and a risk prediction model of pneumothorax after CT-guided needle biopsy,” Journal of Thoracic Disease, vol. 9, no. 11, pp. 4750-4757 (2017), doi:10.21037/jtd.2017.09.47, hereinafter Zhao, and further in view of Amarasingham et al. (US 2013/0262357 A1), hereinafter Amarasingham, and further in view of Huo et al. (Huo YR, Chan MV, Habib AR, Lui I, Ridley L., "Pneumothorax rates in CT-guided lung biopsies: a comprehensive systematic review and meta-analysis of risk factors," Br J Radiol. 2020 Apr 1;93(1108):20190866, doi:10.1259/bjr.20190866, Epub 2020 Jan 3), hereinafter Huo, and further in view of Gerard et al., “FissureNet: A Deep Learning Approach For Pulmonary Fissure Detection in CT Images,” IEEE Transactions on Medical Imaging, vol. 38, no. 1, pp. 156-166 (2019), doi:10.1109/TMI.2018.2858202, hereinafter Gerard.
The modified Yeung teaches claim 54 as described above.
Regarding claim 55, the modified Yeung does not fully teach wherein the training of the data analysis algorithm further comprises training one or more individual models and using one or more predictions generated by the one or more individual models as input for training the data analysis algorithm, wherein when the clinical condition is pneumothorax, the one or more individual models comprise at least two of: a model for predicting a patient pose during an instrument steering procedure, a model for estimating pleural cavity volume, a model for estimating fissure crossing, a model for estimating bulla crossing, and a model for predicting respiration anomalies during an instrument insertion and steering procedure, through at least the patient-pose and fissure-crossing alternatives. Yeung alone does not teach both relied-on pneumothorax-specific individual models.
Rather, Yeung teaches training an ANN using image-derived input values including the location of the lumen center and a confidence level produced by a combined lumen-detection method (Yeung, ¶[0219]). Boddington further teaches an AI engine having multiple trained classifiers for respective tasks and teaches providing multiple trained classifiers with respective weighted contributions to a final surgical outcome prediction (Boddington, ¶[0106]-[0107]). Thus, the cumulative art teaches a downstream trained prediction model that uses information generated by separate analytical models, but does not teach at least two of the recited pneumothorax-specific individual models.
Boddington teaches a learned classifier-based pose technique in which anatomical landmarks are identified based on classifiers learned from medical-image datasets and the computing platform determines an optimal pose. Boddington further teaches that the appropriate pose can be acquired by adjusting the position of the anatomy or subject or by adjusting the imaging equipment (Boddington, ¶[0119]-[0122]).
Huo teaches that patient position and fissure crossing are procedure-related factors affecting pneumothorax during CT-guided lung biopsy. Huo identifies patient positioning as a modifiable risk factor and reports significantly different pneumothorax risks for different positions. Huo separately reports that crossing a fissure increased pneumothorax incidence from 24.6% to 52.8%, with an odds ratio of 3.75 (Huo, p. 1, Abstract; p. 6, section "Fissure crossed"; p. 7, Table 4).
Gerard et al. teach a supervised, coarse-to-fine cascade of two convolutional neural networks for locating pulmonary fissures in CT images. The first Seg3DNet is trained to detect a fissure region of interest, and the second Seg3DNet is trained to determine the precise fissure location within that region (Gerard et al., pp. 2-3).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung by applying Boddington's learned pose-classification technique and Gerard et al.'s trained fissure-location technique to provide at least a patient-pose model and a fissure-crossing model, and to use the predictions from those models as inputs when training the pneumothorax-probability data analysis algorithm established regarding claim 54. The modified system already obtains medical images and establishes the instrument trajectory during steering. Boddington's learned landmark and pose analysis therefore can determine patient pose from the intraoperative images. Gerard et al.'s trained fissure-location output can be compared with the planned or updated instrument trajectory to output whether the trajectory crosses a pulmonary fissure; the trained fissure-location cascade and that trajectory comparison together provide the individual fissure-crossing model. Huo supplies a specific reason to use both predictions because patient position and fissure crossing are recognized pneumothorax risk factors, with fissure crossing substantially increasing the observed risk. Yeung's training architecture already accepts values produced by upstream image analysis as ANN inputs, and Boddington teaches combining multiple trained classifiers in a final outcome-prediction architecture. The modification therefore would train the same pneumothorax-probability algorithm using predictions from at least the patient-pose and fissure-crossing individual models.
Claim 57 is rejected under 35 U.S.C. 103 as being unpatentable over Yeung et al. (US 2018/0296281 A1), hereinafter Yeung, and further in view of Ben-David et al., “Evaluation of a CT-Guided Robotic System for Precise Percutaneous Needle Insertion,” Journal of Vascular and Interventional Radiology (2018), pp. 1-7, doi:10.1016/j.jvir.2018.01.002, hereinafter Ben-David, and further in view of Konh et al. (US 2020/0060772 A1), hereinafter Konh, and further in view of Komaki et al., “Robotic CT-guided out-of-plane needle insertion: comparison of angle accuracy with manual insertion in phantom and measurement of distance accuracy in animals,” European Radiology, vol. 30, pp. 1342-1349 (2020), doi:10.1007/s00330-019-06477-1, hereinafter Komaki, and further in view of Boddington et al. (US 2021/0177522 A1), hereinafter Boddington, and further in view of Zhao et al., “Logistic regression analysis and a risk prediction model of pneumothorax after CT-guided needle biopsy,” Journal of Thoracic Disease, vol. 9, no. 11, pp. 4750-4757 (2017), doi:10.21037/jtd.2017.09.47, hereinafter Zhao, and further in view of Do et al., “Automated Quantification of Pneumothorax in CT,” Computational and Mathematical Methods in Medicine, vol. 2012, Article ID 736320, 7 pages (2012), doi:10.1155/2012/736320, hereinafter Do, and further in view of Taylor et al., “Automated detection of moderate and large pneumothorax on frontal chest X-rays using deep convolutional neural networks: A retrospective study,” PLoS Medicine, vol. 15, no. 11, e1002697 (2018), doi:10.1371/journal.pmed.1002697, hereinafter Taylor, and further in view of Huo et al. (Huo YR, Chan MV, Habib AR, Lui I, Ridley L., "Pneumothorax rates in CT-guided lung biopsies: a comprehensive systematic review and meta-analysis of risk factors," Br J Radiol. 2020 Apr 1;93(1108):20190866, doi:10.1259/bjr.20190866, Epub 2020 Jan 3), hereinafter Huo.
The modified Yeung teaches claim 56 as described above.
Regarding claim 57, the modified Yeung further teaches wherein the clinical condition the data analysis algorithm is trained to provide the prediction and/or detection thereof is pneumothorax, the output of the data analysis algorithm comprises a probability of pneumothorax occurrence. As incorporated regarding claim 45, Zhao teaches a pneumothorax risk-prediction model for CT-guided needle biopsy and expressly states that “a pneumothorax risk prediction model was created using the regression coefficients to determine the predictive probability (PP) of pneumothorax” (Zhao, p. 4752). Zhao further evaluates the relationship between the predictive probability and pneumothorax incidence using an ROC curve and reports the resulting predictive performance (Zhao, pp. 4754-4755, FIG. 4).
The modified Yeung further teaches the one or more new datasets comprise one or more images of a region of interest (Yeung, ¶[0205]: “FIG. 13 shows an exemplary field-of-view comprising an object of interest 1301 and a tool for surgical operations,” wherein the object of interest may be detected or recognized using automated image feature extraction; ¶[0071], wherein image data collected by the image sensors are processed and supplied to the machine-learning architecture).
Also, regarding claim 57, the modified Yeung does not fully teach detecting one or more critical tissues in the one or more images; detecting pleural cavity volume. The modified Yeung already provides CT imaging in which tissue interfaces and critical anatomy are relevant to the insertion procedure. Ben-David teaches selecting checkpoints based on “tissue interfaces and critical anatomy” and placing checkpoints before “critical structures” such as the spine and aorta, with a new CT scan obtained for course-correction planning (Ben-David, p. 3). Boddington further teaches medical-image feature detection, annotation, and segmentation as image-processing functions used by its computing platform (Boddington, ¶[0079]). However, the modified Yeung does not fully teach automated CT segmentation providing the claimed pleural-cavity volumetric determination.
Do teaches an automated CAD algorithm for pneumothorax analysis from CT images. Do identifies lung tissue and air through automated segmentation and applies three-dimensional connectivity analysis so that an air component is included as pneumothorax only when it has “finite boundaries within the pleural cavity.” Do then calculates the volumes of the regions designated as air and lung tissue, Vair and Vlung, and calculates relative pneumothorax volume as Vair/(Vair + Vlung) (Do, pp. 3-4). Do further reports the pneumothorax measurement as a percentage of the “total pleural space” (Do, p. 5, FIG. 3). Thus, Do's automated processing segments the relevant thoracic tissue and determines the volumetric quantities from which the total pleural-space volume, Vair + Vlung, is obtained.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung in view of Do to apply Do's automated CT segmentation and volumetric analysis to the CT images used during the insertion procedure, thereby automatically detecting relevant thoracic tissues in the images and determining pleural cavity volume from the segmented lung and pleural-air regions. The modification is technically compatible because the modified Yeung already uses CT images during the insertion procedure, while Do performs its segmentation and volumetric analysis on CT image datasets. A person of ordinary skill would have been motivated to make the modification because Do teaches that automated CT analysis provides rapid, objective volumetric information for pneumothorax assessment, while Ben-David teaches that tissue interfaces and critical anatomy are relevant to planning and correcting the needle path. The modification would have constituted application of a known CT segmentation and volumetric-analysis technique to CT images already available to the modified system.
Also, regarding claim 57, the modified Yeung does not fully teach determining if the probability of pneumothorax occurrence is above a predetermined threshold, and if the probability of pneumothorax occurrence is determined to be above the predetermined threshold, generating an alert, and providing a recommendation of one or more mitigating actions to reduce the probability of pneumothorax occurrence.
Taylor teaches automated classifiers that classify chest images as positive or negative for pneumothorax and teaches flagging images suspicious for moderate or large pneumothorax so that they may be prioritized for rapid review. Taylor further explains that prioritization of suspected pneumothorax images may result in earlier treatment and identifies development of effective clinician alerts for potentially critical pneumothorax findings as an implementation objective (Taylor, pp. 1-2). Boddington likewise teaches presenting a Failure Risk Score as a confidence percentage and generating a hazard alert when the analyzed condition indicates a potentially suboptimal situation (Boddington, ¶[0079], ¶[0145]-[0146]).
Huo teaches modifiable procedural measures that reduce pneumothorax risk in CT-guided lung biopsy. In the Discussion, Huo explains that placing patients in a lateral decubitus position with the target lesion in the dependent lung significantly reduced pneumothorax rates (Huo, p. 11, Discussion, paragraph beginning "A logistically simpler technique"). Huo separately recommends placing a patient in a prone position following an anterior needle entry to reduce pneumothorax risk (Huo, p. 12, Discussion, paragraph ending "Therefore, we suggest placing patient").
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the modified Yeung to compare Zhao's calculated pneumothorax probability with a predetermined decision threshold selected using Zhao's ROC analysis and, when the threshold is exceeded, generate Boddington's hazard alert and provide Huo's mitigating recommendation. Zhao supplies a scalar probability and ROC analysis, Taylor teaches positive or negative classification and flagging for rapid review, Boddington teaches risk-based alerts, and Huo teaches specific risk-reducing actions. The modification would have required applying a known decision boundary to a numeric model output and associating digital alert and recommendation outputs with the positive result. Those functions are technically compatible with the established model and would have been expected to operate without changing its upstream probability calculation. A person of ordinary skill would have been motivated to make the modification so that elevated pneumothorax risk would be brought to the operator's attention while an opportunity for mitigation remained.
Response to Arguments
35 U.S.C. §103
Applicant's arguments filed 7/24/2026, pages 9-14, regarding the previous 103 rejections of claims 45-57 and 60-64 have been fully considered but are not persuasive or are moot as there are new grounds of rejection. Applicant's arguments directed specifically to Lu and He are moot because neither reference is relied upon in the present rejections. Applicant's remaining arguments concerning retained references and inherited limitations are addressed below.
Specifically, claims 45-48, 50-53, 56, and 60-64 are presently rejected over Yeung in view of Ben-David, Konh, Komaki, Boddington, and Zhao; claims 49 and 54 are presently rejected over Yeung in view of Ben-David, Konh, Komaki, Boddington, Zhao, and Amarasingham; claim 55 is presently rejected over Yeung in view of Ben-David, Konh, Komaki, Boddington, Zhao, Amarasingham, Huo, and Gerard; and claim 57 is presently rejected over Yeung in view of Ben-David, Konh, Komaki, Boddington, Zhao, Do, Taylor, and Huo.
Applicant's Argument: Applicant argues that Yeung uses its sensor and steering-history data for closed-loop steering control rather than clinical-condition prediction, while Boddington generates surgical-risk predictions from EHR, medical-image, and known-predictor data rather than device steering telemetry. Applicant argues that the references therefore do not teach or suggest generating the claimed clinical-condition prediction based at least in part on steering-phase operational data generated during execution of the non-linear trajectory and that supplying Yeung's steering telemetry to Boddington's risk-prediction system would constitute hindsight without an articulated reason or reasonable expectation of success
Examiner's Response: Applicant’s arguments are not persuasive as they do not overcome the present rejection. The present rejection does not rely merely on Yeung as a source of steering data and Boddington as an unrelated source of clinical prediction. Ben-David bridges the relationship. Ben-David teaches a CT-guided robotic system that steers a percutaneous medical instrument during insertion while making intraoperative needle-error corrections and trajectory updates (Ben-David, p. 2, section "Robotic and Navigation System"), and further teaches that excessive angulation when correcting the needle path in the lung can cause tearing and pneumothorax formation (Ben-David, p. 7, section "Discussion"). Boddington teaches applying machine-learning techniques to calculate surgical decision risks, predict problems, and provide guidance in real-time situations (Boddington, ¶[0069]). Zhao teaches prediction of pneumothorax probability using procedure-related variables during CT-guided needle biopsy (Zhao, pp. 4752, 4754). The art therefore supplies the reason to use risk-relevant steering-phase operational information generated during the insertion as an input to the clinical-condition prediction, and that reason does not originate in Applicant's disclosure.
Applicant's premise that Boddington expects EHR and image-based known predictors is also not commensurate with the reference. Boddington does not define its predictor datasets by data type. Boddington states that "[t]hese datasets are configured to include information that will potentially have an impact on the outcome of the procedure" (Boddington, ¶[0106]). Boddington further teaches that its AI Engine includes "multiple CNNs based classifiers which can be selected using the specific dataset (one or more dataset, most importantly uncorrelated data that make the CNN learn new relevant features)" (Boddington, ¶[0106]). Boddington therefore does not merely tolerate a predictor drawn from a different data source; it identifies uncorrelated data as most important because such data cause the classifier to learn new relevant features. Steering-phase operational data generated by the automated medical device is uncorrelated with the patient-record and image predictors Applicant identifies, and is outcome-relevant for the reason Ben-David gives.
Boddington further teaches that information from independent datasets is selected during a surgical event and supplied to multiple trained classifiers, each having a weighted contribution to the final surgical outcome prediction, and that the resulting multiple-prediction model uses datasets sharing uncorrelated or partially correlated predictors of a specific outcome (Boddington, ¶[0107]). Yeung already generates the steering-phase information as digital procedural data, and Boddington's trained outcome classifiers operate on digital procedural datasets, such that the proposed use of the steering information requires no alteration of Yeung's steering function. Accordingly, a person of ordinary skill would have had a reasonable expectation that the risk-relevant steering information identified by Ben-David could be incorporated as an additional weighted predictor and contribute to the pneumothorax prediction.
Accordingly, both the reason for the modification and the reasonable expectation of success are supplied by the prior art rather than by Applicant's disclosure.
Applicant's Argument: Applicant argues that Yeung's optical robotic endoscope does not generate a total radiation dose or checkpoint errors and that the previous combination fails to provide a percutaneous, image-guided, checkpoint-based steering device. Applicant further argues that the references do not teach deriving a clinical-condition prediction from radiation-dose and checkpoint-error data generated during execution of a non-linear trajectory.
Examiner's Response: Applicant’s arguments are not persuasive as they do not overcome the present rejection. Ben-David is now relied upon for a CT-guided percutaneous robotic system that steers the medical instrument during insertion, performs intraoperative needle-error corrections and trajectory updates, and uses a plurality of predetermined checkpoints for CT-based assessment and course correction (Ben-David, pp. 2-4, sections "Robotic and Navigation System," "Biopsy Methodology," and "Results"). Konh is relied upon for determining deviation of the actual needle position relative to the planned path (Konh, ¶[0012]-[0013], ¶[0077], ¶[0092]), and Komaki is relied upon for cumulative radiation-dose information generated during CT-guided insertion and corrective adjustment (Komaki, p. 1347, Table 4). The manner in which these teachings are combined to provide steering-phase operational data comprising a total radiation dose and checkpoint errors of a plurality of checkpoints along the non-linear trajectory is set forth in the present rejection.
Applicant's argument that the clinical-condition prediction must be derived specifically from the radiation dose and checkpoint errors also is not commensurate with the scope of claims 45 and 60. As explained in the Claim Interpretation section above, the claims require the steering-phase operational data to include both a total radiation dose in the steering phase and checkpoint errors of a plurality of checkpoints, and separately require the prediction or detection to be based at least in part on the steering-phase operational data. The claims do not require every item contained in the steering-phase operational data, or specifically both the radiation dose and checkpoint errors, to be used in generating the prediction or detection. The present rejection therefore establishes both the required contents of the steering-phase operational data and the use of steering-phase operational data in generating the clinical-condition prediction.
Applicant's Argument: Regarding claim 54, Applicant argues that Boddington estimates probabilities from EHR and prior-outcome datasets rather than steering-phase operational data generated during execution of a non-linear trajectory. Applicant argues that Boddington therefore does not satisfy claim 54 as dependent from amended claim 45.
Examiner's Response: Applicant’s arguments are not persuasive as they do not overcome the present rejection. The requirement that the current clinical-condition prediction be based at least in part on steering-phase operational data is established by the combination applied to amended claim 45. For claim 54's additional limitations, Zhao is relied upon for developing a pneumothorax risk-prediction model from 864 previous CT-guided needle-biopsy procedures having recorded pneumothorax outcomes, using pneumothorax occurrence as the dependent training target, and calculating the predictive probability of pneumothorax (Zhao, pp. 4751-4752, 4754). Boddington further confirms that known surgical outcomes are suitable training data for a classifier that outputs an outcome prediction with a statistical likelihood or confidence level (Boddington, ¶[0102]-[0104]). Claim 54 does not require the prior-procedure target parameters themselves to be steering-phase operational data. The prior occurrence labels are used during supervised training, while the current steering-phase operational data are supplied as instance-specific inputs during inference. Thus, the same algorithm may be trained using prior pneumothorax occurrence labels and express its output as a probability while remaining configured to generate the current prediction during insertion based at least in part on the steering-phase operational data established regarding claim 45. The claim 54 rejection was reassessed because Applicant's amendment and argument placed this inherited current-procedure linkage directly at issue.
Applicant's Argument: Regarding claim 55, Applicant argues that Huo merely identifies statistical pneumothorax risk factors and does not teach constructing individual predictive models whose predictions are used as inputs for training a further data analysis algorithm. Applicant further argues that the previous rejection relied on only a single individual model while amended claim 55 requires at least two of the enumerated condition-specific individual models. Applicant also argues that the combination does not teach those individual models operating on steering-phase operational data.
Examiner's Response: Applicant’s arguments are not persuasive as they do not overcome the present rejection of claim 55. The present rejection does not rely on Huo as teaching either of the two required individual models. Boddington is relied upon for a learned pose-classification technique (Boddington, ¶[0119]-[0122]), and Gerard is relied upon for a trained pulmonary-fissure detection technique (Gerard, pp. 2-3). The fissure location determined using Gerard's technique is compared with the established instrument trajectory to provide the recited fissure-crossing determination. Huo is relied upon for the clinical significance of patient position and fissure crossing to pneumothorax risk, including the reported increase in pneumothorax incidence when a fissure is crossed (Huo, p. 1, Abstract; p. 6, section "Fissure crossed"; p. 7, Table 4), and therefore provides a reason to use predictions from the patient-pose and fissure-crossing individual models when training the pneumothorax-probability data analysis algorithm. The present rejection therefore addresses amended claim 55's requirement for at least two of the recited individual models rather than relying on the single-model theory challenged by Applicant.
Claim 55 also does not require each individual model itself to operate on steering-phase operational data. Claim 55 requires predictions generated by the individual models to be used as input for training the data analysis algorithm. Through dependency from claim 45, that data analysis algorithm is required to generate the clinical-condition prediction or detection based at least in part on steering-phase operational data. The present rejection maintains that requirement for the same pneumothorax-probability data analysis algorithm while additionally using predictions from the patient-pose and fissure-crossing individual models as predictor inputs. Applicant's argument therefore imposes an additional requirement on the individual models that is not recited in claim 55.
Applicant does not present separate substantive arguments directed to the additional limitations of claims 46-53, 56-57, and 61-64 apart from asserting their patentability by virtue of dependency from claims 45 or 60. Because the arguments concerning claims 45 and 60 do not overcome the presently stated grounds, and the additional limitations of the dependent claims are addressed in their respective rejections above, the arguments based solely on dependency do not provide a separate basis for withdrawal of those rejections.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the
examiner should be directed to AARON MERRIAM whose telephone number is (703) 756-
5938. The examiner can normally be reached M-F 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing
using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is
encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s
supervisor, Jason Sims can be reached on (571)272-4867. The fax phone number for the
organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained
from Patent Center. Unpublished application information in Patent Center is available to
registered users. To file and manage patent submissions in Patent Center, visit:
https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for
more information about Patent Center and https://www.uspto.gov/patents/docx for
information about filing in DOCX format. For additional questions, contact the Electronic
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO
Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON MERRIAM/Examiner, Art Unit 3791
/MATTHEW KREMER/Primary Examiner, Art Unit 3791
Prosecution Insights