Notice of Pre-AIA or AIA Status
Claims 1-11, 13-14, 16-18 and 21-22 are currently presented for Examination.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 03/02/2026 has been entered and considered by the examiner. By the
amendment, claims 1-2, 10-11, 13, 14 and 16 are amended. Following Applicants arguments and amendments made, Examiner modify the 103 rejections. See office action for detail.
Response to 103 Arguments
Applicant arguments on claim 1 and 11
Gliner does not disclose or suggest "determining a first plurality of weights for the set of matches based on a distance between each match of the set of matches and an individual anatomic location," as recited by claim 1. (Emphasis added.) Instead, Gliner states that a distance is determined between each anatomical point corresponding to a predefined location and the closest point on a bone of the patient's face. As Gliner explains, the point on the bone of the patient's face to which a distance is measured is different for each anatomical point. For example, when the anatomical point is on the patient's forehead, a distance is measured between the forehead point and a point on the patient's forehead bone. When the anatomical point is on the patient's cheek, a distance is measured between the cheek point and a point on the patient's cheek bone. A distance is not measured between the cheek point and the point on the patient's forehead bone, or vice versa. Therefore, as discussed during the interview, Gliner does not determine a distance between each anatomical point and the same individual anatomic location.
Examiner response
Examiner agrees that Gliner does not disclose determine a distance between each anatomical point and the same individual anatomic location. In view of amendment and arguments made regarding the same individual anatomic location, Examiner cited the new references Lin et al. (US20140247993A1) that explicitly discloses the same landmark/individual anatomic location. The landmarks may include eyes, a nose, a mouth, a chin, ears, a jaw line, and/or any other landmark of the face.
Applicant arguments for claim 16
The Applicant respectfully submits that the cited portions of Commonwick does not disclose or suggest "wherein the second anatomic area shares a border with the first anatomic area," as recited by amended claim 16.
Examiner response
Applicant arguments regarding the newly added limitation" wherein the second anatomic area shares a border with the first anatomic area. Examiner do not rely on Commonwick to reject this claim limitation. However, Primary reference Donhowe teaches “dividing the anatomic structure into a plurality of anatomic areas, wherein the second anatomic area shares a border with the first anatomic area”. (See Donhowe para 65-66- As shown in FIG. 5B, the centerline segmented model 504 includes several branch points, some of which are highlighted for visibility in FIG. 5B. The branch points A, B, C, D, and E are shown at each of several of the branch points. The branch point A may represent the point in the model at which the trachea divides into the left and right principal bronchi. The right principal bronchus may be identified in the centerline segment model 504 as being located between branch points A and B. Similarly, secondary bronchi are identified by the branch points B and C and between the branch points B and E. Another generation may be defined between branch points C and D. It is the Examiner position under BRI, first airway area corresponds to between branch point B and C and second airway area corresponds to between branch point C and D. These two areas share the common branch point C which is a shared interface/border between adjoining airway regions. Thus, rejection on claim 16 is still maintained.
Applicant arguments for claim 18
Applicant respectfully submits that the cited portions of Donhowe and Commonwick do not disclose or suggest at least that "dividing the anatomic structure into the plurality of anatomic areas includes: dividing the anatomic structure based on structure rigidities of the plurality of anatomic areas respectively," as recited by claim 18.
Examiner response
Examiner respectfully disagrees. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). In this case, Donhowe teaches dividing the anatomic structure into the plurality of anatomic areas such as airway segments. Commonwick teaches local affine registration for anatomic areas and transformations incorporating “rigid structures”. It would have obvious to define the plurality of anatomic regions of Donhowe according to rigidity characteristics because Commonwick recognizes different anatomic structures may behave rigidly and benefit from localized affine treatment. See Commonwick Appendix A.1- “In this approach, pairs of rigid structures are selected in the input images, along with linear transformations. This results in applying the affine transforms to the user-defined structures while ensuring a smooth interpolation in between them.” It is the Examiner position that under BRI selecting rigid structures means the regions are defined according to the rigidity behavior, therefore the regional division is based on structural rigidity.
Claim Rejections - 35 USC § 103
4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
6. This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
7. Claims 1-6 and 8-11 are rejected under 35 U.S.C. 103 as being unpatentable over Donhowe et al. (WO2017030915A1) in view of Lin et al. (US20140247993A1)
Regarding claim 1
Donhowe teaches a system comprising: a non-transitory memory; one or more processors coupled to the non-transitory memory and configured to read instructions to cause the system to perform operations comprising: (see fig 1 and para 146-One or more elements in embodiments of the invention may be implemented in software to execute on a processor of a computer system such as control system 112. When implemented in software, the elements of the embodiments of the invention are essentially the code segments to perform the necessary tasks. The program or code segments can be stored in a non-transitory processor readable storage medium or device, including any medium that can store information including an optical medium, semiconductor medium, and magnetic medium)
accessing a set of model points of a model of an anatomic structure of a patient, the model points being associated with a model space; ( see para 008-In addition exemplary method may include accessing a set of model points of a model of one or more passageways of a patient. See para 67-n some embodiments, the centerline segmented model 504 is represented in data as a cloud, set, or collection of points in three-dimensional space, rather than as continuous lines. FIG. 5C illustrates the centerline segmented model 504 as a set of points 506. In data, each of the points of the set of model points may include coordinates such as a set of XM, YM, and ZM, coordinates, or other coordinates that identify the location of each point in the three-dimensional space.)
collecting, via a medical instrument, a set of measured points of the medical instrument as the medical instrument traverses the anatomic structure of the patient, the measured points being associated with a patient space; (see para 35-Other motorized drive systems may move the distal end of the medical instrument in multiple degrees of freedom, which may include three degrees of linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) and in three degrees of rotational motion (e.g., rotation about the X, Y, Z Cartesian axes). see para 48-Additionally or alternatively the medical instrument system 200 may be used to gather (i.e., measure) a set of data points corresponding to locations with patient anatomic passageways. See para 69-FIGS. 6A and 6B illustrate an exemplary surgical environment 600 according to some embodiments, with a surgical coordinate system Xs, Ys, Zs, in which a patient P is positioned on a platform 602. The patient P may be stationary within the surgical environment)
wherein movement of the medical instrument as the medical instrument traverses the anatomic structure of the patient is controlled by a teleoperated manipulator; (see para 33-34-Referring to FIG. 1 of the drawings, a teleoperated medical system for use in, for example, surgical, diagnostic, therapeutic, or biopsy procedures, is generally indicated by the reference numeral 100. As shown in FIG. 1, the teleoperated system 100 generally includes a teleoperational manipulator assembly 102 for operating a medical instrument 104 in performing various procedures on the patient P. The assembly 102 is mounted to or near an operating table O. A master assembly 106 allows the clinician or surgeon S to view the interventional site and to control the slave manipulator assembly 102.)
determining a set of matches between the set of model points and the set of measured points, wherein each match of the set of matches comprises a model point of the set of model points and a measured point of the set of measured points; (See para 006-The method may further include matching each measured point to a model point to generate a set of matches. See para 0093- For example, each of the measured data points D may be matched with the closest point in the anatomic model information 550)
determining a first plurality of weights; (see para 93- The registration algorithm identifies matches between closest points in the gathered data points D and in the set of anatomic model points. See para 127- 128-As shown in FIG. 13, in some embodiment, the control system 112 may adjust weights associated with one or more of the points to alter their effects as factors in the computation of the corrective motions. The weights may be adjusted based on one or more factors. In some embodiments of the method 700 of FIG. 7, the registration process 712 may include an additional process in which weights for measured points are altered according to parameters or rules.)
registering the set of model points to the set of measured points based on the first plurality of weights to generate a first registration. (See para 007-performing an initial registration of the set of measured points to a set of modeled points obtained from the model. See para 54 and fig 4-At a process 456, the anatomic model data is registered to the patient anatomy prior to and/or during the course of an image-guided surgical procedure on the patient. Generally, registration involves the matching of measured point to points of the model through the use of rigid and/or non-rigid transforms. The measured points may be generated for use in an iterative closest point (ICP) technique described in detail at FIG. 6 and elsewhere in this disclosure. Other point set registration methods may also be used in registration processes within the scope of this disclosure. See para 125- In some embodiments, other heuristics may be used to assigned weights to measured points. Additionally in some embodiments, the control system 112 may guide a clinician in obtaining measured points to use in registering and anatomic model. For example, certain passageways in the upper lobe of each lung may provide particularly reliable and useful information for registering and anatomic model to a patient undergoing a procedure.)
based on the registration and in response to one or more received user inputs, controlling a drive system of the teleoperated manipulator to actuate the medical instrument within the anatomic structure and move the medical instrument toward the individual anatomic location to examine, diagnose, biopsy, or treat the individual anatomic location. (See para 34-38- The control devices may include any number of a variety of input devices, such as joysticks, trackballs, data gloves, trigger-guns, hand-operated controllers, voice recognition devices, body motion or presence sensors, or the like. The teleoperational assembly 102 includes plurality of actuators or motors that drive inputs on the medical instrument system 104 in response to commands from the control system (e.g., a control system 112). The motors include drive systems that when coupled to the medical instrument system 104 may advance the medical instrument into a naturally or surgically created anatomic orifice. Other motorized drive systems may move the distal end of the medical instrument in multiple degrees of freedom, which may include three degrees of linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) and in three degrees of rotational motion (e.g., rotation about the X, Y, Z Cartesian axes). The display 110 and the operator input system 106 may be oriented so the operator can control the medical instrument system 104 and the operator input system 106 with the perception of telepresence. See para 44-The servo controller(s) may also transmit signals instructing teleoperational assembly 102 to move the medical instrument system(s) 104 which extend into an internal surgical site within the patient body via openings in the body. See para 60- As shown in greater detail in FIG. 2B, medical tool(s) 228 for such procedures as surgery, biopsy, ablation, illumination, irrigation, or suction can be deployed through the channel 221 of the flexible body 216 and used at a target location within the anatomy. If, for example, the tool 228 is a biopsy instrument, it may be used to remove sample tissue or a sampling of cells from a target anatomic location. See para 135-In other embodiments, registration may be initiated by the control system 112 after motion in the tracking device 624 is detected.)
Donhowe does not teach determining a first plurality of weights for the set of matches based on a distance between each match of the set of matches and an individual anatomic location, wherein a first match of the set of matches is a first distance from the individual anatomic location, wherein a second match of the set of matches is a second distance from the individual anatomic location, wherein the second distance is less than the first distance, wherein a first weight determined for the first match is less than a second weight determined for the second match;
In the related field of invention, Lin teaches determining a first plurality of weights for the set of matches based on a distance between each match of the set of matches and an individual anatomic location, wherein a first match of the set of matches is a first distance from the individual anatomic location, wherein a second match of the set of matches is a second distance from the individual anatomic location, wherein the second distance is less than the first distance, wherein a first weight determined for the first match is less than a second weight determined for the second match; (see para 19- For example, the test image may depict a face and the landmarks may include eyes, a nose, a mouth, a chin, ears, a jaw line, and/or any other landmark of the face as shown in FIG. 1. See para 23- he scores depicted or represented in a voting map can be weighted based on distance from the matched feature to the landmark in the object image such that a feature closer to the landmark will have a higher contribution to the score. See para 41- Additionally, the similarity voting service 139 may apply a weight to matching features between the respective object image 159 and the test image 153 that are closer to the respective landmark than those that are farther away. see para 65- In particular, the similarity voting service 139 applies a weight on matching features based on the relative distance between the matching feature and the respective landmark in generating the similarity voting map for the landmark. For instance, features that are closer to the landmark are given a higher weight than any features farther away from the landmark.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date
of the claimed invention to modify the method of registering sets of anatomical data for use during a surgical procedure as disclosed by Donhowe to include determining a first plurality of weights for the set of matches based on a distance between each match of the set of matches and an individual anatomic location, wherein a first match of the set of matches is a first distance from the individual anatomic location, wherein a second match of the set of matches is a second distance from the individual anatomic location, wherein the second distance is less than the first distance, wherein a first weight determined for the first match is less than a second weight determined for the second match as taught by Lin in the system of Donhowe for identifying feature matches between each of a plurality of object images and a test image thus weighting improve the accuracy of the landmark location estimation. (see abstract and para 065, Lin)
Regarding claim 2
Donhowe in view of Lin discloses the system of claim 1. Donhowe further teaches wherein the individual anatomic location is associated with the model space. (See para 003-006- Through these natural orifices or incisions clinicians may insert minimally invasive medical instruments (including surgical, diagnostic, therapeutic, or biopsy instruments) to reach a target tissue location. To assist with reaching the target tissue location, the location and movement of the medical instruments may be correlated with pre-operative or intra-operative images of the patient anatomy. The method may further include matching each measured point to a model point to generate a set of matches, a value of each of the matches depending on the assigned weight of the measured point in the match. See para 0084- Through the user interface, the clinician may identify the anatomic location of the selected landmarks so that a corresponding location in the anatomic model information may be approximated. See para 129- In some embodiments, the weight of a given measured point may be based on the generation of the passageway in which the point was obtained. For example, as shown in FIG. 11 A, the anatomic model 1104 includes several segments 1104 A, 1104B, and 1104C and others that are associated with specific generations of passageways. Because the trachea is a broader passageway, and so provides less information with which to register the measured points to the modeled points, measured points that match to the segment 1104 A be having a relatively reduced weight. Similarly, measured points that match to the segment 1104C and other of the same more distal generation may be assigned a relatively reduced weight as well. The measured points associated with the more distal generations may be assigned less weight because these more distal passageways are more likely to deform due to the forces exerted by the point gather instrument on the passageways. See also para 102-104)
Regarding claim 3
Donhowe in view of Lin discloses the system of claim 2. Donhowe further teaches determining a second plurality of weights for the set of model points or the set of measured points. (see para 127-128-As shown in FIG. 13, in some embodiment, the control system 112 may adjust weights associated with one or more of the points to alter their effects as factors in the computation of the corrective motions. The weights may be adjusted based on one or more factors. Shown in FIG. 13 is a set of measured points as stored in a point pool 1300 in memory of the control system 112. The exemplary point pool 1300 contains data representing the measured points obtained from within the patient P. In the point pool 1300, each point includes a point identifier, a set of coordinates, a timestamp, a sensor ID, a phase marker, and a weight. This data may be formatted in many different ways. As shown in FIG. 13, the phase marker is a binary value, either 0 or 1, depending on whether the phase is inhalation or exhalation, respectively. In some embodiments of the method 700 of FIG. 7, the registration process 712 may include an additional process in which weights for measured points are altered according to parameters or rules.)
Regarding claim 4
Donhowe in view of Lin discloses the system of claim 3. Donhowe further teaches wherein registering the set of model points to the set of model points is further based on the second plurality of weights. (see para 135-Due to movement of the patient P, a previous registration between an anatomic model and measured points may become less accurate. Accordingly information obtained from and/or displayed in connection with the anatomic model, such as a lesion or tumor, may not be accurately communication to a clinician. In some embodiments, after a satisfactory registration has been obtained and movement of the patient P is detected, the registration process may begin again. In some embodiments, a change in displacement and/or orientation measured by the tracking device 624 may be used to update the registration. In some embodiments, the registration process 712 may be performed again beginning with a seeding process. In other embodiments, the registration process 712 may be performed without performing a new seeding process. For example, if the movement of the patient P is determined to be small, discarding older measured points from the point pool 1300 (or decreasing their relative weighting substantially in favor of points obtained after the detection of the movement of patient P) and collecting new measured points using the catheter. Thus in some embodiments, a new set of points is collected and used to register the model to the moved patient P or a mixed weighting of new and old points may be used.)
Regarding claim 5
Donhowe in view of Lin discloses the system of claim 1. Donhowe further teaches obtaining a first distal end location of a distal end of the medical instrument inserted into the anatomic structure. (see para 49-The instrument system 200 includes a catheter system 202 coupled to an instrument body 204. The catheter system 202 includes an elongated flexible catheter body 216 having a proximal end 217 and a distal end or tip portion 218. The catheter system 202 may optionally include a shape sensor 222 for determining the position, orientation, speed, velocity, pose, and/or shape of the catheter tip at distal end 218 and/or of one or more segments 224 along the body 216. The entire length of the body 216, between the distal end 218 and the proximal end 217, may be effectively divided into the segments 224.)
Regarding claim 6
Donhowe in view of Lin discloses the system of claim 5. Donhowe further teaches wherein the determining the first plurality of weights includes: for each measured point, determining a distal end distance between the measured point and the first distal end location; (see para 48-49-the medical instrument system 200 may be used to gather (i.e., measure) a set of data points corresponding to locations with patient anatomic passageways. The instrument system 200 includes a catheter system 202 coupled to an instrument body 204. The catheter system 202 includes an elongated flexible catheter body 216 having a proximal end 217 and a distal end or tip portion 218. The catheter system 202 may optionally include a shape sensor 222 for determining the position, orientation, speed, velocity, pose, and/or shape of the catheter tip at distal end 218 and/or of one or more segments 224 along the body 216. The entire length of the body 216, between the distal end 218 and the proximal end 217, may be effectively divided into the segments 224. see para 71-FIG. 6A shows the instrument body 612 and carriage 606 in a retracted position along the insertion stage 608. In this retracted position, the proximal point 616 is at a position Lo on the axis A. In this position along the insertion stage 608 an Xs component of the location of the point 616 may be set to a zero or original value. With this retracted position of the instrument body 612 and carriage 606, the distal end 618 of the catheter may be positioned just inside an entry orifice of the patient P. Also in this position, the position measuring device may be set to a zero or original value (e.g. 1=0). In FIG. 6B, the instrument body 612 and the carriage 606 have advanced along the linear track of the insertion stage 608 and the distal end of the catheter 610 has advanced into the patient P. In this advanced position, the proximal point 616 is at a position L1 on the axis A. see para 106-The measured data point 1102C matches to the model point 1106C. The model point 1106C is a "terminal point" in that it is the most distal point in the segment 1104G. see para 115-Referring now to FIGS. 1IE and 1IF, some embodiments of the medical system may implement a heuristic of a maximum distance threshold such that measured points that match to model points but that have a separation distance)
and determining a weight for the measured point based on the distal end distance. (see para 115-Referring now to FIGS. 1 IE and 1 IF, some embodiments of the medical system may implement a heuristic of a maximum distance threshold such that measured points that match to model points but that have a separation distance. see para 125- In some embodiments, other heuristics may be used to assigned weights to measured points. see para 133-135-In some embodiments, machine learning may be used to identify qualities of the most reliable points. The control system may then apply weights accordingly. Each of the described factors may be used to determine the weight of a single point. Thus, while a single factor may be used to determine the weight of a given point in some embodiments, in other embodiments multiple factors may be used by the control system to adjust the weight of one or more of the points in the point pool 1300. Referring again to FIGS. 6A and 6B, shown therein is a tracking device 624 temporarily affixed to the patient P. By monitoring the tracking device 624 the control system may determine whether the patient P moves. The tracking device 624 may be a device capable of generating position and or movement data, such as a set of EM sensors/transmitters, accelerometers, etc. For example, if the movement of the patient P is determined to be small, discarding older measured points from the point pool 1300 (or decreasing their relative weighting substantially in favor of points obtained after the detection of the movement of patient P) and collecting new measured points using the catheter. Thus in some embodiments, a new set of points is collected and used to register the model to the moved patient P or a mixed weighting of new and old points may be used)
Regarding claim 8
Donhowe in view of Lin discloses the system of claim 6. Donhowe further teaches wherein a first measured point has a first distance from the first distal end location, wherein a second measured point has a second distance from the first distal end location, the second distance being less than the first distance, and wherein a first weight determined for the first measured point is less than a second weight determined for the second measured point. (see para 118-119-As illustrated in FIG. 1IE, the measured point 1102M is matched to a point on the segment 1104A. However, the separation distance between the measured point 1102M and the segment 1104 A is greater than the maximum distance threshold 1116A and so the match may be ignored when computing the corrective motion to be applied to the set of measured points 1102. Similarly, the measured point 1102N is further away from the segment 1104C than the maximum distance threshold 1116B and so may also be ignored. Referring now to FIG. 1 IF, shown therein is the result of the corrective motion computed based on the set of measured points 1102 and the anatomic model 1104 as illustrated in FIG. 1IE. Due to the corrective motion, the measured points 1102M and 1102N are position within maximum distance threshold 1116A and 1116B, respectively. Accordingly, the matches between the measured points 1102M and 1102N and the closest points in segments 1104A and 1104C, respectively, may be included by the maximum distance heuristic as factors in the computation of the corrective motion of the subsequent iteration. see para 127- As shown in FIG. 13, in some embodiment, the control system 112 may adjust weights associated with one or more of the points to alter their effects as factors in the computation of the corrective motions. see para 130-133-Similarly, if the catheter is in a controlled state such that the distal end of the catheter is actively positioned in a central portion of the passageway through which is the catheter is passing the measured point may have a relatively higher weight than if the catheter is in a flaccid state. In some embodiments, the measured points may be collected into the point pool 1300 only when the catheter is actively being steered and controlled in order to collect points from the center of the passageways. When in the flaccid state the catheter may be more likely to pass along the bottom of the passageway than when in the active, controlled state. In some embodiments, the measured points collected in the flaccid or passive state may be compensated with an adjustment to make it as if the points were collected closer to the center of the passageway. This may be done by altering the coordinates of the collected points to move the point toward the center. In some embodiments, machine learning may be used to identify qualities of the most reliable points. The control system may then apply weights accordingly. Each of the described factors may be used to determine the weight of a single point. Thus, while a single factor may be used to determine the weight of a given point in some embodiments, in other embodiments multiple factors may be used by the control system to adjust the weight of one or more of the points in the point pool 130)
Regarding claim 9
Donhowe in view of Lin discloses the system of claim 6. Donhowe further teaches wherein the controlling includes the moving the distal end of the medical instrument to a second distal end location (see para 141-143-The movement of the rigid instrument body 612 may provide an indication of the movement of the distal end 618 of the catheter 610. At the process 1404, the data representing the points may be added to the point pool 1300, stored in memory, as shown in FIG. 13. This location information represented by the measured points may be collected exclusively at the distal end 618 of the catheter 610, from a plurality of discrete points along the length of the catheter, or obtained from shape-sensor information continuously along the length of the catheter 610.)
wherein the operations further comprise determining a second plurality of weights for the set of measured points respectively based on the second distal end location; (See para 127-As shown in FIG. 13, in some embodiment, the control system 112 may adjust weights associated with one or more of the points to alter their effects as factors in the computation of the corrective motions. The weights may be adjusted based on one or more factors.)
and registering the set of model points to the set of measured points based on the second plurality of weights to generate a second registration. (see para 006-Weights may be assigned to one or more of the measured points. The method may further include matching each measured point to a model point to generate a set of matches, a value of each of the matches depending on the assigned weight of the measured point in the match, and moving the set of measured points relative to the set of model points based on the set of matches. see para 009-generating a second registration between a second set of measured points and the set of model points, and then determining whether to implement the second registration in place of the first registration.)
Regarding claim 10
Donhowe in view of Lin discloses the system of claim 6. Donhowe further teaches for each measured point, determining a target distance between the measured point and the individual anatomic location; (see para 48-49-the medical instrument system 200 may be used to gather (i.e., measure) a set of data points corresponding to locations with patient anatomic passageways. The instrument system 200 includes a catheter system 202 coupled to an instrument body 204. The catheter system 202 includes an elongated flexible catheter body 216 having a proximal end 217 and a distal end or tip portion 218. The catheter system 202 may optionally include a shape sensor 222 for determining the position, orientation, speed, velocity, pose, and/or shape of the catheter tip at distal end 218 and/or of one or more segments 224 along the body 216. The entire length of the body 216, between the distal end 218 and the proximal end 217, may be effectively divided into the segments 224. See para 60-62 As shown in greater detail in FIG. 2B, medical tool(s) 228 for such procedures as surgery, biopsy, ablation, illumination, irrigation, or suction can be deployed through the channel 221 of the flexible body 216 and used at a target location within the anatomy. If, for example, the tool 228 is a biopsy instrument, it may be used to remove sample tissue or a sampling of cells from a target anatomic location. At a process 454, computer software alone or in combination with manual input is used to convert the recorded images into a segmented two-dimensional or three-dimensional composite representation or model of a partial or an entire anatomic organ or anatomic region. See para 0084- Through the user interface, the clinician may identify the anatomic location of the selected landmarks so that a corresponding location in the anatomic model information may be approximated. See para 104-As illustrated in FIG. 11 A, the measured data point 1102A matches to a model point 1106A. A distance 1107 characterizes a distance or error value associated with the matched pair of the measured data point 1102A and the model point 1106A. These points may be matched because the distance 1107 is the shortest distance between the measured data point 1102A and any other model point in the anatomic model 1104. Other matches illustrated in FIG. 11A include a match between the measured point 1102B and model point 1106B, another match between the measured point 1102C and model point 1106C, and another match between the measured point 1102D and model point 1106D. The measured point 1102D is also matched with the model point 1106E. The distance between the measured point 1102D and the model point 1106D and the distance between the measured point 1102D and the model point 1106E may be identical or may be substantially similar so as to be considered identical. For example, when the distances between a measured point and two model points differ by less than a threshold percentage (e.g., 10%, 5%, 3%, etc.), the distances may be deemed identical for certain purposes. In some embodiments, the distance between the model points 1106D and 1106E may be calculated.)
and determining the weight for the measured point based on at least one of the distal end distance and the target distance. (see para 133-135-In some embodiments, machine learning may be used to identify qualities of the most reliable points. The control system may then apply weights accordingly. Each of the described factors may be used to determine the weight of a single point. Thus, while a single factor may be used to determine the weight of a given point in some embodiments, in other embodiments multiple factors may be used by the control system to adjust the weight of one or more of the points in the point pool 1300. Referring again to FIGS. 6A and 6B, shown therein is a tracking device 624 temporarily affixed to the patient P. By monitoring the tracking device 624 the control system may determine whether the patient P moves. The tracking device 624 may be a device capable of generating position and or movement data, such as a set of EM sensors/transmitters, accelerometers, etc. For example, if the movement of the patient P is determined to be small, discarding older measured points from the point pool 1300 (or decreasing their relative weighting substantially in favor of points obtained after the detection of the movement of patient P) and collecting new measured points using the catheter. Thus in some embodiments, a new set of points is collected and used to register the model to the moved patient P or a mixed weighting of new and old points may be used. See para 144-Thee accuracy metric may be the percentage of measured points successfully matched to the model or may be the average distance between the measured points and the matched model points.)
Regarding claim 11
Donhowe teaches a system comprising: a non-transitory memory; one or more processors coupled to the non-transitory memory and configured to read instructions to cause the system to perform operations comprising: (see fig 1 and para 146-One or more elements in embodiments of the invention may be implemented in software to execute on a processor of a computer system such as control system 112. When implemented in software, the elements of the embodiments of the invention are essentially the code segments to perform the necessary tasks. The program or code segments can be stored in a non-transitory processor readable storage medium or device, including any medium that can store information including an optical medium, semiconductor medium, and magnetic medium)
accessing a set of model points of a model of an anatomic structure of a patient, the model points being associated with a model space; (see para 008-In addition exemplary method may include accessing a set of model points of a model of one or more passageways of a patient. See para 67-n some embodiments, the centerline segmented model 504 is represented in data as a cloud, set, or collection of points in three-dimensional space, rather than as continuous lines. FIG. 5C illustrates the centerline segmented model 504 as a set of points 506. In data, each of the points of the set of model points may include coordinates such as a set of XM, YM, and ZM, coordinates, or other coordinates that identify the location of each point in the three-dimensional space.)
collecting, via a medical instrument, a set of measured points of the medical instrument as the medical instrument traverses the anatomic structure of the patient, the measured points being associated with a patient space; (see para 35-Other motorized drive systems may move the distal end of the medical instrument in multiple degrees of freedom, which may include three degrees of linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) and in three degrees of rotational motion (e.g., rotation about the X, Y, Z Cartesian axes). see para 48-Additionally or alternatively the medical instrument system 200 may be used to gather (i.e., measure) a set of data points corresponding to locations with patient anatomic passageways. See para 69-FIGS. 6A and 6B illustrate an exemplary surgical environment 600 according to some embodiments, with a surgical coordinate system Xs, Ys, Zs, in which a patient P is positioned on a platform 602. The patient P may be stationary within the surgical environment)
wherein movement of the medical instrument as the medical instrument traverses the anatomic structure of the patient is controlled by a teleoperated manipulator; (see para 33-34-Referring to FIG. 1 of the drawings, a teleoperated medical system for use in, for example, surgical, diagnostic, therapeutic, or biopsy procedures, is generally indicated by the reference numeral 100. As shown in FIG. 1, the teleoperated system 100 generally includes a teleoperational manipulator assembly 102 for operating a medical instrument 104 in performing various procedures on the patient P. The assembly 102 is mounted to or near an operating table O. A master assembly 106 allows the clinician or surgeon S to view the interventional site and to control the slave manipulator assembly 102.)
determining a first plurality of weights; (see para 93- The registration algorithm identifies matches between closest points in the gathered data points D and in the set of anatomic model points. See para 127- 128-As shown in FIG. 13, in some embodiment, the control system 112 may adjust weights associated with one or more of the points to alter their effects as factors in the computation of the corrective motions. The weights may be adjusted based on one or more factors. In some embodiments of the method 700 of FIG. 7, the registration process 712 may include an additional process in which weights for measured points are altered according to parameters or rules.)
registering the set of model points to the set of measured points based on the first plurality of weights to generate a registration. (see para 007-performing an initial registration of the set of measured points to a set of modeled points obtained from the model. See para 54 and fig 4-At a process 456, the anatomic model data is registered to the patient anatomy prior to and/or during the course of an image-guided surgical procedure on the patient. Generally, registration involves the matching of measured point to points of the model through the use of rigid and/or non-rigid transforms. The measured points may be generated for use in an iterative closest point (ICP) technique described in detail at FIG. 6 and elsewhere in this disclosure. Other point set registration methods may also be used in registration processes within the scope of this disclosure. See para 79-In some embodiments, the model may include one or more landmark points to match to the seed points DLI, DL2, and DL3. See para 125- In some embodiments, other heuristics may be used to assigned weights to measured points. Additionally in some embodiments, the control system 112 may guide a clinician in obtaining measured points to use in registering and anatomic model. For example, certain passageways in the upper lobe of each lung may provide particularly reliable and useful information for registering and anatomic model to a patient undergoing a procedure.)
based on the registration and in response to one or more received user inputs, controlling a drive system of the teleoperated manipulator to actuate the medical instrument within the anatomic structure and move the medical instrument toward the individual anatomic location to examine, diagnose, biopsy, or treat the individual anatomic location. (See para 34-38- The control devices may include any number of a variety of input devices, such as joysticks, trackballs, data gloves, trigger-guns, hand-operated controllers, voice recognition devices, body motion or presence sensors, or the like. The teleoperational assembly 102 includes plurality of actuators or motors that drive inputs on the medical instrument system 104 in response to commands from the control system (e.g., a control system 112). The motors include drive systems that when coupled to the medical instrument system 104 may advance the medical instrument into a naturally or surgically created anatomic orifice. Other motorized drive systems may move the distal end of the medical instrument in multiple degrees of freedom, which may include three degrees of linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) and in three degrees of rotational motion (e.g., rotation about the X, Y, Z Cartesian axes). The display 110 and the operator input system 106 may be oriented so the operator can control the medical instrument system 104 and the operator input system 106 with the perception of telepresence. See para 44-The servo controller(s) may also transmit signals instructing teleoperational assembly 102 to move the medical instrument system(s) 104 which extend into an internal surgical site within the patient body via openings in the body. See para 60- As shown in greater detail in FIG. 2B, medical tool(s) 228 for such procedures as surgery, biopsy, ablation, illumination, irrigation, or suction can be deployed through the channel 221 of the flexible body 216 and used at a target location within the anatomy. If, for example, the tool 228 is a biopsy instrument, it may be used to remove sample tissue or a sampling of cells from a target anatomic location. See para 135-In other embodiments, registration may be initiated by the control system 112 after motion in the tracking device 624 is detected.)
Donhowe does not teach determining a first plurality of weights for the set of model points based on a distance between each model point of the set of model points and an individual anatomic location, wherein a first model point of the set of model points is a first distance from the individual anatomic location, wherein a second model points of the set of model points is a second distance from the individual anatomic location, wherein the second distance is less than the first distance, wherein a first weight determined for the first model point is less than a second weight determined for the second model point;
In the related field of invention, Lin teaches determining a first plurality of weights for the set of matches based on a distance between each match of the set of matches and an individual anatomic location, wherein a first match of the set of matches is a first distance from the individual anatomic location, wherein a second match of the set of matches is a second distance from the individual anatomic location, wherein the second distance is less than the first distance, wherein a first weight determined for the first match is less than a second weight determined for the second match; (see para 19- For example, the test image may depict a face and the landmarks may include eyes, a nose, a mouth, a chin, ears, a jaw line, and/or any other landmark of the face as shown in FIG. 1. See para 23- he scores depicted or represented in a voting map can be weighted based on distance from the matched feature to the landmark in the object image such that a feature closer to the landmark will have a higher contribution to the score. See para 41- Additionally, the similarity voting service 139 may apply a weight to matching features between the respective object image 159 and the test image 153 that are closer to the respective landmark than those that are farther away. see para 65- In particular, the similarity voting service 139 applies a weight on matching features based on the relative distance between the matching feature and the respective landmark in generating the similarity voting map for the landmark. For instance, features that are closer to the landmark are given a higher weight than any features farther away from the landmark.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date
of the claimed invention to modify the method of registering sets of anatomical data for use during a surgical procedure as disclosed by Donhowe to include determining a first plurality of weights for the set of matches based on a distance between each match of the set of matches and an individual anatomic location, wherein a first match of the set of matches is a first distance from the individual anatomic location, wherein a second match of the set of matches is a second distance from the individual anatomic location, wherein the second distance is less than the first distance, wherein a first weight determined for the first match is less than a second weight determined for the second match as taught by Lin in the system of Donhowe for identifying feature matches between each of a plurality of object images and a test image thus weighting improve the accuracy of the landmark location estimation. (see abstract and para 065, Lin)
8. Claims 7 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Donhowe et al. (WO2017030915A1) in view of Lin et al. (US20140247993A1) and further in view of Barbagli et al. (WO2016077419A1)
Regarding claim 7
Donhowe in view of Lin discloses the system of claim 6. Donhowe further teaches wherein the determining the weight for the measured point based on the distal end distance includes: determining that the distal end distance is greater than a predetermined distal end distance threshold; and also teaches determining the weight to the measured point;(see para 93-The registration algorithm identifies matches between closest points in the gathered data points D and in the set of anatomic model points. In various alternatives, matching may be accomplished by using brute force techniques, KD tree techniques, etc. Some matches may be discarded based on maximum distance threshold calculations, maximum angle threshold calculations, or other metrics employed to filter out matches that are not deemed to be reliable enough for inclusion in the model. See para 115-Referring now to FIGS. 1IE and 1IF, some embodiments of the medical system may implement a heuristic of a maximum distance threshold such that measured points that match to model points but that have a separation distance that is greater than the maximum distance threshold are excluded from the computation of the corrective motion for an iteration.)
The combination of Donhowe and Lin does not teach determining the weight has a value of zero for the measured point.
In the related field of invention, Barbagli teaches determining the weight has a value of zero for the measured point. (see para 63-For example, spatial data record 502 describes the position PI and orientation 01 of the distal end of the instrument at a time Tl. A weighting factor Wl may be associated with the spatial data record 502. The weighting factors may be applied after a filtering process is performed for the set of data. For example, a weight Wl of 1 may indicate that the spatial data record 502 is usable or not otherwise objectionable for purposes of registration. A weight Wl of 0 may indicate that the spatial data record 502 is discarded or not to be used in registration of the instrument.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date
of the claimed invention to modify the method of registering sets of anatomical data for use during a surgical procedure as disclosed by Donhowe to include teaches determining the weight has a value of zero for the measured point as taught by Barbagli in the system of Donhowe and Lin in order to filter a set of spatial information from the tracking system by selecting, removing, or providing a weighting factor to the spatial data records that are deemed to be redundant, outliers, of a lower confidence or quality, from an undesirable time period (e.g., too old or too new), obtained during a certain cycle of cyclic anatomical motion, associated with a deformed shape, or other reasons that may cause a particular spatial data record or subset of spatial data records to be objectionable or preferable. And, at a process 458, the filtered set of spatial data records from the tracking system and the spatial information from the model may be registered. Improved techniques for filtering spatial information are needed to improve instrument localization with respect to the anatomy. (see para 62 and 003, Barbagli)
Regarding claim 13
Donhowe in view of Lin discloses the system of claim 11. The combination of Donhowe and Lin does not teach wherein the determining the first plurality of weights includes: for each model point, determining a navigation path distance between the model point and a predetermined navigation path to the individual anatomic location; and determining the weight for each model point based on at least the distance between each model point and the individual anatomic location and the navigation path distance.
However, Barbagli further teaches wherein the determining the first plurality of weights includes: for each model point, determining a navigation path distance between the model point and a predetermined navigation path to the individual anatomic location; (see para 58- At a process 406, an operator or an automated control system may plan a path through the model to a target structure or region (e.g., a tumor or an occlusion. See para 62-Fig. 4B is a flowchart showing an illustrative method 450 for performing a registration using filtered data. At an optional process 452, a set of spatial information is received from a model of the patient anatomy. At a process 454, a set of spatial information is obtained from the instrument tracking system. This set of spatial information may be a set of spatial data records obtained from the tracking system providing information about the time or order in which consecutive records were created and may include position, orientation, shape, movement or other spatial information about the instrument. See para 67- In another embodiment, redundancy may be based upon a spatial relationship such as a measured distance between the data records. For example, redundancy may be determined by ordering a plurality of spatial data records in collection time-gathered order. For example, as shown in FIG. 6B, a distance Dl between a first consecutive spatial data record 610 and a second consecutive spatial data record 612 may be measured. If the distance Dl is below a threshold value (e.g. too close), the second consecutive record 612 may be considered redundant of the first value 610. One of the first or second consecutive records, therefore, may be filtered by being discarded (e.g., give a weighting value of zero))
and determining the weight for each model point based on at least the distance between each model point and the individual anatomic location and the navigation path distance; (see para 67-In another embodiment, redundancy may be based upon a spatial relationship such as a measured distance between the data records. (see para 63-67- FIG. 5B illustrates a table 510 of the spatial data records 502, 504, 506, 508. The spatial data records may be obtained, for example, from a sensor (e.g. an EM sensor) located on a distal portion of the instrument. Each spatial data record provides spatial information about the distal portion of the instrument at various times. For example, spatial data record 502 describes the position PI and orientation 01 of the distal end of the instrument at a time Tl. A weighting factor Wl may be associated with the spatial data record 502. The weighting factors may be applied after a filtering process is performed for the set of data. For example, a weight Wl of 1 may indicate that the spatial data record 502 is usable or not otherwise objectionable for purposes of registration. A weight Wl of 0 may indicate that the spatial data record 502 is discarded or not to be used in registration of the instrument. A weight Wl between 0 and 1 may be applied if the confidence level in the spatial data record 502 is uncertain or low For example, redundancy may be determined by ordering a plurality of spatial data records in collection time-gathered order. For example, as shown in FIG. 6B, a distance Dl between a first consecutive spatial data record 610 and a second consecutive spatial data record 612 may be measured. If the distance Dl is below a threshold value (e.g. too close), the second consecutive record 612 may be considered redundant of the first value 610. One of the first or second consecutive records, therefore, may be filtered by being discarded (e.g., give a weighting value of zero) or provided with a low weighting value (e.g., less than one). This process of evaluating consecutive records may be optimized using a k- dimensional tree analysis.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date
of the claimed invention to modify the method of registering sets of anatomical data for use during a surgical procedure as disclosed by Donhowe to include wherein the determining the first plurality of weights includes: for each model point, determining a navigation path distance between the model point and a predetermined navigation path to the individual anatomic location; and determining the weight for each model point based on at least the distance between each model point and the individual anatomic location and the navigation path distance as taught by Barbagli in the system of Donhowe and Lin in order to filter a set of spatial information from the tracking system by selecting, removing, or providing a weighting factor to the spatial data records that are deemed to be redundant, outliers, of a lower confidence or quality, from an undesirable time period (e.g., too old or too new), obtained during a certain cycle of cyclic anatomical motion, associated with a deformed shape, or other reasons that may cause a particular spatial data record or subset of spatial data records to be objectionable or preferable. And, at a process 458, the filtered set of spatial data records from the tracking system and the spatial information from the model may be registered. Improved techniques for filtering spatial information are needed to improve instrument localization with respect to the anatomy. (see para 62, 003, Barbagli)
Regarding claim 14
Donhowe in view of Lin discloses the system of claim 11. Donhowe further teaches wherein the determining the weight for the model point includes: wherein determining that the distance between each model point and the individual anatomic location is greater than a predetermined target distance threshold; (see para 93-The registration algorithm identifies matches between closest points in the gathered data points D and in the set of anatomic model points. In various alternatives, matching may be accomplished by using brute force techniques, KD tree techniques, etc. Some matches may be discarded based on maximum distance threshold calculations, maximum angle threshold calculations, or other metrics employed to filter out matches that are not deemed to be reliable enough for inclusion in the model. See para 115-Referring now to FIGS. 1IE and 1IF, some embodiments of the medical system may implement a heuristic of a maximum distance threshold such that measured points that match to model points but that have a separation distance that is greater than the maximum distance threshold are excluded from the computation of the corrective motion for an iteration.)
The combination Donhowe and Lin does not teach determining the weight has a value of zero for the model point.
In the related field of invention, Barbagli teaches determining the weight has a value of zero for the model point. (see para 63-For example, spatial data record 502 describes the position PI and orientation 01 of the distal end of the instrument at a time Tl. A weighting factor Wl may be associated with the spatial data record 502. The weighting factors may be applied after a filtering process is performed for the set of data. For example, a weight Wl of 1 may indicate that the spatial data record 502 is usable or not otherwise objectionable for purposes of registration. A weight Wl of 0 may indicate that the spatial data record 502 is discarded or not to be used in registration of the instrument.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date
of the claimed invention to modify the method of registering sets of anatomical data for use during a surgical procedure as disclosed by Donhowe to include determining the weight has a value of zero for the measured point as taught by Barbagli in the system of Donhowe and Lin in order to filter a set of spatial information from the tracking system by selecting, removing, or providing a weighting factor to the spatial data records that are deemed to be redundant, outliers, of a lower confidence or quality, from an undesirable time period (e.g., too old or too new), obtained during a certain cycle of cyclic anatomical motion, associated with a deformed shape, or other reasons that may cause a particular spatial data record or subset of spatial data records to be objectionable or preferable. And, at a process 458, the filtered set of spatial data records from the tracking system and the spatial information from the model may be registered. Improved techniques for filtering spatial information are needed to improve instrument localization with respect to the anatomy. (see para 62, 003, Barbagli)
11. Claims 16-18 and 21-22 are rejected under 35 U.S.C. 103 as being unpatentable over Donhowe et al. (WO2017030915A1) in view of Commowick, Olivier, et al. "An efficient locally affine framework for the smooth registration of anatomical structures." Medical Image Analysis 12.4 (2008): 427-441.
Regarding claim 16
Donhowe teaches a method performed by a computing system comprising: (see fig 1)
accessing a set of model points of a model of an anatomic structure of a patient, the model points being associated with a model space; (see para 008-In addition exemplary method may include accessing a set of model points of a model of one or more passageways of a patient. See para 67-n some embodiments, the centerline segmented model 504 is represented in data as a cloud, set, or collection of points in three-dimensional space, rather than as continuous lines. FIG. 5C illustrates the centerline segmented model 504 as a set of points 506. In data, each of the points of the set of model points may include coordinates such as a set of XM, YM, and ZM, coordinates, or other coordinates that identify the location of each point in the three-dimensional space.)
collecting, via a medical instrument, a set of measured points of the medical instrument as the medical instrument traverses the anatomic structure of the patient, the measured points being associated with a patient space; (see para 35-Other motorized drive systems may move the distal end of the medical instrument in multiple degrees of freedom, which may include three degrees of linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) and in three degrees of rotational motion (e.g., rotation about the X, Y, Z Cartesian axes). see para 48-Additionally or alternatively the medical instrument system 200 may be used to gather (i.e., measure) a set of data points corresponding to locations with patient anatomic passageways. See para 69-FIGS. 6A and 6B illustrate an exemplary surgical environment 600 according to some embodiments, with a surgical coordinate system Xs, Ys, Zs, in which a patient P is positioned on a platform 602. The patient P may be stationary within the surgical environment)
wherein movement of the medical instrument as the medical instrument traverses the anatomic structure of the patient is controlled by a teleoperated manipulator; (see para 33-34-Referring to FIG. 1 of the drawings, a teleoperated medical system for use in, for example, surgical, diagnostic, therapeutic, or biopsy procedures, is generally indicated by the reference numeral 100. As shown in FIG. 1, the teleoperated system 100 generally includes a teleoperational manipulator assembly 102 for operating a medical instrument 104 in performing various procedures on the patient P. The assembly 102 is mounted to or near an operating table O. A master assembly 106 allows the clinician or surgeon S to view the interventional site and to control the slave manipulator assembly 102.)
registering the set of model points with the set of measured points to generate a first registration. (see para 007-performing an initial registration of the set of measured points to a set of modeled points obtained from the model. See para 54 and fig 4-At a process 456, the anatomic model data is registered to the patient anatomy prior to and/or during the course of an image-guided surgical procedure on the patient. Generally, registration involves the matching of measured point to points of the model through the use of rigid and/or non-rigid transforms. The measured points may be generated for use in an iterative closest point (ICP) technique described in detail at FIG. 6 and elsewhere in this disclosure. Other point set registration methods may also be used in registration processes within the scope of this disclosure. See para 79-In some embodiments, the model may include one or more landmark points to match to the seed points DLI, DL2, and DL3. See para 125- In some embodiments, other heuristics may be used to assigned weights to measured points. Additionally in some embodiments, the control system 112 may guide a clinician in obtaining measured points to use in registering and anatomic model. For example, certain passageways in the upper lobe of each lung may provide particularly reliable and useful information for registering and anatomic model to a patient undergoing a procedure.)
dividing the anatomic structure into a plurality of anatomic areas, a first anatomic area and a second anatomic area, wherein the second anatomic area shares a border with the first anatomic area; (see para 65-66- As shown in FIG. 5B, the centerline segmented model 504 includes several branch points, some of which are highlighted for visibility in FIG. 5B. The branch points A, B, C, D, and E are shown at each of several of the branch points. The branch point A may represent the point in the model at which the trachea divides into the left and right principal bronchi. The right principal bronchus may be identified in the centerline segment model 504 as being located between branch points A and B. Similarly, secondary bronchi are identified by the branch points B and C and between the branch points B and E. Another generation may be defined between branch points C and D. see para 78-With reference to FIG. 9, a set of anatomic passageways 900 includes main carinas Ci, C2, C3 where the passageways 900 fork. See also [0129] In some embodiments, the weight of a given measured point may be based on the generation of the passageway in which the point was obtained. For example, as shown in FIG. 11 A, the anatomic model 1104 includes several segments 1104 A, 1104B, and 1104C and others that are associated with specific generations of passageways. See para 83-For example, the major "Y" formation provided by the trachea and the left and right main bronchii shown in FIG. 9 may be used as explained herein.)
Examiner note: First airway area between branch point B and C and second airway area between branch point C and D. These two areas share the common branch point C which is a shared interface/border between adjoining airway regions.
based on the updated first registration and in response to one or more received user inputs, controlling a drive system of the teleoperated manipulator to actuate the medical instrument within the anatomic structure and move the medical instrument toward the individual anatomic location to examine, diagnose, biopsy, or treat the individual anatomic location (See para 34-38- The control devices may include any number of a variety of input devices, such as joysticks, trackballs, data gloves, trigger-guns, hand-operated controllers, voice recognition devices, body motion or presence sensors, or the like. The teleoperational assembly 102 includes plurality of actuators or motors that drive inputs on the medical instrument system 104 in response to commands from the control system (e.g., a control system 112). The motors include drive systems that when coupled to the medical instrument system 104 may advance the medical instrument into a naturally or surgically created anatomic orifice. Other motorized drive systems may move the distal end of the medical instrument in multiple degrees of freedom, which may include three degrees of linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) and in three degrees of rotational motion (e.g., rotation about the X, Y, Z Cartesian axes). The display 110 and the operator input system 106 may be oriented so the operator can control the medical instrument system 104 and the operator input system 106 with the perception of telepresence. See para 44-The servo controller(s) may also transmit signals instructing teleoperational assembly 102 to move the medical instrument system(s) 104 which extend into an internal surgical site within the patient body via openings in the body. See para 60- As shown in greater detail in FIG. 2B, medical tool(s) 228 for such procedures as surgery, biopsy, ablation, illumination, irrigation, or suction can be deployed through the channel 221 of the flexible body 216 and used at a target location within the anatomy. If, for example, the tool 228 is a biopsy instrument, it may be used to remove sample tissue or a sampling of cells from a target anatomic location. See para 135-In other embodiments, registration may be initiated by the control system 112 after motion in the tracking device 624 is detected.)
Donhowe does not teach generating a first local area registration for a first anatomic area of the plurality of anatomic areas; generating a second local area registration for a second anatomic area of the plurality of anatomic areas; combining the first local area registration and the second local area registration to generate an updated first registration.
In the related field of invention, Commowick teaches generating a first local area registration for the first anatomic area of the plurality of anatomic areas; generating a second local area registration for the second anatomic area of the plurality of anatomic areas; (See section Introduction- The registration is done by optimizing together local affine transformations and using a new Log-Euclidean regularization of these transformations, therefore, ensuring coherency between them. The global transformation is then parameterized using these affine components Ai, associated to user-defined areas Ri. See section 2- The goal of our method is then to compute an affine transformation Ai for each region Ri that best matches this region in the two images. Finally, a global transformation T is interpolated from the affine components Ai.)
combining the first local area registration and the second local area registration to generate an updated first registration. (See section 2- The goal of our method is then to compute an affine transformation Ai for each region Ri that best matches this region in the two images. Finally, a global transformation T is interpolated from the affine components Ai. To use our algorithm, we first need to define the regions Ri we want to register. We define them on the reference image. This image is indeed never deformed and the regions can thus be defined once and for all. In our case, we choose to have entire areas adopting the same affine behavior. For example, in Fig. 1, we want the transformation to be roughly affine for each one. We, therefore, put only one region over each eye. To ensure a smooth interpolation between the regions we have defined, we need a minimal distance between them. We have chosen to implement the weighting function for each component as a function of the minimal distance to the corresponding region. At each iteration l, we update the transformation Tl by looking for local affine transformations Ai so that we get a better correspondence between the images over the regions Ri. See equation A.1)
Examiner note: Defining multiple regions (Ri), determining a local affine transformation (Ai) for each region, and then combining these to update a global transformation (Tl). This describes the process of finding local affine transformations (Ai) for each region (Ri)), which corresponds to generating a local area registration for each region. The updated transformation (Tl) is a single, overall registration (which is considered an "updated first registration" in the context of an iterative process), which is a function of the combined local transformations.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date
of the claimed invention to modify the method of image registration during a surgical procedure as disclosed by Donhowe to include generating a first local area registration for the first anatomic area of the plurality of anatomic areas; generating a second local area registration for the second anatomic area of the plurality of anatomic areas; combining the first local area registration and the second local area registration to generate an updated first registration as taught by Commowick in the system of Donhowe for registering the images on user-defined areas, using a set of affine transformations, which are optimized together in a very efficient manner that ensures a smooth and coherent and shows a significant improvement in computation time, which is crucial for clinical applications. (see abstract, Commowick)
Regarding claim 17
Donhowe in view of Commonwick discloses the method of claim 16. Donhowe further teaches wherein the generating the first local area registrations includes: determining a subset of measured points in the first anatomic area based on the first registration; (see para 102- Referring now to FIGS. 11A-H, shown therein are several different rules or heuristics that may be used in selecting a subset of measured data points D that are then used to match to an anatomic model to compute a corrective motion. By identifying points that include superior information and ignoring points that include inferior information, a control system performing registration may operate more efficiently. see para 109- FIG. 11B shows the set of measured points 1102 and the anatomic model 1104 after a corrective motion is applied to transform the measured points 1102 for a first iteration of the registration process. For a subsequent iteration of the registration process, the measured point 1102A is now matched to a model point 1106F to which is it now closer. After the first iteration, the measured point 1102E is further away from the segment 1104A. For the first iteration, the measured point 1102C matched to the terminal model point 1106C. For the subsequent iteration, the measured point 1102C matches to a different terminal model point, point 1106G.)
and registering the model points in the first anatomic area with the subset of measured points in the first anatomic area. (see para 80-Referring again to FIG. 7, at a process 712 registration of the anatomic model information 550 with the set of gathered data points D from the surgical environment 600 is performed. Registration may be accomplished using a point set registration algorithm such as an iterative closest point (ICP) technique as described in processes 512-520, or by implementation of another registration algorithm. Prior to the process 712, at process 710, the ICP registration is seeded with known information about the displacement and orientation relationship between the patient surgical environment and the anatomic model. In this embodiment (FIG. 9), for example, the carina landmarks Ci, C2, C3 are identified in the anatomic model information as points ML1, ML2, ML3. This seeding process, based on a few landmark points, provides an initial coarse registration of the gathered data points D to the anatomic model information 550.)
Regarding claim 18
Donhowe in view of Commonwick teaches a method of claim 16. Donhowe in view of Commonwick discloses the method of claim 16. Donhowe further teaches wherein the dividing the anatomic structure into the plurality of anatomic areas includes: (see para 65-66- As shown in FIG. 5B, the centerline segmented model 504 includes several branch points, some of which are highlighted for visibility in FIG. 5B. The branch points A, B, C, D, and E are shown at each of several of the branch points. The branch point A may represent the point in the model at which the trachea divides into the left and right principal bronchi. The right principal bronchus may be identified in the centerline segment model 504 as being located between branch points A and B. Similarly, secondary bronchi are identified by the branch points B and C and between the branch points B and E. Another generation may be defined between branch points C and D.)
Donhowe does not teach dividing the anatomic structure based on structure rigidities of the plurality of anatomic areas respectively.
In the related field of invention, Commowick teaches dividing the anatomic structure based on structure rigidities of the plurality of anatomic areas respectively. (see section A.1.-Weight functions are basically defined as an inverse distance to the region Ri and renormalized to give the wi. In this approach, pairs of rigid structures are selected in the input images, along with linear transformations. A number of pairs of outer landmarks further constrain the interpolation scheme, which uses Hardy multi-quadric basis functions to interpolate in between the areas. This results in applying the affine transforms to the user-defined structures while ensuring a smooth interpolation in between them.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date
of the claimed invention to modify the method of image registration during a surgical procedure as disclosed by Donhowe to include dividing the anatomic structure based on structure rigidities of the plurality of anatomic areas respectively as taught by Commowick in the system of Donhowe for registering the images on user-defined areas, using a set of affine transformations, which are optimized together in a very efficient manner that ensures a smooth and coherent and shows a significant improvement in computation time, which is crucial for clinical applications. (see abstract, Commowick)
Regarding claim 21
Donhowe in view of Commonwick teaches a method of claim 16. Donhowe further teaches wherein the first anatomic area is adjacent to the second anatomic area, (see para 65-66- As shown in FIG. 5B, the centerline segmented model 504 includes several branch points, some of which are highlighted for visibility in FIG. 5B. The branch points A, B, C, D, and E are shown at each of several of the branch points. The branch point A may represent the point in the model at which the trachea divides into the left and right principal bronchi. The right principal bronchus may be identified in the centerline segment model 504 as being located between branch points A and B. Similarly, secondary bronchi are identified by the branch points B and C and between the branch points B and E. Another generation may be defined between branch points C and D. Each of these generations may be associated with a representation of the diameter of the lumen of the corresponding passageway. see para 78-With reference to FIG. 9, a set of anatomic passageways 900 includes main carinas Ci, C2, C3 where the passageways 900 fork. See also [0129] In some embodiments, the weight of a given measured point may be based on the generation of the passageway in which the point was obtained. For example, as shown in FIG. 11 A, the anatomic model 1104 includes several segments 1104 A, 1104B, and 1104C and others that are associated with specific generations of passageways. See para 83-For example, the major "Y" formation provided by the trachea and the left and right main bronchii shown in FIG. 9 may be used as explained herein.)
Donhowe does not teach wherein combining the first local area registration and the second local area registration to generate an updated first registration includes: blending the first and second local area registrations for the adjacent first and second anatomic areas in a transition area of the adjacent first and second anatomic areas.
In the related field of invention, Commowick further teaches wherein combining the first local area registration and the second local area registration to generate an updated first registration includes: blending the first and second local area registrations for the adjacent first and second anatomic areas in a transition area of the adjacent first and second anatomic areas. (See equation A.1 and section 2.2.2-2.2.3-To ensure a smooth interpolation between the regions we have defined, we need a minimal distance between them. The binary areas can indeed overlap each other after definition. We then need to erode them as little as possible to have a minimal distance between regions while keeping them as close as possible to the original region. We then associate to each region a weighting function, defining the relative influence of the region at point x. We have chosen to implement the weighting function for each component as a function of the minimal distance to the corresponding region. See section 2.5-To ensure a smooth and invertible transformation everywhere, we, therefore, choose to use the third method of interpolation (Log-Euclidean polyaffine, to build the final transformation)
PNG
media_image1.png
53
373
media_image1.png
Greyscale
Examiner note: The final transformation is the blending of the first and second local registration. Overlapping weight functions create a transition area where both local transforms influence the final transformation.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date
of the claimed invention to modify the method of image registration during a surgical procedure as disclosed by Donhowe to include wherein combining the first local area registration and the second local area registration to generate an updated first registration includes: blending the first and second local area registrations for the adjacent first and second anatomic areas in a transition area of the adjacent first and second anatomic areas as taught by Commowick in the system of Donhowe for registering the images on user-defined areas, using a set of affine transformations, which are optimized together in a very efficient manner that ensures a smooth and coherent and shows a significant improvement in computation time, which is crucial for clinical applications. (see abstract, Commowick)
Regarding claim 22
Donhowe further teaches wherein the updated first registration includes at least one of a deflection estimate and a rotation estimate for the first anatomic area. (see para 87-While the orientation of the patient P to the point gathering instrument 604 may already be known, by navigating the distal end 618 into either the left or right primary bronchus (shown in FIG. 6C) additional information may be gathered that can be used to seed the initial transformation. For example, using shape sensor or EM sensor data, information characterizing a first roughly right angle may be collected between the entrance of the ET tube and exit of the ET tube into the length of the trachea of the patient P. This first angle may identify a plane that may be expected to bisect the anatomic model information 550. When the distal end 618 transitions from the trachea into either the left or right primary bronchus, a second angle defining a second plane may be identified. The first and second places are roughly orthogonal. By using the first and second angles, the medical system may identify a right-left orientation of the patient which may be used to seed the registration process by roughly orienting the data points D with the anatomic model information 550.)
Conclusion
13. Claims 1-11, 13-14, 16-18 and 21-22 is/are rejected.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Donhowe et al. " US10706543B2.
Discussing the method that includes accessing a set of model points of a model of one or more passageways of a patient; detecting a point collection condition in image data obtained from an image-capture device of a catheter based upon machine vision; automatically initiating collection of a set of measured points based on detection of the point collection condition in the image data obtained from the image-capture device; performing a point set registration algorithm using the set of model points of the model of one or more passageways of the patient and the set of measured points to produce a registered set of model points; and displaying a visual representation of the registered set of model points in a user interface provided by a display system.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PURSOTTAM GIRI whose telephone number is (469)295-9101. The examiner can normally be reached 7:30-5:30 PM, Monday to Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, RENEE CHAVEZ can be reached at 5712701104. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PURSOTTAM GIRI/Examiner, Art Unit 2186
/RENEE D CHAVEZ/Supervisory Patent Examiner, Art Unit 2186
Prosecution Insights