DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is in response to the remarks filed on 4/01/2026. The amendments filed on 4/01/2026 are entered.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-13 and 15-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the procedure" in line 5. There is insufficient antecedent basis for this limitation in the claim.
Claim 10 recites the limitation "the procedure" in lines 7-8. There is insufficient antecedent basis for this limitation in the claim.
Claim 13 recites the limitation "the procedure" in line 4. There is insufficient antecedent basis for this limitation in the claim.
Claim 15 recites the limitation "the procedure" in line 4. There is insufficient antecedent basis for this limitation in the claim.
Claims dependent upon rejected claims are also rejected for indefiniteness. Therefore, dependent claims 2-9, 11-12, and 16-21 are also rejected.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 4, 8-13, 15-16, 18-19, and 21 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shina (U.S. Pub. No. 20060036167) hereinafter Shina.
Regarding claim 1, Shina teaches:
A device for guiding an interventional imaging device (abstract), the device comprising:
a processor ([0083]-[0085], controller 210) configured to:
obtain first image data as first data from a first imaging device, wherein the first image data comprises a first fluoroscopy image acquired during the procedure, wherein the first image data comprises representation of the interventional imaging device inserted within a vessel structure of a subject, and wherein the first image data comprises image data relating to a first point in time ([0086], generating catheter positions; [0089], complement data acquired using IVUS; [0097], angiography carried out using the angiography imaging method; [0098]-[0099], fluoroscopy is optionally the imaging modality used in conjunction with the IVUS catheter procedure, and therefore would be the imaging modality in reference to the angiography images; [0101]-[0103], first angiography image (x-ray image) is acquired as a dataset from the x-ray imager as the first image data and corresponds to the acquired x-ray image of paragraph [0124]; [0120], “estimating the position of the IVUS head 220” based on imaging data of the IVUS head; [0122], estimating an initial position of head 220 forms a first point in time; [0124], “Alternatively, the position is optionally detected automatically, for example by automatic detection of a radio-opaque marker built into head 220. As head 220 is known to be in a blood vessel, a search is made for a blood vessel that, when projected onto the 2D image encloses head 220. Optionally, if there is more than one possibility, a user may select the vessel on the 3D image, for example, being given the option to select among multiple possibilities.” and “Optionally, catheter 118 also includes one or more radio-opaque markers, which are used to further limit the possible vessels to be considered (as head 220 and catheter 118 must lie in a same vessel in a normal procedure).” In the first image data from the x-ray imaging system; see also [0125]-[0131] and [0132]-[0139], 3D IVUS generation; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image);
obtain second data relating to a movement of the interventional imaging device from the first point in time to at least a second point in time, wherein the second data relates to the first point in time and to at least a second point in time ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]);
estimate a pose of the interventional imaging device in the first image data ([0086], generating catheter positions; [0089], complement data acquired using IVUS; [0097], angiography carried out using the angiography imaging method; [0120], “estimating the position of the IVUS head 220” based on imaging data of the IVUS head; [0122], estimating an initial position of head 220 forms a first point in time; [0124], “Alternatively, the position is optionally detected automatically, for example by automatic detection of a radio-opaque marker built into head 220. As head 220 is known to be in a blood vessel, a search is made for a blood vessel that, when projected onto the 2D image encloses head 220. Optionally, if there is more than one possibility, a user may select the vessel on the 3D image, for example, being given the option to select among multiple possibilities.” and “Optionally, catheter 118 also includes one or more radio-opaque markers, which are used to further limit the possible vessels to be considered (as head 220 and catheter 118 must lie in a same vessel in a normal procedure).” In the first image data from the x-ray imaging system; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; position and orientation determination forms an estimate of pose);
track a relative motion of the interventional imaging device based on the second data ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image);
compute an updated pose estimate of the interventional imaging device relating to at least the second point in time based on the estimated pose and the tracked relative motion ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image; position and orientation determination forms an estimate of pose);
generate an updated indicator of the interventional imaging device representing the updated pose estimate relating to at least the second point in time based on the computed updated pose estimate ([0127], “The angiography image may be updated by annotating it to include the current position of head 220.”);
augment the first image data with the updated indicator relating to the first point in time with the updated indicator representing the updated pose estimate relating to the at least second point in time ([0127], “The angiography image may be updated by annotating it to include the current position of head 220.”); and
provide the augmented first image data without continuous fluoroscopy image acquisition ([0127], “The angiography image may be updated by annotating it to include the current position of head 220.”; note [0125]-[0126] teaches to limiting x-ray exposure from the imaging system, which with the pose estimation and updating feature, forms a teaching to without continuous fluoroscopy image acquisition).
Regarding claim 2, Shina teaches all of the limitations of claim 1. Shina further teaches:
wherein the second data is second image data from a second imaging device provided by the interventional imaging device ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]);
wherein the second image data comprises a representation of the interior within the vessel structure ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]); and
wherein the processor is further configured to track the relative motion of the interventional imaging device within the vessel structure is based on the second image data ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image).
Regarding claim 4, Shina teaches all of the limitations of claim 2. Shina further teaches:
wherein to estimate the pose of the interventional imaging device in the first image data, the processor is configured to use one or more images of the second image data used to generate pose estimates relating to at least the first point in time for adapting the estimated pose of the interventional imaging device in the first image data ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image).
Regarding claim 8, Shina teaches all of the limitations of claim 1. Shina further teaches:
Wherein the first image data comprises X-ray image data ([0083],x-ray system; [0097]; [0120], x-ray image device; [0124], x-ray imaging device); and
wherein the second image data comprises at least one of the group of: optical camera image data, ultrasound image data and optical coherence tomography image data ([0083], IVUS ultrasound; [0120]-[0139], IVUS ultrasound imaging device).
Regarding claim 9, Shina teaches all of the limitations of claim 1. Shina further teaches:
provide a confidence estimate related to the relative motion estimate from the second image data ([0129], error related to threshold forms a confidence estimate of the motion estimate of the second position as in “Changes in obliqueness of the vessel may be used to indicate an orientation problem. Changes in vessel diameter may indicate an axial positioning problem. The position may be adjusted until an error is minimal. Optionally, the comparison takes into account parts of the data set when an increased error may be expected, for example, at bends. Optionally, if the error is above a threshold, the user is notified.”); and
wherein the output interface is configured to provide a confidence indicator to the user ([0129], “Optionally, if the error is above a threshold, the user is notified.”).
Regarding claim 10, Shina teaches all of the limitations of claim 1. Shina further teaches:
A system for guiding an interventional imaging device (abstract), the system comprising:
a first data arrangement comprising a first imaging device ([0086], generating catheter positions; [0089], complement data acquired using IVUS; [0097], angiography carried out using the angiography imaging method; [0120], “estimating the position of the IVUS head 220” based on imaging data of the IVUS head; [0122], estimating an initial position of head 220 forms a first point in time; [0124], “Alternatively, the position is optionally detected automatically, for example by automatic detection of a radio-opaque marker built into head 220. As head 220 is known to be in a blood vessel, a search is made for a blood vessel that, when projected onto the 2D image encloses head 220. Optionally, if there is more than one possibility, a user may select the vessel on the 3D image, for example, being given the option to select among multiple possibilities.” and “Optionally, catheter 118 also includes one or more radio-opaque markers, which are used to further limit the possible vessels to be considered (as head 220 and catheter 118 must lie in a same vessel in a normal procedure).” In the first image data from the x-ray imaging system; see also [0125]-[0131] and [0132]-[0139], 3D IVUS generation; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image);
a second data arrangement ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]); and
a device for guiding an interventional imaging device according to claim 1(see rejection of claim 1 above);
wherein the first imaging device is configured to generate the first image data as the first data, wherein the first image data comprises a first fluoroscopy image acquired during the procedure ([0086], generating catheter positions; [0089], complement data acquired using IVUS; [0097], angiography carried out using the angiography imaging method; [0098]-[0099], fluoroscopy is optionally the imaging modality used in conjunction with the IVUS catheter procedure, and therefore would be the imaging modality in reference to the angiography images; [0101]-[0103], first angiography image (x-ray image) is acquired as a dataset from the x-ray imager as the first image data and corresponds to the acquired x-ray image of paragraph [0124]; [0120], “estimating the position of the IVUS head 220” based on imaging data of the IVUS head; [0122], estimating an initial position of head 220 forms a first point in time; [0124], “Alternatively, the position is optionally detected automatically, for example by automatic detection of a radio-opaque marker built into head 220. As head 220 is known to be in a blood vessel, a search is made for a blood vessel that, when projected onto the 2D image encloses head 220. Optionally, if there is more than one possibility, a user may select the vessel on the 3D image, for example, being given the option to select among multiple possibilities.” and “Optionally, catheter 118 also includes one or more radio-opaque markers, which are used to further limit the possible vessels to be considered (as head 220 and catheter 118 must lie in a same vessel in a normal procedure).” In the first image data from the x-ray imaging system; see also [0125]-[0131] and [0132]-[0139], 3D IVUS generation; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image); and
wherein the second data arrangement is configured to generate the second data ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]).
Regarding claim 11, Shina teaches all of the limitations of claim 10. Shina further teaches:
Wherein an interventional imaging device is provided ([0083], IVUS ultrasound; [0120]-[0139], IVUS ultrasound imaging device); and
wherein the second data arrangement is provided as a second imaging device provided by the interventional imaging device ([0083], IVUS ultrasound; [0120]-[0139], IVUS ultrasound imaging device).
Regarding claim 12, Shina teaches all of the limitations of claim 10. Shina further teaches:
the first imaging device is provided as an X-ray imaging device ([0083],x-ray system; [0097]; [0120], x-ray image device; [0124], x-ray imaging device); and
wherein the second imaging device is provided as at least one of bronchoscope, endoscope, colonoscope, intravascular ultrasound, intracardiac echocardiography, endobronchial ultrasound or radial endobronchial ultrasound, and optical coherence tomography ([0083], IVUS ultrasound; [0120]-[0139], IVUS ultrasound imaging device).
Regarding claim 13, Shina teaches:
A method for guiding an interventional imaging device, (abstract) the method comprising the following steps:
providing first image data as first data from a first imaging, wherein the first image data comprises a first fluoroscopy image acquired during the procedure, wherein the first image data comprises a representation of the interventional imaging device inserted within a vessel structure of a subject, and wherein the first image data comprises image data relating to a first point in time ([0086], generating catheter positions; [0089], complement data acquired using IVUS; [0097], angiography carried out using the angiography imaging method; [0098]-[0099], fluoroscopy is optionally the imaging modality used in conjunction with the IVUS catheter procedure, and therefore would be the imaging modality in reference to the angiography images; [0101]-[0103], first angiography image (x-ray image) is acquired as a dataset from the x-ray imager as the first image data and corresponds to the acquired x-ray image of paragraph [0124]; [0120], “estimating the position of the IVUS head 220” based on imaging data of the IVUS head; [0122], estimating an initial position of head 220 forms a first point in time; [0124], “Alternatively, the position is optionally detected automatically, for example by automatic detection of a radio-opaque marker built into head 220. As head 220 is known to be in a blood vessel, a search is made for a blood vessel that, when projected onto the 2D image encloses head 220. Optionally, if there is more than one possibility, a user may select the vessel on the 3D image, for example, being given the option to select among multiple possibilities.” and “Optionally, catheter 118 also includes one or more radio-opaque markers, which are used to further limit the possible vessels to be considered (as head 220 and catheter 118 must lie in a same vessel in a normal procedure).” In the first image data from the x-ray imaging system; see also [0125]-[0131] and [0132]-[0139], 3D IVUS generation; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image);
estimating a pose of the interventional imaging device in the first image data ([0086], generating catheter positions; [0089], complement data acquired using IVUS; [0097], angiography carried out using the angiography imaging method; [0120], “estimating the position of the IVUS head 220” based on imaging data of the IVUS head; [0122], estimating an initial position of head 220 forms a first point in time; [0124], “Alternatively, the position is optionally detected automatically, for example by automatic detection of a radio-opaque marker built into head 220. As head 220 is known to be in a blood vessel, a search is made for a blood vessel that, when projected onto the 2D image encloses head 220. Optionally, if there is more than one possibility, a user may select the vessel on the 3D image, for example, being given the option to select among multiple possibilities.” and “Optionally, catheter 118 also includes one or more radio-opaque markers, which are used to further limit the possible vessels to be considered (as head 220 and catheter 118 must lie in a same vessel in a normal procedure).” In the first image data from the x-ray imaging system; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; position and orientation determination forms an estimate of pose);
providing second data relating to a movement of the interventional imaging device from the first point in time to at least a second point in time, wherein the second data relates to the first point in time and to at least a second point in time ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]);
tracking a relative motion of the interventional imaging device based on the second data ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image);
computing an updated pose estimate of the interventional imaging device relating to at least the second point in time based on the estimated pose and the tracked relative motion ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image);
generating an updated indicator of the interventional imaging device representing the updated pose estimate relating to at least the second point in time based on the computed updated pose estimate ([0127], “The angiography image may be updated by annotating it to include the current position of head 220.”; see also figure 4, [0168]);
augmenting the first image data relating to the first point in time with the updated indicator representing the updated pose estimate relating to at least the second point in time ([0127], “The angiography image may be updated by annotating it to include the current position of head 220.”; see also figure 4, [0168]); and
providing the augmented first image data without continuous fluoroscopy image acquisition ([0127], “The angiography image may be updated by annotating it to include the current position of head 220.”; note [0125]-[0126] teaches to limiting x-ray exposure from the imaging system, which with the pose estimation and updating feature, forms a teaching to without continuous fluoroscopy image acquisition).
Regarding claim 15, Shina teaches:
A non-transitory computer readable medium having stored a computer program having instructions which, when executed by a processor (abstract), cause the processor to:
obtain first image data as first data from a first imaging device, wherein the first image data comprises a first fluoroscopy image acquired during the procedure, wherein the first image data comprises a representation of the interventional imaging device inserted within a vessel structure of a subject, and wherein the first image data comprises image data relating to a first point in time ([0086], generating catheter positions; [0089], complement data acquired using IVUS; [0097], angiography carried out using the angiography imaging method; [0098]-[0099], fluoroscopy is optionally the imaging modality used in conjunction with the IVUS catheter procedure, and therefore would be the imaging modality in reference to the angiography images; [0101]-[0103], first angiography image (x-ray image) is acquired as a dataset from the x-ray imager as the first image data and corresponds to the acquired x-ray image of paragraph [0124]; [0120], “estimating the position of the IVUS head 220” based on imaging data of the IVUS head; [0122], estimating an initial position of head 220 forms a first point in time; [0124], “Alternatively, the position is optionally detected automatically, for example by automatic detection of a radio-opaque marker built into head 220. As head 220 is known to be in a blood vessel, a search is made for a blood vessel that, when projected onto the 2D image encloses head 220. Optionally, if there is more than one possibility, a user may select the vessel on the 3D image, for example, being given the option to select among multiple possibilities.” and “Optionally, catheter 118 also includes one or more radio-opaque markers, which are used to further limit the possible vessels to be considered (as head 220 and catheter 118 must lie in a same vessel in a normal procedure).” In the first image data from the x-ray imaging system; see also [0125]-[0131] and [0132]-[0139], 3D IVUS generation; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image);
obtain second data relating to a movement of the interventional imaging device from the first point in time to at least a second point in time, wherein the second data relates to the first point in time and to at least a second point in time ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]);
estimate a pose of the interventional imaging device in the first image data ([0086], generating catheter positions; [0089], complement data acquired using IVUS; [0097], angiography carried out using the angiography imaging method; [0120], “estimating the position of the IVUS head 220” based on imaging data of the IVUS head; [0122], estimating an initial position of head 220 forms a first point in time; [0124], “Alternatively, the position is optionally detected automatically, for example by automatic detection of a radio-opaque marker built into head 220. As head 220 is known to be in a blood vessel, a search is made for a blood vessel that, when projected onto the 2D image encloses head 220. Optionally, if there is more than one possibility, a user may select the vessel on the 3D image, for example, being given the option to select among multiple possibilities.” and “Optionally, catheter 118 also includes one or more radio-opaque markers, which are used to further limit the possible vessels to be considered (as head 220 and catheter 118 must lie in a same vessel in a normal procedure).” In the first image data from the x-ray imaging system; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”);
track a relative motion of the interventional imaging device based on the second data ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image);
compute an updated pose estimate of the interventional imaging device relating to at least the second point in time based on the estimated pose and the tracked relative motion ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image);,
generate an updated indicator of the interventional imaging device representing the updated pose estimate relating to at least the second point in time based on the computed updated pose estimate ([0127], “The angiography image may be updated by annotating it to include the current position of head 220.”);
augment the first image data relating to the first point in time with the updated indicator representing the updated pose estimate relating to at least the second point in time ([0127], “The angiography image may be updated by annotating it to include the current position of head 220.”); and
provide the augmented first image data without continuous fluoroscopy image acquisition ([0127], “The angiography image may be updated by annotating it to include the current position of head 220.”; note [0125]-[0126] teaches to limiting x-ray exposure from the imaging system, which with the pose estimation and updating feature, forms a teaching to without continuous fluoroscopy image acquisition).
Regarding claim 16, Shina teaches all of the limitations of claim 15. Shina further teaches:
wherein the second data is second image data from a second imaging device provided by the interventional imaging device ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]);
wherein the second image data comprises a representation of the interior within the vessel structure ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]); and
wherein instruction, when executed by the processor, further cause the processor to track the relative motion of the interventional imaging device within the vessel structure is based on the second image data ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image).
Regarding claim 18, Shina teaches all of the limitations of claim 15. Shina further teaches:
wherein, to estimate the pose of the interventional imaging device in the first image data, the instructions, when executed by the processor, further cause the processor to use one or more images of the second image data used to generate pose estimates relating to at least the first point in time for adapting the estimated pose of the interventional imaging device in the first image data ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image).
Regarding claim 19, Shina teaches all of the limitations of claim 13. Shina further teaches:
wherein the second data is second image data from a second imaging device provided by the interventional imaging device ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]);
wherein the second image data comprises a representation of the interior within the vessel structure ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]); and
the method further comprising tracking the relative motion of the interventional imaging device within the vessel structure is based on the second image data ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image).
Regarding claim 21, Shina teaches all of the limitations of claim 13. Shina further teaches:
wherein estimating the pose of the interventional imaging device in the first image data further comprises using one or more images of the second image data used to generate pose estimates relating to at least the first point in time for adapting the estimated pose of the interventional imaging device in the first image data ([0123], estimating a current position of head 220 forms a second data relating to the movement of the interventional imaging device from the first point in time (initial position) to a second point in time (current position); [0124]-[0125]; [0126], “it is desired to further minimize exposure to radiation and/or contrast material. In an exemplary embodiment of the invention, a current head position is estimated based on the initial position and a length of pull-back of the IVUS head during imaging”; [0127], current position of the head 220; [0128], “the orientation of IVUS head 220 (and thus the UVUS slice orientation, which is typically perpendicular) is estimated by assuming the head lies along the vessel centerline at which the head is located. Optionally, this estimate is verified using the projection of the head on the 2D image”; [0129], “IVUS head orientation is determined or verified by aligning the IVUS image with a matching spatial section of the 3D data set”; [0132], “IVUS head 220 generates trans-axial images (slices) of the vessel in which it is located. These slices may be collected using various methods and formed into a 3D image data set. One method of collection comprises aligning the images based on the head position and/or orientation and interpolating values for data set elements that lie between the slices based on the distance between images (e.g., based on relative motion) and pixels in those images.”; see also [0130]-[0139]; [0189], locating a catheter and/or catheter tip in a 2D and/or 3D image).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shina as applied to claims 1 or 15 or 13 above, and further in view of Sun et al. (U.S. Pub. No. 20170032512) hereinafter Sun.
Regarding claim 3, primary reference Shina teaches all of the limitations of claim 1. Primary reference Shina further fails to teach:
provide the computing of the updated pose estimate comprising at least one of:
a correction of the out-of-plane pose estimate from the first image data using pose estimate from the second image data; and a correction of the in-plane pose estimate from the second image data using pose estimate from the first image data
However, the analogous art of Sun of a probe pose detection for use within a sequence of fluoroscopic medical images (abstract) teaches:
provide the computing of the updated pose estimate comprising at least one of:
a correction of the out-of-plane pose estimate from the first image data using pose estimate from the second image data; and a correction of the in-plane pose estimate from the second image data using pose estimate from the first image data ([0039], “The image processor 26 is configured to detect in-plane pose with machine-learnt classification, detect out-of-plane pose with template matching, and use visual tracking for one or both of in-plane and out-of-plane pose selection. In one embodiment, markers on the probe 18 are detected, and the detection is used to increase accuracy of the out-of-plane pose detection. The detection of the out-of-plane pose is weighted by the detection of the markers on the probe.”; [0043]-[0049]; [0052]; [0055]-[0057], out-of-plane tracking provides for a correction using the correlated frame of data or the region of interest of the frame data; [0080]-[0089]; [0096]-[0102]; figure 6).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the pose estimation device for use with an interventional imaging system of Shina to incorporate the correction of the out-of-plane pose estimate from the second data as taught by Sun because fully automated tracking uses temporal information to more reliably detect the probe pose (Sun, [0021]). This leads to higher quality tracking of the pose estimates based upon the real-time position ad improved diagnostic output quality.
Regarding claim 17, primary reference Shina teaches all of the limitations of claim 15. Primary reference Shina further fails to teach:
wherein the instructions, when executed by the processor, further cause the processor to:
provide the computing of the updated pose estimate comprising at least one of:
a correction of the out-of-plane pose estimate from the first image data using pose estimate from the second image data; and a correction of the in-plane pose estimate from the second image data using pose estimate from the first image data.
However, the analogous art of Sun of a probe pose detection for use within a sequence of fluoroscopic medical images (abstract) teaches:
wherein the instructions, when executed by the processor, further cause the processor to:
provide the computing of the updated pose estimate comprising at least one of:
a correction of the out-of-plane pose estimate from the first image data using pose estimate from the second image data; and a correction of the in-plane pose estimate from the second image data using pose estimate from the first image data ([0039], “The image processor 26 is configured to detect in-plane pose with machine-learnt classification, detect out-of-plane pose with template matching, and use visual tracking for one or both of in-plane and out-of-plane pose selection. In one embodiment, markers on the probe 18 are detected, and the detection is used to increase accuracy of the out-of-plane pose detection. The detection of the out-of-plane pose is weighted by the detection of the markers on the probe.”; [0043]-[0049]; [0052]; [0055]-[0057], out-of-plane tracking provides for a correction using the correlated frame of data or the region of interest of the frame data; [0080]-[0089]; [0096]-[0102]; figure 6).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the pose estimation device for use with an interventional imaging system of Shina to incorporate the correction of the out-of-plane pose estimate from the second data as taught by Sun because fully automated tracking uses temporal information to more reliably detect the probe pose (Sun, [0021]). This leads to higher quality tracking of the pose estimates based upon the real-time position ad improved diagnostic output quality.
Regarding claim 20, primary reference Shina teaches all of the limitations of claim 13. Primary reference Shina further fails to teach:
providing the computing of the updated pose estimate comprising at least one of: a correction of the out-of-plane pose estimate from the first image data using pose estimate from the second image data; a correction of the out-of-plane pose estimate from the first image data using pose estimate from the second image data; and a correction of the in-plane pose estimate from the second image data using pose estimate from the first image data
However, the analogous art of Sun of a probe pose detection for use within a sequence of fluoroscopic medical images (abstract) teaches:
providing the computing of the updated pose estimate comprising at least one of: a correction of the out-of-plane pose estimate from the first image data using pose estimate from the second image data; a correction of the out-of-plane pose estimate from the first image data using pose estimate from the second image data; and a correction of the in-plane pose estimate from the second image data using pose estimate from the first image data ([0039], “The image processor 26 is configured to detect in-plane pose with machine-learnt classification, detect out-of-plane pose with template matching, and use visual tracking for one or both of in-plane and out-of-plane pose selection. In one embodiment, markers on the probe 18 are detected, and the detection is used to increase accuracy of the out-of-plane pose detection. The detection of the out-of-plane pose is weighted by the detection of the markers on the probe.”; [0043]-[0049]; [0052]; [0055]-[0057], out-of-plane tracking provides for a correction using the correlated frame of data or the region of interest of the frame data; [0080]-[0089]; [0096]-[0102]; figure 6).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the pose estimation device for use with an interventional imaging system of Shina to incorporate the correction of the out-of-plane pose estimate from the second data as taught by Sun because fully automated tracking uses temporal information to more reliably detect the probe pose (Sun, [0021]). This leads to higher quality tracking of the pose estimates based upon the real-time position ad improved diagnostic output quality.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Shina as applied to claim 2 above, and further in view of Rauniyar et al. (U.S. Pub. No. 20210090226) hereinafter Rauniyar.
Regarding claim 5, primary reference Shina teaches all of the limitations of claim 2. Primary reference Shina further fails to teach:
wherein the second image data comprises a stream of second images; and
wherein the processor is configured to provide the tracking of the relative motion of the interventional imaging device for consecutive images of the stream of second images
However, the analogous art of Rauniyar of an endoscope system for tracking image frames across a coordinate space (abstract) teaches:
wherein the second image data comprises a stream of second images ([0053], “SFM generally refers to a mapping of three-dimensional structures from a succession of images captured by a moving POV, where the changing position and orientation (pose) of the imager is tracked. The depth map is continuously populated as new images are gathered”; [0064]-[0070], SFM techniques are utilized to determine the position and orientation pose of the imager; [0072]-[0078], figure 8; [0084]-[0089]; figure 11); and
wherein the processor is configured to provide the tracking of the relative motion of the interventional imaging device for consecutive images of the stream of second images ([0053], “SFM generally refers to a mapping of three-dimensional structures from a succession of images captured by a moving POV, where the changing position and orientation (pose) of the imager is tracked. The depth map is continuously populated as new images are gathered”; [0064]-[0070], SFM techniques are utilized to determine the position and orientation pose of the imager; [0072]-[0078], figure 8; [0084]-[0089]; figure 11).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the pose estimation device for use with an interventional imaging system of Shina to incorporate the tracking of relative motion across a stream of images of an interventional imager as taught by Rauniyar because SFM techniques may be employed to inform a process for tracking an endoscope position during an endoscopic procedure (Rauniyar, [0064]). This provides for position tracking using the built-in imaging sensor of the device, leading to direct mapping of structures within a moving point of view (Rauniyar, [0053]).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Shina as applied to claim 1 above, and further in view of Averbuch et al. (U.S. Pub. No. 20200046214) hereinafter Averbuch.
Regarding claim 6, primary reference Shina teaches all of the limitations of claim 1. Primary reference Shina further fails to teach:
compute a trajectory of the interventional imaging device based on the updated pose estimate
i) generate, based on the computed trajectory and the updated pose of the interventional imaging device, a projection of the trajectory of the interventional imaging device; and to augment the first image data based on the projection of the trajectory of the interventional imaging device to provide an updated virtual first image data; or ii) to project the computed trajectory onto the first image data
However, the analogous art of Averbuch of a device for use during a medical imaging process within a patient (abstract) teaches:
compute a trajectory of the interventional imaging device based on the updated pose estimate ([0066], “tracking the radiopaque instrument for: identifying a trajectory, and using the trajectory as a further geometric constraint, wherein the radiopaque instrument comprises an endoscope, an endo-bronchial tool, or a robotic arm.”; 0156], “This method uses anatomical elements detected both in the Fluoroscopic image and in the CT scan as an input to a pose estimation algorithm that produces a Fluoroscopic device Pose (e.g., orientation and position) with respect to the CT scan. The following extends this method by adding 3D space trajectories, corresponding to an endo-bronchial device position, to the inputs of the registration method. These trajectories can be acquired by several means, such as: attaching positional sensors along a scope or by using a robotic endoscopic arm. Such an endo-bronchial device will be referred from now on as Tracked Scope.”; [0157]-[0158]; [0159], “The projected pathway to the target lesion provides the physician with only two-dimensional information, resulting in a depth ambiguity, that is to say several airways segmented on CT may correspond to the same projection on the 2D Fluoroscopic image.”); and
i) generate, based on the computed trajectory and the updated pose of the interventional imaging device, a projection of the trajectory of the interventional imaging device; and to augment the first image data based on the projection of the trajectory of the interventional imaging device to provide an updated virtual first image data; or ii) to project the computed trajectory onto the first image data ([0066], “tracking the radiopaque instrument for: identifying a trajectory, and using the trajectory as a further geometric constraint, wherein the radiopaque instrument comprises an endoscope, an endo-bronchial tool, or a robotic arm.”; [0156], “This method uses anatomical elements detected both in the Fluoroscopic image and in the CT scan as an input to a pose estimation algorithm that produces a Fluoroscopic device Pose (e.g., orientation and position) with respect to the CT scan. The following extends this method by adding 3D space trajectories, corresponding to an endo-bronchial device position, to the inputs of the registration method. These trajectories can be acquired by several means, such as: attaching positional sensors along a scope or by using a robotic endoscopic arm. Such an endo-bronchial device will be referred from now on as Tracked Scope.”; [0157]-[0158]; [0159], “The projected pathway to the target lesion provides the physician with only two-dimensional information, resulting in a depth ambiguity, that is to say several airways segmented on CT may correspond to the same projection on the 2D Fluoroscopic image.”; Tracked trajectories must be within the bronchial airways segmented from the images which forms the pathway to the lesion that is projected to the fluoroscopic live video as in [0159]. This provides for a generated trajectory on the image data).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the pose estimation device for use with an interventional imaging system of Shina to incorporate the trajectory tracking and display of the trajectory pathway as taught by Averbuch because the trajectories serve as additional constraints to the pose estimation method, since the estimated pose is constrained by the condition that the trajectories must fit the bronchial airways segmented from the registered CT scan (Averbuch, [0158]). This reduces the possibility for side effects with damage to surrounding tissues and leads to improvements in clinical outcomes.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Shina as applied to claim 1 above, and further in view of Tremblay et al. (U.S. Pub. No. 20210118166) hereinafter Tremblay.
Regarding claim 7, primary reference Shina teaches all of the limitations of claim 1. Primary reference Shina further fails to teach:
wherein the data processor is further configured to use a trained generative neural network to generate, based on the first image data at the first point in time and an updated pose of the interventional imaging device at a second point in time, a realistic synthetic image rendering the updated pose of interventional imaging device
However, the analogous art of Tremblay of a prediction and determination of pose position for a tracked autonomous object (abstract) teaches:
wherein the data processor is further configured to use a trained generative neural network to generate, based on the pose image data and an updated pose of the autonomous device at a second point in time, a realistic synthetic image rendering the updated pose of autonomous device ([0049], “as a base coordinate or other feature of this robot can be accurately identified in a camera space, or camera coordinate system”; [0050], neural network utilized in inference of features in combination with generative renderer to provide a synthetic image rendering the updated pose of the autonomous device (interventional IVUS device in the combined prior art invention with Shina); [0051]-[0052]; [0053], “a renderer can be utilized 404 to render a virtual version of a robot using a provided model and kinematic data. In at least one embodiment, this renderer can provide a control interface that enables 406 posing of this virtual robot in various poses, where image can be captured or generated for specific poses of this virtual robot. In at least one embodiment, a set of synthetic images can be generated with pose data, where each image represents a robot in a specified pose”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the pose estimation device for use with an interventional imaging system of Shina to incorporate the neural network based synthetic image of updated pose of the autonomous device as taught by Tremblay because these synthetic images can then serve as training data for a neural network, as images include representations of a type of robot in specific poses, and corresponding pose data can serve as ground truth data for training (Tremblay, [0050]). This leads to higher quality tracking and improved accuracy of device pose estimation.
Response to Arguments
Applicant's arguments filed 4/01/2026 have been fully considered but they are not persuasive. Responses to each of the applicant’s arguments are detailed below.
Regarding the applicant’s arguments on pages 11-12 of the remarks, the applicant argues that the Shina reference does not disclose “first image data” as claimed. The applicant argues that the Shina reference does not teach to the first image data being fluoroscopy data and being a first fluoroscopy image acquired during the procedure. In the current rejections the additional cited portions of Shina that include [0098]-[0099], teach to optionally include fluoroscopy as the x-ray imaging of choice for the disclosed systems and methods, wherein the initial image acquired during the procedure that corresponds to the “x-ray image” of [0124] that corresponds to the angiography image of [0127] teaches to the first image as claimed. Examiner notes that “the procedure” is indefinite in the current 112(b) rejections above and without antecedent basis it is unclear as to what procedure is being referred to. Furthermore, the use of the language “a first fluoroscopy image” does not meaningfully limit the image to being the first image acquired by the device after the device is initially turned on and the procedure begins. Any image within a set of “first” and then “second” images would be sufficient to teach to the broadest reasonable interpretation of the “First fluoroscopy image” as currently claimed, even when not considering the current 112(b) rejections above. For these reasons, the cited portions of the Shina reference teach to an initially acquired x-ray image that shows the location of the IVUS interventional imaging device, with the updated pose estimation after movement not requiring an additional x-ray image to achieve the further pose determination based upon that initial “first image”.
The applicant further argues that the Shina reference fails to teach to the second data related to the second point in time from the first point in time. In the current rejections, the teachings of Shina to the initial position (a first point in time corresponding to the location from the first x-ray image) and second current IVUS head position (second data relating to a movement to at least a second point in time) are sufficient to teach to the tracking of relative motion between the two time points. Shina teaches in [0126] that exposure to radiation is reduced by using a tracking methodology that doesn’t require additional subsequent x-ray images to determine the IVUS head current position. This teaches to the tracking of a relative motion and using both the estimated pose and tracked relative motion to determine an updated pose estimate of the IVUS device. Estimation from pull-back length and/or sensor data fully teaches to the same methodology as claimed, because it is utilized in conjunction with the initial x-ray determined first position to form the updated pose estimate and produce an annotated updated pose estimate on the image relating to the current second point in time updated pose estimate.
For these reasons, the applicant’s arguments have been considered but are not persuasive.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN A FRITH whose telephone number is (571)272-1292. The examiner can normally be reached M-Th 8:00-5:30 Second Fri 8:00-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Keith Raymond can be reached at 571-270-1790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN A FRITH/Primary Examiner, Art Unit 3798
Prosecution Insights