Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on July 10, 2023, is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
Applicant is reminded of the proper content of an abstract of the disclosure.
A patent abstract is a concise statement of the technical disclosure of the patent and should include that which is new in the art to which the invention pertains. The abstract should be in narrative form and generally limited to a single paragraph within the range of 50 to 150 words in length. See MPEP § 608.01(b) for guidelines for the preparation of patent abstracts.
The abstract of the disclosure is objected to because it contains more than 150 words. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4 and 8-17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Philipp et al. (US 8,781193 B2), hereafter Philipp.
Regarding claim 1, Philipp teaches a computer-implemented pathological vessel guidance method comprising:
generating a vessel roadmap library, the vessel roadmap library including a plurality of vessel roadmaps ([Col. 26, lines 42-44] “FIG. 1 is a flow chart, at least some of the steps of which are used to automatically generate a road map, in accordance with some applications of the present invention;”),
wherein each vessel roadmap comprises a vessel roadmap image and first (The first alignment data is ECG data. [Col. 30, lines 65-67 – Col. 31, lines 1-8] “In Phase 1 of the automatic road map generation, a fluoroscopic image stream of the coronary arteries is acquired. Typically, during the acquisition of the image stream, a contrast agent is administered to the subject. Optionally, the image stream is gated, tracked, and/or stabilized by other means. For example, selected image frames corresponding to a given phase in the motion cycle of the heart may be identified by means of a physiological signal. For some applications, the physiological signal applied is the subject's ECG and the image frames are selected by means of gating to the ECG….”) and second alignment data (The second alignment data includes contrast agent segmentation information. [Col. 33, lines 7-16] “…changes in the image may include a relatively abrupt change in the color and/or grayscale level (i.e., darkness) of a relatively large number and/or portion of image pixels, or the appearance of vessel-like features in the image, or any combination thereof. It is noted that by assessing a change in the darkness level to identify the time of injection of the contrast agent, the automatic road map generation processor may identify a darker area of the image or a lighter area of the image, depending on whether the contrast agent is represented as dark or light.” [Col. 33, lines 50-54] “…identifying decreased vesselness, for example, by means of a filter that performs enhancement and/or detection and/or segmentation of curvilinear structures, a Frangi filter, and/or a Frangi-like filter.” Additionally, roadmaps are displayed based on the respiratory cycle; thus, the roadmaps must include second alignment data related to the cycles. [Col. 42, lines 44-49] “For some applications, and in the case of images pertaining to a cyclically-moving organ, different road maps are displayed, typically in a cyclical manner, in conjunction with the then-current phase of the organ's motion cycle. For example, a dynamic road map as described may be correlated to the cardiac cycle, and/or the respiratory cycle.”);
detecting, within the vessel roadmap images of the vessel roadmap library, a pathological vessel ([Col. 13, lines 5-7] “For some applications, identifying the target portion includes identifying a portion of the blood vessel that corresponds to a lesion.” Additionally, Philipp discloses automatically identifying a lesion in a user-designated area of an image. [Col. 13, lines 44-48] “in response to a user designating a single point on the image: automatically identifying a portion of a blood vessel in a vicinity of the designated point that corresponds to a lesion, by…”);
obtaining a real-time fluoroscopy image and corresponding real-time first (Cardiac cycles and ECG signal (which is first fluoroscopy information) are recorded during live image tracking for guidance of the tool during fluoroscopic imaging. [Col. 52, lines 14-16] “For some applications, identification of the end-diastolic and end-systolic points within a cardiac cycle is determined from an ECG signal, and/or by means of image processing.”) and second fluoroscopy information (Tool position tracking information is second fluoroscopy information. [Col. 48, lines 64-67] “Once the markers have been identified in one or more image frames, then the system typically continues to identify (i.e., detect) those markers automatically in the subsequent image frames along the image stream or a segment thereof, and displays a tracked image stream.” [Col. 12, lines 2-6] “subsequently, inserting a tool into the blood vessel; while the tool is inside the blood vessel, determining a position of the tool; and modifying the road map to account for the determined position of the tool.”);
selecting a vessel roadmap from the roadmap library based on comparing the first real-time fluoroscopy information with the first alignment data of each vessel roadmap of the roadmap library (One embodiment teaches generating the roadmap lines and overlaying them over a live fluoroscopy image feed. [Col. 10, lines 17-22] “For some applications, generating the lines includes generating lines corresponding to the paths of the blood vessels during a given phase of a motion cycle of the blood vessels, and generating the output includes overlaying the lines on an image stream of the blood vessels that is gated to the given phase.”);
overlaying the real-time fluoroscopy image with the selected vessel roadmap image of the vessel roadmap library ([Col. 40, lines 27-30] “For some applications, the road map is generally displayed side-by-side with the fluoroscopic image stream, and from time to time is momentarily overlaid upon the fluoroscopic image stream.”);
aligning the vessel roadmap image of the selected vessel roadmap and the real-time fluoroscopy image based on the second alignment data and the real time second fluoroscopy information (The vessel roadmap and live fluoroscopy image stream is aligned and rotated based on the position of the tool (second fluoroscopy data) and considers the organ’s motion cycle (second alignment data). [Col. 50, lines 49-55] “For some applications, a virtual line that connects the markers is aligned such that it remains at the same or a similar relative position throughout the image stream. For example, the alignment may include translating individual image frames such that the markers (or the virtual line connecting the markers) remain at a same or similar location within the image frames throughout the tracked image stream. Alternatively or additionally, the alignment includes rotating individual image frames such that the angle of the virtual line connecting the markers is same or similar throughout the tracked image stream.” [Col. 51, lines 12-15] “Typically, the aforementioned image tracking results in the tool being displayed as relatively stable, while the surrounding anatomy is displayed as moving relative to the generally-stable tool in the course of the organ's motion cycle.”); and
providing guidance for a fluoroscopy object to the pathological vessel based on the selected vessel roadmap and the second fluoroscopy information ([Col. 12, lines 1-6] “…a method, including: generating a road map of a blood vessel; subsequently, inserting a tool into the blood vessel; while the tool is inside the blood vessel, determining a position of the tool; and modifying the road map to account for the determined position of the tool.”).
Regarding claim 2, Philipp teaches wherein detecting the pathological vessel includes:
determining centerlines of the vessels included in each vessel roadmap image of the vessel roadmap library ([Col. 26, lines 56-58] “FIG. 5 shows center lines constructed automatically along a portion of the blood vessels, in accordance with some applications of the present invention;”);
determining, based on the centerlines, lumina of the vessels included in each vessel roadmap image of the vessel roadmap library ([Col. 44, lines 22-42] “For some applications, a lesion is automatically detected in accordance with the following procedure. Scan lines are generated perpendicular to the centerline of a segment of the vessel that is sampled… For each new scan line, vessel diameter is defined as a distance between the two points where the scan line intersects vessel boundaries.” [Col. 44, lines 48-56] “Thus, for some applications, measurements are provided in absolute units, such as, lesion length, the diameter of the vessel at each point along the centerline, and/or minimum lumen diameter (which is also known as the MLD). For some applications, the level of occlusion (which is typically provided as a percentage) at the minimum lumen diameter is determined by comparing the diameter of the vessel at that point, to the diameter of the vessel at reference points of the vessel.”); and
detecting the pathological vessel based on the lumina of the vessels included in each vessel roadmap image of the vessel roadmap library (Philipp discloses automatically identifying a lesion in a user-designated area of an image. [Col. 13, lines 44-48] “in response to a user designating a single point on the image: automatically identifying a portion of a blood vessel in a vicinity of the designated point that corresponds to a lesion, by…” [Col. 45, lines 62-65] “Alternatively, or additionally, the user receives an indication of the minimum lumen diameter, or of reference diameters of a lesion in the vicinity of the current location of the cursor.” Additionally, figure 18 shows a lesion 64 and the lumina diameter 68 of the lesion.).
Regarding claim 3, Philipp teaches wherein generating the vessel roadmap library includes:
recording, for each vessel roadmap image of the vessel roadmap library, one or more imaging parameters, the imaging parameters indicating one or more parameters associated with an imaging method used to obtain the vessel roadmap image ([Col. 33, lines 64-67] “In Phase 5 of the automatic generation of the road map, the angiographic image frames (also known as angiograms) corresponding to a given angiographic sequence are automatically analyzed.” [Col. 34, lines 33-37] For some applications, the determination of vesselness of image pixels is made while accounting for the specific viewing angle at which the images are generated.” Philipp teaches that the viewing angles for the captured images are considered when developing the roadmap.).
Regarding claim 4, Philipp teaches wherein the one or more imaging parameters include at least one of an angiography angle and a contrast medium dosage ([Col. 33, lines 64-67] “In Phase 5 of the automatic generation of the road map, the angiographic image frames (also known as angiograms) corresponding to a given angiographic sequence are automatically analyzed.” [Col. 34, lines 33-37] “For some applications, the determination of vesselness of image pixels is made while accounting for the specific viewing angle at which the images are generated.” Philipp teaches that the viewing angles for the captured images are considered when developing the roadmap.).
Regarding claim 8, Philipp teaches wherein the fluoroscopy object is a robot-controlled fluoroscopy object and wherein the guidance is provided to the robot-controlled fluoroscopy object to enable the robot- controlled fluoroscopy object to be guided to the pathological vessel ([Col. 28, lines 56-67 – Col. 29, lines 1-21] “The terms “medical tool,” “tool”, “device,” and “probe” refer to… a cardiovascular catheter, a stent delivery and/or placement and/or retrieval tool, a balloon delivery and/or placement and/or retrieval tool, a valve delivery and/or repair and/or placement and/or retrieval tool, a graft delivery and/or placement and/or retrieval tool… a robotic tool, a tool that is controlled remotely, or any combination thereof.”).
Regarding claim 9, Philipp teaches wherein the second alignment data is derived from the corresponding vessel roadmap image ([Col. 32, lines 14-24] “…the baseline image frame is analyzed such that the degree of “vesselness” (i.e., the extent to which a given pixel is likely to be an element of an image of a vessel) in applicable areas of the image frame is determined. For example, vesselness may be determined by means of a filter, such as the filter described in the article by Frangi (a “Frangi filter”), cited hereinabove, which is incorporated herein by reference, and/or by means of a filter that performs enhancement and/or detection and/or segmentation of curvilinear structures. For some applications, a filter is used that is similar to a Frangi filter.”).
Regarding claim 10, Philipp teaches wherein generating the vessel roadmap further includes:
obtaining a vessel image sequence using an imaging method based on an inflow of a contrast medium into a vessel tree via a contrast application object and imaging physiological information associated with the vessel image sequence ([Col. 31, lines 47-63] “In Phase 2 of the automatic road map generation, a baseline fluoroscopic image frame is identified, typically automatically, the baseline image frame having been acquired prior to the contrast agent having been administered to the subject… For some applications, the baseline image frame is gated to a given phase of the subject's cardiac cycle (i.e., it selected based on its having been acquired at the given phase of the subject's cardiac cycle) … For some applications, the baseline image frame is used a reference image frame, to which to compare subsequent image frames, in order to determine when an angiographic sequence has commenced,”);
detecting, within the vessel image sequence, contrasted vessel images ([Col. 32, lines 14-24] “For some applications, the baseline image frame is analyzed such that the degree of “vesselness” (i.e., the extent to which a given pixel is likely to be an element of an image of a vessel) in applicable areas of the image frame is determined. For example, vesselness may be determined by means of a filter, such as the filter described in the article by Frangi (a “Frangi filter”), cited hereinabove, which is incorporated herein by reference, and/or by means of a filter that performs enhancement and/or detection and/or segmentation of curvilinear structures.” In Phase 3 and 4 [Cols. 32-33], Phillip teaches identifying an angiographic sequence, which is a series of images taken from the time the contrast is injected to the time when it dissipates. Contrast application object segmentation is performed to identify vessels [Col. 32, lines 56-60], and it is determined that the sequence is over when contrasted vessels are no longer being detected [Col. 33, lines 40-57]. In Phase 5, the image with the most identifiable contrasted vessels is selected for the roadmap [Col. 33, lines 64-67 – Col. 34, lines 1-5]. Thus, the position of the contrast over time is analyzed.);
performing vessel segmentation on the contrasted vessel images to generate vessel segmentation data (Figs. 5-6 show determining centerlines and segments from the contrast image for generating a road map. Fig. 7 shows methods for connecting the segments to create a complete roadmap.);
performing contrast application object segmentation on the contrasted vessel images to generate contrast application object segmentation data identifying a position of the contrast application object in the contrasted vessel images (The position of the catheter which injects contrast is considered. [Col. 33, lines 2-7] “Suitable image processing techniques include the analysis of changes in the current image, and/or, specifically, changes in the image region at the distal end of the catheter from which the contrast agent enters the subject's vasculature (such as a guiding catheter in the case of coronary road mapping).” Additionally, Phillip teaches segmenting objects such as catheters for identification of its location in the vessels. See Fig. 11; label 30 show the detected catheter.); and
for each contrasted vessel image, generating a vessel roadmap (Phillips teaches selecting images with high vessel visibility for generating roadmaps. [Col. 34, lines 9-13] “Such an angiogram with the greatest visibility of coronary arteries is typically the most suitable for serving as the basis for the most informative road map in situations wherein the greatest amount of vasculature should be observed.”),
the generated vessel roadmap comprising:
the contrasted vessel image (Contrasted images with high vessel visibility are used for a roadmap. [Col. 34, lines 9-13] “Such an angiogram with the greatest visibility of coronary arteries is typically the most suitable for serving as the basis for the most informative road map in situations wherein the greatest amount of vasculature should be observed.”) and the vessel segmentation data (Phases 7-9 of the automatic road map generation teach segmenting the vessels [Cols. 35-36]. Figs. 5-6 show determining centerlines and segments from the contrast image for generating a road map. Fig. 7 shows methods for connecting the segments to create a complete roadmap.) as the vessel roadmap image (Fig. 12 shows a vessel roadmap image, which is the vessel segmentation road map overlaid over the contrasted vessel image.);
the imaging physiological information in the first alignment data (The image sequence considers the motion cycle of the heart using ECG data. [Col. 30, line 67 – Col. 31, lines 1-20] “Typically, during the acquisition of the image stream, a contrast agent is administered to the subject. Optionally, the image stream is gated, tracked, and/or stabilized by other means. For example, selected image frames corresponding to a given phase in the motion cycle of the heart may be identified by means of a physiological signal… For some applications, a processor that performs the automatic generation of the road map, or a dedicated processor, identifies the selected phase of the ECG signal.”); and
the contrast application object segmentation data in the second alignment data (Figs. 5-7 show the vessel segmentation data that makes up the roadmap. Regarding object segmentation data, the location of the object used for injecting the contrast is also known. See Fig. 11, label 30.).
Regarding claim 11, Philipp teaches wherein: the imaging physiological information includes an electrocardiogram (ECG) ([Col. 31, lines 14-17] “For some applications, the ECG signal is received from an ECG monitor. Alternatively, or additionally, the ECG signal is received from a Cardiac Rhythm Management (CRM) device such as a pacer, or a defibrillator.”), and
generating the vessel roadmap further includes:
identifying one or more cardiac cycles within the vessel image sequence based on the ECG, wherein the generated vessel roadmap further comprises the identified one or more cardiac cycles in the first alignment data (The image sequence considers the motion cycle of the heart using ECG data. [Col. 30, line 67 – Col. 31, lines 1-20] “Typically, during the acquisition of the image stream, a contrast agent is administered to the subject. Optionally, the image stream is gated, tracked, and/or stabilized by other means. For example, selected image frames corresponding to a given phase in the motion cycle of the heart may be identified by means of a physiological signal… For some applications, a processor that performs the automatic generation of the road map, or a dedicated processor, identifies the selected phase of the ECG signal.”).
Regarding claim 12, Philipp teaches wherein obtaining a real-time fluoroscopy image and corresponding real-time first and second fluoroscopy information includes:
obtaining fluoroscopy physiological information associated with the fluoroscopy image (ECG data is recorded in real time and synchronized to the fluoroscopy image feed. [Col. 31, lines 14-17] “The ECG signal is typically received from an ECG monitor, or from a cardiac rhythm management (CRM) device, such as a pacer, or a defibrillator.”); and
performing fluoroscopy object segmentation on the fluoroscopy image to generate fluoroscopy object segmentation data identifying a position of the fluoroscopy object in the fluoroscopy image ([Col. 40, lines 20-26] “For some applications, image tracking of the fluoroscopic image stream is performed with respect to the guiding catheter or with respect to a segment thereof, as described hereinbelow. Alternatively or additionally, image tracking is performed with respect to radiopaque markers or segments of a tool (e.g., a balloon, a stent, a valve, or a different tool), as described hereinbelow.”),
wherein the fluoroscopy physiological information is included in the first real-time fluoroscopy information (Philipp teaches that the real-time physiological information is used alongside the live image stream and needed for some applications. For example: [Col. 54, lines 7-8] “For some applications, inflation is synchronized specifically to a selected phase in the subject's ECG signal.”), and
wherein the generated fluoroscopy object segmentation data is included in the second real-time fluoroscopy information (Philipp teaches use of the object segmentation data. For example: Fig. 23A shows edge lines added to the object, which in this case is a balloon, for visualization of the balloon within the vessel.).
Regarding claim 13, Philipp teaches wherein the fluoroscopy physiological information includes an electrocardiogram (ECG) ([Col. 31, lines 14-17] “The ECG signal is typically received from an ECG monitor, or from a cardiac rhythm management (CM) device, such as a pacer, or a defibrillator.”) and
wherein obtaining the real-time fluoroscopy image and corresponding real-time first and second fluoroscopy information further comprises identifying one or more cardiac cycles based on the ECG, wherein the identified one or more cardiac cycles are included in the first real-time fluoroscopy information (Philipp teaches that live fluoroscopy streams are gated and tracked based on the cardiac cycle. [Col. 31, lines 1-8] “Optionally, the image stream is gated, tracked, and/or stabilized by other means. For example, selected image frames corresponding to a given phase in the motion cycle of the heart may be identified by means of a physiological signal. For some applications, the physiological signal applied is the subject's ECG and the image frames are selected by means of gating to the ECG.” For example, an action as part of a surgery may be done based on the cardiac phase. [Col. 54, lines 7-8] “For some applications, inflation is synchronized specifically to a selected phase in the subject's ECG signal.”).
Regarding claim 14, Phillip teaches wherein aligning the vessel roadmap image and the real-time fluoroscopy image based on the second alignment data and the real time second fluoroscopy information comprises aligning the position of the contrast application object with the position of the fluoroscopy object ([Col. 64, lines 62-65] “Further typically, fiducials are identified within the road map and within the fluoroscopic image stream, in order to facilitate the registration of the road map to the fluoroscopic image stream.” [Col. 40, lines 38-44] “Fiducials are typically chosen that are observable even in images of the fluoroscopic image stream that are generated in the absence of a contrast agent. For example, the registration may be performed by means of a portion(s) (e.g., a marker, or a radiopaque portion) of a tool that is observable both in the road map and in the fluoroscopic images.”).
Regarding claim 15, Philipp teaches a non-transitory computer-readable medium storing instructions configured to be executed by a computer including at least one processor ([Col. 9, lines 13-21] “There is further provided, in accordance with some applications of the present invention, apparatus, including: an image-acquisition device configured to acquire a set of images of blood vessels of a Subject; a display; and at least one processor,”), the instructions causing the processor to:
generate a vessel roadmap library, the vessel roadmap library including a plurality of vessel roadmaps ([Col. 26, lines 42-44] “FIG. 1 is a flow chart, at least some of the steps of which are used to automatically generate a road map, in accordance with some applications of the present invention;”),
wherein each vessel roadmap comprises a vessel roadmap image and first (The first alignment data is ECG data. [Col. 30, lines 65-67 – Col. 31, lines 1-8] “In Phase 1 of the automatic road map generation, a fluoroscopic image stream of the coronary arteries is acquired. Typically, during the acquisition of the image stream, a contrast agent is administered to the subject. Optionally, the image stream is gated, tracked, and/or stabilized by other means. For example, selected image frames corresponding to a given phase in the motion cycle of the heart may be identified by means of a physiological signal. For some applications, the physiological signal applied is the subject's ECG and the image frames are selected by means of gating to the ECG….”) and second alignment data (The second alignment data includes contrast agent segmentation information. [Col. 33, lines 7-16] “…changes in the image may include a relatively abrupt change in the color and/or grayscale level (i.e., darkness) of a relatively large number and/or portion of image pixels, or the appearance of vessel-like features in the image, or any combination thereof. It is noted that by assessing a change in the darkness level to identify the time of injection of the contrast agent, the automatic road map generation processor may identify a darker area of the image or a lighter area of the image, depending on whether the contrast agent is represented as dark or light.” [Col. 33, lines 50-54] “…identifying decreased vesselness, for example, by means of a filter that performs enhancement and/or detection and/or segmentation of curvilinear structures, a Frangi filter, and/or a Frangi-like filter.” Additionally, roadmaps are displayed based on the respiratory cycle; thus, the roadmaps must include second alignment data related to the cycles. [Col. 42, lines 44-49] “For some applications, and in the case of images pertaining to a cyclically-moving organ, different road maps are displayed, typically in a cyclical manner, in conjunction with the then-current phase of the organ's motion cycle. For example, a dynamic road map as described may be correlated to the cardiac cycle, and/or the respiratory cycle.”);
detect, within the vessel roadmap images of the vessel roadmap library, a pathological vessel ([Col. 13, lines 5-7] “For some applications, identifying the target portion includes identifying a portion of the blood vessel that corresponds to a lesion.” Additionally, Philipp discloses automatically identifying a lesion in a user-designated area of an image. [Col. 13, lines 44-48] “in response to a user designating a single point on the image: automatically identifying a portion of a blood vessel in a vicinity of the designated point that corresponds to a lesion, by…”);
obtain a real-time fluoroscopy image and corresponding real-time first (Cardiac cycles and ECG signal (which is first fluoroscopy information) are recorded during live image tracking for guidance of the tool during fluoroscopic imaging. [Col. 52, lines 14-16] “For some applications, identification of the end-diastolic and end-systolic points within a cardiac cycle is determined from an ECG signal, and/or by means of image processing.”) and second fluoroscopy information (Tool position tracking information is second fluoroscopy information. [Col. 48, lines 64-67] “Once the markers have been identified in one or more image frames, then the system typically continues to identify (i.e., detect) those markers automatically in the subsequent image frames along the image stream or a segment thereof, and displays a tracked image stream.” [Col. 12, lines 2-6] “subsequently, inserting a tool into the blood vessel; while the tool is inside the blood vessel, determining a position of the tool; and modifying the road map to account for the determined position of the tool.”);
select a vessel roadmap from the roadmap library based on comparing the first real-time fluoroscopy information with the first alignment data of each vessel roadmap of the roadmap library (One embodiment teaches generating the roadmap lines and overlaying them over a live fluoroscopy image feed. [Col. 10, lines 17-22] “For some applications, generating the lines includes generating lines corresponding to the paths of the blood vessels during a given phase of a motion cycle of the blood vessels, and generating the output includes overlaying the lines on an image stream of the blood vessels that is gated to the given phase.”);
overlay the real-time fluoroscopy image with the selected vessel roadmap image of the vessel roadmap library ([Col. 40, lines 27-30] “For some applications, the road map is generally displayed side-by-side with the fluoroscopic image stream, and from time to time is momentarily overlaid upon the fluoroscopic image stream.”);
align the vessel roadmap image of the selected vessel roadmap and the real-time fluoroscopy image based on the second alignment data and the real time second fluoroscopy information (The vessel roadmap and live fluoroscopy image stream is aligned and rotated based on the position of the tool (second fluoroscopy data) and considers the organ’s motion cycle (second alignment data). [Col. 50, lines 49-55] “For some applications, a virtual line that connects the markers is aligned such that it remains at the same or a similar relative position throughout the image stream. For example, the alignment may include translating individual image frames such that the markers (or the virtual line connecting the markers) remain at a same or similar location within the image frames throughout the tracked image stream. Alternatively or additionally, the alignment includes rotating individual image frames such that the angle of the virtual line connecting the markers is same or similar throughout the tracked image stream.” [Col. 51, lines 12-15] “Typically, the aforementioned image tracking results in the tool being displayed as relatively stable, while the surrounding anatomy is displayed as moving relative to the generally-stable tool in the course of the organ's motion cycle.”); and
provide guidance for a fluoroscopy object to the pathological vessel based on the selected vessel roadmap and the second fluoroscopy information ([Col. 12, lines 1-6] “…a method, including: generating a road map of a blood vessel; subsequently, inserting a tool into the blood vessel; while the tool is inside the blood vessel, determining a position of the tool; and modifying the road map to account for the determined position of the tool.”).
Regarding claim 16, Philipp teaches wherein generation of the vessel roadmap library includes:
recordaction, for each vessel roadmap image of the vessel roadmap library, one or more imaging parameters, the imaging parameters indicating one or more parameters associated with an imaging method used to obtain the vessel roadmap image ([Col. 33, lines 64-67] “In Phase 5 of the automatic generation of the road map, the angiographic image frames (also known as angiograms) corresponding to a given angiographic sequence are automatically analyzed.” [Col. 34, lines 33-37] For some applications, the determination of vesselness of image pixels is made while accounting for the specific viewing angle at which the images are generated.” Philipp teaches that the viewing angles for the captured images are considered when developing the roadmap.).
Regarding claim 17, Philipp teaches wherein the one or more imaging parameters include at least one of an angiography angle and a contrast medium dosage ([Col. 33, lines 64-67] “In Phase 5 of the automatic generation of the road map, the angiographic image frames (also known as angiograms) corresponding to a given angiographic sequence are automatically analyzed.” [Col. 34, lines 33-37] “For some applications, the determination of vesselness of image pixels is made while accounting for the specific viewing angle at which the images are generated.” Philipp teaches that the viewing angles for the captured images are considered when developing the roadmap.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Philipp (US 8,781193 B2), further in view of Berg (Three-Dimensional Image Overlay to Assist Endovascular Procedures. Vascular Disease Management. 10(9) 179-184.).
Regarding claim 5, Phillip teaches overlaying the roadmap of the fluoroscopy image steam ([Col. 40, lines 6-10] “FIG. 14, which shows a road map 36 displayed side-by-side with a stabilized fluoroscopic image stream 38, edge lines of the road-map also being overlaid upon the fluoroscopic image stream, in accordance with some applications of the present invention.”), and Phillip teaches that by identifying fiducials, such as anatomical locations or markers on tools, the roadmap can be oriented and overlayed onto the image stream ([Col. 40, lines 35-44] “Further typically, fiducials are identified within the road map and within the fluoroscopic image stream, in order to facilitate the registration of the road map to the fluoroscopic image stream. Fiducials are typically chosen that are observable even in images of the fluoroscopic image stream that are generated in the absence of a contrast agent. For example, the registration may be performed by means of a portion(s) (e.g., a marker, or a radiopaque portion) of a tool that is observable both in the road map and in the fluoroscopic images.”). Additionally, Philips mentions that the angle is known and considered when creating a roadmap ([Col. 31-32] “For some applications, the determination of vesselness of image pixels is made while accounting for the specific viewing angle at which the images are generated.”)
Phillip does not specifically mention comparing saved angles between the image stream and the roadmap. Thus, Phillip fails to teach wherein the one or more imaging parameters includes at least an angiography angle used to obtain each vessel roadmap image; further comprising comparing the angiography angle with a fluoroscopy angle, the fluoroscopy angle being used to obtain the real-time fluoroscopy image; and when the angiography angle and the fluoroscopy angle differ by more than an angle difference threshold, the refraining from overlaying the real-time fluoroscopy image with the selected vessel roadmap image.
However, Berg teaches wherein the one or more imaging parameters includes at least an angiography angle used to obtain each vessel roadmap image (Berg teaches that when analyzing a 3D roadmap, the viewing angle can be stored. [p. 180-181, Section 2] “The projection angles can be stored, and later recalled to automatically steer the C-arm to the predefined rotation/angulation.”);
further comprising comparing the angiography angle with a fluoroscopy angle, the fluoroscopy angle being used to obtain the real-time fluoroscopy image (The viewing angle of the roadmap is the angiographic angle, and it is used to position the C-arm during the subsequent fluoroscopic imaging. Thus, the angles are compared. [p. 180-181, Section 2] “The projection angles can be stored, and later recalled to automatically steer the C-arm to the predefined rotation/angulation.”); and
when the angiography angle and the fluoroscopy angle differ by more than an angle difference threshold, the refraining from overlaying the real-time fluoroscopy image with the selected vessel roadmap image (Berg teaches receiving the stored angiographic angle and using that angle to guide the C-arm, thus determining the fluoroscopy angle [p. 180-181, Section 2]. Berg also teaches the process of guiding the C-arm so that the fluoroscopy stream can be overlaid with the roadmap. This “registration step” ensured that the fluoroscopy and angiographic angles are close enough so that the overlaid result is accurate.” [p. 181, Section 3(b)] “The registration step ensures that the fluoroscopy stream can be accurately superimposed on the 3D volume of the CTA.”).
Philipp and Berg are analogous in the art to the claimed invention because both teach methods of receiving a vessel roadmap and overlaying the vessel roadmap with a live fluoroscopy image feed to guide an operator during a coronary operation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Philipp’s invention by additionally using the angle the roadmap was captured at (and not just the fiducials alone) to ensure the live image feed and the roadmap are overlayed in sync. This modification would allow for the operator to plan a viewing angle ahead of time using the roadmap images (Berg [p. 180-181, Section 2] “In the planning step, the user can interact with the (segmented) CTA to find optimal working projections to be used during the intervention.”), and it would ensure a degree of accuracy when aligning the image stream and the roadmap (Berg [p. 181, Section 3(b)] “The registration step ensures that the fluoroscopy stream can be accurately superimposed on the 3D volume of the CTA.”).
Regarding claim 18, Philipp fails to teach wherein the one or more imaging parameters includes the angiography angle used to obtain each vessel roadmap image; wherein the instructions further comprise comparison of the angiography angle with a fluoroscopy angle, the fluoroscopy angle being used to obtain the real-time fluoroscopy image; and when the angiography angle and the fluoroscopy angle differ by more than an angle difference threshold, the refraining from overlaying the real-time fluoroscopy image with the selected vessel roadmap image.
However, Berg teaches wherein the one or more imaging parameters includes the angiography angle used to obtain each vessel roadmap image (Berg teaches that when analyzing a 3D roadmap, the viewing angle can be stored. [p. 180-181, Section 2] “The projection angles can be stored, and later recalled to automatically steer the C-arm to the predefined rotation/angulation.”);
wherein the instructions further comprise comparison of the angiography angle with a fluoroscopy angle, the fluoroscopy angle being used to obtain the real-time fluoroscopy image; (The viewing angle of the roadmap is the angiographic angle, and it is used to position the C-arm during the subsequent fluoroscopic imaging. Thus, the angles are compared. [p. 180-181, Section 2] “The projection angles can be stored, and later recalled to automatically steer the C-arm to the predefined rotation/angulation.”); and
when the angiography angle and the fluoroscopy angle differ by more than an angle difference threshold, the refraining from overlaying the real-time fluoroscopy image with the selected vessel roadmap image (Berg teaches receiving the stored angiographic angle and using that angle to guide the C-arm, thus determining the fluoroscopy angle [p. 180-181, Section 2]. Berg also teaches the process of guiding the C-arm so that the fluoroscopy stream can be overlaid with the roadmap. This “registration step” ensured that the fluoroscopy and angiographic angles are close enough so that the overlaid result is accurate.” [p. 181, Section 3(b)] “The registration step ensures that the fluoroscopy stream can be accurately superimposed on the 3D volume of the CTA.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Philipp’s invention by additionally using the angle the roadmap was captured at (and not just the fiducials alone) to ensure the live image feed and the roadmap are overlayed in sync. This modification would allow for the operator to plan a viewing angle ahead of time using the roadmap images (Berg [p. 180-181, Section 2] “In the planning step, the user can interact with the (segmented) CTA to find optimal working projections to be used during the intervention.”), and it would ensure a degree of accuracy when aligning the image stream and the roadmap (Berg [p. 181, Section 3(b)] “The registration step ensures that the fluoroscopy stream can be accurately superimposed on the 3D volume of the CTA.”).
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Philipp (US 8,781193 B2), further in view of Liao et al. (US 2008/0275467 A1), hereafter Liao.
Regarding claim 6, Philipp teaches displaying the vessel roadmap and segmentation data over the fluoroscopy image feed, so a doctor performing a medical operation could visualize the vessels and determine a path to guide his/her tool to a pathological vessel (See Fig. 20 for example). However, Philipp does not teach that the medical system is able to determine a guiding path itself; thus, Philip fails to teach wherein providing guidance for the fluoroscopy object to the pathological vessel includes determining a path from the fluoroscopy object to the pathological vessel based on the second real-time fluoroscopy information and vessel segmentation data.
However, Liao teaches wherein providing guidance for the fluoroscopy object to the pathological vessel ([0072] “…the user marks the target blood vessel 46 for the catheter or guidewire on the 3D images.”) includes determining a path from the fluoroscopy object to the pathological vessel (Fig. 3 shows a path determined between the main vessel 48 and an area of interest 46. This path is followed by a catheter or other tool to reach the target vessel 46.)
based on the second real-time fluoroscopy information (The real-time position of the tool in the vessel is the second fluoroscopy information. [0075-0076] “Given the 3D location of the tracked catheter or interventional device, an exemplary embodiment of the present invention is able to determine the distance between this location and the closest node in the selected path. If this distance is greater than a preset threshold (e.g., 5 mm or the maximum vessel diameter), an audible warning (e.g., an intermittent beeping sound) and/or a visual warning (e.g., flashing the catheter or device location via a small red colored sphere on the fluoroscopic image monitor), may be provided. With such a warning, the user may be able to return the tracked catheter or device back to the last branching point in order to follow the correct branch along the planned path.”) and vessel segmentation data (The vessel tree 40 is the vessel segmentation data. [0070] “FIG. 3… shows a vessel tree as would be available from a 3D angiographic medical image showing blood vessels, e.g., MRA, 3D X-ray Angiography, or CT angiography.”).
Philipp and Liao are analogous in the art to the claimed invention, because both teach methods of obtaining a vessel roadmap and overlaying the roadmap onto a fluoroscopic live feed for guiding a tool through a patient’s blood vessels. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Philipp’s invention by utilizing the shortest-path finding tools (such as Dikstra’s algorithm mentioned by Philipp in [Col. 44, line 36]) for determining and displaying a guiding path for the surgical tool, such as a catheter, to get to the pathological vessel. This modification would allow for easier navigation of the surgical tool by an operator amid many branching vessels visible in the roadmap (Liao [0071] “Using the vessel tree 40 as a roadmap by overlaying it on 2D fluoroscopy or X-ray angiography may be confusing or unclear to the doctor, since the vessels have many branchings and may overcross each other when the images are viewed from a particular angle.” [0073] “Having this symbolic path overlaid on the 2D interventional images, such as X-ray angiography or fluoroscopy, helps the user navigate the catheter or guidewire and helps clarify where the branching points 50 are, and where the planned path 42 is, specially when there are overcrossing vessels, such as vessel 52.”).
Regarding claim 7, Philipp fails to teach displaying the path on a display of a medical imaging system to guide an operator operating the fluoroscopy object to the pathological vessel.
However, Liao teaches displaying the path on a display of a medical imaging system to guide an operator operating the fluoroscopy object to the pathological vessel ([0073] “Having this symbolic path overlaid on the 2D interventional images, such as X-ray angiography or fluoroscopy, helps the user navigate the catheter or guidewire and helps clarify where the branching points 50 are,”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Philipp’s invention by determining a guiding path for the surgical tool, such as a catheter, to get to the pathological vessel and displaying the path to guide the operator of the tool. This modification would allow for easier navigation of the surgical tool by an operator amid many branching vessels visible in the roadmap (Liao [0071] “Using the vessel tree 40 as a roadmap by overlaying it on 2D fluoroscopy or X-ray angiography may be confusing or unclear to the doctor, since the vessels have many branchings and may overcross each other when the images are viewed from a particular angle.” [0073] “Having this symbolic path overlaid on the 2D interventional images, such as X-ray angiography or fluoroscopy, helps the user navigate the catheter or guidewire and helps clarify where the branching points 50 are, and where the planned path 42 is, specially when there are overcrossing vessels, such as vessel 52.”).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Philipp (US 8,781193 B2), in view of Berg (Three-Dimensional Image Overlay to Assist Endovascular Procedures. Vascular Disease Management. 10(9) 179-184.), further in view of Liao (US 2008/0275467 A1).
Regarding claim 19, Philipp teaches displaying the vessel roadmap and segmentation data over the fluoroscopy image feed, so a doctor performing a medical operation could visualize the vessels and determine a path to guide his/her tool to a pathological vessel (See Fig. 20 for example). However, Philipp does not teach that the medical system is able to determine a guiding path itself; thus, Philip fails to teach wherein providing guidance for the fluoroscopy object to the pathological vessel includes determining a path from the fluoroscopy object to the pathological vessel based on the second real-time fluoroscopy information and vessel segmentation data.
However, Liao teaches wherein provision of the guidance for the fluoroscopy object to the pathological vessel ([0072] “…the user marks the target blood vessel 46 for the catheter or guidewire on the 3D images.”) includes determination of a path from the fluoroscopy object to the pathological vessel (Fig. 3 shows a path determined between the main vessel 48 and an area of interest 46. This path is followed by a catheter or other tool to reach the target vessel 46.)
based on the second real-time fluoroscopy information (The real-time position of the tool in the vessel is the second fluoroscopy information. [0075-0076] “Given the 3D location of the tracked catheter or interventional device, an exemplary embodiment of the present invention is able to determine the distance between this location and the closest node in the selected path. If this distance is greater than a preset threshold (e.g., 5 mm or the maximum vessel diameter), an audible warning (e.g., an intermittent beeping sound) and/or a visual warning (e.g., flashing the catheter or device location via a small red colored sphere on the fluoroscopic image monitor), may be provided. With such a warning, the user may be able to return the tracked catheter or device back to the last branching point in order to follow the correct branch along the planned path.”) and vessel segmentation data (The vessel tree 40 is the vessel segmentation data. [0070] “FIG. 3… shows a vessel tree as would be available from a 3D angiographic medical image showing blood vessels, e.g., MRA, 3D X-ray Angiography, or CT angiography.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Philipp’s invention by utilizing the shortest-path finding tools (such as Dikstra’s algorithm mentioned by Philipp in [Col. 44, line 36]) for determining and displaying a guiding path for the surgical tool, such as a catheter, to get to the pathological vessel. This modification would allow for easier navigation of the surgical tool by an operator amid many branching vessels visible in the roadmap (Liao [0071] “Using the vessel tree 40 as a roadmap by overlaying it on 2D fluoroscopy or X-ray angiography may be confusing or unclear to the doctor, since the vessels have many branchings and may overcross each other when the images are viewed from a particular angle.” [0073] “Having this symbolic path overlaid on the 2D interventional images, such as X-ray angiography or fluoroscopy, helps the user navigate the catheter or guidewire and helps clarify where the branching points 50 are, and where the planned path 42 is, especially when there are overcrossing vessels, such as vessel 52.”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Elion (US 4,878,115 A) teaches generating a dynamic coronary roadmap. The roadmap sequence is then replayed while overlaid over a live fluoroscopic image feed of the heart, and the sequence is synchronized to the live feed based on the receipt of an ECG R-wave.
Li et al. (US 8,428,690 B2) teaches a system for generating and displaying a 4D model of an imaged anatomy. The model provides a roadmap of the anatomy which can be used in image-guided surgery. The system includes a controller for a real-time tracking system and an imaging probe, and the tracking system directs the probe to acquire 3D image data.
Dascal et al. (US 9,351,698 B2) teaches a system for processing, tracking, and registering angiographic images using optical coherence tomography. Methods are included for storing and retrieving images along with segmenting images and tracking vessel centerlines.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC JAMES SHOEMAKER whose telephone number is (571)272-6605. The examiner can normally be reached Monday through Friday from 8am to 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JENNIFER MEHMOOD, can be reached at (571)272-2976. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Eric Shoemaker/
Patent Examiner
/JENNIFER MEHMOOD/ Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights