DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s Amendments filed on February 13, 2026, has been entered and made of record.
Currently pending Claim(s) 1-19
Independent Claim(s) 1 and 15
Amended Claim(s) 16
Response to Arguments
This office action is responsive to Applicant’s Arguments/Remarks Made in an Amendment received on February 13, 2026.
In view of amendments filed on February 13, 2026, the Applicant has amended the abstract to fall within the range of 50-150 words; thus, the objection to the abstract is overcome. Regarding the claims, the Applicant has amended claim 16 to correct a typo, and no other claims have been amended. Rather, the Applicant provides arguments against the Examiner’s prior art rejections made in the Non-Final Rejection dated November 26, 2025.
In view of Applicant Arguments/Remarks filed February 13, 2026, with respect to the claims, the Applicant first argued (Remarks pages 9-11) that none of the cited prior art teaches "selecting a vessel roadmap from the roadmap library based on comparing the first real-time fluoroscopy information with the first alignment data of each vessel roadmap of the roadmap library," "overlaying the real-time fluoroscopy image with the selected vessel roadmap image of the vessel roadmap library," and "aligning the vessel roadmap image of the selected vessel roadmap and the real-time fluoroscopy image based on the second alignment data and the real time second fluoroscopy information." The Applicant argued that Phillip (US 8,781,193 B2) at [Col. 50, lines 49-55] and [Col. 51, lines 12-15] instead teaches aligning and stabilizing successive frames of a continuous fluoroscopy stream, and Phillip does not teach aligning a previously generated static roadmap image (which is dynamically selected from a roadmap library) with a fluoroscopy stream based on second alignment data and second fluoroscopy information. Upon review of the prior art and the Applicant’s arguments, the Examiner finds this argument to be persuasive. The quoted passage [Col. 50, lines 49-55] differs from the claimed invention by teaching stabilization of a stream rather than alignment of a roadmap to the stream.
However, upon completing further searching focused on the alignment discussed above, the Examiner has found one additional reference that teaches these limitations. Walsum (US 2020/0222018 A1) teaches the alignment step using the second alignment data and the second fluoroscopy information ([0184] “Within step 107, the location of the device (e.g. catheter tip) in current X-ray fluoroscopic frame, as a result of step 106, and the device location (e.g. catheter tip) from the selected roadmap frame as a result of step 105 is used to obtain a transformation function to align the selected roadmap with the current X-ray fluoroscopic image frame.”), and overall, Walsum teaches nearly the same invention as the claimed invention, including roadmap generation, selection, overlay, and alignment. New rejections are presented using Walsum instead of Phillip, and accordingly, this action is made non-final.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4 and 8-17 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Walsum et al. (US 2020/0222018 A1), hereafter Walsum.
Regarding claim 1, Walsum teaches A computer-implemented pathological vessel guidance method (See Fig. 1 outlining the overall method.) comprising:
generating a vessel roadmap library, the vessel roadmap library including a plurality of vessel roadmaps (See block 100 of Fig. 1. [0140-0147] provides several methods for generating a vessel roadmap library.),
wherein each vessel roadmap comprises a vessel roadmap image (Fig. 35) and first (The ECG signal data is the first alignment data. [0145] “Along with the Nafter-contrast ECG signals or the extracted cardiac cycle as a result from step 501 of FIG. 5 corresponding to the frames within Nafter-contrast are retrieved for later to allow selecting a temporal aligned roadmap to a given X-ray fluoroscopic frame during the online phase.”) and second alignment data (The device location (e.g. catheter for injecting contrast) is registered to each roadmap for later alignment with the live feed. The device location is second alignment data. See the device 3502 in Fig. 35. [0139] “the next step (103) is to create the roadmaps. Within step 103, the device location will be integrated in the created roadmaps to serve as reference point for the roadmap transformation.” [0147] “Finally, in step 103 the obtained device location as a result of step 102 is integrated into the created roadmaps… The final roadmap, showing the vascular structures (3503) also contains the device location (3502).” Also see Fig. 5.);
detecting, within the vessel roadmap images of the vessel roadmap library, a pathological vessel ([0118] “One or more of the roadmaps may also contain information of clinical interest, e.g. location and percentage of vessel obstruction, diameter and area, pressure, blood velocity, fractional flow reserve, wall shear stress, the curvature of the vessel, amount of foreshortening, the location and amount and type of coronary plaque (e.g. calcified, soft plaque, mixed-plaque), location and extent of coronary total occlusion, or location and extent of coronary obstruction.” [0193] “These geometrical parameters can be derived for example from the vessel model created in step 103. Another example of quantitative parameters are pathological parameters such as the amount of calcified plaque in the lumen which is derived in step 103 and in more detail by the flowchart description of FIG. 28.” Furthermore, [0146] teaches that image analysis methods can be performed to determine pathological information and provides references for performing the image analysis methods.);
obtaining a real-time fluoroscopy image ([0148] “The first step in the online phase (110), is the retrieval of the X-ray fluoroscopic image data represented by step 104 of FIG. 1. Within a preferred embodiment the X-ray fluoroscopic image sequence is acquired real time as a live stream.”) and corresponding real-time first ([0148] “Simultaneously, the ECG signal of the patient can be acquired in real time and time synchronized to the X-ray fluoroscopic image sequence.”) and second fluoroscopy information (Step 106 of Fig. 1 and [0156-0183] teach methods of tracking a fluoroscopic device (such as a catheter, pacemaker, or anatomical landmark) during the live fluoroscopic stream.);
selecting a vessel roadmap from the roadmap library based on comparing the first real-time fluoroscopy information with the first alignment data of each vessel roadmap of the roadmap library ([0150] “Roadmap selection represented by step 104 in FIG. 1 can be achieved by comparing the ECG signal associated with the online fluoroscopic image and the ECG of the offline angiographic sequence, such that the most suitable candidate roadmap is selected where the best match of the ECG signals is found. The selected roadmap has the same (or very similar) cardiac phase with the online X-ray fluoroscopic image, which compensates the difference of vessel shape and pose induced by cardiac motion.”);
overlaying the real-time fluoroscopy image with the selected vessel roadmap image of the vessel roadmap library ([0187] “The dynamic roadmap that results from the cardiac matching and transformation of the roadmap is rendered and integrated as an overlay on the corresponding X-ray fluoroscopic or angiographic image data frame of the live angiographic image data image data stream of the patient.”);
aligning the vessel roadmap image of the selected vessel roadmap and the real-time fluoroscopy image based on the second alignment data and the real time second fluoroscopy information ([0184] “Within step 107, the location of the device (e.g. catheter tip) in current X-ray fluoroscopic frame, as a result of step 106, and the device location (e.g. catheter tip) from the selected roadmap frame as a result of step 105 is used to obtain a transformation function to align the selected roadmap with the current X-ray fluoroscopic image frame.”); and
providing guidance for a fluoroscopy object to the pathological vessel based on the selected vessel roadmap and the second fluoroscopy information (Walsum teaches that the purpose of the dynamic roadmaps is to provide guidance to the physician using a fluoroscopy object ([0116] “…in order to provide visual guidance to the physician to provide support for placement of one or more medical instruments inserted through a guiding catheter.” For more specific limitations, such as determining a path for the fluoroscopic object to the pathological vessel and displaying that path on a screen, see the 35 USC 103 rejection to claims 6-7.).
Regarding claim 2, Walsum teaches the pathological vessel guidance method of claim 1, wherein detecting the pathological vessel includes: determining centerlines of the vessels included in each vessel roadmap image of the vessel roadmap library; determining, based on the centerlines, lumina of the vessels included in each vessel roadmap image of the vessel roadmap library ([0118] “A roadmap includes information that characterizes properties of the coronary arteries for a given phase of the cardiac cycle. Such information can include centerlines for the coronary arteries, contours (luminal boundary) of the coronary arteries over respective lengths of the coronary arteries, and/or a mask image that represents the coronary arteries (e.g., an image that covers the space occupied by the coronary arteries).” Furthermore, [0186] discusses aligning a roadmap to a live fluoroscopic image and mentions that the roadmap may be represented as centerline. [0186] “The transformation function T can be, for example, a displacement function, rotation function, scaling function, etc. For instance, when the roadmap represents the vessel model as centerlines, or contours, the above transformation can be applied to each two dimensional coordinate (x,y) of the centerlines or contours.”); and
detecting the pathological vessel based on the lumina of the vessels included in each vessel roadmap image of the vessel roadmap library ([0118] “One or more of the roadmaps may also contain information of clinical interest, e.g. location and percentage of vessel obstruction, diameter and area, pressure, blood velocity, fractional flow reserve, wall shear stress, the curvature of the vessel, amount of foreshortening, the location and amount and type of coronary plaque (e.g. calcified, soft plaque, mixed-plaque), location and extent of coronary total occlusion, or location and extent of coronary obstruction.” [0193] “These geometrical parameters can be derived for example from the vessel model created in step 103. Another example of quantitative parameters are pathological parameters such as the amount of calcified plaque in the lumen which is derived in step 103 and in more detail by the flowchart description of FIG. 28.” Furthermore, [0146] teaches that image analysis methods can be performed to determine pathological information and provides references for performing the image analysis methods.).
Regarding claim 3, Walsum teaches the pathological vessel guidance method of claim 1, wherein generating the vessel roadmap library includes:
recording, for each vessel roadmap image of the vessel roadmap library, one or more imaging parameters, the imaging parameters indicating one or more parameters associated with an imaging method used to obtain the vessel roadmap image (In [0031-0032], Walsum teaches that when using 2D roadmaps, it is advantageous to create the roadmaps and capture the live stream from the same angiography angle. When using 3D roadmaps, images from different angles are taken to generate the 3D roadmap. Thus, imaging viewpoints/angles must be saved as imaging parameters. Furthermore, Fig. 6 and [0126] teach tracking the contrast over multiple frames.).
Regarding claim 4, Walsum teaches the pathological vessel guidance method of claim 3, wherein the one or more imaging parameters include at least one of an angiography angle and a contrast medium dosage (In [0031-0032], Walsum teaches that when using 2D roadmaps, it is advantageous to create the roadmaps and capture the live stream from the same angiography angle. When using 3D roadmaps, images from different angles are taken to generate the 3D roadmap. Thus, imaging viewpoints/angles must be saved as imaging parameters. Furthermore, Fig. 6 and [0126] teach tracking the contrast over multiple frames.).
Regarding claim 8, Walsum teaches the pathological vessel guidance method of claim 1, wherein the fluoroscopy object is a robot-controlled fluoroscopy object and wherein the guidance is provided to the robot-controlled fluoroscopy object to enable the robot-controlled fluoroscopy object to be guided to the pathological vessel ([0301] “Alternatively, the embodiments, with special focus on the extract roadmaps as described by FIG. 1, FIG. 15, FIG. 18 and FIG. 26 may also be used in the field of robotic assisted percutaneous coronary interventions (PCI) or peripheral vascular interventions.”).
Regarding claim 9, Walsum teaches the pathological vessel guidance method of claim 1, wherein the second alignment data is derived from the corresponding vessel roadmap image (The device location (e.g. catheter for injecting contrast) is registered to each roadmap for later alignment with the live feed. The device location is second alignment data. See the device 3502 in Fig. 35. [0139] “the next step (103) is to create the roadmaps. Within step 103, the device location will be integrated in the created roadmaps to serve as reference point for the roadmap transformation.” [0147] “Finally, in step 103 the obtained device location as a result of step 102 is integrated into the created roadmaps… The final roadmap, showing the vascular structures (3503) also contains the device location (3502).” Also see Fig. 5.).
Regarding claim 10, Walsum teaches the pathological vessel guidance method of claim 1, wherein generating the vessel roadmap further includes:
obtaining a vessel image sequence using an imaging method based on an inflow of a contrast medium into a vessel tree via a contrast application object and imaging physiological information associated with the vessel image sequence (See Step 101 of Fig. 1. [0123-0124] describes methods for capturing the angiographic images and recording ECG data for cardiac phase matching.);
detecting, within the vessel image sequence, contrasted vessel images; performing vessel segmentation on the contrasted vessel images to generate vessel segmentation data (See [0143-0144] teaching methods for separating the vessels from the base fluoroscopic image(s).);
performing contrast application object segmentation on the contrasted vessel images to generate contrast application object segmentation data identifying a position of the contrast application object in the contrasted vessel images (Fig. 12 shows a neural network for joint segmentation and detection for a catheter in the angiographic images. [0137] “Another method to detect the device (e.g. the catheter tip) may be performed by model based detection, or use of convolution neural networks. Detection of the catheter tip might be improved by incorporating temporal information in the preprocessing or post processing. A fully automatic catheter segmentation technique based on convolutional neural network.”); and
for each contrasted vessel image, generating a vessel roadmap (Step 103 of Fig. 1. Also see the section starting at [0140].), the generated vessel roadmap comprising:
the contrasted vessel image and the vessel segmentation data as the vessel roadmap image (Fig. 8 shows the images that comprise a roadmap. 801 is the contrasted vessel image, and the image is separated into a breathing structure layer 802, a background layer 803, and a vessel structure layer 804. Thus, a contrasted vessel image 801 and an image with the vessel structures 804 are present in a vessel roadmap. See [0143-0144] teaching methods for separating the vessels from the base fluoroscopic image.);
the imaging physiological information in the first alignment data ([0129-0134] and Fig. 7 teach methods for extracting the cardiac cycle information for use with the roadmaps.); and
the contrast application object segmentation data in the second alignment data (Fig. 5 shows the steps for determining the contrast application object location in a roadmap. Referring to Fig. 1, [0147] “Finally, in step 103 the obtained device location as a result of step 102 is integrated into the created roadmaps.” Referring to Fig. 35, [0147] “The final roadmap, showing the vascular structures (3503) also contains the device location (3502).” The device is the contrast application object.).
Regarding claim 11, Walsum teaches the pathological vessel guidance method of claim 10, wherein: the imaging physiological information includes an electrocardiogram (ECG), and generating the vessel roadmap further includes:identifying one or more cardiac cycles within the vessel image sequence based on the ECG, wherein the generated vessel roadmap further comprises the identified one or more cardiac cycles in the first alignment data (See Step 101 of Fig. 1. [0123-0124] describes methods for capturing the angiographic images and recording ECG data for cardiac phase matching. [0129-0134] and Fig. 7 teach methods for extracting the cardiac cycle information for use with the roadmaps.).
Regarding claim 12, Walsum teaches the pathological vessel guidance method of claim 1, wherein obtaining a real-time fluoroscopy image and corresponding real-time first and second fluoroscopy information (Online Phase 110 of Fig. 1. Also see the section starting at [0148].) includes:
obtaining fluoroscopy physiological information associated with the fluoroscopy image ([0148] “The first step in the online phase (110), is the retrieval of the X-ray fluoroscopic image data represented by step 104 of FIG. 1… Simultaneously, the ECG signal of the patient can be acquired in real time and time synchronized to the X-ray fluoroscopic image sequence.”); and
performing fluoroscopy object segmentation on the fluoroscopy image to generate fluoroscopy object segmentation data identifying a position of the fluoroscopy object in the fluoroscopy image ([0148] “Simultaneously, the device (e.g. the catheter tip location) in the acquired X-ray fluoroscopic image stream is tracked for instance by using a deep learning based Bayesian filtering method in as described further by step 106.” [0156-0183] teaches many different methods for tracking the location of the fluoroscopy object during the live fluoroscopy image stream.),
wherein the fluoroscopy physiological information is included in the first real-time fluoroscopy information ([0154] “In the online phase (block 110 of FIG. 1), for acquisition of each image, a block of NECG latest ECG signal points is constantly stored. These NECG ECG signal points are considered as the ECG signal corresponding to the online fluoroscopic frame.” [0155] discussed comparing ECG data between a roadmap and a live fluoroscopic image (thus comparing first alignment data and first fluoroscopy information).), and
wherein the generated fluoroscopy object segmentation data is included in the second real-time fluoroscopy information ([0158-0183] teach methods for obtaining the location of the fluoroscopy object in the live fluoroscopy stream. Furthmore, in [0182] Walsum teaches that the fluoroscopy object location in the live stream is compared to the location of the contrast agent device in the selected roadmap for alignment purposes. Thus, device location (e.g. contrast injecting catheter) from the roadmap is second alignment data, and the fluoroscopy object location from the livestream is the second fluoroscopy information.).
Regarding claim 13, Walsum teaches the pathological vessel guidance method of claim 12, wherein the fluoroscopy physiological information includes an electrocardiogram (ECG) and wherein obtaining the real-time fluoroscopy image and corresponding real-time first and second fluoroscopy information further comprises identifying one or more cardiac cycles based on the ECG, wherein the identified one or more cardiac cycles are included in the first real-time fluoroscopy information ([0148] “It is preferred to acquire the ECG signal simultaneously with the acquisition of the X-ray frames. During the online phase (110), the ECG signal will be used to match the acquired frames with the corresponding roadmaps as extracted by step 103, to represent the same cardiac phase in the cardiac cycle.” [0154] “In the online phase (block 110 of FIG. 1), for acquisition of each image, a block of NECG latest ECG signal points is constantly stored. These NECG ECG signal points are considered as the ECG signal corresponding to the online fluoroscopic frame.”).
Regarding claim 14, Walsum teaches the pathological vessel guidance method of claim 12, wherein aligning the vessel roadmap image and the real-time fluoroscopy image based on the second alignment data and the real time second fluoroscopy information comprises aligning the position of the contrast application object with the position of the fluoroscopy object (Step 107 of Fig. 1 teaches aligning a selected roadmap to a live image based on the catheter tip location in the roadmap and in the live image. See the Section starting at [0184]. [0184] “Within step 107, the location of the device (e.g. catheter tip) in current X-ray fluoroscopic frame, as a result of step 106, and the device location (e.g. catheter tip) from the selected roadmap frame as a result of step 105 is used to obtain a transformation function to align the selected roadmap with the current X-ray fluoroscopic image frame.” [0184-0186] discuss possible methods for this alignment.).
Regarding claim 15, Walsum teaches a non-transitory computer-readable medium storing instructions configured to be executed by a computer including at least one processor ([0050]), the instructions causing the processor to:
generate a vessel roadmap library, the vessel roadmap library including a plurality of vessel roadmaps (See block 100 of Fig. 1. [0140-0147] provides several methods for generating a vessel roadmap library.), wherein each vessel roadmap comprises a vessel roadmap image (Fig. 35) and first (The ECG signal data is the first alignment data. [0145] “Along with the Nafter-contrast ECG signals or the extracted cardiac cycle as a result from step 501 of FIG. 5 corresponding to the frames within Nafter-contrast are retrieved for later to allow selecting a temporal aligned roadmap to a given X-ray fluoroscopic frame during the online phase.”) and second alignment data (The device location (e.g. catheter for injecting contrast) is registered to each roadmap for later alignment with the live feed. The device location is second alignment data. See the device 3502 in Fig. 35. [0139] “the next step (103) is to create the roadmaps. Within step 103, the device location will be integrated in the created roadmaps to serve as reference point for the roadmap transformation.” [0147] “Finally, in step 103 the obtained device location as a result of step 102 is integrated into the created roadmaps… The final roadmap, showing the vascular structures (3503) also contains the device location (3502).” Also see Fig. 5.);
detect, within the vessel roadmap images of the vessel roadmap library, a pathological vessel ([0118] “One or more of the roadmaps may also contain information of clinical interest, e.g. location and percentage of vessel obstruction, diameter and area, pressure, blood velocity, fractional flow reserve, wall shear stress, the curvature of the vessel, amount of foreshortening, the location and amount and type of coronary plaque (e.g. calcified, soft plaque, mixed-plaque), location and extent of coronary total occlusion, or location and extent of coronary obstruction.” [0193] “These geometrical parameters can be derived for example from the vessel model created in step 103. Another example of quantitative parameters are pathological parameters such as the amount of calcified plaque in the lumen which is derived in step 103 and in more detail by the flowchart description of FIG. 28.” Furthermore, [0146] teaches that image analysis methods can be performed to determine pathological information and provides references for performing the image analysis methods.);
obtain a real-time fluoroscopy image ([0148] “The first step in the online phase (110), is the retrieval of the X-ray fluoroscopic image data represented by step 104 of FIG. 1. Within a preferred embodiment the X-ray fluoroscopic image sequence is acquired real time as a live stream.”) and corresponding real-time first ([0148] “Simultaneously, the ECG signal of the patient can be acquired in real time and time synchronized to the X-ray fluoroscopic image sequence.”) and second fluoroscopy information (Step 106 of Fig. 1 and [0156-0183] teach methods of tracking a fluoroscopic device (such as a catheter, pacemaker, or anatomical landmark) during the live fluoroscopic stream.);
select a vessel roadmap from the roadmap library based on comparing the first real- time fluoroscopy information with the first alignment data of each vessel roadmap of the roadmap library ([0150] “Roadmap selection represented by step 104 in FIG. 1 can be achieved by comparing the ECG signal associated with the online fluoroscopic image and the ECG of the offline angiographic sequence, such that the most suitable candidate roadmap is selected where the best match of the ECG signals is found. The selected roadmap has the same (or very similar) cardiac phase with the online X-ray fluoroscopic image, which compensates the difference of vessel shape and pose induced by cardiac motion.”);
overlay the real-time fluoroscopy image with the selected vessel roadmap image of the vessel roadmap library ([0187] “The dynamic roadmap that results from the cardiac matching and transformation of the roadmap is rendered and integrated as an overlay on the corresponding X-ray fluoroscopic or angiographic image data frame of the live angiographic image data image data stream of the patient.”);
align the vessel roadmap image of the selected vessel roadmap and the real-time fluoroscopy image based on the second alignment data and the real time second fluoroscopy information ([0184] “Within step 107, the location of the device (e.g. catheter tip) in current X-ray fluoroscopic frame, as a result of step 106, and the device location (e.g. catheter tip) from the selected roadmap frame as a result of step 105 is used to obtain a transformation function to align the selected roadmap with the current X-ray fluoroscopic image frame.”); and
provide guidance for a fluoroscopy object to the pathological vessel based on the selected vessel roadmap and the second fluoroscopy information (Walsum teaches that the purpose of the dynamic roadmaps is to provide guidance to the physician using a fluoroscopy object ([0116] “…in order to provide visual guidance to the physician to provide support for placement of one or more medical instruments inserted through a guiding catheter.” For more specific limitations, such as determining a path for the fluoroscopic object to the pathological vessel and displaying that path on a screen, see the 35 USC 103 rejection to claims 6-7.).
Regarding claim 16, Walsum teaches the non-transitory computer readable medium of claim 15, wherein generation of the vessel roadmap library includes:
recording, for each vessel roadmap image of the vessel roadmap library, one or more imaging parameters, the imaging parameters indicating one or more parameters associated with an imaging method used to obtain the vessel roadmap image (In [0031-0032], Walsum teaches that when using 2D roadmaps, it is advantageous to create the roadmaps and capture the live stream from the same angiography angle. When using 3D roadmaps, images from different angles are taken to generate the 3D roadmap. Thus, imaging viewpoints/angles must be saved as imaging parameters. Furthermore, Fig. 6 and [0126] teach tracking the contrast over multiple frames.).
Regarding claim 17, Walsum teaches the non-transitory computer readable medium of claim 16, wherein the one or more imaging parameters include at least one of an angiography angle and a contrast medium dosage (In [0031-0032], Walsum teaches that when using 2D roadmaps, it is advantageous to create the roadmaps and capture the live stream from the same angiography angle. When using 3D roadmaps, images from different angles are taken to generate the 3D roadmap. Thus, imaging viewpoints/angles must be saved as imaging parameters. Furthermore, Fig. 6 and [0126] teach tracking the contrast over multiple frames.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Walsum (US 2020/0222018 A1), further in view of Berg (Three-Dimensional Image Overlay to Assist Endovascular Procedures. Vascular Disease Management. 10(9) 179-184.).
Regarding claim 5, Walsum teaches the pathological vessel guidance method of claim 3, wherein the one or more imaging parameters includes at least an angiography angle used to obtain each vessel roadmap image (In [0031-0032], Walsum teaches that when using 2D roadmaps, it is advantageous to create the roadmaps and capture the live stream from the same angiography angle. When using 3D roadmaps, images from different angles are taken to generate the 3D roadmap. Thus, the angle used to capture the roadmaps is considered in Walsum’s methods.);
further comprising comparing the angiography angle with a fluoroscopy angle, the fluoroscopy angle being used to obtain the real-time fluoroscopy image ([0031] “For example, the three-dimensional roadmap can be transformed according to viewpoint (e.g., viewing angles) used to acquire the second image data. This allows the second image data to be acquired from viewpoints different from the first image data.” For 2D roadmaps, the angiography and fluoroscopy angles are the same. [0032] “In other embodiments, the roadmap selected in iii) can be a two-dimensional roadmap that is transformed to generate at least one two-dimensional dynamic roadmap for overlay on the second image data. In this case, the first image data and the second image data can be acquired from a common viewpoint.”).
Walsum teaches determining the angiography angle and fluoroscopy angle so that 3D roadmaps can be transformed and 2D roadmaps can be properly aligned at the same angle. However, Walsum does not teach comparing these angles to ensure that the difference is not greater than a threshold (although this process likely occurs implicitly). Rather, Walsum teaches detecting movement and drift by comparing the fluoroscopy angle of different frames of the same cardiac phase within the live fluoroscopy stream (See Step 2608 of Fig. 6 and the section starting at [0278].). Therefore, Walsum fails to teach when the angiography angle and the fluoroscopy angle differ by more than an angle difference threshold, the refraining from overlaying the real-time fluoroscopy image with the selected vessel roadmap image.
However, Berg teaches when the angiography angle and the fluoroscopy angle differ by more than an angle difference threshold, the refraining from overlaying the real-time fluoroscopy image with the selected vessel roadmap image (The viewing angle of the roadmap is the angiographic angle, and it is used to position the C-arm during the subsequent fluoroscopic live imaging. Thus, the angles are compared. [p. 180-181, Section 2] “The projection angles can be stored, and later recalled to automatically steer the C-arm to the predefined rotation/angulation.” Berg teaches receiving the stored angiographic angle and using that angle to guide the C-arm, thus determining the fluoroscopy angle [p. 180-181, Section 2]. Berg also teaches the process of guiding the C-arm so that the fluoroscopy stream can be overlaid with the roadmap. This “registration step” ensured that the fluoroscopy and angiographic angles are close enough so that the overlaid result is accurate.” [p. 181, Section 3(b)] “The registration step ensures that the fluoroscopy stream can be accurately superimposed on the 3D volume of the CTA.”).
Walsum and Berg are analogous in the art to the claimed invention because both teach methods of overlaying a vessel roadmap with a live fluoroscopy image feed to guide an operator during a coronary operation. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Walsum’s invention by continuously checking if the angiography angle and the fluoroscopy angle are within a threshold. This modification would further enhance Walsum’s method by allowing for the operator to plan a viewing angle ahead of time using the roadmap images (Berg [p. 180-181, Section 2] “In the planning step, the user can interact with the (segmented) CTA to find optimal working projections to be used during the intervention.”), and it would ensure a degree of accuracy when aligning the image stream and the roadmap (Berg [p. 181, Section 3(b)] “The registration step ensures that the fluoroscopy stream can be accurately superimposed on the 3D volume of the CTA.”).
Regarding claim 18, Walsum teaches the non-transitory computer readable medium of claim 17, wherein the one or more imaging parameters includes the angiography angle used to obtain each vessel roadmap image (In [0031-0032], Walsum teaches that when using 2D roadmaps, it is advantageous to create the roadmaps and capture the live stream from the same angiography angle. When using 3D roadmaps, images from different angles are taken to generate the 3D roadmap. Thus, the angle used to capture the roadmaps is considered in Walsum’s methods.);
wherein the instructions further comprise comparison of the angiography angle with a fluoroscopy angle, the fluoroscopy angle being used to obtain the real-time fluoroscopy image ([0031] “For example, the three-dimensional roadmap can be transformed according to viewpoint (e.g., viewing angles) used to acquire the second image data. This allows the second image data to be acquired from viewpoints different from the first image data.” For 2D roadmaps, the angiography and fluoroscopy angles are the same. [0032] “In other embodiments, the roadmap selected in iii) can be a two-dimensional roadmap that is transformed to generate at least one two-dimensional dynamic roadmap for overlay on the second image data. In this case, the first image data and the second image data can be acquired from a common viewpoint.”).
Walsum teaches determining the angiography angle and fluoroscopy angle so that 3D roadmaps can be transformed and 2D roadmaps can be properly aligned at the same angle. However, Walsum does not teach comparing these angles to ensure that the difference is not greater than a threshold (although this process likely occurs implicitly). Rather, Walsum teaches detecting movement and drift by comparing the fluoroscopy angle of different frames of the same cardiac phase within the live fluoroscopy stream (See Step 2608 of Fig. 6 and the section starting at [0278].). Therefore, Walsum fails to teach when the angiography angle and the fluoroscopy angle differ by more than an angle difference threshold, the refraining from overlaying the real-time fluoroscopy image with the selected vessel roadmap image.
However, Berg teaches when the angiography angle and the fluoroscopy angle differ by more than an angle difference threshold, the refraining from overlaying the real-time fluoroscopy image with the selected vessel roadmap image (The viewing angle of the roadmap is the angiographic angle, and it is used to position the C-arm during the subsequent fluoroscopic live imaging. Thus, the angles are compared. [p. 180-181, Section 2] “The projection angles can be stored, and later recalled to automatically steer the C-arm to the predefined rotation/angulation.” Berg teaches receiving the stored angiographic angle and using that angle to guide the C-arm, thus determining the fluoroscopy angle [p. 180-181, Section 2]. Berg also teaches the process of guiding the C-arm so that the fluoroscopy stream can be overlaid with the roadmap. This “registration step” ensured that the fluoroscopy and angiographic angles are close enough so that the overlaid result is accurate.” [p. 181, Section 3(b)] “The registration step ensures that the fluoroscopy stream can be accurately superimposed on the 3D volume of the CTA.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Walsum’s invention by continuously checking if the angiography angle and the fluoroscopy angle are within a threshold. This modification would further enhance Walsum’s method by allowing for the operator to plan a viewing angle ahead of time using the roadmap images (Berg [p. 180-181, Section 2] “In the planning step, the user can interact with the (segmented) CTA to find optimal working projections to be used during the intervention.”), and it would ensure a degree of accuracy when aligning the image stream and the roadmap (Berg [p. 181, Section 3(b)] “The registration step ensures that the fluoroscopy stream can be accurately superimposed on the 3D volume of the CTA.”).
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Walsum (US 2020/0222018 A1), further in view of Liao et al. (US 2008/0275467 A1), hereafter Liao.
Regarding claim 6, Walsum teaches the pathological vessel guidance method of claim 1. Furthermore, Walsum teaches displaying the vessel roadmap over the fluoroscopy image feed, so a doctor performing a medical operation could visualize the vessels and determine a path to guide his/her tool to a pathological vessel. However, Walsum does not teach that the medical system is able to determine and display a guiding path itself; thus, Walsum fails to teach wherein providing guidance for the fluoroscopy object to the pathological vessel includes determining a path from the fluoroscopy object to the pathological vessel based on the second real-time fluoroscopy information and vessel segmentation data.
However, Liao teaches wherein providing guidance for the fluoroscopy object to the pathological vessel ([0072] “…the user marks the target blood vessel 46 for the catheter or guidewire on the 3D images.”) includes determining a path from the fluoroscopy object to the pathological vessel (Fig. 3 shows a path determined between the main vessel 48 and an area of interest 46. This path is followed by a catheter or other tool to reach the target vessel 46.)
based on the second real-time fluoroscopy information (The real-time position of the tool in the vessel is the second fluoroscopy information. [0075-0076] “Given the 3D location of the tracked catheter or interventional device, an exemplary embodiment of the present invention is able to determine the distance between this location and the closest node in the selected path. If this distance is greater than a preset threshold (e.g., 5 mm or the maximum vessel diameter), an audible warning (e.g., an intermittent beeping sound) and/or a visual warning (e.g., flashing the catheter or device location via a small red colored sphere on the fluoroscopic image monitor), may be provided. With such a warning, the user may be able to return the tracked catheter or device back to the last branching point in order to follow the correct branch along the planned path.”) and vessel segmentation data (The vessel tree 40 is the vessel segmentation data. [0070] “FIG. 3… shows a vessel tree as would be available from a 3D angiographic medical image showing blood vessels, e.g., MRA, 3D X-ray Angiography, or CT angiography.”).
Walsum and Liao are analogous in the art to the claimed invention, because both teach methods of obtaining a vessel roadmap and overlaying the roadmap onto a fluoroscopic live feed for guiding a tool through a patient’s blood vessels. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Walsum’s invention by utilizing the shortest-path finding tools for determining and displaying a guiding path for the surgical tool, such as a catheter, to get to the pathological vessel. This modification would allow for easier navigation of the surgical tool by an operator amid many branching vessels visible in the roadmap ([Liao 0071] “Using the vessel tree 40 as a roadmap by overlaying it on 2D fluoroscopy or X-ray angiography may be confusing or unclear to the doctor, since the vessels have many branchings and may overcross each other when the images are viewed from a particular angle.” [Liao 0073] “Having this symbolic path overlaid on the 2D interventional images, such as X-ray angiography or fluoroscopy, helps the user navigate the catheter or guidewire and helps clarify where the branching points 50 are, and where the planned path 42 is, specially when there are overcrossing vessels, such as vessel 52.”). Furthermore, Walsum’s invention motivated this modification, since its purpose is to provide guidance to a physician ([Walsum 0116] “A visual representation of the dynamic roadmap is overlaid on the live fluoroscopic image stream of the X-ray fluoroscopic image sequence in order to provide visual guidance to the physician to provide support for placement of one or more medical instruments inserted through a guiding catheter.”).
Regarding claim 7, Walsum and Liao teach the pathological vessel guidance method of claim 6. Liao further teaches further comprising: displaying the path on a display of a medical imaging system to guide an operator operating the fluoroscopy object to the pathological vessel ([Liao 0073] “Having this symbolic path overlaid on the 2D interventional images, such as X-ray angiography or fluoroscopy, helps the user navigate the catheter or guidewire and helps clarify where the branching points 50 are,”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Walsum’s invention by determining a guiding path for the surgical tool, such as a catheter, to get to the pathological vessel and displaying the path to guide the operator of the tool. This modification would allow for easier navigation of the surgical tool by an operator amid many branching vessels visible in the roadmap ([Liao 0071] “Using the vessel tree 40 as a roadmap by overlaying it on 2D fluoroscopy or X-ray angiography may be confusing or unclear to the doctor, since the vessels have many branchings and may overcross each other when the images are viewed from a particular angle.” [Liao 0073] “Having this symbolic path overlaid on the 2D interventional images, such as X-ray angiography or fluoroscopy, helps the user navigate the catheter or guidewire and helps clarify where the branching points 50 are, and where the planned path 42 is, specially when there are overcrossing vessels, such as vessel 52.”).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Walsum (US 2020/0222018 A1), in view of Berg (Three-Dimensional Image Overlay to Assist Endovascular Procedures. Vascular Disease Management. 10(9) 179-184.), and further in view of Liao (US 2008/0275467 A1).
Regarding claim 19, Walsum and Berg teach the non-transitory computer readable medium of claim 18. Furthermore, Walsum teaches displaying the vessel roadmap over the fluoroscopy image feed, so a doctor performing a medical operation could visualize the vessels and determine a path to guide his/her tool to a pathological vessel. However, Walsum does not teach that the medical system is able to determine and display a guiding path itself; thus, Walsum fails to teach wherein provision of the guidance for the fluoroscopy object to the pathological vessel includes determination of a path from the fluoroscopy object to the pathological vessel based on the second real-time fluoroscopy information and vessel segmentation data.
However, Liao teaches wherein provision of the guidance for the fluoroscopy object to the pathological vessel ([0072] “…the user marks the target blood vessel 46 for the catheter or guidewire on the 3D images.”) includes determination of a path from the fluoroscopy object to the pathological vessel (Fig. 3 shows a path determined between the main vessel 48 and an area of interest 46. This path is followed by a catheter or other tool to reach the target vessel 46.)
based on the second real-time fluoroscopy information (The real-time position of the tool in the vessel is the second fluoroscopy information. [0075-0076] “Given the 3D location of the tracked catheter or interventional device, an exemplary embodiment of the present invention is able to determine the distance between this location and the closest node in the selected path. If this distance is greater than a preset threshold (e.g., 5 mm or the maximum vessel diameter), an audible warning (e.g., an intermittent beeping sound) and/or a visual warning (e.g., flashing the catheter or device location via a small red colored sphere on the fluoroscopic image monitor), may be provided. With such a warning, the user may be able to return the tracked catheter or device back to the last branching point in order to follow the correct branch along the planned path.”) and vessel segmentation data (The vessel tree 40 is the vessel segmentation data. [0070] “FIG. 3… shows a vessel tree as would be available from a 3D angiographic medical image showing blood vessels, e.g., MRA, 3D X-ray Angiography, or CT angiography.”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Walsum’s invention by utilizing the shortest-path finding tools for determining and displaying a guiding path for the surgical tool, such as a catheter, to get to the pathological vessel. This modification would allow for easier navigation of the surgical tool by an operator amid many branching vessels visible in the roadmap ([Liao 0071] “Using the vessel tree 40 as a roadmap by overlaying it on 2D fluoroscopy or X-ray angiography may be confusing or unclear to the doctor, since the vessels have many branchings and may overcross each other when the images are viewed from a particular angle.” [Liao 0073] “Having this symbolic path overlaid on the 2D interventional images, such as X-ray angiography or fluoroscopy, helps the user navigate the catheter or guidewire and helps clarify where the branching points 50 are, and where the planned path 42 is, specially when there are overcrossing vessels, such as vessel 52.”). Furthermore, Walsum’s invention motivated this modification, since its purpose is to provide guidance to a physician ([Walsum 0116] “A visual representation of the dynamic roadmap is overlaid on the live fluoroscopic image stream of the X-ray fluoroscopic image sequence in order to provide visual guidance to the physician to provide support for placement of one or more medical instruments inserted through a guiding catheter.”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Wagner et al. (US 11,087,464 B2) teaches a system and method for creating motion-compensated images so that static roadmaps can be displayed during coronary operations.
Auvray et al. (US 9583075) teaches methods for overlaying a coronary roadmap based on the current cardiac phase during a live fluoroscopy image stream.
Piayada et al. (Dynamic coronary roadmapping during percutaneous coronary intervention: a feasibility study. Eur J Med Res. 23(1):36. doi: 10.1186/s40001-018-0333-x.) teaches a feasibility study of dynamic coronary roadmap software during percutaneous coronary interventions.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC JAMES SHOEMAKER whose telephone number is (571)272-6605. The examiner can normally be reached Monday through Friday from 8am to 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner' s supervisor, JENNIFER MEHMOOD, can be reached at (571)272-2976. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Eric Shoemaker/
Patent Examiner
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights