DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement
2. The information disclosure statements (IDS) submitted on the following dates are in compliance with the provisions of 37 CFR 1.97 and are being considered by the Examiner: 03/14/2025.
Claim Rejections - 35 USC § 103
3. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1-7, 9-10 and 12-18 are rejected under 35 U.S.C. 103 as being unpatentable over Wagner et al., (“Wagner”) [US-2020/0410666-A1] in view of Spahn (“Spahn”) [US-2011/0235889-A1]
Regarding claim 1, Wagner discloses an X-ray imaging method (Wagner- ¶0010, at least discloses a method is provided for creating motion-adjusted images of a patient to guide an interventional medical procedure) comprising:
obtaining at least two X-ray images of a region to be depicted in an object (Wagner- ¶0011, at least discloses an x-ray source assembly coupled at one end and a x-ray detector array assembly […] The computer system is further programmed to generate a static roadmap of vasculature of the patient using the first plurality of images and the second plurality of images, generate a motion model of the patient using the first plurality of images and the second plurality of images, and control the x-ray source assembly and the x-ray detector array assembly to acquire a third plurality of images of the patient with an interventional medical device deployed within the patient; Figs. 2-3 and ¶0055-0058, at least disclose a process flow 200 for creating motion-adjusted images of a patient to guide an interventional medical procedure. The process 200 includes acquiring (e.g., via the x-ray imaging system 100) a plurality of images 202 having non-contrast enhanced vasculature, and acquiring a second plurality of images 204 from the patient having contrast enhanced vasculature […] the process 220 includes first acquiring non-contrast images 222 (e.g., mask images) and contrast enhanced images 224 of the patient during a full respiratory cycle […] Once the non-contrast images 222 and the contrast images 224 have been acquired and selected (e.g., utilizing only contrast images during the arterial phase), the process 220 generates subtracted images, which are formed by the subtraction of the contrast images 224 from the non-contrast images 222 […] The non-contrast images 222 and the contrast images 224 are also used to generate the vasculature motion model 230, which is a specific implementation of the motion model 208; ¶0061, at least discloses At step 252, the non-contrast image sequence can be defined as It N(x), and the contrast image sequence can be defined as It F(x). In both image sequence identifiers, t represents the acquisition time for a given image; ¶0123, at least discloses a connected component analysis can be performed to find all connected regions in the binarized volume. In some cases, only the largest region can be used to represent the vasculature [object]. For example, smaller regions can be manually parsed out and deleted, as they are typically noise and artifacts; Fig. 4 and ¶0124, at least discloses The guidance system 400 includes generating a respiratory motion model 414, which utilizes the difference in brightness between regions within fluoroscopy images, where the regions can include locations above and below the diaphragm […] the mean brightness in the x-dimension can be calculated for a given fluoroscopic image within the image set 404), wherein the at least two X-ray images correspond to different first acquisition timeframes (Wagner- Figs. 2-3 and ¶0055-0058, at least disclose a process flow 200 for creating motion-adjusted images of a patient to guide an interventional medical procedure. The process 200 includes acquiring (e.g., via the x-ray imaging system 100) a plurality of images 202 having non-contrast enhanced vasculature, and acquiring a second plurality of images 204 from the patient having contrast enhanced vasculature […] the process 220 includes first acquiring non-contrast images 222 (e.g., mask images) and contrast enhanced images 224 of the patient during a full respiratory cycle […] Once the non-contrast images 222 and the contrast images 224 have been acquired and selected (e.g., utilizing only contrast images during the arterial phase), the process 220 generates subtracted images, which are formed by the subtraction of the contrast images 224 from the non-contrast images 222 […] The non-contrast images 222 and the contrast images 224 are also used to generate the vasculature motion model 230, which is a specific implementation of the motion model 208; ¶0061, at least discloses At step 252, the non-contrast image sequence [timeframes] can be defined as It N(x), and the contrast image sequence can be defined as It F(x). In both image sequence identifiers, t represents the acquisition time [different first acquisition timeframes] for a given image) and represent a vascular structure of the object in different contrast-agent filling states (Wagner- Figs. 2-3 and ¶0055-0058, at least disclose a process flow 200 for creating motion-adjusted images of a patient to guide an interventional medical procedure. The process 200 includes acquiring (e.g., via the x-ray imaging system 100) a plurality of images 202 having non-contrast enhanced vasculature [vascular structure], and acquiring a second plurality of images 204 from the patient having contrast enhanced vasculature […] the process 220 includes first acquiring non-contrast images 222 (e.g., mask images) and contrast enhanced images 224 [different contrast-agent] of the patient during a full respiratory cycle […] Once the non-contrast images 222 and the contrast images 224 have been acquired and selected (e.g., utilizing only contrast images during the arterial phase), the process 220 generates subtracted images, which are formed by the subtraction of the contrast images 224 from the non-contrast images 222 […] The non-contrast images 222 and the contrast images 224 are also used to generate the vasculature motion model 230, which is a specific implementation of the motion model 208; Fig. 11 and ¶0090, at least disclose overlays of the ground-truth and estimated vessel masks for different respiratory states [different states]. Specifically, the first row of images within FIG. 11, are native image frames used to estimate breathing states for three different time frames);
generating a vessel mask, which represents the vascular structure, based on the at least two X-ray images (Wagner- ¶0041, at least disclose after acquiring the vascular morphology, as described above, and during a real-time imaging procedure, the curvilinear feature detection is applied to the real-time fluoroscopy image to determine the vessel mask to display. This creates a dynamic motion-adjusted vessel mask, which can be superimposed on the real-time fluoroscopic images that include the interventional medical device; ¶0057, at least discloses the process 220 includes first acquiring non-contrast images 222 (e.g., mask images) and contrast enhanced images 224 of the patient during a full respiratory cycle (e.g., starting at inhalation and completing at exhalation) […] the contrast enhanced images 224 can be acquired using a specific procedure, which ensures that the contrast enhanced images 224 are only acquired during the arterial phase, such that the vasculature of interest is filled with contrast and is well visible in all captured frames; ¶0084, at least discloses depending on the display choice (e.g., implemented via a user selection), the translation vector(s) for each pixel can be applied to a static roadmap of the vasculature to generate a dynamic vessel mask 288 to motion compensate the vasculature; ¶0098, at least discloses After training based on the dataset above, the neural network 300 was evaluated by comparing the segmentation results over the test dataset to a mask-subtraction-based segmentation algorithm. A corresponding mask image was created for each test image containing only the anatomic background, assuming no motion occurred between the acquisitions of mask and test image);
obtaining at least two live images of the region to be depicted, wherein the at least two live images correspond to different second acquisition timeframes (Wagner- Fig. 3 and ¶0059-0060, at least disclose The process 220 further includes acquiring a plurality of live images 234 that include an interventional medical instrument in the images […] The plurality of live images 234 are also used to extract the interventional medical instrument form the plurality of live images 234 […] The process 220 can generate a real-time display 242 of the live images 234 […] In the second display, the live images 234 and the image of the medical device are displayed, and the transformed roadmap 238 is the motion compensation applied to the static roadmap 228 to show patient movement; ¶0083, at least discloses The real-time system 280 starts with the acquiring of live images 282 (e.g., similar to live images 234). Then, a given live image within the live images 282 is optimized within the respiratory motion model 260 to analyze and extract curvilinear features to determine the respiratory state, as indicated by step 284; Fig. 20 and ¶0131-0132, at least disclose real-time fluoroscopy images 418 are acquired via the x-ray imaging system 100, which include the instrument acquired from different planes. Then, these images within the real-time fluoroscopy images 418 are processed to segment the instrument, and create a 3D volume of the instrument (e.g., at 420) […] for all subsequent frames (e.g., within the real-time fluoroscopy images 418), the instrument can be tracked using 3D to 2D registration; ¶0134, at least discloses the real-time fluoroscopy images 418 can be directed to the respiratory motion model 414 to determine the respiratory state r(t), indicated by reference numeral 422; Fig. 11 and ¶0090, at least disclose overlays of the ground-truth and estimated vessel masks for different respiratory states [different states]. Specifically, the first row of images within FIG. 11, are native image frames used to estimate breathing states for three different time frames); and
generating, for each live image of the at least two live images, a corresponding overlay image for displaying on a display device (Wagner- ¶0007, at least discloses The DSA vascular image can be overlaid or shown side by side with real-time fluoroscopic images, while the instruments are manipulated until the shape of the instrument coincides with the path of the desired vascular branch; ¶0041, at least disclose during a real-time imaging procedure, the curvilinear feature detection is applied to the real-time fluoroscopy image to determine the vessel mask to display. This creates a dynamic motion-adjusted vessel mask, which can be superimposed on the real-time fluoroscopic images that include the interventional medical device; Fig. 3 and ¶0060, at least disclose the live images 234 and the static roadmap 228 are displayed and the transformed roadmap 238 is the motion compensation of the image of the medical device, which is overlaid on the roadmap 238. In the second display, the live images 234 and the image of the medical device are displayed, and the transformed roadmap 238 is the motion compensation applied to the static roadmap 228 to show patient movement).
Wagner does not explicitly disclose generating a shared vessel mask, which represents the vascular structure, based on the at least two X-ray images; generating, for each live image of the at least two live images, depending on the shared vessel mask, a corresponding overlay image for displaying on a display device.
However, Spahn discloses
generating a shared vessel mask, which represents the vascular structure, based on the at least two X-ray images (Spahn- ¶0017, at least discloses a device for identifying a position displacement determines a displacement between the X-ray images and the projection images of the parts of the body that are not blood vessels. A display plays back the X-ray images and the projection image of the blood vessels; ¶0033, at least discloses The method proposes that firstly, in a first phase, X-ray images with pure anatomy are recorded during the system dose regulation phase and then, during a second phase, the fill phase, in which the vessels are filled with contrast agent, X-ray images are recorded, the mask image being produced from both sets of images; ¶0035, at least discloses c) displacing (35) the subtraction image (22) by at least one pixel in the x- and/or y-direction and subsequent summation in order to generate a modified vessel image (39) as a mask [vessel mask] having a substantially improved signal-to-noise ratio, d) segmenting (40) the vascular tree (12) in the modified vessel image (39) in order to generate a segmentation image (41), e) processing (42) the modified vessel image (39), the segmentation image (41) with vascular tree (12), and at least one native image (13) and/or further image (29, 13) in order to generate at least one composite image (43); ¶0047, at least discloses During the processing for generating at least one composite image the modified vessel image as the new mask image M′ and at least one further image as the fluoroscopic image series Fn can advantageously be overlaid with the aid of the binary information B of the segmentation image in a precisely targeted manner by merging in an overlay reference method; ¶0054, at least discloses According to the invention the imaging system additionally has a subtraction stage for subtracting the empty image and a mask image such that a subtraction image is obtained in which only the vascular tree can be seen, a displacement stage for displacing the subtraction image by at least one pixel in the x- and/or y-direction, an addition stage for subsequent summation of the displaced subtraction images for a modified vessel image as a mask [vessel mask] which has a substantially improved signal-to-noise ratio, a segmentation stage of the modified vessel image for generating a segmentation image with extracted vascular tree and a further image processing stage for the modified vessel image, the segmentation image with vascular tree, and at least one further image for generating at least one composite image; ¶0056, at least discloses Alternatively, in order to generate at least one composite image, the further image processing stage can overlay the modified vessel image as a new mask image M′ and at least one further image as fluoroscopic image series Fn with the aid of the binary information B of the segmentation image in a precisely targeted manner by merging; ¶0066-0067, at least discloses modified vessel image 39 is converted by means of a segmentation stage 40 into a binary image B or segmentation image 41. The segmentation in the segmentation stage 40 can be efficiently performed by threshold value formation, for example, since the signal, the vascular tree 12, and the noise are more strongly separated. Thus, a binary image can be generated in which e.g. the entry “0” stands for background and “1” for vascular tree [Wingdings font/0xE0] suggests a shared vessel mask);
generating, depending on the shared vessel mask, a corresponding overlay image for displaying on a display device (Spahn- ¶0047, at least discloses During the processing for generating at least one composite image the modified vessel image as the new mask image M′ and at least one further image as the fluoroscopic image series Fn can advantageously be overlaid with the aid of the binary information B of the segmentation image in a precisely targeted manner by merging in an overlay reference method; ¶0056, at least discloses Alternatively, in order to generate at least one composite image, the further image processing stage can overlay the modified vessel image as a new mask image M′ and at least one further image as fluoroscopic image series Fn with the aid of the binary information B of the segmentation image in a precisely targeted manner by merging).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Wagner to incorporate the teachings of Spahn, and apply the modified vessel image as the new mask image or the binary image into the Wagner’s teachings for generating a shared vessel mask, which represents the vascular structure, based on the at least two X-ray images; generating, for each live image of the at least two live images, depending on the shared vessel mask, a corresponding overlay image for displaying on a display device.
Doing so would provide an enhanced visualization of objects in interventional angiographic examination.
Regarding claim 2, Wagner in view of Spahn, discloses the X-ray imaging method of claim 1, and discloses the method further comprising:
obtaining at least one mask image of the region to be depicted (Wagner- Fig. 3 and ¶0057, at least discloses the process 220 includes first acquiring non-contrast images 222 (e.g., mask images) and contrast enhanced images 224 of the patient during a full respiratory cycle (e.g., starting at inhalation and completing at exhalation);
obtaining at least two contrast-agent images of the region to be depicted are obtained (Wagner- Fig. 3 and ¶0057, at least discloses the process 220 includes first acquiring non-contrast images 222 (e.g., mask images) and contrast enhanced images 224 of the patient during a full respiratory cycle (e.g., starting at inhalation and completing at exhalation), wherein the at least two contrast-agent images represent the vascular structure in the different contrast-agent filling states (Wagner- Figs. 2-3 and ¶0055, at least disclose The process 200 includes acquiring (e.g., via the x-ray imaging system 100) a plurality of images 202 having non-contrast enhanced vasculature, and acquiring a second plurality of images 204 from the patient having contrast enhanced vasculature. Both of these plurality of images can be acquired throughout a full breathing cycle (e.g., beginning during inhalation and completing after exhalation); ¶0057, at least discloses the process 220 includes first acquiring non-contrast images 222 (e.g., mask images) and contrast enhanced images 224 of the patient during a full respiratory cycle (e.g., starting at inhalation and completing at exhalation).); and
generating, for each contrast-agent image of the at least two contrast-agent images, a subtraction image based on the at least one mask image (Wagner- Fig. 3 and ¶0058, at least disclose Once the non-contrast images 222 and the contrast images 224 have been acquired and selected (e.g., utilizing only contrast images during the arterial phase), the process 220 generates subtracted images, which are formed by the subtraction of the contrast images 224 from the non-contrast images 222. These subtracted images can be used to generate a static roadmap 228 of the vasculature of the patient),
wherein the at least two X-ray images are given by the subtraction images (Wagner- Fig. 3 and ¶0055-0058, at least disclose The process 200 includes acquiring (e.g., via the x-ray imaging system 100) a plurality of images 202 having non-contrast enhanced vasculature, and acquiring a second plurality of images 204 from the patient having contrast enhanced vasculature […] Once the non-contrast images 222 and the contrast images 224 have been acquired and selected (e.g., utilizing only contrast images during the arterial phase), the process 220 generates subtracted images, which are formed by the subtraction of the contrast images 224 from the non-contrast images 222. These subtracted images can be used to generate a static roadmap 228 of the vasculature of the patient; Figs. 2-3 and ¶0055-0058, at least disclose a process flow 200 for creating motion-adjusted images of a patient to guide an interventional medical procedure. The process 200 includes acquiring (e.g., via the x-ray imaging system 100) a plurality of images 202 having non-contrast enhanced vasculature, and acquiring a second plurality of images 204 from the patient having contrast enhanced vasculature […] the process 220 includes first acquiring non-contrast images 222 (e.g., mask images) and contrast enhanced images 224 of the patient during a full respiratory cycle).
Regarding claim 3, Wagner in view of Spahn, discloses the X-ray imaging method of claim 2, and further discloses wherein the at least one mask image comprises at least two mask images, wherein the at least two mask images correspond to different further acquisition timeframes (Wagner- Fig. 3 and ¶0057, at least disclose the process 220 includes first acquiring non-contrast images 222 (e.g., mask images) and contrast enhanced images 224 of the patient during a full respiratory cycle (e.g., starting at inhalation and completing at exhalation); ¶0061, at least discloses At step 252, the non-contrast image sequence [timeframes] can be defined as It N(x), and the contrast image sequence can be defined as It F(x). In both image sequence identifiers, t represents the acquisition time [different acquisition timeframes] for a given image) ,
wherein, for each contrast-agent image of the at least two contrast-agent images (Wagner- Fig. 3 and ¶0057, at least disclose the process 220 includes first acquiring non-contrast images 222 (e.g., mask images) and contrast enhanced images 224 of the patient during a full respiratory cycle (e.g., starting at inhalation and completing at exhalation)), a corresponding summation image is generated by weighted summation of the at least two mask images (Spahn- ¶0035, at least discloses c) displacing (35) the subtraction image (22) by at least one pixel in the x- and/or y-direction and subsequent summation in order to generate a modified vessel image (39) as a mask having a substantially improved signal-to-noise ratio; ¶0054, at least discloses the imaging system additionally has a subtraction stage for subtracting the empty image and a mask image such that a subtraction image is obtained in which only the vascular tree can be seen, a displacement stage for displacing the subtraction image by at least one pixel in the x- and/or y-direction, an addition stage for subsequent summation of the displaced subtraction images for a modified vessel image as a mask which has a substantially improved signal-to-noise ratio, a segmentation stage of the modified vessel image for generating a segmentation image with extracted vascular tree and a further image processing stage for the modified vessel image, the segmentation image with vascular tree, and at least one further image for generating at least one composite image; Fig. 4 and ¶0064, at least disclose a roadmap method using double subtraction is employed and wherein, by displacement of the original mask by one or more pixels in the x- and/or y-direction and subsequent summation, a mask is produced which has a substantially improved signal-to-noise ratio and from which the vascular tree can be segmented; ¶0074, at least discloses the roadmap method, a first subtraction image 22 or mask image M is produced and, by means of the described method, a segmented image B (41) is generated by way of the segmentation stage by displacement (35) and summation (38) of the vessel images 37 displaced relative to one another in a modified vessel image 39 or a new mask M′; Fig. 6 and ¶0075, at least disclose a variant of a segmentation image 41 or binary image B described with reference to FIG. 4, having a softer transition between the vascular and the non-vascular region. In this case the vessels are assigned a weighting of w=2, the transition regions a weighting of w=1, and the remaining region a weighting of w=0), and
wherein, for each contrast-agent image of the at least two contrast-agent images, the associated subtraction image is generated by subtracting the associated summation image from the particular contrast-agent image (Wagner- Fig. 3 and ¶0057, at least disclose Once the non-contrast images 222 and the contrast images 224 have been acquired and selected (e.g., utilizing only contrast images during the arterial phase), the process 220 generates subtracted images, which are formed by the subtraction of the contrast images 224 from the non-contrast images 222. These subtracted images can be used to generate a static roadmap 228 of the vasculature of the patient).
It would have been obvious to one of ordinary in the art before the effective filing date of the claimed invention to have modified Wagner to incorporate the teachings of Spahn, and apply the weighted summation into the Wagner’s teachings for obtaining at least two contrast-agent images of the region to be depicted are obtained, wherein the at least two contrast-agent images represent the vascular structure in the different contrast-agent filling states; and generating, for each contrast-agent image of the at least two contrast-agent images, a subtraction image based on the at least one mask image, wherein the at least two X-ray images are given by the subtraction images.
The same motivation that was utilized in the rejection of claim 1 applies equally to this claim.
Regarding claim 4, Wagner in view of Spahn, discloses the X-ray imaging method of claim 1, and further discloses wherein the generating of the shared vessel mask (see Claim 1 rejection for detailed analysis) comprises applying a trained first machine-learning model to first input data, which depends on the at least two X-ray images (Wagner- ¶0093-0094, at least disclose machine learning based approaches have been proposed, which use classification approaches to identify small segments which are then combined using linear programming, or hierarchical shape models based on principle component analysis […] the previous machine learning approaches inputted a fluoroscopic image containing a medical instrument and outputted a disconnected and non-continuous image of the medical instrument; ¶0097, at least discloses In order to improve image quality, the neural network 300 can be trained by providing the current image, and the past two frames, as an input to the neural network 300. Advantageously, this allows the neural network 300 to learn to track moving parts in the image, which are often easier to detect then a static object due to noise. In other non-limiting examples, the neural network 300 can be trained by providing the current image, along with the past four frames).
Regarding claim 5, Wagner in view of Spahn, discloses the X-ray imaging method of claim 4, and further discloses wherein the trained first machine-learning model comprises a convolutional neural network or a transformer network (Wagner- Fig. 12 and ¶0094-0095, at least disclose The deep learning approach [transformer network] proposed in the present disclosure overcomes the problems associated with prior attempts […] The encoder 302 is followed generally by thirteen layers 308, where each layer 308 includes a convolutional layer [convolutional neural network], a batch normalization layer, and a rectifier linear unit (“ReLU”).).
5=12
Regarding claim 6, Wagner in view of Spahn, discloses the X-ray imaging method of claim 2, and discloses the method further comprising:
generating, for each X-ray image of the at least two X-ray images, a provisional vessel mask by applying a trained first machine-learning model to first input data, which depends on the particular X-ray image network (Wagner- Fig. 12 and ¶0097-0098, at least disclose the neural network 300 can be trained by providing the current image, and the past two frames, as an input to the neural network 300 […] After training based on the dataset above, the neural network 300 was evaluated by comparing the segmentation results over the test dataset to a mask-subtraction-based segmentation algorithm. A corresponding mask image was created for each test image containing only the anatomic background, assuming no motion occurred between the acquisitions of mask and test image; ¶0103-0104, at least disclose The neural network 300 can be trained using training images 316, which can be, for example, previously acquired fluoroscopy images that include the instrument intended to be used during the procedure […] Once the neural network 300 is prepared, real-time fluoroscopy images 326 are acquired (e.g., via the x-ray imaging system 100) as the guidewire is inserted into the patient (i.e., the procedure has begun) […] Correspondingly, a given image within the real-time fluoroscopy images 326 is directed to both the neural network 300 and the respiratory motion model 260. As discussed previously, the given image within the real-time fluoroscopy images 326 that is inputted into the neural network 300, will output a continuous and connected image of the guidewire 328, which can be displayed on a display 330 (e.g., similar to the display 150, the display 124, etc.)),
wherein the shared vessel mask is generated depending on the provisional vessel masks (Spahn- 0035, at least discloses c) displacing (35) the subtraction image (22) by at least one pixel in the x- and/or y-direction and subsequent summation in order to generate a modified vessel image (39) as a mask [vessel mask] having a substantially improved signal-to-noise ratio, d) segmenting (40) the vascular tree (12) in the modified vessel image (39) in order to generate a segmentation image (41), e) processing (42) the modified vessel image (39), the segmentation image (41) with vascular tree (12), and at least one native image (13) and/or further image (29, 13) in order to generate at least one composite image (43); ¶0047, at least discloses During the processing for generating at least one composite image the modified vessel image as the new mask image M′ [provisional vessel masks] and at least one further image as the fluoroscopic image series Fn can advantageously be overlaid with the aid of the binary information B of the segmentation image in a precisely targeted manner by merging in an overlay reference method; ¶0054, at least discloses According to the invention the imaging system additionally has a subtraction stage for subtracting the empty image and a mask image such that a subtraction image is obtained in which only the vascular tree can be seen, a displacement stage for displacing the subtraction image by at least one pixel in the x- and/or y-direction, an addition stage for subsequent summation of the displaced subtraction images for a modified vessel image as a mask [vessel mask] which has a substantially improved signal-to-noise ratio, a segmentation stage of the modified vessel image for generating a segmentation image with extracted vascular tree and a further image processing stage for the modified vessel image, the segmentation image with vascular tree, and at least one further image for generating at least one composite image; ¶0056, at least discloses Alternatively, in order to generate at least one composite image, the further image processing stage can overlay the modified vessel image as a new mask image M′ and at least one further image as fluoroscopic image series Fn with the aid of the binary information B of the segmentation image in a precisely targeted manner by merging; ¶0066-0067, at least discloses modified vessel image 39 is converted by means of a segmentation stage 40 into a binary image B or segmentation image 41. The segmentation in the segmentation stage 40 can be efficiently performed by threshold value formation, for example, since the signal, the vascular tree 12, and the noise are more strongly separated. Thus, a binary image can be generated in which e.g. the entry “0” stands for background and “1” for vascular tree [Wingdings font/0xE0] suggests a shared vessel mask).
Regarding claim 7, Wagner in view of Spahn, discloses the X-ray imaging method of claim 6, and further discloses wherein the shared vessel mask is selected as one of the provisional vessel masks according to a defined rule (Spahn- 0035, at least discloses c) displacing (35) the subtraction image (22) by at least one pixel in the x- and/or y-direction and subsequent summation in order to generate a modified vessel image (39) as a mask [vessel mask] having a substantially improved signal-to-noise ratio, d) segmenting (40) the vascular tree (12) in the modified vessel image (39) in order to generate a segmentation image (41), e) processing (42) the modified vessel image (39), the segmentation image (41) with vascular tree (12), and at least one native image (13) and/or further image (29, 13) in order to generate at least one composite image (43); ¶0047, at least discloses During the processing for generating at least one composite image the modified vessel image as the new mask image M′ [provisional vessel masks] and at least one further image as the fluoroscopic image series Fn can advantageously be overlaid with the aid of the binary information B of the segmentation image in a precisely targeted manner by merging in an overlay reference method; ¶0054, at least discloses According to the invention the imaging system additionally has a subtraction stage for subtracting the empty image and a mask image such that a subtraction image is obtained in which only the vascular tree can be seen, a displacement stage for displacing the subtraction image by at least one pixel in the x- and/or y-direction, an addition stage for subsequent summation of the displaced subtraction images for a modified vessel image as a mask [vessel mask] which has a substantially improved signal-to-noise ratio, a segmentation stage of the modified vessel image for generating a segmentation image with extracted vascular tree and a further image processing stage for the modified vessel image, the segmentation image with vascular tree, and at least one further image for generating at least one composite image; ¶0056, at least discloses Alternatively, in order to generate at least one composite image, the further image processing stage can overlay the modified vessel image as a new mask image M′ and at least one further image as fluoroscopic image series Fn with the aid of the binary information B of the segmentation image in a precisely targeted manner by merging; ¶0066-0067, at least discloses modified vessel image 39 is converted by means of a segmentation stage 40 into a binary image B or segmentation image 41. The segmentation in the segmentation stage 40 can be efficiently performed by threshold value formation, for example, since the signal, the vascular tree 12, and the noise are more strongly separated. Thus, a binary image can be generated in which e.g. the entry “0” stands for background and “1” for vascular tree [Wingdings font/0xE0] suggests a shared vessel mask), or
wherein the shared vessel mask is generated by applying a trained second machine-learning model to second input data, which depends on the provisional vessel masks.
Regarding claim 9, Wagner in view of Spahn, discloses the X-ray imaging method of claim 1, and discloses the method further comprising:
generating, for each X-ray image of the at least two X-ray images, a provisional vessel mask by applying a trained first machine-learning model to first input data, which depends on the particular X-ray image (Spahn- 0035, at least discloses c) displacing (35) the subtraction image (22) by at least one pixel in the x- and/or y-direction and subsequent summation in order to generate a modified vessel image (39) as a mask [vessel mask] having a substantially improved signal-to-noise ratio, d) segmenting (40) the vascular tree (12) in the modified vessel image (39) in order to generate a segmentation image (41); ¶0047, at least discloses During the processing for generating at least one composite image the modified vessel image as the new mask image M′ [provisional vessel masks] and at least one further image as the fluoroscopic image series Fn can advantageously be overlaid with the aid of the binary information B of the segmentation image in a precisely targeted manner by merging in an overlay reference method; ¶0054, at least discloses According to the invention the imaging system additionally has a subtraction stage for subtracting the empty image and a mask image such that a subtraction image is obtained in which only the vascular tree can be seen, a displacement stage for displacing the subtraction image by at least one pixel in the x- and/or y-direction, an addition stage for subsequent summation of the displaced subtraction images for a modified vessel image as a mask [vessel mask] which has a substantially improved signal-to-noise ratio, a segmentation stage of the modified vessel image for generating a segmentation image with extracted vascular tree and a further image processing stage for the modified vessel image, the segmentation image with vascular tree, and at least one further image for generating at least one composite image; ¶0066-0067, at least discloses modified vessel image 39 is converted by means of a segmentation stage 40 into a binary image B or segmentation image 41. The segmentation in the segmentation stage 40 can be efficiently performed by threshold value formation, for example, since the signal, the vascular tree 12, and the noise are more strongly separated. Thus, a binary image can be generated in which e.g. the entry “0” stands for background and “1” for vascular tree [Wingdings font/0xE0] suggests a shared vessel mask),
wherein the shared vessel mask is generated depending on the provisional vessel masks (Spahn- 0035, at least discloses c) displacing (35) the subtraction image (22) by at least one pixel in the x- and/or y-direction and subsequent summation in order to generate a modified vessel image (39) as a mask [vessel mask] having a substantially improved signal-to-noise ratio, d) segmenting (40) the vascular tree (12) in the modified vessel image (39) in order to generate a segmentation image (41); ¶0047, at least discloses During the processing for generating at least one composite image the modified vessel image as the new mask image M′ [provisional vessel masks] and at least one further image as the fluoroscopic image series Fn can advantageously be overlaid with the aid of the binary information B of the segmentation image in a precisely targeted manner by merging in an overlay reference method; ¶0054, at least discloses According to the invention the imaging system additionally has a subtraction stage for subtracting the empty image and a mask image such that a subtraction image is obtained in which only the vascular tree can be seen, a displacement stage for displacing the subtraction image by at least one pixel in the x- and/or y-direction, an addition stage for subsequent summation of the displaced subtraction images for a modified vessel image as a mask [vessel mask] which has a substantially improved signal-to-noise ratio, a segmentation stage of the modified vessel image for generating a segmentation image with extracted vascular tree and a further image processing stage for the modified vessel image, the segmentation image with vascular tree, and at least one further image for generating at least one composite image; ¶0066-0067, at least discloses modified vessel image 39 is converted by means of a segmentation stage 40 into a binary image B or segmentation image 41. The segmentation in the segmentation stage 40 can be efficiently performed by threshold value formation, for example, since the signal, the vascular tree 12, and the noise are more strongly separated. Thus, a binary image can be generated in which e.g. the entry “0” stands for background and “1” for vascular tree [Wingdings font/0xE0] suggests a shared vessel mask).
Regarding claim 10, Wagner in view of Spahn, discloses the X-ray imaging method of claim 9, and further discloses wherein the shared vessel mask is selected as one of the provisional vessel masks according to a defined rule (see Claim 7 rejection for detailed analysis), or
wherein the shared vessel mask is generated by applying a trained second machine-learning model to second input data, which depends on the provisional vessel masks.
Regarding claim 12, Wagner in view of Spahn, discloses the X-ray imaging method of claim 9, and further discloses wherein the trained first machine-learning model comprises a convolutional neural network or a transformer network (see Claim 5 rejection for detailed analysis).
Regarding claim 13, Wagner in view of Spahn, discloses the X-ray imaging method of claim 1, and further discloses wherein the at least two X-ray images are generated (see Claim 1 rejection for detailed analysis) during administration of a contrast agent into the vascular structure (Wagner- ¶0006, at least discloses a reference of the vascular system is created by injecting contrast agent to the vasculature region, acquiring a fluoroscopic image of the contrast enhanced vasculature, and displaying a static 2D digital subtraction angiography (“DSA”) on the contrast enhanced vasculature, prior to placing or manipulating the instruments; Fig. 3 and ¶0057, at least disclose process 220 includes a step, which automatically detects the arrival of contrast agent in the vasculature to determine the arterial, tissue, and venous phases of the injection; ¶0121, at least discloses In order to inject the contrast agent, an angiographic catheter (e.g., sized 5Fr) can be placed in the superior mesenteric artery under fluoroscopic image guidance, and the contrast agent (e.g., iodine) can be injected followed by a saline injection; Spahn- ¶0009, at least discloses interventional imaging with fluoroscopy or real-time X-ray imaging, wherein primarily the positioning of catheters, guide wires, balloon catheters, stents, etc. is carried out at a low X-ray dose, this method also being employed purely diagnostically in order to position a catheter for administration of contrast agent, and roadmapping, wherein, similarly to DSA, a mask, a native image with contrast-agent-filled vascular tree, is produced initially).
Regarding claim 14, Wagner in view of Spahn, discloses the X-ray imaging method of claim 1, and further discloses wherein the at least two X-ray images represent the vascular structure during different movement states of the object (Wagner- Fig. 16 and ¶0112-0113, at least disclose the vasculature in two different respiratory states. In the illustrated example, the same vasculature is shown in a first position in a first respiratory state 350 and a second position in a second respiratory state 352 […] the above-described systems and methods are able to perform motion compensation or motion adjustment across a full range of movement that occurs during time in a patient, including compression, expansion, deformation, translation, rotation; ¶0137, at least discloses The movement data (e.g., a translation matrix/3-element translation vector above) that relates to the spatial manipulation of the different portions of the vasculature from one respiratory state to another, can be used to generate a 3D motion compensated representation of the vasculature, or alternatively, a 3D motion compensated representation of the medical instrument. For example, if the respiratory state is known, then the movement data indicating how the vasculature should move is also known for that respiratory state. This movement data can then be applied to a 3D volume of the vasculature (e.g., at r(t) for a fully inhaled or exhaled state) to manipulate the 3D volume of the vasculature within the viewing plane, such that the 3D volume represents the realistic orientation of the vasculature at that respiratory state).
Regarding claim 15, Wagner in view of Spahn, discloses the X-ray imaging method of claim 1, and further discloses wherein at least one live image of the at least two live images (see Claim 1 rejection for detailed analysis) represents an instrument for vascular intervention in the region to be depicted (Wagner- ¶0059, at least discloses The process 220 further includes acquiring a plurality of live images 234 that include an interventional medical instrument in the images, and which are used to track the respiratory state 236, which is a specific form of motion tracking data (e.g., with regard to the process 200). The plurality of live images 234 are also used to extract the interventional medical instrument form the plurality of live images 234. Once extracted, the image of the medical instrument can be enhanced (e.g., compressed, elongated, changed in color, etc.) as indicated at step 240).
Regarding claim 16, Wagner in view of Spahn, discloses a data processing apparatus (Wagner- ¶0039, at least discloses systems and methods that display a static roadmap and the plurality of images to show an interventional medical device aligned on the static roadmap using a motion transformation; Fig. 1 and ¶0042-0055, at least disclose the C-arm x-ray imaging system 100) comprising:
at least one computing unit (Wagner- Fig. 1 and ¶0048, at least discloses The C-arm x-ray imaging system 100 also includes an operator workstation 122, which typically includes a display 124 […] and a computer processor 128. The computer processor 128 may include a commercially available programmable machine running a commercially available operating system) configured to perform the method of claim 1.
Regarding claim 17, Wagner in view of Spahn, discloses an X-ray imaging system (Wagner- Fig. 1 and ¶0042, at least disclose the C-arm x-ray imaging system 100) comprising:
an X-ray source and an X-ray detector (Wagner- Fig. 1 and ¶0043, at least disclose the C-arm x-ray imaging system 100 includes a gantry 102 having a C-arm to which an x-ray source assembly 104 is coupled on one end and an x-ray detector array assembly 106 is coupled at its other end. The gantry 102 enables the x-ray source assembly 104 and detector array assembly 106 to be oriented in different positions and angles around a subject 108, such as a medical patient or an object undergoing examination that is positioned on a table 110.) for generating at least two live images (see Claim 1 rejection for detailed analysis); and
a data processing apparatus having at least one computing unit (Wagner- Fig. 1 and ¶0048-0052, at least disclose The computer processor 128 may include a commercially available programmable machine running a commercially available operating system […] The DAS 144 samples data from the one or more x-ray detectors in the x-ray detector array assembly 106 and converts the data to digital signals for subsequent processing […] The image reconstruction system 130 may include a commercially available computer processor, or may be a highly parallel computer architecture, such as a system that includes multiple-core processors and massively parallel, high-density computing devices.) configured to perform the method of claim 1.
Regarding claim 18, Wagner in view of Spahn, discloses the X-ray imaging system of claim 17, and discloses the system further comprising:
the display device (Wagner- Fig. 1 and ¶0048, at least disclose The C-arm x-ray imaging system 100 also includes an operator workstation 122, which typically includes a display 124) configured to display the corresponding overlay image (Wagner- Figs. 1, 14 and ¶0104, at least disclose the real-time fluoroscopy images 322 can be displayed on a display 331 (e.g., similar to the display 150, the display 124, etc.), if the interventional radiologist desires […] Correspondingly, a given image within the real-time fluoroscopy images 326 is directed to both the neural network 300 and the respiratory motion model 260. As discussed previously, the given image within the real-time fluoroscopy images 326 that is inputted into the neural network 300, will output a continuous and connected image of the guidewire 328, which can be displayed on a display 330 (e.g., similar to the display 150, the display 124, etc.) […] The dynamic vessel mask 334 can be displayed on a display 336 (e.g., similar to the display 150, the display 124, etc.); Fig. 15 and ¶0108, at least disclose This motion compensated image of the medical instrument is overlaid with the static vasculature roadmap (e.g., also outputted from the vasculature motion model 230). This output from the first state is displayed on the display 346 (e.g., similar to the display 150, the display 124, etc.); Spahn- Fig. 1 and ¶0008, at least disclose The X-ray images can then be viewed on a monitor 9; ¶0041, at least discloses playing back the at least one composite image (33)).
Allowable Subject Matter
5. Claims 8 and 11 are objected to as being dependent upon a rejected base
claim, but would be allowable if rewritten in independent form including all of the
limitations of the base claim and any intervening claims.
6. The following is a statement of reasons for the indication of allowable subject matter:
Regarding Claim 8, the combination of prior arts teaches the method of Claim 1.
However in the context of claim 1, 2, 6 and 8 as a whole, the combination of prior arts does
not teach identifying a contrast-agent image of the contrast-agent images by comparing the
particular live image with the at least two contrast-agent images; registering, for the generating of the overlay image, the shared vessel mask with the particular live image by the registration instruction that corresponds to the identified contrast-agent image. Therefore, Claim 9 in the context of claim 1, 2, 6 as a whole does comprise allowable subject matter.
Regarding Claim 11, the combination of prior arts teaches the method of Claim 1.
However in the context of claim 1, 9 and 11 as a whole, the combination of prior arts does
not teach identifying an X-ray image of the at least two X-ray images by comparing the
particular live image with the at least two X-ray images; registering, for the generating of the overlay image, the shared vessel mask with the particular live image by the registration instruction that corresponds to the identified X-ray image. Therefore, Claim 11 in the context of claim 1, 9 as a whole does comprise allowable subject matter.
Conclusion
7. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. They are as recited in the attached PTO-892 form.
8. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL LE whose telephone number is (571)272-5330. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL LE/Primary Examiner, Art Unit 2614
Prosecution Insights