DETAILED ACTION
Notice of Pre-AIA or AIA Status
Claims 1-20 are pending in this application. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on non-statutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a non-statutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are provisionally rejected on the ground of non-statutory double patenting as being unpatentable over claims 1-20 of co-pending Application No. 18/971062 (reference application). Although the claims at issue are not identical, they are not patentably distinct from each other because they are both directed towards image registration and subtraction of an area of interest. The claimed inventions are co-extensive in scope especially in light of the dependent claims presented. This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Florent et al. (US Patent 2024/0164733, filed March 14, 2022), hereby referred to as “Florent”, in view of Dascal et al. (US Patent 2017/0140532, November 18, 2016), hereby referred to as “Dascal”.
Consider Claims 1 and 13 and 20.
Florent teaches:
1. A method for correcting a digital subtraction angiography image, the method comprising: / 13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the processor is caused to: / 20. A non-volatile computer readable storage medium comprising a computer program stored thereon, wherein when the computer program is executed by a processor, the processor is caused to: (Florent: abstract, The present invention relates to subtraction imaging. In order provide further improved accuracy of masking images, a device (10) for digital subtraction imaging is provided comprising an image data input (12), a data processor (14) and an output interface (16). 3D image data (22) of a region of interest of an object is received that comprises a 3D representation of the object based on a reconstruction from a plurality of 2D projection images. Further, a 2D live X-ray image (24) of the region of interest is received. The 3D image data and the 2D live X-ray image are registered, wherein a matching pose of the 3D image data corresponding to the 2D live X-ray image is determined. A digitally reconstructed radiography is computed from the 3D image data based on the determined matching pose to generate a 2D mask image. For the 2D mask image, current data related to the 2D live X-ray image is used to achieve an adapted 2D mask image, wherein the data related to 10 the 2D live X-ray image comprises 2D live acquisition parameters and/or data of the 2D live X-ray image. The generated adapted 2D mask is subtracted image from the 2D live X-ray image a digital image highlighting changes in the region of interest is provided. [0037]-[0047], Figures 1-3)
1. performing a first registration on a first scan image and a reference image to obtain a second scan image; / 13. perform a first registration on a first scan image and a reference image to obtain a second scan image; / 20. perform a first registration on a first scan image and a reference image to obtain a second scan image; (Florent: [0047] FIG. 3 shows basic steps of an example of a method 100 for digital subtraction imaging. The method 100 comprises the following steps: In a first step 102, 3D image data of a region of interest of an object is received. The 3D image data comprises a 3D representation of the object that is based on a reconstruction from a plurality of 2D projection images of the region of interest. In a second step 104, a 2D live X-ray image of the region of interest is received. In a third step 106, the 3D image data and the 2D live X-ray image are registered. A matching pose of the 3D image data corresponding to the 2D live X-ray image is determined.)
1. performing a second registration on a region of interest of the second scan image and a region of interest of the reference image to obtain a third scan image; / 13. perform a second registration on a region of interest of the second scan image and a region of interest of the reference image to obtain a third scan image; / 20. perform a second registration on a region of interest of the second scan image and a region of interest of the reference image to obtain a third scan image; (Florent: [0047], Figure 3, In a third step 106, the 3D image data and the 2D live X-ray image are registered. A matching pose of the 3D image data corresponding to the 2D live X-ray image is determined. In a fourth step 108, a digitally reconstructed radiography is computed from the 3D image data based on the determined matching pose to generate a 2D mask image. For the generating of the 2D mask image, current data related to the 2D live X-ray image is used to achieve an adapted 2D mask image, wherein the data related to the 2D live X-ray image comprises at least one of the group of 2D live acquisition parameters and data of the 2D live X-ray image. In a fifth step 110, the generated adapted 2D mask image is subtracted from the 2D live X-ray image. In a sixth step 112, a digital image highlighting changes in the region of interest is provided.)
1. calculating a buffer area between the third scan image and the second scan image, and performing transition processing on the buffer area, to obtain a processed buffer area image; / 13. calculate a buffer area between the third scan image and the second scan image, and perform transition processing on the buffer area, to obtain a processed buffer area image; / 20. calculate a buffer area between the third scan image and the second scan image, and perform transition processing on the buffer area, to obtain a processed buffer area image; (Examiner Note: for purposes of examination “buffer area” is being interpreted to be analogous in scope as a matching pose as it is also pertinent to the spatial arrangement of the features of interest and corresponds to data that is computed through a 3D-2D registration process Florent: [0097] In an example of the device, not shown in detail, the data processor 14 is configured to estimate a pose of the object in the 2D live X-ray image. For the registration of the 3D image data and the 2D live X-ray image, the data processor 14 is configured to align the 3D representation of the object of the 3D image data to the estimated pose. Alternatively or in addition, for the reconstruction of the projection data of the region of interest, the data processor 14 is configured to consider the estimated pose for the reconstruction. [0098] The term “pose” relates to a spatial arrangement of the imaging arrangement in relation to a subject to be imaged. The term pose thus relates to a spatial identification. [ 0099] In an example, the matching pose of 3D data corresponding to the 2D live image is determined from the 3D reconstructed data and the live image. In an example, this is provided as an example of a 3D-2D registration, with the possible constraint that this operation may ignore the presence of injected structures (vessels) in one or the other data source (usually in the 2D live data). [0101] In an example of the device, not shown in detail, for finding a matching pose the data processor 14 is configured to provide at least one of the group of: i) a pose estimation that is based on a system-based geometric registration; ii) a segmentation-based pose estimation; iii) a constraint that the presence of contrast injected vascular structures is ignored in the 2D live X-ray image and/or in the 3D image data; and iv) a generation of a plurality of digitally reconstructed radiographies for different projection directions of the 3D image data and use of the plurality of digitally reconstructed radiographies for identification of possible matching with the 2D live image.)
1. and fusing the second scan image, the third scan image, and the processed buffer area image to obtain a corrected scan image. / 13. and fuse the second scan image, the third scan image, and the processed buffer area image to obtain a corrected scan image. / 20. and fuse the second scan image, the third scan image, and the processed buffer area image to obtain a corrected scan image. (Florent: [0116] In an example, the 2D scatter estimation is using the angiographic image as input. [0117] In an example of the device, not shown in detail, as correction for the reconstruction of the 3D image data, the data processor 14 is configured to provide i) a virtual heel effect correction. In addition or alternatively, the data processor 14 is configured to provide ii) a virtual inverse detector gain correction. [0118] The heel effect correction and detector gain correction are used to create virtually a homogeneous X-ray intensity. Due to the heel effect, the emission intensity of X-rays is slightly different for every emission angle. The detector X-ray detection efficiency varies for every pixel, due to various factors like the internal detector composition. In the 2D live data, these effects may or may not be corrected. Furthermore, if a correction is applied the correction may be a simplification of the real effect, and thus not fully accurate. For the DRR generation these correction steps can be inversely applied if no correction is used on the 2D live data. Thus, in the DRR a virtual inhomogeneity can be created similar to the 2D live data. Alternatively, the heel effect and detector gain can be taken into account during DRR generation simulating the physical reality of the system, and then the simplified correction of the 2D live data is applied in order to achieve the same correction with limited accuracy is created on the DRR. [0119] In an example, for both effects, calibration data from the image reconstruction pipeline is used. [0120] In an example, if the digitally reconstructed radiography, after restoration, is not geometrically matching the live image, a 2D registration process may be provided. As an example, at least a part of out-of-plane motions are corrected by the adapting steps. The subsequent 2D registration process can thus produce good results, even in the presence of harsh patient movements. Subtraction can then be applied, producing a clean and improved digital subtraction angiography.)
Even if Florent does not specifically teach: “buffer area”
Dascal teaches:
1. A method for correcting a digital subtraction angiography image, the method comprising: / 13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the processor is caused to: / 20. A non-volatile computer readable storage medium comprising a computer program stored thereon, wherein when the computer program is executed by a processor, the processor is caused to: (Dascal: abstract, The disclosure relates generally to the field of vascular system and peripheral vascular system data collection, imaging, image processing and feature detection relating thereto. In part, the disclosure more specifically relates to methods for detecting position and size of contrast cloud in an x-ray image including with respect to a sequence of x-ray images during intravascular imaging. Methods of detecting and extracting metallic wires from x-ray images are also described herein such as guidewires used in coronary procedures. Further, methods for of registering vascular trees for one or more images, such as in sequences of x-ray images, are disclosed. In part, the disclosure relates to processing, tracking and registering angiography images and elements in such images. The registration can be performed relative to images from an intravascular imaging modality such as, for example, optical coherence tomography (OCT) or intravascular ultrasound (IVUS). [0027] In one embodiment, the method further includes co-registering the plurality of frames of angiography image data and the plurality of frames of optical coherence tomography data using a co-registration table, the co-registration table including angiography image frames, a plurality of per frame OCT time stamps, a plurality of per frame angiography time stamps, and optical coherence tomography image frames including a score measurement for each co-registered position. The method further includes displaying a stent representation in an OCT image and an angiography image in a user interface using the co-registration table and a computing device. The method further includes identifying a side branch in one or more OCT images or angiography images using the co-registration table and a user interface configured to display the side branch. [0028] In one embodiment, the method further includes displaying a plurality of cross-frame registered angiography images using a diagnostic system, where the plurality of cross-frame registered angiography images is selected from the set.)
1. performing a first registration on a first scan image and a reference image to obtain a second scan image; / 13. perform a first registration on a first scan image and a reference image to obtain a second scan image; / 20. perform a first registration on a first scan image and a reference image to obtain a second scan image; (Dascal: [0069] As an example of such error reducing methods and other angiography or peripheral vascular system imaging enhancements, several are discussed in detail herein. These embodiments relate to contrast cloud detection, extracting or identifying wires in frames of x-ray image data and tracking or registering features and devices relative to the vascular system including with respect to angled branches and bifurcations or guidewires. These embodiments reduce errors that can propagate through other registration processes and lead to additional errors and inaccuracies. Ultimately, such errors can preclude proper cross-frame registration between angiography frames and any co-registration of other imaging modalities with such angiography frames. The errors can also interfere with tracking and co-registering probe movements for probes that include one or more markers such as radiopaque markers. [0074] As a result, intravascular imaging technologies such as optical coherence tomography (OCT) and acoustic technologies such as intravascular ultrasound (IVUS) and others are also described herein. For example, such blood vessel imaging is used by physicians to diagnose, locate and treat blood vessel disease during interventions such as bypass surgery or stent placement. FIG. 1 shows an exemplary system 2 for implementing one or more embodiments of the invention that includes an x-ray imaging system 4 such as an angiography system. [0075] The data collection system 2 includes a noninvasive imaging system such as a nuclear magnetic resonance, x-ray, computer aided tomography, or other suitable noninvasive imaging technology indicated by system 4. As shown as a non-limiting example of such a noninvasive imaging system, an angiography system 4 such as suitable for generating cines is shown. The angiography system 4 can include a fluoroscopy system. Angiography system 4 is configured to noninvasively image the subject S such that frames of angiography data, typically in the form of frames of image data, are generated. This x-ray imaging occurs while a pullback procedure is performed using a probe such that a blood vessel in region R of subject S is imaged using angiography and one or more imaging technologies such as OCT or IVUS, for example. The imaging results of a non-invasive scan (left and right images in display 7) and intravascular imaging results such as from OCT or IVUS are shown in the middle panel of display 7. In addition to the display, the probe used to collect intravascular data can be disposable and connect to a patient interface unit or PIU as part of system 2.)
1. performing a second registration on a region of interest of the second scan image and a region of interest of the reference image to obtain a third scan image; / 13. perform a second registration on a region of interest of the second scan image and a region of interest of the reference image to obtain a third scan image; / 20. perform a second registration on a region of interest of the second scan image and a region of interest of the reference image to obtain a third scan image; (Dascal: [0082] In one embodiment, the data collection system 18 and the angiography system 4 have a shared clock or other timing signals configured to synchronize angiography video frame time stamps and OCT image frame time stamps. In one embodiment, angiography system 12 runs various image processing and feature detection and other software-based processes as shown by 15 a, 15 b and 15 c. In one embodiment, angiography system 12 runs various image processing and feature detection and other software-based processes as shown by 15 a, 15 b and 15 c. These processes can include contrast cloud detection processes, feature extraction processes, wire detection and feature extraction relative thereto, interframe registration processes, cross frame registration process and other processes, methods and steps as described herein. [0083] In general, software-based processes 15 a, 15 b and 15 c are designed to reduce errors in cross-frame registration and to perform other processes described herein such as detecting a feature in an x-ray image and flagging it for use or exclusion in subsequent processing steps. Thus, a contrast cloud can be detected and then flagged by such software processes so that the region of the cloud is not used for processes that will be negatively impacted by the positional uncertainty and noise in the region. [0092] The data collection system 2 can include one or more displays 7 to show angiography frames of data, an OCT frames, user interfaces for OCT and angiography data. The co-registration of angiography frames relative other angiography frames allows. The displays 7 can also show other controls and features of interest. [0093] The noninvasive image data generated using angiography image analysis and processing system 12 can be transmitted to, stored in, and processed by one or more servers or workstations which can be system 12 or system 18 as shown in FIG. 1. Intravascular image processing system 16 can be in electrical communication with the PIU and an image processing subsystem 18. The subsystem 18 includes various software modules to track marker positions and perform co-registration between intravascular image frames and x-ray image frames.)
1. calculating a buffer area between the third scan image and the second scan image, and performing transition processing on the buffer area, to obtain a processed buffer area image; / 13. calculate a buffer area between the third scan image and the second scan image, and perform transition processing on the buffer area, to obtain a processed buffer area image; / 20. calculate a buffer area between the third scan image and the second scan image, and perform transition processing on the buffer area, to obtain a processed buffer area image; (Dascal: [0075] The data collection system 2 includes a noninvasive imaging system such as a nuclear magnetic resonance, x-ray, computer aided tomography, or other suitable noninvasive imaging technology indicated by system 4. As shown as a non-limiting example of such a noninvasive imaging system, an angiography system 4 such as suitable for generating cines is shown. The angiography system 4 can include a fluoroscopy system. Angiography system 4 is configured to noninvasively image the subject S such that frames of angiography data, typically in the form of frames of image data, are generated. This x-ray imaging occurs while a pullback procedure is performed using a probe such that a blood vessel in region R of subject S is imaged using angiography and one or more imaging technologies such as OCT or IVUS, for example. The imaging results of a non-invasive scan (left and right images in display 7) and intravascular imaging results such as from OCT or IVUS are shown in the middle panel of display 7. In addition to the display, the probe used to collect intravascular data can be disposable and connect to a patient interface unit or PIU as part of system 2. [0076] The angiography system 4 is in communication with an angiography data storage and image management system 12, which can be implemented as a workstation or server in one embodiment. In one embodiment, the data processing relating to the collected angiography signal is performed directly on the detector of the angiography system 4. The images from system 4 are stored and managed by the angiography data storage and image management 12. In one embodiment, a subsystem, a server or workstation handle the functions of system 12. In one embodiment, the entire system 4 generates electromagnetic radiation, such as x-rays. The system 4 also receives such radiation after passing through the subject S. In turn, the data processing system 12 uses the signals from the angiography system 4 to image one or more regions of the subject S including region R. In one embodiment, system 12 and an intravascular system 18 are all part of one integrated system.)
1. and fusing the second scan image, the third scan image, and the processed buffer area image to obtain a corrected scan image. / 13. and fuse the second scan image, the third scan image, and the processed buffer area image to obtain a corrected scan image. / 20. and fuse the second scan image, the third scan image, and the processed buffer area image to obtain a corrected scan image. (Dascal: [0107] FIG. 3 illustrates a flowchart 100 of a method for detecting a contrast cloud for each input image. In step A1, an image is input into a system for processing for the detection of a contrast cloud. In step A2, the image is denoised to produce a smoother image. This step can be an optional step, but can improve the image and can be important for noisy x-ray images. If the image is one of better quality, step A2 can be skipped. FIG. 4 illustrates an exemplary image of an x-ray image 150 of a vessel 170 with a contrast agent 160 region after image denoising has been performed. Knowing the location of cloud 160 allows the improvement of the consistency and stability of vessel centerlines which leads to improved accuracy of the co-registration and tracking of the marker of the probe. [0108]-[0111], [0112] In steps B1-B4, optional processing steps can be used for fusing cloud masks from multiple images. In step B1, cloud masks from multiple images are used together to create a single fused mask. A pixel-wise OR operator can be used to obtain a merged contrast cloud mask incorporating information from multiple x-ray frames, in step B2. After obtaining the merged mask, another component-based filter can be used to remove small components or components that are out of the region of interest in step B3. The use of multiple x-ray frames is advantageous given the expansion and dispersal of the cloud over a time period following the contrast solutions initial delivery. [0113] In step B4, the cloud masks from each frame can be fused, as illustrated in FIG. 8 which illustrates an exemplary image 160 of a fused contrast cloud mask 410 from an x-ray image sequence. This mask can be generated and applied to the image to identify the contrast cloud regions, which may include a buffer zone around them as a safety factor. These identified contrast cloud regions can then be stored in memory as ignore/avoid regions when performing additional image processing. In one embodiment, the fused cloud mask of FIG. 8 is derived by taking a pixel wise OR between multiple cloud masks.)
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify Florent’s method and system for subtraction imaging to use Dascal’s algorithm for x-ray image feature detection and registration as they are both directed towards the same field of endeavor for medical image analysis and processing. The determination of obviousness is predicated upon the following findings: One skilled in the art would have been motivated to modify Florent in order to improve the digital subtraction and image masking method and system by including improved algorithms for feature detection and image registration in order to improve the overall accuracy of the image subtraction process. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface or programming techniques, without changing a “fundamental” operating principle of Florent, while the teaching of Dascal continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of processing, tracking and registering features of interest in medical imaging analysis and reconstruction. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Consider Claims 2 and 14.
The combination of Florent and Dascal teaches:
2. The method of claim 1, wherein the first scan image comprises a first mask image, the second scan image comprises a second mask image, the third scan image comprises a third mask image, the reference image comprises a contrast image, and the corrected scan image comprises a corrected mask image. / 14. The computer device of claim 13, wherein the first scan image comprises a first mask image, the second scan image comprises a second mask image, the third scan image comprises a third mask image, the reference image comprises a contrast image, and the corrected scan image comprises a corrected mask image. (Florent: [0116] In an example, the 2D scatter estimation is using the angiographic image as input. [0117] In an example of the device, not shown in detail, as correction for the reconstruction of the 3D image data, the data processor 14 is configured to provide i) a virtual heel effect correction. In addition or alternatively, the data processor 14 is configured to provide ii) a virtual inverse detector gain correction. [0118] The heel effect correction and detector gain correction are used to create virtually a homogeneous X-ray intensity. Due to the heel effect, the emission intensity of X-rays is slightly different for every emission angle. The detector X-ray detection efficiency varies for every pixel, due to various factors like the internal detector composition. In the 2D live data, these effects may or may not be corrected. Furthermore, if a correction is applied the correction may be a simplification of the real effect, and thus not fully accurate. For the DRR generation these correction steps can be inversely applied if no correction is used on the 2D live data. Thus, in the DRR a virtual inhomogeneity can be created similar to the 2D live data. Alternatively, the heel effect and detector gain can be taken into account during DRR generation simulating the physical reality of the system, and then the simplified correction of the 2D live data is applied in order to achieve the same correction with limited accuracy is created on the DRR. [0119] In an example, for both effects, calibration data from the image reconstruction pipeline is used. [0120] In an example, if the digitally reconstructed radiography, after restoration, is not geometrically matching the live image, a 2D registration process may be provided. As an example, at least a part of out-of-plane motions are corrected by the adapting steps. The subsequent 2D registration process can thus produce good results, even in the presence of harsh patient movements. Subtraction can then be applied, producing a clean and improved digital subtraction angiography. Dascal: [0107] FIG. 3 illustrates a flowchart 100 of a method for detecting a contrast cloud for each input image. In step A1, an image is input into a system for processing for the detection of a contrast cloud. In step A2, the image is denoised to produce a smoother image. This step can be an optional step, but can improve the image and can be important for noisy x-ray images. If the image is one of better quality, step A2 can be skipped. FIG. 4 illustrates an exemplary image of an x-ray image 150 of a vessel 170 with a contrast agent 160 region after image denoising has been performed. Knowing the location of cloud 160 allows the improvement of the consistency and stability of vessel centerlines which leads to improved accuracy of the co-registration and tracking of the marker of the probe. [0108]-[0111], [0112] In steps B1-B4, optional processing steps can be used for fusing cloud masks from multiple images. In step B1, cloud masks from multiple images are used together to create a single fused mask. A pixel-wise OR operator can be used to obtain a merged contrast cloud mask incorporating information from multiple x-ray frames, in step B2. After obtaining the merged mask, another component-based filter can be used to remove small components or components that are out of the region of interest in step B3. The use of multiple x-ray frames is advantageous given the expansion and dispersal of the cloud over a time period following the contrast solutions initial delivery. [0113] In step B4, the cloud masks from each frame can be fused, as illustrated in FIG. 8 which illustrates an exemplary image 160 of a fused contrast cloud mask 410 from an x-ray image sequence. This mask can be generated and applied to the image to identify the contrast cloud regions, which may include a buffer zone around them as a safety factor. These identified contrast cloud regions can then be stored in memory as ignore/avoid regions when performing additional image processing. In one embodiment, the fused cloud mask of FIG. 8 is derived by taking a pixel wise OR between multiple cloud masks.)
Consider Claims 3 and 4.
The combination of Florent and Dascal teaches:
3. The method of claim 2, wherein the first registration comprises at least one of: translating and/or rotating the first mask image, performing an affine transformation on the first mask image, or performing an elastic registration on the first mask image and the contrast image. / 4. The method of claim 2, wherein the second registration comprises at least one of: translating and/or rotating the second mask image, performing an affine transformation on the second mask image, or performing an elastic registration on the second mask image and the contrast image. (Florent: [0100] For pose estimation, numerous computer vision or machine-learning methods are provided to solve this task. In an example, the reconstruction operation is adapted and external radio-opaque markers implanted on the patient are used to make this operation faster or more robust. For example, markers are present in both the 3D and the 2D live data, and a geometrical transform matching their projections from 3D to their life locations in 2D determines the targeted pose. [0101] In an example of the device, not shown in detail, for finding a matching pose the data processor 14 is configured to provide at least one of the group of: i) a pose estimation that is based on a system-based geometric registration; ii) a segmentation-based pose estimation; iii) a constraint that the presence of contrast injected vascular structures is ignored in the 2D live X-ray image and/or in the 3D image data; and iv) a generation of a plurality of digitally reconstructed radiographies for different projection directions of the 3D image data and use of the plurality of digitally reconstructed radiographies for identification of possible matching with the 2D live image. [0103] n an example, for the matching pose, radio-opaque markers are provided attached to the subject, which markers are present in both the 3D image data and the 2D live X-ray image. A geometrical transform that is matching their projections from 3D to their locations in 2D determines the targeted pose. [0106] In an example, the registering of the 3D image data and the 2D live image is provided dependent of the target anatomy. For rigid anatomic structures, a rigid transformation is provided. For non-rigid structures, an elastic deformation is provided.)
Consider Claims 5 and 15.
The combination of Florent and Dascal teaches:
5. The method of claim 2, wherein calculating the buffer area between the third scan image and the second scan image comprises: calculating the buffer area between the third mask image and the second mask image based on an artifact degree of a boundary between the third mask image and the second mask image./ 15. The computer device of claim 14, wherein calculating the buffer area between the third scan image and the second scan image comprises: calculating the buffer area between the third mask image and the second mask image based on an artifact degree of a boundary between the third mask image and the second mask image.(Examiner Note: for purposes of examination “buffer area” is being interpreted to be analogous in scope as a matching pose as it is also pertinent to the spatial arrangement of the features of interest and corresponds to data that is computed through a 3D-2D registration process Florent: [0097] In an example of the device, not shown in detail, the data processor 14 is configured to estimate a pose of the object in the 2D live X-ray image. For the registration of the 3D image data and the 2D live X-ray image, the data processor 14 is configured to align the 3D representation of the object of the 3D image data to the estimated pose. Alternatively or in addition, for the reconstruction of the projection data of the region of interest, the data processor 14 is configured to consider the estimated pose for the reconstruction. [0098] The term “pose” relates to a spatial arrangement of the imaging arrangement in relation to a subject to be imaged. The term pose thus relates to a spatial identification. [0099] In an example, the matching pose of 3D data corresponding to the 2D live image is determined from the 3D reconstructed data and the live image. In an example, this is provided as an example of a 3D-2D registration, with the possible constraint that this operation may ignore the presence of injected structures (vessels) in one or the other data source (usually in the 2D live data). [0101] In an example of the device, not shown in detail, for finding a matching pose the data processor 14 is configured to provide at least one of the group of: i) a pose estimation that is based on a system-based geometric registration; ii) a segmentation-based pose estimation; iii) a constraint that the presence of contrast injected vascular structures is ignored in the 2D live X-ray image and/or in the 3D image data; and iv) a generation of a plurality of digitally reconstructed radiographies for different projection directions of the 3D image data and use of the plurality of digitally reconstructed radiographies for identification of possible matching with the 2D live image. Dascal: [0075] The data collection system 2 includes a noninvasive imaging system such as a nuclear magnetic resonance, x-ray, computer aided tomography, or other suitable noninvasive imaging technology indicated by system 4. As shown as a non-limiting example of such a noninvasive imaging system, an angiography system 4 such as suitable for generating cines is shown. The angiography system 4 can include a fluoroscopy system. Angiography system 4 is configured to noninvasively image the subject S such that frames of angiography data, typically in the form of frames of image data, are generated. This x-ray imaging occurs while a pullback procedure is performed using a probe such that a blood vessel in region R of subject S is imaged using angiography and one or more imaging technologies such as OCT or IVUS, for example. The imaging results of a non-invasive scan (left and right images in display 7) and intravascular imaging results such as from OCT or IVUS are shown in the middle panel of display 7. In addition to the display, the probe used to collect intravascular data can be disposable and connect to a patient interface unit or PIU as part of system 2. [0076] The angiography system 4 is in communication with an angiography data storage and image management system 12, which can be implemented as a workstation or server in one embodiment. In one embodiment, the data processing relating to the collected angiography signal is performed directly on the detector of the angiography system 4. The images from system 4 are stored and managed by the angiography data storage and image management 12. In one embodiment, a subsystem, a server or workstation handle the functions of system 12. In one embodiment, the entire system 4 generates electromagnetic radiation, such as x-rays. The system 4 also receives such radiation after passing through the subject S. In turn, the data processing system 12 uses the signals from the angiography system 4 to image one or more regions of the subject S including region R. In one embodiment, system 12 and an intravascular system 18 are all part of one integrated system.)
Consider Claims 6.
The combination of Florent and Dascal teaches:
6. The method of claim 2, wherein calculating the buffer area between the third scan image and the second scan image comprises: calculating the buffer area between the third mask image and the second mask image based on a motion amplitude of the region of interest. (Florent: [0117], [0118] The heel effect correction and detector gain correction are used to create virtually a homogeneous X-ray intensity. Due to the heel effect, the emission intensity of X-rays is slightly different for every emission angle. The detector X-ray detection efficiency varies for every pixel, due to various factors like the internal detector composition. In the 2D live data, these effects may or may not be corrected. Furthermore, if a correction is applied the correction may be a simplification of the real effect, and thus not fully accurate. For the DRR generation these correction steps can be inversely applied if no correction is used on the 2D live data. Thus, in the DRR a virtual inhomogeneity can be created similar to the 2D live data. Alternatively, the heel effect and detector gain can be taken into account during DRR generation simulating the physical reality of the system, and then the simplified correction of the 2D live data is applied in order to achieve the same correction with limited accuracy is created on the DRR. [0119] In an example, for both effects, calibration data from the image reconstruction pipeline is used. [0120] In an example, if the digitally reconstructed radiography, after restoration, is not geometrically matching the live image, a 2D registration process may be provided. As an example, at least a part of out-of-plane motions are corrected by the adapting steps. The subsequent 2D registration process can thus produce good results, even in the presence of harsh patient movements. Subtraction can then be applied, producing a clean and improved digital subtraction angiography. Dascal: [0129] A method for registering vascular trees extracted from different contrast enhanced x-ray frames, for example during x-ray angiography is described herein. In one embodiment, the vessel centerlines are known or treated as known for each of the vascular branches of interest. The process of detecting such centerlines can be performed using various methods as described in U.S. Pat. No. 9,351,698, the disclosure of which is incorporated by reference herein in its entirety. In one embodiment, a registration method described herein uses bifurcation points (if they exist) and the bending points (if they exist) as “anchors” for matching anatomical positions between different frames. Once the set of matching “anchor points” is obtained, the registration is based on interpolating for matching positions based on relative geodesic distance as measured along the arc-length of the centerlines. Various distance metrics can be used as appropriate. [0130] Furthermore, the “anchors points” fusion can be used to generate an estimation of the three-dimensional cardiac motion and deformation. Using a pair of angiographic images (2D projections), from the same cardiac phase, one can obtain three-dimensional reconstruction of the tree vessels. These 3-D vessels structures reconstructions at multiple phases of the cardiac cycle are of interest for 3D heart motion understanding. The displacements of these anchor points on each view along the image sequences induce a way of computing the motion estimation in the 3D vascular structure. Additional details relating to methods of performing “anchor points” matching for interframe registration of vascular trees are described below and otherwise herein. [0131] FIG. 14 shows an exemplary process flow 630 suitable for registering points associated with a cardiac system such as vascular trees between a first and a second angiography frame. This process flow can be used to detect anatomical features and use them for cross-frame/interframe registration. Cross-frame registration can be also accomplished by other anatomical features or anatomical landmarks found along the vessel. The method 630 can be used to perform interframe registration of vascular trees as shown.)
Consider Claims 7.
The combination of Florent and Dascal teaches:
7. The method of claim 1, wherein performing transition processing on the buffer area comprises compressing or stretching positions of pixel points between an outer boundary of the buffer area and a boundary of the region of interest of the second scan image. (Florent: [0117], [0118] The heel effect correction and detector gain correction are used to create virtually a homogeneous X-ray intensity. Due to the heel effect, the emission intensity of X-rays is slightly different for every emission angle. The detector X-ray detection efficiency varies for every pixel, due to various factors like the internal detector composition. In the 2D live data, these effects may or may not be corrected. Furthermore, if a correction is applied the correction may be a simplification of the real effect, and thus not fully accurate. For the DRR generation these correction steps can be inversely applied if no correction is used on the 2D live data. Thus, in the DRR a virtual inhomogeneity can be created similar to the 2D live data. Alternatively, the heel effect and detector gain can be taken into account during DRR generation simulating the physical reality of the system, and then the simplified correction of the 2D live data is applied in order to achieve the same correction with limited accuracy is created on the DRR. [0119] In an example, for both effects, calibration data from the image reconstruction pipeline is used. [0120] In an example, if the digitally reconstructed radiography, after restoration, is not geometrically matching the live image, a 2D registration process may be provided. As an example, at least a part of out-of-plane motions are corrected by the adapting steps. The subsequent 2D registration process can thus produce good results, even in the presence of harsh patient movements. Subtraction can then be applied, producing a clean and improved digital subtraction angiography. Dascal: [0129] A method for registering vascular trees extracted from different contrast enhanced x-ray frames, for example during x-ray angiography is described herein. In one embodiment, the vessel centerlines are known or treated as known for each of the vascular branches of interest. The process of detecting such centerlines can be performed using various methods as described in U.S. Pat. No. 9,351,698, the disclosure of which is incorporated by reference herein in its entirety. In one embodiment, a registration method described herein uses bifurcation points (if they exist) and the bending points (if they exist) as “anchors” for matching anatomical positions between different frames. Once the set of matching “anchor points” is obtained, the registration is based on interpolating for matching positions based on relative geodesic distance as measured along the arc-length of the centerlines. Various distance metrics can be used as appropriate. [0130] Furthermore, the “anchors points” fusion can be used to generate an estimation of the three-dimensional cardiac motion and deformation. Using a pair of angiographic images (2D projections), from the same cardiac phase, one can obtain three-dimensional reconstruction of the tree vessels. These 3-D vessels structures reconstructions at multiple phases of the cardiac cycle are of interest for 3D heart motion understanding. The displacements of these anchor points on each view along the image sequences induce a way of computing the motion estimation in the 3D vascular structure. Additional details relating to methods of performing “anchor points” matching for interframe registration of vascular trees are described below and otherwise herein. [0131] FIG. 14 shows an exemplary process flow 630 suitable for registering points associated with a cardiac system such as vascular trees between a first and a second angiography frame. This process flow can be used to detect anatomical features and use them for cross-frame/interframe registration. Cross-frame registration can be also accomplished by other anatomical features or anatomical landmarks found along the vessel. The method 630 can be used to perform interframe registration of vascular trees as shown.)
Consider Claims 8.
The combination of Florent and Dascal teaches:
8. The method of claim 2, wherein performing transition processing on the buffer area comprises: acquiring matching point pairs of a plurality of pixel points in the buffer area before and after performing the second registration; and performing transition processing on the buffer area based on coordinates of the matching point pairs. (Florent: [0117], [0118] The heel effect correction and detector gain correction are used to create virtually a homogeneous X-ray intensity. Due to the heel effect, the emission intensity of X-rays is slightly different for every emission angle. The detector X-ray detection efficiency varies for every pixel, due to various factors like the internal detector composition. In the 2D live data, these effects may or may not be corrected. Furthermore, if a correction is applied the correction may be a simplification of the real effect, and thus not fully accurate. For the DRR generation these correction steps can be inversely applied if no correction is used on the 2D live data. Thus, in the DRR a virtual inhomogeneity can be created similar to the 2D live data. Alternatively, the heel effect and detector gain can be taken into account during DRR generation simulating the physical reality of the system, and then the simplified correction of the 2D live data is applied in order to achieve the same correction with limited accuracy is created on the DRR. [0119] In an example, for both effects, calibration data from the image reconstruction pipeline is used. [0120] In an example, if the digitally reconstructed radiography, after restoration, is not geometrically matching the live image, a 2D registration process may be provided. As an example, at least a part of out-of-plane motions are corrected by the adapting steps. The subsequent 2D registration process can thus produce good results, even in the presence of harsh patient movements. Subtraction can then be applied, producing a clean and improved digital subtraction angiography. Dascal: [0129] A method for registering vascular trees extracted from different contrast enhanced x-ray frames, for example during x-ray angiography is described herein. In one embodiment, the vessel centerlines are known or treated as known for each of the vascular branches of interest. The process of detecting such centerlines can be performed using various methods as described in U.S. Pat. No. 9,351,698, the disclosure of which is incorporated by reference herein in its entirety. In one embodiment, a registration method described herein uses bifurcation points (if they exist) and the bending points (if they exist) as “anchors” for matching anatomical positions between different frames. Once the set of matching “anchor points” is obtained, the registration is based on interpolating for matching positions based on relative geodesic distance as measured along the arc-length of the centerlines. Various distance metrics can be used as appropriate. [0130] Furthermore, the “anchors points” fusion can be used to generate an estimation of the three-dimensional cardiac motion and deformation. Using a pair of angiographic images (2D projections), from the same cardiac phase, one can obtain three-dimensional reconstruction of the tree vessels. These 3-D vessels structures reconstructions at multiple phases of the cardiac cycle are of interest for 3D heart motion understanding. The displacements of these anchor points on each view along the image sequences induce a way of computing the motion estimation in the 3D vascular structure. Additional details relating to methods of performing “anchor points” matching for interframe registration of vascular trees are described below and otherwise herein. [0131] FIG. 14 shows an exemplary process flow 630 suitable for registering points associated with a cardiac system such as vascular trees between a first and a second angiography frame. This process flow can be used to detect anatomical features and use them for cross-frame/interframe registration. Cross-frame registration can be also accomplished by other anatomical features or anatomical landmarks found along the vessel. The method 630 can be used to perform interframe registration of vascular trees as shown.)
Consider Claims 9 and 17.
The combination of Florent and Dascal teaches:
9. The method of claim 8, wherein acquiring matching point pairs of the plurality of pixel points in the buffer area before and after performing the second registration comprises: dividing the buffer area to obtain a plurality of sub-areas; and acquiring a matching point pair of a target point of a sub-area, based on an offset of the target point of the sub-area after performing the second registration, the matching point pair comprising the target point of the sub-area before performing the second registration and the target point of the sub- area after performing the second registration. / 17. The computer device of claim 16, wherein acquiring matching point pairs of the plurality of pixel points in the buffer area before and after performing the second registration comprises: dividing the buffer area to obtain a plurality of sub-areas; and acquiring a matching point pair of a target point of a sub-area, based on an offset of the target point of the sub-area after performing the second registration, sub-area the matching point pair comprising the target point of the sub-area before performing the second registration and the target point of the sub-area after performing the second registration. (Florent: [0117], [0118] The heel effect correction and detector gain correction are used to create virtually a homogeneous X-ray intensity. Due to the heel effect, the emission intensity of X-rays is slightly different for every emission angle. The detector X-ray detection efficiency varies for every pixel, due to various factors like the internal detector composition. In the 2D live data, these effects may or may not be corrected. Furthermore, if a correction is applied the correction may be a simplification of the real effect, and thus not fully accurate. For the DRR generation these correction steps can be inversely applied if no correction is used on the 2D live data. Thus, in the DRR a virtual inhomogeneity can be created similar to the 2D live data. Alternatively, the heel effect and detector gain can be taken into account during DRR generation simulating the physical reality of the system, and then the simplified correction of the 2D live data is applied in order to achieve the same correction with limited accuracy is created on the DRR. [0119] In an example, for both effects, calibration data from the image reconstruction pipeline is used. [0120] In an example, if the digitally reconstructed radiography, after restoration, is not geometrically matching the live image, a 2D registration process may be provided. As an example, at least a part of out-of-plane motions are corrected by the adapting steps. The subsequent 2D registration process can thus produce good results, even in the presence of harsh patient movements. Subtraction can then be applied, producing a clean and improved digital subtraction angiography. Dascal: [0129] A method for registering vascular trees extracted from different contrast enhanced x-ray frames, for example during x-ray angiography is described herein. In one embodiment, the vessel centerlines are known or treated as known for each of the vascular branches of interest. The process of detecting such centerlines can be performed using various methods as described in U.S. Pat. No. 9,351,698, the disclosure of which is incorporated by reference herein in its entirety. In one embodiment, a registration method described herein uses bifurcation points (if they exist) and the bending points (if they exist) as “anchors” for matching anatomical positions between different frames. Once the set of matching “anchor points” is obtained, the registration is based on interpolating for matching positions based on relative geodesic distance as measured along the arc-length of the centerlines. Various distance metrics can be used as appropriate. [0130] Furthermore, the “anchors points” fusion can be used to generate an estimation of the three-dimensional cardiac motion and deformation. Using a pair of angiographic images (2D projections), from the same cardiac phase, one can obtain three-dimensional reconstruction of the tree vessels. These 3-D vessels structures reconstructions at multiple phases of the cardiac cycle are of interest for 3D heart motion understanding. The displacements of these anchor points on each view along the image sequences induce a way of computing the motion estimation in the 3D vascular structure. Additional details relating to methods of performing “anchor points” matching for interframe registration of vascular trees are described below and otherwise herein. [0131] FIG. 14 shows an exemplary process flow 630 suitable for registering points associated with a cardiac system such as vascular trees between a first and a second angiography frame. This process flow can be used to detect anatomical features and use them for cross-frame/interframe registration. Cross-frame registration can be also accomplished by other anatomical features or anatomical landmarks found along the vessel. The method 630 can be used to perform interframe registration of vascular trees as shown. [0132] In one embodiment, the process of detecting anatomical features such as bifurcation points such as the split of an artery into a first and a second blood vessel or bends for a given vascular tree and the associated process of grouping such points can be implemented using various data transformation and image processing steps. Initially, the method determines a sequence of x-ray images and associated centerlines Step C1 for processing such as by user selection or other criteria. [0133] Centerlines can be determined as described herein. Theses x-ray images undergo preprocessing Step C2. An example of such a preprocessed image 640 is shown in FIG. 15A. In one embodiment, a skeleton image is generated around each centerline of interest as the preprocessing step. Various arterial branches and the associated bends and take off junctions and angles thereof are evident and detectable as a result of the lightening of peripheral features and the darkening of arterial features as shown in FIG. 15A. [0134] FIGS. 15B and 15C are an original angiography image and a skeleton of one the vessels in that image and a portion of its periphery environment, respectfully, after the application of image processing and data analysis in accordance with an illustrative embodiment of the disclosure. The skeleton image of FIG. 15C corresponds to the output of Step C3. Still referring to FIG. 14, the method includes steps that can be grouped into two processing paths or categories. In one embodiment, the two processing paths or categories can relate to a first anatomical feature or anatomical landmark and one relating to bifurcations and one relating to bend points. [0135] In one embodiment, the bifurcation related portion of the method includes the steps of detecting bifurcations on all or a subset of all frames Step C4 and grouping of bifurcations by clustering Step C5. The bend related portion of the method includes the steps of detecting “bend” points detection on all or a subset of all frames Step C6 and grouping of the detected bend points Step C7. These groupings of bifurcations and bend points are in turn used to perform interframe registration of the vascular trees Step C8 or other vascular structures or subsets thereof. In general, any groups of a first anatomical features and a second anatomical feature can be grouped or clustered as described herein and in turn used to perform interframe registration of the vascular trees Step C8 or other vascular structures or subsets thereof. [0136]-[0145] Bifurcation Detection—Feature Extraction Related Features)
Consider Claims 10.
The combination of Florent and Dascal teaches:
10. The method of claim 9, wherein dividing the buffer area comprises: dividing the buffer area into polygons, the sub-area comprising a polygonal area. (Florent: [0133] In an example of the method, for finding a matching pose it is provided at least one of the group of:
i) a pose estimation that is based on a system-based geometric registration; ii) a segmentation-based pose estimation; iii) a constraint that the presence of contrast injected vascular structures is ignored in the 2D live X-ray image and/or in the 3D image data; and iv) generating a plurality of digitally reconstructed radiographies for different projection directions of the 3D image data and using the plurality of digitally reconstructed radiographies for identifying possible matching with the 2D live image. Dascal: [0101] During x-ray guided procedures, physicians use x-ray scans combined with contrast agents to visualize blood vessels and cardiac chambers. During contrast injection, a contrast cloud can form near the contrast-leading catheter. The contrast cloud is typically amorphic and varies in shape and size in different image frames collected during a scan. A contrast cloud has the potential to block or hide underlying structures and potentially lead to decreased performance of various image processing and computer vision algorithms. Detecting the location and extent of a contrast cloud from a single or multiple image frames establish a refined region of interest when applying image processing and computer vision algorithms. [0109] In step A5, for each image pixel in the binary image created in step A3, the number of bright pixels inside a neighborhood area that is similar to the typical sizes of the contrast clouds that need detection is counted. Typical neighborhood areas surrounding the contrast cloud can be disk shaped, rectangular shaped, or any arbitrary shape. In one embodiment, a dimension of the neighborhood is less than about 5 mm. In one embodiment, a dimension of the neighborhood range from about 1 mm to about 4 cm. In one embodiment, the dimension is a diameter, a chord, or a line segment. FIG. 6 illustrates an exemplary image 290 that results from counting the bright white pixels 300, 325, 330, 335 in the bright white regions of the image of FIG. 5, including in a predefined neighborhood surrounding the contrast cloud 300.)
Consider Claims 11 and 18.
The combination of Florent and Dascal teaches:
11. The method of claim 8, wherein performing transition processing on the buffer area based on the coordinates of the matching point pairs comprises: calculating position coordinates of remaining pixel points in the buffer area based on the coordinates of the matching point pairs, coordinate characteristics of the pixel points whose distance from the region of interest does not exceed a first preset value being approximate to coordinate characteristics of pixel points of the third mask image, and coordinate characteristics of the pixel points whose distance from the region of interest exceeds the first preset value being approximate to coordinate characteristics of pixel points of the contrast image./ 18. The computer device of claim 16, wherein performing transition processing on the buffer area based on the coordinates of the matching point pairs comprises: calculating position coordinates of remaining pixel points in the buffer area based on the coordinates of the matching point pairs, coordinate characteristics of the pixel points whose distance from the region of interest does not exceed a first preset value being approximate to coordinate characteristics of pixel points of the third mask image, and the coordinate characteristics of the pixel points whose distance from the region of interest exceeds the first preset value being approximate to coordinate characteristics of pixel points of the contrast image. (Florent: [0117], [0118] The heel effect correction and detector gain correction are used to create virtually a homogeneous X-ray intensity. Due to the heel effect, the emission intensity of X-rays is slightly different for every emission angle. The detector X-ray detection efficiency varies for every pixel, due to various factors like the internal detector composition. In the 2D live data, these effects may or may not be corrected. Furthermore, if a correction is applied the correction may be a simplification of the real effect, and thus not fully accurate. For the DRR generation these correction steps can be inversely applied if no correction is used on the 2D live data. Thus, in the DRR a virtual inhomogeneity can be created similar to the 2D live data. Alternatively, the heel effect and detector gain can be taken into account during DRR generation simulating the physical reality of the system, and then the simplified correction of the 2D live data is applied in order to achieve the same correction with limited accuracy is created on the DRR. [0119] In an example, for both effects, calibration data from the image reconstruction pipeline is used. [0120] In an example, if the digitally reconstructed radiography, after restoration, is not geometrically matching the live image, a 2D registration process may be provided. As an example, at least a part of out-of-plane motions are corrected by the adapting steps. The subsequent 2D registration process can thus produce good results, even in the presence of harsh patient movements. Subtraction can then be applied, producing a clean and improved digital subtraction angiography. Dascal: [0129] A method for registering vascular trees extracted from different contrast enhanced x-ray frames, for example during x-ray angiography is described herein. In one embodiment, the vessel centerlines are known or treated as known for each of the vascular branches of interest. The process of detecting such centerlines can be performed using various methods as described in U.S. Pat. No. 9,351,698, the disclosure of which is incorporated by reference herein in its entirety. In one embodiment, a registration method described herein uses bifurcation points (if they exist) and the bending points (if they exist) as “anchors” for matching anatomical positions between different frames. Once the set of matching “anchor points” is obtained, the registration is based on interpolating for matching positions based on relative geodesic distance as measured along the arc-length of the centerlines. Various distance metrics can be used as appropriate. [0130] Furthermore, the “anchors points” fusion can be used to generate an estimation of the three-dimensional cardiac motion and deformation. Using a pair of angiographic images (2D projections), from the same cardiac phase, one can obtain three-dimensional reconstruction of the tree vessels. These 3-D vessels structures reconstructions at multiple phases of the cardiac cycle are of interest for 3D heart motion understanding. The displacements of these anchor points on each view along the image sequences induce a way of computing the motion estimation in the 3D vascular structure. Additional details relating to methods of performing “anchor points” matching for interframe registration of vascular trees are described below and otherwise herein. [0131] FIG. 14 shows an exemplary process flow 630 suitable for registering points associated with a cardiac system such as vascular trees between a first and a second angiography frame. This process flow can be used to detect anatomical features and use them for cross-frame/interframe registration. Cross-frame registration can be also accomplished by other anatomical features or anatomical landmarks found along the vessel. The method 630 can be used to perform interframe registration of vascular trees as shown. [0132] In one embodiment, the process of detecting anatomical features such as bifurcation points such as the split of an artery into a first and a second blood vessel or bends for a given vascular tree and the associated process of grouping such points can be implemented using various data transformation and image processing steps. Initially, the method determines a sequence of x-ray images and associated centerlines Step C1 for processing such as by user selection or other criteria. [0133] Centerlines can be determined as described herein. Theses x-ray images undergo preprocessing Step C2. An example of such a preprocessed image 640 is shown in FIG. 15A. In one embodiment, a skeleton image is generated around each centerline of interest as the preprocessing step. Various arterial branches and the associated bends and take off junctions and angles thereof are evident and detectable as a result of the lightening of peripheral features and the darkening of arterial features as shown in FIG. 15A. [0134] FIGS. 15B and 15C are an original angiography image and a skeleton of one the vessels in that image and a portion of its periphery environment, respectfully, after the application of image processing and data analysis in accordance with an illustrative embodiment of the disclosure. The skeleton image of FIG. 15C corresponds to the output of Step C3. Still referring to FIG. 14, the method includes steps that can be grouped into two processing paths or categories. In one embodiment, the two processing paths or categories can relate to a first anatomical feature or anatomical landmark and one relating to bifurcations and one relating to bend points. [0135] In one embodiment, the bifurcation related portion of the method includes the steps of detecting bifurcations on all or a subset of all frames Step C4 and grouping of bifurcations by clustering Step C5. The bend related portion of the method includes the steps of detecting “bend” points detection on all or a subset of all frames Step C6 and grouping of the detected bend points Step C7. These groupings of bifurcations and bend points are in turn used to perform interframe registration of the vascular trees Step C8 or other vascular structures or subsets thereof. In general, any groups of a first anatomical features and a second anatomical feature can be grouped or clustered as described herein and in turn used to perform interframe registration of the vascular trees Step C8 or other vascular structures or subsets thereof. [0136]-[0145] Bifurcation Detection—Feature Extraction Related Features)
Consider Claims 12 and 19.
The combination of Florent and Dascal teaches:
12. The method of claim 2, wherein fusing the second scan image, the third scan image, and the processed buffer area image to obtain the corrected scan image comprises: stitching a part of the second mask image excluding the region of interest and the buffer area, the third mask image, and the processed buffer area image together to obtain the corrected mask image. / 19. The computer device of claim 14, wherein fusing the second scan image the third scan image, and the processed buffer area image to obtain the corrected scan image comprises: stitching a part of the second mask image excluding the region of interest and the buffer area, the third mask image, and the processed buffer area image together to obtain the corrected mask image. (Florent: [0116] In an example, the 2D scatter estimation is using the angiographic image as input. [0117] In an example of the device, not shown in detail, as correction for the reconstruction of the 3D image data, the data processor 14 is configured to provide i) a virtual heel effect correction. In addition or alternatively, the data processor 14 is configured to provide ii) a virtual inverse detector gain correction. [0118] The heel effect correction and detector gain correction are used to create virtually a homogeneous X-ray intensity. Due to the heel effect, the emission intensity of X-rays is slightly different for every emission angle. The detector X-ray detection efficiency varies for every pixel, due to various factors like the internal detector composition. In the 2D live data, these effects may or may not be corrected. Furthermore, if a correction is applied the correction may be a simplification of the real effect, and thus not fully accurate. For the DRR generation these correction steps can be inversely applied if no correction is used on the 2D live data. Thus, in the DRR a virtual inhomogeneity can be created similar to the 2D live data. Alternatively, the heel effect and detector gain can be taken into account during DRR generation simulating the physical reality of the system, and then the simplified correction of the 2D live data is applied in order to achieve the same correction with limited accuracy is created on the DRR. [0119] In an example, for both effects, calibration data from the image reconstruction pipeline is used. [0120] In an example, if the digitally reconstructed radiography, after restoration, is not geometrically matching the live image, a 2D registration process may be provided. As an example, at least a part of out-of-plane motions are corrected by the adapting steps. The subsequent 2D registration process can thus produce good results, even in the presence of harsh patient movements. Subtraction can then be applied, producing a clean and improved digital subtraction angiography. Dascal: [0107] FIG. 3 illustrates a flowchart 100 of a method for detecting a contrast cloud for each input image. In step A1, an image is input into a system for processing for the detection of a contrast cloud. In step A2, the image is denoised to produce a smoother image. This step can be an optional step, but can improve the image and can be important for noisy x-ray images. If the image is one of better quality, step A2 can be skipped. FIG. 4 illustrates an exemplary image of an x-ray image 150 of a vessel 170 with a contrast agent 160 region after image denoising has been performed. Knowing the location of cloud 160 allows the improvement of the consistency and stability of vessel centerlines which leads to improved accuracy of the co-registration and tracking of the marker of the probe. [0108]-[0111], [0112] In steps B1-B4, optional processing steps can be used for fusing cloud masks from multiple images. In step B1, cloud masks from multiple images are used together to create a single fused mask. A pixel-wise OR operator can be used to obtain a merged contrast cloud mask incorporating information from multiple x-ray frames, in step B2. After obtaining the merged mask, another component-based filter can be used to remove small components or components that are out of the region of interest in step B3. The use of multiple x-ray frames is advantageous given the expansion and dispersal of the cloud over a time period following the contrast solutions initial delivery. [0113] In step B4, the cloud masks from each frame can be fused, as illustrated in FIG. 8 which illustrates an exemplary image 160 of a fused contrast cloud mask 410 from an x-ray image sequence. This mask can be generated and applied to the image to identify the contrast cloud regions, which may include a buffer zone around them as a safety factor. These identified contrast cloud regions can then be stored in memory as ignore/avoid regions when performing additional image processing. In one embodiment, the fused cloud mask of FIG. 8 is derived by taking a pixel wise OR between multiple cloud masks.)
Conclusion
The prior art made of record in form PTO-892 and not relied upon is considered pertinent to applicant's disclosure.
PNG
media_image1.png
316
883
media_image1.png
Greyscale
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TAHMINA ANSARI whose telephone number is 571-270-3379. The examiner can normally be reached on IFP Flex - Monday through Friday 9 to 5.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’NEAL MISTRY can be reached on 313-446-4912. The fax phone numbers for the organization where this application or proceeding is assigned are 571-273-8300 for regular communications and 571-273-8300 for After Final communications. TC 2600’s customer service number is 571-272-2600.
Any inquiry of a general nature or relating to the status of this application or proceeding should be directed to the receptionist whose telephone number is 571-272-2600.
2674
September 16, 2026
/TAHMINA N ANSARI/Primary Examiner, Art Unit 2674
Prosecution Insights