DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over SHELLY (WO 2021209287 A1, as provided in IDS), and in view of MANSI (US 20190223819 A1, as provided in IDS), and further in view of HARKS (US 20190357987 A1, as provided in IDS).
Re Claim 1, SHELLY discloses an apparatus for synthetizing medical images (see SHELLY: e.g., --to generate a synthetic image of a subject…In block iii) the processor is then caused to create the synthetic medical image--, in line 28, pp. 5 through line 26, pp. 6), wherein the apparatus comprises: at least a process (see SHELLY: e.g., --to generate a synthetic image of a subject…In block iii) the processor is then caused to create the synthetic medical image--, in line 28, pp. 5 through line 26, pp. 6); wherein the apparatus comprises:
at least a process; and a memory communicatively connected to the at least a processor (see SHELLY: e.g., --The apparatus comprises a memory comprising instruction data representing a set of instructions, and a processor configured to communicate with the memory and to execute the set of instructions. The set of instructions, when executed by the processor, cause the processor to obtain first and second medical images, determine an elastic deformation that can be used to register the first medical image onto the second medical image, and create the synthetic medical image by weighting the determined elastic deformation and applying the weighted elastic deformation to a portion of the first medical image.--, in abstract, and in lines 7-14, pp. 2), wherein the memory contains instructions configuring the at least a processor to:
receive an ultrasound image of a patient's organ (see SHELLY: e.g., Fig. 1, and, --the first medical image may comprise a medical image of a human or animal subject e.g. a “real” human or animal subject. The first medical image may have been obtained, for example, during a medical examination, or scan. The first medical image may comprise a stock image from a database of medical images….the first and second medical images may comprise a common landmark or landmarks that enable the first medical image to be registered to the second medical image (in step ii) as described below). For example, both the first and second medical images may comprise images of the same anatomical feature. Examples of anatomical features include, but are not limited to a lung, a heart, a ventricle in a heart, a brain, a fetus, etc…..The first and second medical images may comprise medical images of any imaging modality, including but not limited to: a computed tomography (CT) image (for example, from a CT scan) such as a C-arm CT image, a spectral CT image or a phase contrast CT Image, an x-ray image (for example, from an x-ray scan), a magnetic resonance (MR) image (for example, from an MR scan), an ultrasound (US) image (for example, from an ultrasound scan), fluoroscopy images, nuclear medicine images, or any other type of medical images.--, in line 4, page 4 through line 12, page 5);
SHELLY discloses generate an organ model related to the patient's organ (see SHELLY: e.g., --when executed by the processor, further cause the processor to segment the first medical image to produce a segmentation. In such embodiments, in block iii), the processor being caused to create the synthetic image may comprise the processor being caused to apply the weighted elastic deformation to a portion of the first medical image comprising a region of interest, based on the segmentation……the processor may determine the location of the region of interest from the segmentation….In block iii), the synthetic medical image may be created by weighting the determined elastic deformation and applying the weighted elastic deformation to a portion of the first medical image labelled as a thoracic region in the segmentation.
converting the image into constituent blocks or “segments”, the pixels or voxels in each segment having a common attribute. In some methods, image segmentation may comprise fitting a model to one or more features in an image.
One method of image segmentation is Model-Based Segmentation (MBS), whereby a triangulated mesh of a target structure (such as, for example, a heart, brain, lung etc.) is adapted in an iterative fashion to features in an image. Segmentation models typically encode population-based appearance features and shape information. Such information describes permitted shape variations based on real-life shapes of the target structure in members of the population. Shape variations may be encoded, for example, in the form of Eigenmodes which describe the manner in which changes to one part of a model are constrained, or dependent, on the shapes of other parts of a model. Model- based segmentation has been used in various applications to segment one or multiple target organs from medical images, see for example, the paper by Ecabert, O., et al. 2008 entitled “ Automatic Model-Based Segmentation of the Heart in CT Images IEEE Trans. Med. Imaging 27 (9), 1189— 1201--, in line 25, pp. 7 through line 21, pp. 8)
SHELLY however does not explicitly disclose above generating an organ model related to the patient's organ as a function of the ultrasound image;
HARKS discloses generate an organ model related to the patient's organ as a function of the ultrasound image (see HARKS: e.g., ., --the tracking unit 7 evaluates the US signals sensed by the US sensor 6 while the US probe 2 images the volume of interest by emitting US beam pulses under different azimuth angles and, in case of a 3D US probe 2, also under different elevation angles. In order to determine the angular position of the US sensor 6 with respect to the US probe, the tracking unit 7 compares the responses to the emitted US beams sensed by the US sensor 6 and determines the azimuth angle and, in case of a 3D US probe 2, also the elevation angle under which the beam(s) resulting in the maximum response(s) have been emitted. The determined angle(s) define(s) the relative angular position of the US sensor 6 with respect to the US probe 2.--, in [0059], also see: e.g., --the system may generate visualizations in which the live ultrasound (US) images and indications of the position and/or orientation of the medical device are overlaid over the model. In addition or as an alternative, the system may generate visualizations which include a part of the model included in the field of view of a virtual eye at the tip of the medical device 1. This part of the model may further be overlaid by the live ultrasound (US) images in the visualizations…The three-dimensional model of the relevant region of the patient is preferably created prior to the actual interventional procedure during which the live ultrasound (US) images are acquired and stored in a 3D model providing unit 5 for use during the actual interventional procedure. By way of example, a corresponding model 21 of the left atrium of the heart is schematically illustrated in FIG. 2--, in [0041]-[0043], and, -- it may be imaged from the right atrium through the interatrial septum. For this purpose, the US probe 2 is placed at an appropriate position in the right atrium and is operated to acquire a series of ultrasound (US) images of the left atrium under different viewing angles so that the left atrium is imaged essentially completely. On the basis of these images, a model of the left atrium may then be created in the 3D model providing unit 5 by stitching the acquired US images. As an alternative, the US probe 2 may be positioned within the left atrium for acquiring the series of images of the left atrium under different viewing angles.--, in [0044]-[0046]);
SHELLY and HARKS are combinable as they are in the same field of endeavor: image processing and analysis heart features using models and neural network and generate medical image/map. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify SHELLY’s apparatus using HARKS’ teachings by including generate an organ model related to the patient's organ as a function of the ultrasound image to SHELLY’s segmented heart model in order to generate visualizations which include a part of the model to assess heart anatomy and function (see HARKS: e.g. in abstract, and [0041]-[0046]);
SHELLY as modified by HARKS however still do not explicitly disclose identify a region of interest within the organ model, wherein identifying the region of interest comprises: locating at least a point of view on the organ model;
MANSI teaches a heart model, and identify a region of interest {such as temporal region of interest, which changes over time} within the organ model, wherein identifying the region of interest comprises: locating at least a point of view on the organ model (see MANSI: e.g., -- epicardial, endocardial or myocardial EP maps (e.g., local activation time, potentials, deactivation, conduction velocity, and/or others) are generated. A 3D avatar of the patient is estimated using an optical or RGBD camera, or any another imager. The 3D avatar is a digital representation of patient's body, including at least the thorax. ECG electrodes on the patient are localized on the 3D avatar using the RGBD camera or imager. The ECG electrodes may be placed anywhere on the patient torso, such as placements guided by the system or based on other criteria, rather than standardized locations. A 3D heart model is estimated from the heart shadow seen in 2D x-ray images--, in [0017], [0025]-[0028], and, --The heart, optionally the lung, 3D surface…..EP mapping over time. Alternatively, acts 10-13 are repeated for different times to account for patient or other motion. A sequence of 3D heart surfaces, 3D exterior surfaces, electrode positions, and optionally lung locations are estimated. Alternatively, the acts are performed initially, and the surfaces or locations are tracked over time using other processes, such as correlation. [0048] In act 15, an ECG monitor measures electric potential. Each of the electrodes is used to measure potential at the skin of the patient. The potential is measured at one time or measured over time….[0050] In act 16, the image processor generates an EP map. Any EP map may be generated, such as electric potential, local activation time, deactivation time, or convection velocity. The EP map may represent EP operation of the heart at a given time or over time. A sequence of EP maps representing EP operation of the heart at different times may be generated. Any part of the heart may be included or used for the EP map, such as the myocardium or epicardium--, in [0047]-[0050] {apparently, above mentioned “such as the myocardium or epicardium” as the identified ROI, which reads on “a temporal region of interest, which changes over time”}; also see: -- The 3D model of the heart is regressed from the heart shadow or shadows and the 3D avatar.
[0043] The regression model is machine trained. A large set of CT cardiac scans are gathered. The heart is segmented using automated or manual methods from the CT scans. These segmentations provide a ground truth. Hyper-realistic digitally reconstructed radiographs (DRRs) are generated from the CT scans using projection and generative deep learnt models to create 2D x-ray images from different view directions. A 2D mask per x-ray view of the 3D heart model is also projected to form the ground truth--, in [0042]-[0043], and [0050]-[0052]);
SHELLY (as modified by HARKS) and MANSI are combinable as they are in the same field of endeavor: image processing and analysis heart features using models and neural network and generate medical image/map. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify SHELLY (as modified by HARKS)’s apparatus using MANSI’s teachings by including identify a region of interest {such as temporal region of interest, which changes over time} within the organ model, wherein identifying the region of interest comprises: locating at least a point of view on the organ model to SHELLY (as modified by HARKS)’s heart model in order to assess heart anatomy, function and features and to segment the heart is segmented using automated or manual methods from the CT scans, and these segmentations provide a ground truth (see MANSI: e.g. in [0017], [0025]-[0028], [0042]-[0043], and [0050]-[0052]);
SHELLY as modified by HARKS and MANSI further disclose determining a view angle corresponding to the at least a point of view, wherein the at least a point of view and the corresponding view angle define at least one field of view that include at least a portion of the organ model (see HARKS: e.g., --the tracking unit 7 evaluates the US signals sensed by the US sensor 6 while the US probe 2 images the volume of interest by emitting US beam pulses under different azimuth angles and, in case of a 3D US probe 2, also under different elevation angles. In order to determine the angular position of the US sensor 6 with respect to the US probe, the tracking unit 7 compares the responses to the emitted US beams sensed by the US sensor 6 and determines the azimuth angle and, in case of a 3D US probe 2, also the elevation angle under which the beam(s) resulting in the maximum response(s) have been emitted. The determined angle(s) define(s) the relative angular position of the US sensor 6 with respect to the US probe 2.--, in [0059], also see: e.g., --the system may generate visualizations in which the live US images and indications of the position and/or orientation of the medical device are overlaid over the model. In addition or as an alternative, the system may generate visualizations which include a part of the model included in the field of view of a virtual eye at the tip of the medical device 1. This part of the model may further be overlaid by the live US images in the visualizations…The three-dimensional model of the relevant region of the patient is preferably created prior to the actual interventional procedure during which the live US images are acquired and stored in a 3D model providing unit 5 for use during the actual interventional procedure. By way of example, a corresponding model 21 of the left atrium of the heart is schematically illustrated in FIG. 2--, in [0041]-[0043], and, -- it may be imaged from the right atrium through the interatrial septum. For this purpose, the US probe 2 is placed at an appropriate position in the right atrium and is operated to acquire a series of US images of the left atrium under different viewing angles so that the left atrium is imaged essentially completely. On the basis of these images, a model of the left atrium may then be created in the 3D model providing unit 5 by stitching the acquired US images. As an alternative, the US probe 2 may be positioned within the left atrium for acquiring the series of images of the left atrium under different viewing angles.--, in [0044]-[0046]);
SHELLY (as modified by MANSI) and HARKS are combinable as they are in the same field of endeavor: image processing and analysis heart features using models and neural network and generate medical image/map. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify SHELLY (as modified by MANSI)’s apparatus using HARKS’ teachings by including determining a view angle corresponding to the at least a point of view, wherein the at least a point of view and the corresponding view angle define at least one field of view that include at least a portion of the heart model to SHELLY (as modified by MANSI)’s segmented heart model in order to generate visualizations which include a part of the model to assess heart anatomy and function (see HARKS: e.g. in abstract, and [0041]-[0046]); and
generate at least a medical image as a function of the region of interest using an image generator, wherein the at least a medical image captures an anatomical structure of the at least a portion of the organ model (see MANSI: e.g., -- epicardial, endocardial or myocardial EP maps (e.g., local activation time, potentials, deactivation, conduction velocity, and/or others) are generated. A 3D avatar of the patient is estimated using an optical or RGBD camera, or any another imager. The 3D avatar is a digital representation of patient's body, including at least the thorax. ECG electrodes on the patient are localized on the 3D avatar using the RGBD camera or imager. The ECG electrodes may be placed anywhere on the patient torso, such as placements guided by the system or based on other criteria, rather than standardized locations. A 3D heart model is estimated from the heart shadow seen in 2D x-ray images--, in [0017], [0025]-[0028], and, -- The 3D model of the heart is regressed from the heart shadow or shadows and the 3D avatar. [0043] The regression model is machine trained. A large set of CT cardiac scans are gathered. The heart is segmented using automated or manual methods from the CT scans. These segmentations provide a ground truth. Hyper-realistic digitally reconstructed radiographs (DRRs) are generated from the CT scans using projection and generative deep learnt models to create 2D x-ray images from different view directions. A 2D mask per x-ray view of the 3D heart model is also projected to form the ground truth--, in [0042]-[0043], and [0050]-[0052] and, also see: --The heart, optionally the lung, 3D surface…..EP mapping over time. Alternatively, acts 10-13 are repeated for different times to account for patient or other motion. A sequence of 3D heart surfaces, 3D exterior surfaces, electrode positions, and optionally lung locations are estimated. Alternatively, the acts are performed initially, and the surfaces or locations are tracked over time using other processes, such as correlation. [0048] In act 15, an ECG monitor measures electric potential. Each of the electrodes is used to measure potential at the skin of the patient. The potential is measured at one time or measured over time….[0050] In act 16, the image processor generates an EP map. Any EP map may be generated, such as electric potential, local activation time, deactivation time, or convection velocity. The EP map may represent EP operation of the heart at a given time or over time. A sequence of EP maps representing EP operation of the heart at different times may be generated. Any part of the heart may be included or used for the EP map, such as the myocardium or epicardium--, in [0047]-[0050] {apparently, above mentioned “such as the myocardium or epicardium” as the identified ROI, which reads on “a temporal region of interest, which changes over time” and also read on as “real-time visualization of cardiac anatomy”, and apparently above generated EP map, and “Hyper-realistic digitally reconstructed radiographs (DRRs)” align to claimed “wherein the medical image comprises a real-time visualization of cardiac anatomy”}; also see: -- generate an EP map on the heart mesh from measurements of an ECG monitor based on the surface. A display is configured to display the EP map.--, in [0006], and, -- the patient is mapped in real-time--, in [0031]; also see SHELLY: e.g., --to generate a synthetic image of a subject…In block iii) the processor is then caused to create the synthetic medical image--, in line 28, pp. 5 through line 26, pp. 6; --when executed by the processor, further cause the processor to segment the first medical image to produce a segmentation. In such embodiments, in block iii), the processor being caused to create the synthetic image may comprise the processor being caused to apply the weighted elastic deformation to a portion of the first medical image comprising a region of interest, based on the segmentation……the processor may determine the location of the region of interest from the segmentation….In block iii), the synthetic medical image may be created by weighting the determined elastic deformation and applying the weighted elastic deformation to a portion of the first medical image labelled as a thoracic region in the segmentation. converting the image into constituent blocks or “segments”, the pixels or voxels in each segment having a common attribute. In some methods, image segmentation may comprise fitting a model to one or more features in an image. One method of image segmentation is Model-Based Segmentation (MBS), whereby a triangulated mesh of a target structure (such as, for example, a heart, brain, lung etc.) is adapted in an iterative fashion to features in an image. Segmentation models typically encode population-based appearance features and shape information. Such information describes permitted shape variations based on real-life shapes of the target structure in members of the population. Shape variations may be encoded, for example, in the form of Eigenmodes which describe the manner in which changes to one part of a model are constrained, or dependent, on the shapes of other parts of a model. Model- based segmentation has been used in various applications to segment one or multiple target organs from medical images, see for example, the paper by Ecabert, O., et al. 2008 entitled “ Automatic Model-Based Segmentation of the Heart in CT Images IEEE Trans. Med. Imaging 27 (9), 1189— 1201--, in line 25, pp. 7 through line 21, pp. 8; a).
Re Claim 2, SHELLY as modified by MANSI and HARKS further disclose wherein the ultrasound image of the patient's organ comprises one or more of a transesophageal echocardiogram image, transthoracic echocardiogram image, and point-of-care ultrasound image (see HARKS: e.g., -- [0003] Ultrasound (US) imaging is also often used in these procedures, including intracardiac echocardiography (ICE), transesophageal echocardiography (TEE) and transthoracic echocardiography (TTE). US imaging has the advantage that it allows for the visualization of soft-tissue structures and blood flow without harmful scatter radiation. Devices such as catheters and needles can be visualized using ultrasound.--, in [0003], also see: -- [0039] In case the relevant region of the patient body includes a cardiac chamber, the US probe 2 is preferably inserted into the heart to image the relevant cardiac chamber in accordance with an ICE technique. However, the US probe 2 may likewise be configured and utilized in accordance with another echocardiography technique known to a person skilled in the art, such as echocardiographic imaging from the esophagus as in TEE or echocardiographic imaging from a position external to the patient body as in TTE.--, in [0039]).
Re Claim 3, SHELLY as modified by MANSI and HARKS further disclose wherein generating the organ model comprises generating a three-dimensional (3D) data structure representing the patient's organ using an anatomy modeling model (see HARKS: e.g., --the system may generate visualizations in which the live ultrasound (US) images and indications of the position and/or orientation of the medical device are overlaid over the model. In addition or as an alternative, the system may generate visualizations which include a part of the model included in the field of view of a virtual eye at the tip of the medical device 1. This part of the model may further be overlaid by the live ultrasound (US) images in the visualizations…The three-dimensional model of the relevant region of the patient is preferably created prior to the actual interventional procedure during which the live ultrasound (US) images are acquired and stored in a 3D model providing unit 5 for use during the actual interventional procedure. By way of example, a corresponding model 21 of the left atrium of the heart is schematically illustrated in FIG. 2--, in [0041]-[0043], and, -- it may be imaged from the right atrium through the interatrial septum. For this purpose, the US probe 2 is placed at an appropriate position in the right atrium and is operated to acquire a series of ultrasound (US) images of the left atrium under different viewing angles so that the left atrium is imaged essentially completely. On the basis of these images, a model of the left atrium may then be created in the 3D model providing unit 5 by stitching the acquired US images. As an alternative, the US probe 2 may be positioned within the left atrium for acquiring the series of images of the left atrium under different viewing angles.--, in [0044]-[0046]).
Re Claim 4, SHELLY as modified by MANSI and HARKS further disclose wherein generating the 3D data structure representing the patient's organ using the anatomy modeling model comprises: generating anatomy training data (see MANSI: e.g., -- a generative adversarial network (GAN) is trained, where the input is the measured points or locations, the measurements, and the 3D surface. The output is the reconstructed field on the entire torso or 3D surface. The GAN is trained on a large database of dense body surface maps, from which ECG signals are extracted, for other people. The database may be augmented or formed using simulated potentials from a virtual heart model that includes electrophysical modeling.--, in [0055]),
wherein the anatomy training data comprises a plurality of image sets as input and a plurality of anatomical object models as output (see MANSI: e.g., -- a generative adversarial network (GAN) is trained, where the input is the measured points or locations, the measurements, and the 3D surface. The output is the reconstructed field on the entire torso or 3D surface. The GAN is trained on a large database of dense body surface maps, from which ECG signals are extracted, for other people. The database may be augmented or formed using simulated potentials from a virtual heart model that includes electrophysical modeling.--, in [0055]; also see MANSI: e.g., --A 3D heart model is estimated from the heart shadow seen in 2D x-ray images--, in [0017], [0025]-[0028], and, --The heart, optionally the lung, 3D surface…..EP mapping over time. Alternatively, acts 10-13 are repeated for different times to account for patient or other motion. A sequence of 3D heart surfaces, 3D exterior surfaces, electrode positions, and optionally lung locations are estimated. Alternatively, the acts are performed initially, and the surfaces or locations are tracked over time using other processes, such as correlation. [0048] In act 15, an ECG monitor measures electric potential. Each of the electrodes is used to measure potential at the skin of the patient. The potential is measured at one time or measured over time….[0050] In act 16, the image processor generates an EP map. Any EP map may be generated, such as electric potential, local activation time, deactivation time, or convection velocity. The EP map may represent EP operation of the heart at a given time or over time. A sequence of EP maps representing EP operation of the heart at different times may be generated. Any part of the heart may be included or used for the EP map, such as the myocardium or epicardium--, in [0047]-[0050] {apparently, above mentioned “such as the myocardium or epicardium” as the identified ROI, which reads on “a temporal region of interest, which changes over time”}; also see MANSI: e.g., -- The 3D model of the heart is regressed from the heart shadow or shadows and the 3D avatar.[0043] The regression model is machine trained. A large set of CT cardiac scans are gathered. The heart is segmented using automated or manual methods from the CT scans. These segmentations provide a ground truth. Hyper-realistic digitally reconstructed radiographs (DRRs) are generated from the CT scans using projection and generative deep learnt models to create 2D x-ray images from different view directions. A 2D mask per x-ray view of the 3D heart model is also projected to form the ground truth--, in [0042]-[0043], and [0050]-[0052]);
training the anatomy modeling model using the anatomy training data; and generating the 3D data structure using the trained anatomy modeling model (see MANSI: e.g., -- The regression model is machine trained. A large set of CT cardiac scans are gathered. The heart is segmented using automated or manual methods from the CT scans. These segmentations provide a ground truth. Hyper-realistic digitally reconstructed radiographs (DRRs) are generated from the CT scans using projection and generative deep learnt models to create 2D x-ray images from different view directions. A 2D mask per x-ray view of the 3D heart model is also projected to form the ground truth. The regression is then learned, given the 3D avatar, the x-ray image, and the heart segmentation, to provide the 3D heart mesh directly from the 3D surface and the heart shadow. [0044] To facilitate the learning task, the different 3D heart shapes may be projected onto a common shape space parameterized with a few parameters, such as using principle component analysis (PCA). Since the 3D segmentation provides point correspondence across patients, a point-distribution model may straightforwardly be calculated. The output of this step is a reconstructed 3D heart shape, regressed from the x-ray images, heart mask 40, and the 3D avatar. Other embodiments may consider only the x-ray images (no avatar), rely on DynaCT images, preoperative images, or a combination of the above.--, in [0043]-[0044];-- a generative adversarial network (GAN) is trained, where the input is the measured points or locations, the measurements, and the 3D surface. The output is the reconstructed field on the entire torso or 3D surface. The GAN is trained on a large database of dense body surface maps, from which ECG signals are extracted, for other people. The database may be augmented or formed using simulated potentials from a virtual heart model that includes electrophysical modeling.--, in [0055]).
Re Claim 5, SHELLY as modified by MANSI and HARKS further disclose wherein the image generator comprises a generative machine- learning model (see MANSI: e.g., -- epicardial, endocardial or myocardial EP maps (e.g., local activation time, potentials, deactivation, conduction velocity, and/or others) are generated. A 3D avatar of the patient is estimated using an optical or RGBD camera, or any another imager. The 3D avatar is a digital representation of patient's body, including at least the thorax. ECG electrodes on the patient are localized on the 3D avatar using the RGBD camera or imager. The ECG electrodes may be placed anywhere on the patient torso, such as placements guided by the system or based on other criteria, rather than standardized locations. A 3D heart model is estimated from the heart shadow seen in 2D x-ray images--, in [0017], [0025]-[0028], and, --The heart, optionally the lung, 3D surface…..EP mapping over time. Alternatively, acts 10-13 are repeated for different times to account for patient or other motion. A sequence of 3D heart surfaces, 3D exterior surfaces, electrode positions, and optionally lung locations are estimated. Alternatively, the acts are performed initially, and the surfaces or locations are tracked over time using other processes, such as correlation. [0048] In act 15, an ECG monitor measures electric potential. Each of the electrodes is used to measure potential at the skin of the patient. The potential is measured at one time or measured over time….[0050] In act 16, the image processor generates an EP map. Any EP map may be generated, such as electric potential, local activation time, deactivation time, or convection velocity. The EP map may represent EP operation of the heart at a given time or over time. A sequence of EP maps representing EP operation of the heart at different times may be generated. Any part of the heart may be included or used for the EP map, such as the myocardium or epicardium--, in [0047]-[0050] {apparently, above mentioned “such as the myocardium or epicardium” as the identified ROI, which reads on “a temporal region of interest, which changes over time”}; also see: -- The 3D model of the heart is regressed from the heart shadow or shadows and the 3D avatar.
[0043] The regression model is machine trained. A large set of CT cardiac scans are gathered. The heart is segmented using automated or manual methods from the CT scans. These segmentations provide a ground truth. Hyper-realistic digitally reconstructed radiographs (DRRs) are generated from the CT scans using projection and generative deep learnt models to create 2D x-ray images from different view directions. A 2D mask per x-ray view of the 3D heart model is also projected to form the ground truth--, in [0042]-[0043], and [0050]-[0052]).
Re Claim 6, SHELLY as modified by MANSI and HARKS further disclose wherein generating the at least a medical image of the patient's organ comprises:
receiving image training data, wherein the image training data comprises exemplary organ models correlated to exemplary medical images (see MANSI: e.g., -- The regression model is machine trained. A large set of CT cardiac scans are gathered. The heart is segmented using automated or manual methods from the CT scans. These segmentations provide a ground truth. Hyper-realistic digitally reconstructed radiographs (DRRs) are generated from the CT scans using projection and generative deep learnt models to create 2D x-ray images from different view directions. A 2D mask per x-ray view of the 3D heart model is also projected to form the ground truth. The regression is then learned, given the 3D avatar, the x-ray image, and the heart segmentation, to provide the 3D heart mesh directly from the 3D surface and the heart shadow. [0044] To facilitate the learning task, the different 3D heart shapes may be projected onto a common shape space parameterized with a few parameters, such as using principle component analysis (PCA). Since the 3D segmentation provides point correspondence across patients, a point-distribution model may straightforwardly be calculated. The output of this step is a reconstructed 3D heart shape, regressed from the x-ray images, heart mask 40, and the 3D avatar. Other embodiments may consider only the x-ray images (no avatar), rely on DynaCT images, preoperative images, or a combination of the above.--, in [0043]-[0044]; and, -- a generative adversarial network (GAN) is trained, where the input is the measured points or locations, the measurements, and the 3D surface. The output is the reconstructed field on the entire torso or 3D surface. The GAN is trained on a large database of dense body surface maps, from which ECG signals are extracted, for other people. The database may be augmented or formed using simulated potentials from a virtual heart model that includes electrophysical modeling.--, in [0055]);
training the generative machine-learning model using the image training data (see MANSI: e.g., -- The regression model is machine trained. A large set of CT cardiac scans are gathered. The heart is segmented using automated or manual methods from the CT scans. These segmentations provide a ground truth. Hyper-realistic digitally reconstructed radiographs (DRRs) are generated from the CT scans using projection and generative deep learnt models to create 2D x-ray images from different view directions. A 2D mask per x-ray view of the 3D heart model is also projected to form the ground truth. The regression is then learned, given the 3D avatar, the x-ray image, and the heart segmentation, to provide the 3D heart mesh directly from the 3D surface and the heart shadow. [0044] To facilitate the learning task, the different 3D heart shapes may be projected onto a common shape space parameterized with a few parameters, such as using principle component analysis (PCA). Since the 3D segmentation provides point correspondence across patients, a point-distribution model may straightforwardly be calculated. The output of this step is a reconstructed 3D heart shape, regressed from the x-ray images, heart mask 40, and the 3D avatar. Other embodiments may consider only the x-ray images (no avatar), rely on DynaCT images, preoperative images, or a combination of the above.--, in [0043]-[0044]; and, -- a generative adversarial network (GAN) is trained, where the input is the measured points or locations, the measurements, and the 3D surface. The output is the reconstructed field on the entire torso or 3D surface. The GAN is trained on a large database of dense body surface maps, from which ECG signals are extracted, for other people. The database may be augmented or formed using simulated potentials from a virtual heart model that includes electrophysical modeling.--, in [0055]); and
generating the at least a medical image of the patient's organ using the generative machine-learning model (see MANSI: e.g., -- epicardial, endocardial or myocardial EP maps (e.g., local activation time, potentials, deactivation, conduction velocity, and/or others) are generated. A 3D avatar of the patient is estimated using an optical or RGBD camera, or any another imager. The 3D avatar is a digital representation of patient's body, including at least the thorax. ECG electrodes on the patient are localized on the 3D avatar using the RGBD camera or imager. The ECG electrodes may be placed anywhere on the patient torso, such as placements guided by the system or based on other criteria, rather than standardized locations. A 3D heart model is estimated from the heart shadow seen in 2D x-ray images--, in [0017], [0025]-[0028], and, -- The 3D model of the heart is regressed from the heart shadow or shadows and the 3D avatar. [0043] The regression model is machine trained. A large set of CT cardiac scans are gathered. The heart is segmented using automated or manual methods from the CT scans. These segmentations provide a ground truth. Hyper-realistic digitally reconstructed radiographs (DRRs) are generated from the CT scans using projection and generative deep learnt models to create 2D x-ray images from different view directions. A 2D mask per x-ray view of the 3D heart model is also projected to form the ground truth--, in [0042]-[0043], and [0050]-[0052] and, also see: --The heart, optionally the lung, 3D surface…..EP mapping over time. Alternatively, acts 10-13 are repeated for different times to account for patient or other motion. A sequence of 3D heart surfaces, 3D exterior surfaces, electrode positions, and optionally lung locations are estimated. Alternatively, the acts are performed initially, and the surfaces or locations are tracked over time using other processes, such as correlation. [0048] In act 15, an ECG monitor measures electric potential. Each of the electrodes is used to measure potential at the skin of the patient. The potential is measured at one time or measured over time….[0050] In act 16, the image processor generates an EP map. Any EP map may be generated, such as electric potential, local activation time, deactivation time, or convection velocity. The EP map may represent EP operation of the heart at a given time or over time. A sequence of EP maps representing EP operation of the heart at different times may be generated. Any part of the heart may be included or used for the EP map, such as the myocardium or epicardium--, in [0047]-[0050] {apparently, above mentioned “such as the myocardium or epicardium” as the identified ROI, which reads on “a temporal region of interest, which changes over time” and also read on as “real-time visualization of cardiac anatomy”, and apparently above generated EP map, and “Hyper-realistic digitally reconstructed radiographs (DRRs)” align to claimed “wherein the medical image comprises a real-time visualization of cardiac anatomy”}; also see: -- generate an EP map on the heart mesh from measurements of an ECG monitor based on the surface. A display is configured to display the EP map.--, in [0006], and, -- the patient is mapped in real-time--, in [0031]; also see SHELLY: e.g., --to generate a synthetic image of a subject…In block iii) the processor is then caused to create the synthetic medical image--, in line 28, pp. 5 through line 26, pp. 6; --when executed by the processor, further cause the processor to segment the first medical image to produce a segmentation. In such embodiments, in block iii), the processor being caused to create the synthetic image may comprise the processor being caused to apply the weighted elastic deformation to a portion of the first medical image comprising a region of interest, based on the segmentation……the processor may determine the location of the region of interest from the segmentation….In block iii), the synthetic medical image may be created by weighting the determined elastic deformation and applying the weighted elastic deformation to a portion of the first medical image labelled as a thoracic region in the segmentation. converting the image into constituent blocks or “segments”, the pixels or voxels in each segment having a common attribute. In some methods, image segmentation may comprise fitting a model to one or more features in an image. One method of image segmentation is Model-Based Segmentation (MBS), whereby a triangulated mesh of a target structure (such as, for example, a heart, brain, lung etc.) is adapted in an iterative fashion to features in an image. Segmentation models typically encode population-based appearance features and shape information. Such information describes permitted shape variations based on real-life shapes of the target structure in members of the population. Shape variations may be encoded, for example, in the form of Eigenmodes which describe the manner in which changes to one part of a model are constrained, or dependent, on the shapes of other parts of a model. Model- based segmentation has been used in various applications to segment one or multiple target organs from medical images, see for example, the paper by Ecabert, O., et al. 2008 entitled “ Automatic Model-Based Segmentation of the Heart in CT Images IEEE Trans. Med. Imaging 27 (9), 1189— 1201--, in line 25, pp. 7 through line 21, pp. 8; a).
Re Claim 7, SHELLY as modified by MANSI and HARKS further disclose wherein identifying the region of interest within the organ model comprises:
selecting a first set of points from a medical image (see HARKS: e.g., -- [0064] In one embodiment of the registration procedure, the mapping unit 8 may identify fiducial image points in the live US image and map these image points to corresponding points of the model in order to determine the transformation. The mapping of fiducial points can be carried out using known computer vision techniques, such as, for example, scale-invariant feature transform (SIFT). Alternatively, a registration method may be applied which determines the rigid transformation such that the transformed live US image has the largest similarity to the model. Such a registration procedure may be performed on the basis of a segmented version of the live US image, which may be determined using a suitable segmentation procedure known the person skilled in the art. The similarity between the (transformed) US image and the model may again be determined on the basis of a suitable similarity measure, e.g. as explained above.--, in [0063]-[0064]);
determining a second set of points on the organ model corresponding to the first set of points (see HARKS: e.g., -- [0063] In one implementation, the mapping of a live US image onto the model is performed on the basis of the comparison between the live US image and the model. In particular, an image registration between the live US image and the model may be carried out which involves the determination of a rigid transformation for transforming the US image such that it matches a portion of the model. The rigid transformation comprises a rotation and/or a translation.
[0064] In one embodiment of the registration procedure, the mapping unit 8 may identify fiducial image points in the live US image and map these image points to corresponding points of the model in order to determine the transformation. The mapping of fiducial points can be carried out using known computer vision techniques, such as, for example, scale-invariant feature transform (SIFT). Alternatively, a registration method may be applied which determines the rigid transformation such that the transformed live US image has the largest similarity to the model. Such a registration procedure may be performed on the basis of a segmented version of the live US image, which may be determined using a suitable segmentation procedure known the person skilled in the art. The similarity between the (transformed) US image and the model may again be determined on the basis of a suitable similarity measure, e.g. as explained above.--, in [0063]-[0064]); and
mapping a plurality of points of the medical image to the organ model using a
relationship between the first set of points and the second set of points (see HARKS: e.g., -- [0063] In one implementation, the mapping of a live US image onto the model is performed on the basis of the comparison between the live US image and the model. In particular, an image registration between the live US image and the model may be carried out which involves the determination of a rigid transformation for transforming the US image such that it matches a portion of the model. The rigid transformation comprises a rotation and/or a translation.
[0064] In one embodiment of the registration procedure, the mapping unit 8 may identify fiducial image points in the live US image and map these image points to corresponding points of the model in order to determine the transformation. The mapping of fiducial points can be carried out using known computer vision techniques, such as, for example, scale-invariant feature transform (SIFT). Alternatively, a registration method may be applied which determines the rigid transformation such that the transformed live US image has the largest similarity to the model. Such a registration procedure may be performed on the basis of a segmented version of the live US image, which may be determined using a suitable segmentation procedure known the person skilled in the art. The similarity between the (transformed) US image and the model may again be determined on the basis of a suitable similarity measure, e.g. as explained above.--, in [0063]-[0064], [0067]).
Re Claim 8, SHELLY as modified by MANSI and HARKS further disclose, wherein mapping the plurality of points of the medical image to the organ model using the relationship between the first set of points and the second set of points comprises determining a rigid transformation from the first set of points to the second set of points (see HARKS: e.g., -- [0063] In one implementation, the mapping of a live US image onto the model is performed on the basis of the comparison between the live US image and the model. In particular, an image registration between the live US image and the model may be carried out which involves the determination of a rigid transformation for transforming the US image such that it matches a portion of the model. The rigid transformation comprises a rotation and/or a translation.
[0064] In one embodiment of the registration procedure, the mapping unit 8 may identify fiducial image points in the live US image and map these image points to corresponding points of the model in order to determine the transformation. The mapping of fiducial points can be carried out using known computer vision techniques, such as, for example, scale-invariant feature transform (SIFT). Alternatively, a registration method may be applied which determines the rigid transformation such that the transformed live US image has the largest similarity to the model. Such a registration procedure may be performed on the basis of a segmented version of the live US image, which may be determined using a suitable segmentation procedure known the person skilled in the art. The similarity between the (transformed) US image and the model may again be determined on the basis of a suitable similarity measure, e.g. as explained above.--, in [0063]-[0064]); and
mapping a plurality of points of the medical image to the organ model using a
relationship between the first set of points and the second set of points (see HARKS: e.g., -- [0063] In one implementation, the mapping of a live US image onto the model is performed on the basis of the comparison between the live US image and the model. In particular, an image registration between the live US image and the model may be carried out which involves the determination of a rigid transformation for transforming the US image such that it matches a portion of the model. The rigid transformation comprises a rotation and/or a translation.
[0064] In one embodiment of the registration procedure, the mapping unit 8 may identify fiducial image points in the live US image and map these image points to corresponding points of the model in order to determine the transformation. The mapping of fiducial points can be carried out using known computer vision techniques, such as, for example, scale-invariant feature transform (SIFT). Alternatively, a registration method may be applied which determines the rigid transformation such that the transformed live US image has the largest similarity to the model. Such a registration procedure may be performed on the basis of a segmented version of the live US image, which may be determined using a suitable segmentation procedure known the person skilled in the art. The similarity between the (transformed) US image and the model may again be determined on the basis of a suitable similarity measure, e.g. as explained above.--, in [0063]-[0064], [0067]).
.
Re Claim 9, SHELLY as modified by MANSI and HARKS further disclose wherein generating the organ model comprises:
transforming the organ model to a second organ model using a Statistical Shape Model as a function of a plurality of mode changers within the Statistical Shape Model,
wherein each mode changer of the plurality of mode changers is associated with a
model feature of the organ model (see SHELLY: e.g., --when executed by the processor, further cause the processor to segment the first medical image to produce a segmentation. In such embodiments, in block iii), the processor being caused to create the synthetic image may comprise the processor being caused to apply the weighted elastic deformation to a portion of the first medical image comprising a region of interest, based on the segmentation……the processor may determine the location of the region of interest from the segmentation….In block iii), the synthetic medical image may be created by weighting the determined elastic deformation and applying the weighted elastic deformation to a portion of the first medical image labelled as a thoracic region in the segmentation.
converting the image into constituent blocks or “segments”, the pixels or voxels in each segment having a common attribute. In some methods, image segmentation may comprise fitting a model to one or more features in an image.
One method of image segmentation is Model-Based Segmentation (MBS), whereby a triangulated mesh of a target structure (such as, for example, a heart, brain, lung etc.) is adapted in an iterative fashion to features in an image. Segmentation models typically encode population-based appearance features and shape information. Such information describes permitted shape variations based on real-life shapes of the target structure in members of the population. Shape variations may be encoded, for example, in the form of Eigenmodes which describe the manner in which changes to one part of a model are constrained, or dependent, on the shapes of other parts of a model. Model- based segmentation has been used in various applications to segment one or multiple target organs from medical images, see for example, the paper by Ecabert, O., et al. 2008 entitled “ Automatic Model-Based Segmentation of the Heart in CT Images IEEE Trans. Med. Imaging 27 (9), 1189— 1201--, in line 25, pp. 7 through line 21, pp. 8; also see Mansi: e.g., -- The regression model is machine trained. A large set of CT cardiac scans are gathered. The heart is segmented using automated or manual methods from the CT scans. These segmentations provide a ground truth. Hyper-realistic digitally reconstructed radiographs (DRRs) are generated from the CT scans using projection and generative deep learnt models to create 2D x-ray images from different view directions. A 2D mask per x-ray view of the 3D heart model is also projected to form the ground truth. The regression is then learned, given the 3D avatar, the x-ray image, and the heart segmentation, to provide the 3D heart mesh directly from the 3D surface and the heart shadow. [0044] To facilitate the learning task, the different 3D heart shapes may be projected onto a common shape space parameterized with a few parameters, such as using principle component analysis (PCA). Since the 3D segmentation provides point correspondence across patients, a point-distribution model may straightforwardly be calculated. The output of this step is a reconstructed 3D heart shape, regressed from the x-ray images, heart mask 40, and the 3D avatar. Other embodiments may consider only the x-ray images (no avatar), rely on DynaCT images, preoperative images, or a combination of the above.--, in [0043]-[0044]).
Re Claim 10, SHELLY as modified by MANSI and HARKS further disclose wherein: generating the at least a medical image comprises generating a plurality of medical images (see MANSI: e.g., -- generate an EP map on the heart mesh from measurements of an ECG monitor based on the surface. A display is configured to display the EP map.--, in [0006], and, -- the patient is mapped in real-time--, in [0031]; -- epicardial, endocardial or myocardial EP maps (e.g., local activation time, potentials, deactivation, conduction velocity, and/or others) are generated. A 3D avatar of the patient is estimated using an optical or RGBD camera, or any another imager. The 3D avatar is a digital representation of patient's body, including at least the thorax. ECG electrodes on the patient are localized on the 3D avatar using the RGBD camera or imager. The ECG electrodes may be placed anywhere on the patient torso, such as placements guided by the system or based on other criteria, rather than standardized locations. A 3D heart model is estimated from the heart shadow seen in 2D x-ray images--, in [0017], [0025]-[0028], and, -- The 3D model of the heart is regressed from the heart shadow or shadows and the 3D avatar. [0043] The regression model is machine trained. A large set of CT cardiac scans are gathered. The heart is segmented using automated or manual methods from the CT scans. These segmentations provide a ground truth. Hyper-realistic digitally reconstructed radiographs (DRRs) are generated from the CT scans using projection and generative deep learnt models to create 2D x-ray images from different view directions. A 2D mask per x-ray view of the 3D heart model is also projected to form the ground truth--, in [0042]-[0043], and [0050]-[0052] and, also see: --The heart, optionally the lung, 3D surface…..EP mapping over time. Alternatively, acts 10-13 are repeated for different times to account for patient or other motion. A sequence of 3D heart surfaces, 3D exterior surfaces, electrode positions, and optionally lung locations are estimated. Alternatively, the acts are performed initially, and the surfaces or locations are tracked over time using other processes, such as correlation. [0048] In act 15, an ECG monitor measures electric potential. Each of the electrodes is used to measure potential at the skin of the patient. The potential is measured at one time or measured over time….[0050] In act 16, the image processor generates an EP map. Any EP map may be generated, such as electric potential, local activation time, deactivation time, or convection velocity. The EP map may represent EP operation of the heart at a given time or over time. A sequence of EP maps representing EP operation of the heart at different times may be generated. Any part of the heart may be included or used for the EP map, such as the myocardium or epicardium--, in [0047]-[0050] {apparently, above mentioned “such as the myocardium or epicardium” as the identified ROI, which reads on “a temporal region of interest, which changes over time” and also read on as “real-time visualization of cardiac anatomy”, and apparently above generated EP map, and “Hyper-realistic digitally reconstructed radiographs (DRRs)” align to claimed “wherein the medical image comprises a real-time visualization of cardiac anatomy”}; also see: -- generate an EP map on the heart mesh from measurements of an ECG monitor based on the surface. A display is configured to display the EP map.--, in [0006], and, -- the patient is mapped in real-time--, in [0031]; also see SHELLY: e.g., --to generate a synthetic image of a subject…In block iii) the processor is then caused to create the synthetic medical image--, in line 28, pp. 5 through line 26, pp. 6; --when executed by the processor, further cause the processor to segment the first medical image to produce a segmentation. In such embodiments, in block iii), the processor being caused to create the synthetic image may comprise the processor being caused to apply the weighted elastic deformation to a portion of the first medical image comprising a region of interest, based on the segmentation……the processor may determine the location of the region of interest from the segmentation….In block iii), the synthetic medical image may be created by weighting the determined elastic deformation and applying the weighted elastic deformation to a portion of the first medical image labelled as a thoracic region in the segmentation. converting the image into constituent blocks or “segments”, the pixels or voxels in each segment having a common attribute. In some methods, image segmentation may comprise fitting a model to one or more features in an image. One method of image segmentation is Model-Based Segmentation (MBS), whereby a triangulated mesh of a target structure (such as, for example, a heart, brain, lung etc.) is adapted in an iterative fashion to features in an image. Segmentation models typically encode population-based appearance features and shape information. Such information describes permitted shape variations based on real-life shapes of the target structure in members of the population. Shape variations may be encoded, for example, in the form of Eigenmodes which describe the manner in which changes to one part of a model are constrained, or dependent, on the shapes of other parts of a model. Model- based segmentation has been used in various applications to segment one or multiple target organs from medical images, see for example, the paper by Ecabert, O., et al. 2008 entitled “ Automatic Model-Based Segmentation of the Heart in CT Images IEEE Trans. Med. Imaging 27 (9), 1189— 1201--, in line 25, pp. 7 through line 21, pp. 8; a); and
the memory contains instructions further configuring the at least a processor to: compile the plurality of medical images into a video; and display the video on a display device (see MANSI: e.g., -- generate an EP map on the heart mesh from measurements of an ECG monitor based on the surface. A display is configured to display the EP map.--, in [0006], and, -- the patient is mapped in real-time--, in [0031]; and, --The image processor 74 generates an image from the EP map. For example, a rendering from a given viewing direction or a bull's-eye view image are generated. Any EP map visualization may be used. [0087] The image processor 74 may output other information. For example, a sensitivity analysis is performed using a virtual heart model personalized to the patient 77. The locations to which electrodes should be moved are output. The image processor 74 may indicate locations of uncertainty or guide intracardiac measurements. [0088] The memory 75 is a graphics processing memory, a video random access memory, a random access memory, system memory, random access memory, cache memory, hard drive, optical media, magnetic media, flash drive, buffer, database, combinations thereof, or other now known or later developed memory device for storing data or video information. The memory 75 is part of the diagnostic scanner 71, part of a computer associated with the image processor 74, part of a database, part of another system, a picture archival memory, or a standalone device.--, in [0086]-[0088]).
Re Claims 11-20, claims 11-20 are the corresponding method claim to claims 1-10 respectively. Thus, claims 11-20 are rejected for the similar reasons as for claims 1-10. Furthermore, SHELLY as modified by MANSI and HARKS further disclose a method for synthetizing medical images (see SHELLY : e.g., --to generate a synthetic image of a subject…In block iii) the processor is then caused to create the synthetic medical image--, in line 28, pp. 5 through line 26, pp. 6), wherein the apparatus comprises: at least a process (see SHELLY: e.g., --to generate a synthetic image of a subject…In block iii) the processor is then caused to create the synthetic medical image--, in line 28, pp. 5 through line 26, pp. 6; see MANSI: e.g., --the heart modeling may use x-ray projection data and the 3D surface. The heart is modeled in three dimensions from 2D x-ray projection data. A single x-ray or multiple x-ray images from different directions relative to the patient may be used. Alternatively, the heart is modeled from the 3D surface without other medical imaging. A heart prior or standard shape may be altered based on the 3D surface in order to model the patient. [0040] The x-ray image or images are used to form a heart mask. The projection data represents the heart in two dimensions. The heart mask indicates locations where the heart is located in those two dimensions. Any segmentation or detection may be used to form the heart mask. In one embodiment, a deep machine-learnt neural network is applied. The x-ray image is input to the neural network, which outputs the heart mask or heart locations. The neural network is trained with deep learning to segment the heart from x-ray images. In one embodiment, a deep convolutional-deconvolutional network (e.g., image-to-image network) architecture is used. Other architectures, such as a dense net, U-net, or generative model, may be used. Any machine-learnt segmentation or non-machine-learnt segmentation may be used.--, in [0039]-[0045]; and, -- a generative adversarial network (GAN) is trained, where the input is the measured points or locations, the measurements, and the 3D surface. The output is the reconstructed field on the entire torso or 3D surface. The GAN is trained on a large database of dense body surface maps, from which ECG signals are extracted, for other people. The database may be augmented or formed using simulated potentials from a virtual heart model that includes electrophysical modeling.--, in [0055]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Chitiboi (US 20210383537 A1) discloses the synthesis of contrast enhanced medical images. Embodiments of the present invention are described herein to give a visual understanding of such methods and systems. A digital image is often composed of digital representations of one or more objects (or shapes). generating synthesized LGE MRI medical images of pathological tissue from cine-MRI medical images of healthy tissue. Such synthesized LGE MRI medical images are generated using a GAN (generative adversarial network) to provide realistic looking synthesized LGE MRI medical images. Advantageously, such synthesized LGE MRI medical images of pathological tissue may be utilized for training machine learning based systems for performing medical image analysis tasks, such as, e.g., segmentation of LGE enhanced regions of LGE MRI medical images. Such machine learning based systems trained using the synthesized LGE MRI medical images in accordance with embodiments described herein are more robust to various types of pathological tissue. (in [0018]-[0020]),
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEIWEN YANG whose telephone number is (571)270-5670. The examiner can normally be reached on Monday-Friday 8:30am-4:30pm east.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEI WEN YANG/Primary Examiner, Art Unit 2662
Prosecution Insights