DETAILED ACTION
Response to Arguments
The amendments filed 9/16/2025 have been entered and made of record.
Applicant's amendments and corresponding arguments filed 9/16/2025 have been fully considered, but are moot in view of the new ground(s) of rejection because the Applicant has substantially amended at least independent claims 1, 10 and 20 by including limitation of “the motion field indicates respective displacements of a plurality of pixels from the first image to the second image”, and, “determine a correction factor for the motion field based on the change to the second tracked contour”, and, “adjust the motion field that indicates the respective displacements of the plurality of pixels from the first image to the second image based on the determined correction factor”, and furthermore, claim 3 has been amended “the processor is configured to determine the correction factor for the motion field using an artificial neural network pre-trained for predicting the correction factor based on a difference between the first segmentation mask and the second segmentation mask”; after further considerations, above limitation is rejected with the new ground(s) of further in view of Wetzl (US 11455734 B2), and further in view of Oliveira Ferreira (US 20210125333 A1) because it is well known in the art {Artificial-Intelligence-Based assessment of Cardiac motion/movement/deformation from a stack of MR adjacent medical images in temporal sequence of Cardiac Magnetic Resonance (CMR) image data}:
Wetzl discloses determine a correction factor based on the change to the second tracked contour {using a trained machine learning network, especially a deep learning network, based on adaptive algorithm } (see Wetzl: e.g., Fig. 4, “motion detector 22” -- the movement data (of the target) is calculated based on an adaptive algorithm trained for calculating movement data on the basis of temporal adjacent images based on the localization data. Such adaptive algorithm could be trained on a plurality of test images with targets, wherein the localization data of the targets are given and the movement of the target is known.--, in lines 35-56, col. 5; and, -- calculating the movement data (of the target), a quantitative motion curve is generated describing the average motion magnitude between temporal adjacent images. …A ground truth for the motion detection algorithm (e.g. a neural network) is computed based on elastic registration between adjacent image time points, i.e. calculating where pixels that belong to the target structure will move from one time point to the next. If the average of this movement is low, this is considered a resting phase. Based on this ground truth, the algorithm is trained to predict this average movement value. So in such method steps no elastic registration is performed, but the trained algorithm provides results characterizing the motion of the target.
(28) According to a method according to an exemplary embodiment, a motion detection is performed by comparing the calculated movement data (of the target) with a predefined threshold movement value.--, in line 58, col. 5 through line 27, col. 6;
-- two or more objects may be localized in the images, wherein two or more landmarks and/or contours are obtained. The motion data is then preferably calculated based on the relative movement between two or more objects…. Thus, it is preferred to track at least one target “globally”. It is particularly preferred that a target is chosen for this “global tracking” that is in a consistent motion state, wherein the relative motions to other structures should be the same. Then it is possible to depict all examined structures … the motion detector comprise a trained machine learning network, especially a deep learning network. This machine learning network is preferably a neural network and/or a convolutional network and/or an end-to-end network.--, in line 45, col. 6 to line 16, col. 7);
Apparently, Wetz discloses determine “a correction factor”, which is the calculated “movement” and “motion data”, because these detected/calculated, or predicted “movement value” and “motion data” read on “a correction factor” as describing the average motion magnitude between temporal adjacent images, and to depict all examined structures;
Wetzl above disclosed “motion data”, “movement values”, and “a quantitative motion curve is generated” describing the average motion magnitude between temporal adjacent images; Wetzl however does not explicitly disclose “a motion field”;
Oliveira Ferreira discloses determining a correction factor for the motion field based on the change to the second tracked contour (see Oliveira Ferreira: e.g., -- [0101] Here, a trained neural network may predict the two ventricular insertion points on a first image frame of a slice or alternatively on all image frames. Then, deformation fields are considered by the trained neural network. These deformation fields are defined as mappings indicating a direction and/or a magnitude of how much each pixel in every image frame moves over time. By thus following or predicting the positions of the two ventricular insertion points along the image frames in time, the movement feature and classification module 33 (in a segmental classification embodiment) may divide the mask into the respective segments.
[0102] Based on the above deformation fields, the movement feature and classification module 33 may compute one or more time series with regard to the mask characteristics. In particular, for different regions of the mask and for at least a pre-determined number of image frames in time, for example a portion or all image frames in a particular slice, a time series of a tracked deformation feature may be considered with regard to one or more of an inner radius of the mask, an outer radius of the mask, an inner curvature of the mask, an outer curvature of the mask, a bloodpool area, and a segment thickness.
[0103] In other words, based on the above deformation fields, the time series tracks a temporal behaviour of a movement feature (e.g., inner radius of the mask, outer radius of the mask, inner curvature of the mask, outer curvature of the mask, bloodpool area, a segment thickness) and extracts this temporal behaviour as the movement feature. This indicates, for example, the amount of movement of a particular segment in a particular anatomic layer of the heart.--, in [0101]-[0103]; and,
-- the deformation direction and/or a deformation magnitude may be determined by considering infinitesimal voxel deformations dx and dy related to the movement of structures present between successive frames in time. Then each image frame may be reduced into a second representation defining the deformation direction and/or the deformation magnitude of the mask over the angle θ, and thus reducing the 4D image data representation of the image frame, I(x, y, z, t), into a second 3D data representation of the image frame, I(θ, z, t) in which a second image intensity now represents the deformation direction and/or the deformation magnitude (of a frame-to-frame motion). This second 3D data representation is illustrated in FIG. 11, indicating for respective angles θ, for respective times t (or correspondingly the frame number), and the number of slices z that the intensity gray value defines a corresponding deformation magnitude or deformation direction. As indicated in FIG. 11, the deformation magnitude or deformation direction indicated based on the infinitesimal voxel deformations dx and dy with regard to the movement of structures present between successive frames in time are different representations with regard to myocardium deformation and motion tracking as compared to the deformation fields defined as mappings indicating a direction and/or a magnitude of how much each pixel in every image frame moves over time.--, in [0114]);
Apparently, Oliveira Ferreira discloses determining “the deformation direction and/or a deformation magnitude” which read on “a correction factor”, that is consistent with above Wetzl’s disclosures of “motion magnitude between temporal adjacent images”, and Oliveira Ferreira discloses “deformation fields” which read on claimed “motion field”;
WANG and Wetzl, and Oliveira Ferreira are combinable as they are in the same field of endeavor: tracking of cardiac/heart, anatomical structure, and soft tissues motion and images processing, rendering and analysis. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify WANG’s apparatus using Wetzl and Oliveira’s teachings by including determining a correction factor for the motion field based on the change to the second tracked contour to WANG’s motion tracking and motion detection in order to determine the amount of movement, and the deformation direction and/or a deformation magnitude related to the movement of structures present between successive frames in time (see Wetzl: e.g., in lines 35-56, col. 5; line 58, col. 5 through line 27, col. 6; and, in line 45, col. 6 to line 16, col. 7; and similarly see Oliveira Ferreira: e.g., in [0101]-[013], and [0114]);
Therefore, amended claims 1-20 are still not patentably distinguishable over the prior art reference(s). Further discussions are addressed in the prior art rejection section below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be /negated by the manner in which the invention was made.
Claims 1, 3-6, 10-11, 13-16, and 20 rejected under 35 U.S.C. 103 as being patentable over WANG (US 10776998 B1); and in view of YU (US 9390514 B2); and further in view of Wetzl (US 11455734 B2), and further in view of Oliveira Ferreira (US 20210125333 A1).
Re Claim 1, WANG discloses an apparatus, comprising: a processor configured to: present a first image of an anatomical structure and a second image of the anatomical structure, wherein the first image indicates a first tracked contour of the anatomical structure, the second image indicates a second tracked contour of the anatomical structure (see WANG: e.g., --tracking cardiac motion and for computing deformation parameters and mechanical properties of the heart from a variety of cardiac MR imaging techniques.--, in lines 55-58, col. 1; Fig. 20, Fig. 21, and Fig. 22 --The reference state was chosen to be the last cardiac phase which had good myocardial-blood pool contrast permitting good contour definition--, in lines 1-52, col. 21), and
the second tracked contour is determined based on the first tracked contour and a motion field between the first image and the second image (see WANG: e.g., --the mid-wall contours at the reference state were deformed to the following phases according to the motion fields obtained from 3D SinMod and results were compared with the deformed contours obtained from 3D HARP [11]. FIG. 20 shows the workflow for comparison of mid-wall contour deformation using motion field from the 3D SinMod method with the 3D HARP method. FIG. 21 shows mid-wall contours at a mid-ventricular slice for all phases. Traversing from top-left to the bottom-right are phases 1 to 20. The red contours are deformed with motion field from 3D SinMod. The green contours are deformed with motion field from 3D HARP. --, in lines 1-52, col. 21),
and the motion field indicates respective displacements of a plurality of pixels from the first image to the second image (see WANG: e.g., --the mid-wall contours at the reference state were deformed to the following phases according to the motion fields obtained from 3D SinMod and results were compared with the deformed contours obtained from 3D HARP [11]. FIG. 20 shows the workflow for comparison of mid-wall contour deformation using motion field from the 3D SinMod method with the 3D HARP method. FIG. 21 shows mid-wall contours at a mid-ventricular slice for all phases. Traversing from top-left to the bottom-right are phases 1 to 20. The red contours are deformed with motion field from 3D SinMod. The green contours are deformed with motion field from 3D HARP. --, in lines 1-52, col. 21);
receive an indication of a change to the second tracked contour (see WANG: e.g., --relative change in length of a mid-wall contour with respect to the reference length at end-diastole--, and Table. 4, in line 27, col. 22 through line 25, col. 23);
WANG however does not explicitly disclose determine a correction factor for the motion field based on the change to the second tracked contour;
Wetzl discloses determine a correction factor based on the change to the second tracked contour {using a trained machine learning network, especially a deep learning network, based on adaptive algorithm } (see Wetzl: e.g., Fig. 4, “motion detector 22” -- the movement data (of the target) is calculated based on an adaptive algorithm trained for calculating movement data on the basis of temporal adjacent images based on the localization data. Such adaptive algorithm could be trained on a plurality of test images with targets, wherein the localization data of the targets are given and the movement of the target is known.--, in lines 35-56, col. 5; and, -- calculating the movement data (of the target), a quantitative motion curve is generated describing the average motion magnitude between temporal adjacent images. …A ground truth for the motion detection algorithm (e.g. a neural network) is computed based on elastic registration between adjacent image time points, i.e. calculating where pixels that belong to the target structure will move from one time point to the next. If the average of this movement is low, this is considered a resting phase. Based on this ground truth, the algorithm is trained to predict this average movement value. So in such method steps no elastic registration is performed, but the trained algorithm provides results characterizing the motion of the target.
(28) According to a method according to an exemplary embodiment, a motion detection is performed by comparing the calculated movement data (of the target) with a predefined threshold movement value.--, in line 58, col. 5 through line 27, col. 6;
-- two or more objects may be localized in the images, wherein two or more landmarks and/or contours are obtained. The motion data is then preferably calculated based on the relative movement between two or more objects…. Thus, it is preferred to track at least one target “globally”. It is particularly preferred that a target is chosen for this “global tracking” that is in a consistent motion state, wherein the relative motions to other structures should be the same. Then it is possible to depict all examined structures … the motion detector comprise a trained machine learning network, especially a deep learning network. This machine learning network is preferably a neural network and/or a convolutional network and/or an end-to-end network.--, in line 45, col. 6 to line 16, col. 7);
Apparently, Wetz discloses determine “a correction factor”, which is the calculated “movement” and “motion data”, because these detected/calculated, or predicted “movement value” and “motion data” read on “a correction factor” as describing the average motion magnitude between temporal adjacent images, and to depict all examined structures;
Wetzl above disclosed “motion data”, “movement values”, and “a quantitative motion curve is generated” describing the average motion magnitude between temporal adjacent images; Wetzl however does not explicitly disclose “a motion field”;
Oliveira Ferreira discloses determining a correction factor for the motion field based on the change to the second tracked contour (see Oliveira Ferreira: e.g., -- [0101] Here, a trained neural network may predict the two ventricular insertion points on a first image frame of a slice or alternatively on all image frames. Then, deformation fields are considered by the trained neural network. These deformation fields are defined as mappings indicating a direction and/or a magnitude of how much each pixel in every image frame moves over time. By thus following or predicting the positions of the two ventricular insertion points along the image frames in time, the movement feature and classification module 33 (in a segmental classification embodiment) may divide the mask into the respective segments.
[0102] Based on the above deformation fields, the movement feature and classification module 33 may compute one or more time series with regard to the mask characteristics. In particular, for different regions of the mask and for at least a pre-determined number of image frames in time, for example a portion or all image frames in a particular slice, a time series of a tracked deformation feature may be considered with regard to one or more of an inner radius of the mask, an outer radius of the mask, an inner curvature of the mask, an outer curvature of the mask, a bloodpool area, and a segment thickness.
[0103] In other words, based on the above deformation fields, the time series tracks a temporal behaviour of a movement feature (e.g., inner radius of the mask, outer radius of the mask, inner curvature of the mask, outer curvature of the mask, bloodpool area, a segment thickness) and extracts this temporal behaviour as the movement feature. This indicates, for example, the amount of movement of a particular segment in a particular anatomic layer of the heart.--, in [0101]-[0103]; and,
-- the deformation direction and/or a deformation magnitude may be determined by considering infinitesimal voxel deformations dx and dy related to the movement of structures present between successive frames in time. Then each image frame may be reduced into a second representation defining the deformation direction and/or the deformation magnitude of the mask over the angle θ, and thus reducing the 4D image data representation of the image frame, I(x, y, z, t), into a second 3D data representation of the image frame, I(θ, z, t) in which a second image intensity now represents the deformation direction and/or the deformation magnitude (of a frame-to-frame motion). This second 3D data representation is illustrated in FIG. 11, indicating for respective angles θ, for respective times t (or correspondingly the frame number), and the number of slices z that the intensity gray value defines a corresponding deformation magnitude or deformation direction. As indicated in FIG. 11, the deformation magnitude or deformation direction indicated based on the infinitesimal voxel deformations dx and dy with regard to the movement of structures present between successive frames in time are different representations with regard to myocardium deformation and motion tracking as compared to the deformation fields defined as mappings indicating a direction and/or a magnitude of how much each pixel in every image frame moves over time.--, in [0114]);
Apparently, Oliveira Ferreira discloses determining “the deformation direction and/or a deformation magnitude” which read on “a correction factor”, that is consistent with above Wetzl’s disclosures of “motion magnitude between temporal adjacent images”, and Oliveira Ferreira discloses “deformation fields” which read on claimed “motion field”;
WANG and Wetzl, and Oliveira Ferreira are combinable as they are in the same field of endeavor: tracking of cardiac/heart, anatomical structure, and soft tissues motion and images processing, rendering and analysis. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify WANG’s apparatus using Wetzl and Oliveira’s teachings by including determining a correction factor for the motion field based on the change to the second tracked contour to WANG’s motion tracking and motion detection in order to determine the amount of movement, and the deformation direction and/or a deformation magnitude related to the movement of structures present between successive frames in time (see Wetzl: e.g., in lines 35-56, col. 5; line 58, col. 5 through line 27, col. 6; and, in line 45, col. 6 to line 16, col. 7; and similarly see Oliveira Ferreira: e.g., in [0101]-[013], and [0114]);
WANG however does not explicitly disclose adjust the motion field indicates the respective displacements of the plurality of pixels from the first image to the second image based on the determined correction factor;
YU discloses adjust the motion field indicates the respective displacements of the plurality of pixels from the first image to the second image based on the determined correction factor (see YU: e.g., -- According to a first image motion analysis technique a first image is warped according to a locally affine model. The first warped image and a second image are compared and a match between the first warped image and the second image is discovered. A value for a motion parameter is estimated based on the match. According to a second image motion analysis technique, an image sequence is converted into an input matrix.--, in abstract {herein “value for a motion parameter is estimated based on the match” read on “adjust the motion field”}; also see: Elasticity imaging is a non-invasive method for measuring mechanical properties of soft tissue, which facilitates the diagnosis of various pathologies. Images are taken of soft tissue before and after application of a mechanical force. The images are correlated, and motion or deformation of the tissues can be inferred through a motion tracking algorithm. (4) An example of a motion tracking algorithm that can be used with elasticity imaging is speckle tracking. …speckle patterns do change after tissue deformation. Due to the change in speckle patterns after tissue deformation (“feature-motion decorrelation”--, in lines 33-53, col. 1, and, -- 73) The image warping component warps 102 the first image 202 according to a locally affine model to produce a warped first image 206 (or warped first set of images). The warped first image 206 and the second image 204 are fed into the estimation component 104. The estimation component 104 searches for a match between the warped first image 206 and a second image 204 as a function of a pre-defined matching measure (matching component 208), and estimate a motion parameter based on the match (parameter component 210). The estimation component 104 can facilitate the estimation of tissue motion and the removal of tissue pattern variations caused by the tissue motion by applying an inverse motion field to the image to recover tissue motion.
(74) The matching component 208 can employ one or more matching measures to find the match. The matching measures can include a correlation coefficient between the warped first image 206 and the second image 204, a sum of squared distance between the warped first image 206 and the second image 204, a sum of absolute distance between the warped first image 206 and the second image 204.
(75) The parameter component 210 can facilitate the estimation of motion parameter…
optimizes a motion parameter to facilitate compensation of feature motion decorrelation, according to an embodiment of the disclosure. System 300 receives inputs of a first image 202 or set of images and a second image 204 or set of images. The first image 202 and the second image 204 can be taken before and after application of a mechanical force. If the object is biological tissue, the images can be taken before and after tissue motion.--, in line 51, col. 10 through line 62, col. 11; and see:
-- (88) Referring now to FIG. 6, illustrated is an example non-limiting process flow diagram of a method 600 that optimizes a motion parameter to facilitate compensation of feature motion decorrelation, according to an embodiment of the disclosure. The method 600 is an affine warping method. At element 602, a first image is received. At element 604, a second image is received. At element 606, a motion parameter is initialized. At element 606, the first image is warped to a first warped image. Based on the first warped image and the second image, at element 610, a matching pattern between the first warped image and the second image is searched for based on a matching metric. At element 612, a motion parameter is estimated based on the match. At element 614, it is determined whether the motion parameter is optimal. At element 616, if the motion parameter is optimal, the motion parameter is output to facilitate determination of motion and/or mechanical properties of an imaged object. At 618, if the motion parameter is not optimal, the motion parameter is adjusted and acts 608-614 are repeated with an input of the adjusted motion parameter.--, in lines 24-43, col. 13);
WANG (as modified by Wetzl, and Oliveira Ferreira) and YU are combinable as they are in the same field of endeavor: tracking of cardiac/heart, anatomical structure, and soft tissues motion and images processing, rendering and analysis. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify WANG (as modified by Wetzl, and Oliveira Ferreira)’s apparatus using YU’s teachings by including adjust the motion field indicates the respective displacements of the plurality of pixels from the first image to the second image based on the determined correction factor to WANG (as modified by Wetzl, and Oliveira Ferreira)’s motion tracking and motion field in order to optimize motion parameters to facilitate compensation of feature motion decorrelation (see YU: e.g. in abstract, and in lines 33-53, col. 1, line 51, col. 10 through line 62, col. 11, and lines 24-43, col. 13);
WANG as modified by Wetzl and Oliveira Ferreira, and YU further disclose modify the second tracked contour of the anatomical structure in the second image based at least on the adjusted motion field (see WANG: e.g., --the mid-wall contours at the reference state were deformed to the following phases according to the motion fields obtained from 3D SinMod and results were compared with the deformed contours obtained from 3D HARP [11]. FIG. 20 shows the workflow for comparison of mid-wall contour deformation using motion field from the 3D SinMod method with the 3D HARP method. FIG. 21 shows mid-wall contours at a mid-ventricular slice for all phases. Traversing from top-left to the bottom-right are phases 1 to 20. The red contours are deformed with motion field from 3D SinMod. The green contours are deformed with motion field from 3D HARP. --, in lines 1-52, col. 21; also see YU: e.g., --(192) The contour of the valve generated from the automatic method was quantitatively compared to the manual tracing of the valve. The mean absolute distance (MAD) was calculated between the contour of the valve generated from the automatic method and manual tracing.--, in lines 20-38, col. 27 {deformation read on modification of contours}).
Re Claim 3, WANG as modified by Wetzl and Oliveira Ferreira, and YU further disclose wherein the processor is configured to determine the correction factor for the motion field using an artificial neural network pre-trained for predicting the correction factor based on a difference between the first segmentation mask and the second segmentation mask (see Wetzl: e.g., Fig. 4, “motion detector 22” -- the movement data (of the target) is calculated based on an adaptive algorithm trained for calculating movement data on the basis of temporal adjacent images based on the localization data. Such adaptive algorithm could be trained on a plurality of test images with targets, wherein the localization data of the targets are given and the movement of the target is known.--, in lines 35-56, col. 5; and, -- calculating the movement data (of the target), a quantitative motion curve is generated describing the average motion magnitude between temporal adjacent images. …A ground truth for the motion detection algorithm (e.g. a neural network) is computed based on elastic registration between adjacent image time points, i.e. calculating where pixels that belong to the target structure will move from one time point to the next. If the average of this movement is low, this is considered a resting phase. Based on this ground truth, the algorithm is trained to predict this average movement value. So in such method steps no elastic registration is performed, but the trained algorithm provides results characterizing the motion of the target.
(28) According to a method according to an exemplary embodiment, a motion detection is performed by comparing the calculated movement data (of the target) with a predefined threshold movement value.--, in line 58, col. 5 through line 27, col. 6;
-- two or more objects may be localized in the images, wherein two or more landmarks and/or contours are obtained. The motion data is then preferably calculated based on the relative movement between two or more objects…. Thus, it is preferred to track at least one target “globally”. It is particularly preferred that a target is chosen for this “global tracking” that is in a consistent motion state, wherein the relative motions to other structures should be the same. Then it is possible to depict all examined structures … the motion detector comprise a trained machine learning network, especially a deep learning network. This machine learning network is preferably a neural network and/or a convolutional network and/or an end-to-end network.--, in line 45, col. 6 to line 16, col. 7;
also see Oliveira Ferreira: e.g., -- [0101] Here, a trained neural network may predict the two ventricular insertion points on a first image frame of a slice or alternatively on all image frames. Then, deformation fields are considered by the trained neural network. These deformation fields are defined as mappings indicating a direction and/or a magnitude of how much each pixel in every image frame moves over time. By thus following or predicting the positions of the two ventricular insertion points along the image frames in time, the movement feature and classification module 33 (in a segmental classification embodiment) may divide the mask into the respective segments.
[0102] Based on the above deformation fields, the movement feature and classification module 33 may compute one or more time series with regard to the mask characteristics. In particular, for different regions of the mask and for at least a pre-determined number of image frames in time, for example a portion or all image frames in a particular slice, a time series of a tracked deformation feature may be considered with regard to one or more of an inner radius of the mask, an outer radius of the mask, an inner curvature of the mask, an outer curvature of the mask, a bloodpool area, and a segment thickness.
[0103] In other words, based on the above deformation fields, the time series tracks a temporal behaviour of a movement feature (e.g., inner radius of the mask, outer radius of the mask, inner curvature of the mask, outer curvature of the mask, bloodpool area, a segment thickness) and extracts this temporal behaviour as the movement feature. This indicates, for example, the amount of movement of a particular segment in a particular anatomic layer of the heart.--, in [0101]-[0103]; and,
-- the deformation direction and/or a deformation magnitude may be determined by considering infinitesimal voxel deformations dx and dy related to the movement of structures present between successive frames in time. Then each image frame may be reduced into a second representation defining the deformation direction and/or the deformation magnitude of the mask over the angle θ, and thus reducing the 4D image data representation of the image frame, I(x, y, z, t), into a second 3D data representation of the image frame, I(θ, z, t) in which a second image intensity now represents the deformation direction and/or the deformation magnitude (of a frame-to-frame motion). This second 3D data representation is illustrated in FIG. 11, indicating for respective angles θ, for respective times t (or correspondingly the frame number), and the number of slices z that the intensity gray value defines a corresponding deformation magnitude or deformation direction. As indicated in FIG. 11, the deformation magnitude or deformation direction indicated based on the infinitesimal voxel deformations dx and dy with regard to the movement of structures present between successive frames in time are different representations with regard to myocardium deformation and motion tracking as compared to the deformation fields defined as mappings indicating a direction and/or a magnitude of how much each pixel in every image frame moves over time.--, in [0114];
further see WANG: e.g., --the mid-wall contours at the reference state were deformed to the following phases according to the motion fields obtained from 3D SinMod and results were compared with the deformed contours obtained from 3D HARP [11]. FIG. 20 shows the workflow for comparison of mid-wall contour deformation using motion field from the 3D SinMod method with the 3D HARP method. FIG. 21 shows mid-wall contours at a mid-ventricular slice for all phases. Traversing from top-left to the bottom-right are phases 1 to 20. The red contours are deformed with motion field from 3D SinMod. The green contours are deformed with motion field from 3D HARP. --, in lines 1-52, col. 21; also see YU: e.g., --(192) The contour of the valve generated from the automatic method was quantitatively compared to the manual tracing of the valve. The mean absolute distance (MAD) was calculated between the contour of the valve generated from the automatic method and manual tracing.--, in lines 20-38, col. 27 {deformation read on modification of contours}).
Re Claim 4, WANG as modified by Wetzl and Oliveira Ferreira, and YU further disclose the change to the second tracked contour includes a movement of a part of the second tracked contour from a first location to a second location, and wherein the indication of the change to the second tracked contour is received based on a user input that includes at least one of a mouse click, a mouse movement, or a tactile input (see WANG: e.g., -- (141) c. Comparison of Warped with Manually Delineated Tag Lines
(142) All tag lines on 11 slices (from apex to base) and over 11 systolic phases in the same seven 3D CSPAMM tagged data sets were manually delineated. Subsequently, the manually delineated tag lines from each time were warped to time t+1 and the location of the warped tag lines were compared to location of manually delineated tag lines and an average error for all slices at each phase was computed.
(143) FIG. 23 displays the average error as a function of time for each of the 7 data sets. 3D SinMod's average error as a function of time for determining tag line displacements during systole for 7 in-vivo data sets. Results for data sets 1, 2, 3, 4, 5, 6, and 7 are differentiated with red, green, black, yellow, blue, cyan, and magenta colors. Please note that the error for each time point was calculated from the error between tag line locations from time t warped to time t+1 and manually delineated tag lines on all image slices. As may be observed, all errors are in the sub-pixel range.
(144) 9. Results
(145) a. Visualization of Motion Fields
(146) Left ventricular endocardial and epicardial contours were traced manually at all phases for all 3D data sets. The calculated 3D motion fields at apex, apical, mid-cavity, and basal slices are shown in FIG. 24 to FIG. 30 for data set 1 to 7. FIG. 24 is the end systolic 3D motion field for slice 1, 3, 6, and 10 for data set 1. FIG. 25 is the end systolic 3D motion field for slice 1, 3, 6, and 10 for data set 2. FIG. 26 is the end systolic 3D motion field for slice 1, 3, 6, and 10 for data set 3. FIG. 27 is the end systolic 3D motion field for slice 1, 3, 6, and 10 for data set 4. FIG. 28 is the end systolic 3D motion field for slice 1, 3, 6, and 10 for data set 5. FIG. 29 is the end systolic 3D motion field for slice 1, 3, 6, and 10 for data set 6. FIG. 30 is the end systolic 3D motion field for slice 1, 3, 6, and 10 for data set 7.
(147) The projected 2D end-systolic motion fields on selected slices for each data set are shown in FIG. 31 to FIG. 37. FIG. 31 is the end systolic motion field on slices 1, 3, 5, 7, 9, 10 for data set 1. FIG. 32 is the end systolic motion field on slices 1, 3, 5, 7, 9, 10 for data set 2. FIG. 33 is the end systolic motion field on slices 1, 3, 5, 7, 9, 10 for data set 3. FIG. 34 is the end systolic motion field on slices 1, 3, 5, 7, 9, 10 for data set 4. FIG. 35 is the end systolic motion field on slices 1, 3, 5, 7, 9, 10 for data set 5. FIG. 36 is the end systolic motion field on slices 1, 3, 5, 7, 9, 10 for data set 6. FIG. 37 is the end systolic motion field on slices 1, 3, 5, 7, 9, 10 for data set 7. In each case, row 1 is the apex (i.e., apical location), row 2 is the mid-ventricular (i.e., mid-cavity location), and row 3 is the base (i.e., basal location).-- , in line 44, col. 21 through line 26, col. 27;
also see YU: e.g., --(192) The contour of the valve generated from the automatic method was quantitatively compared to the manual tracing of the valve. The mean absolute distance (MAD) was calculated between the contour of the valve generated from the automatic method and manual tracing.--, in lines 20-38, col. 27 {deformation read on modification of contours}).
Re Claim 5, WANG as modified by Wetzl, and Oliveira Ferreira, and YU further disclose wherein the processor being configured determine the correction factor for the motion field based on the change to the second tracked contour comprises the processor being configured to:
identify a first set of feature points associated with the second tracked contour prior to the change to the second tracked contour; identify a second set of feature points associated with the second tracked contour after the change to the second tracked contour (see Wetzl: e.g., Fig. 4, “motion detector 22” -- the movement data (of the target) is calculated based on an adaptive algorithm trained for calculating movement data on the basis of temporal adjacent images based on the localization data. Such adaptive algorithm could be trained on a plurality of test images with targets, wherein the localization data of the targets are given and the movement of the target is known.--, in lines 35-56, col. 5; and, -- calculating the movement data (of the target), a quantitative motion curve is generated describing the average motion magnitude between temporal adjacent images. …A ground truth for the motion detection algorithm (e.g. a neural network) is computed based on elastic registration between adjacent image time points, i.e. calculating where pixels that belong to the target structure will move from one time point to the next. If the average of this movement is low, this is considered a resting phase. Based on this ground truth, the algorithm is trained to predict this average movement value. So in such method steps no elastic registration is performed, but the trained algorithm provides results characterizing the motion of the target.
(28) According to a method according to an exemplary embodiment, a motion detection is performed by comparing the calculated movement data (of the target) with a predefined threshold movement value.--, in line 58, col. 5 through line 27, col. 6;
-- two or more objects may be localized in the images, wherein two or more landmarks and/or contours are obtained. The motion data is then preferably calculated based on the relative movement between two or more objects…. Thus, it is preferred to track at least one target “globally”. It is particularly preferred that a target is chosen for this “global tracking” that is in a consistent motion state, wherein the relative motions to other structures should be the same. Then it is possible to depict all examined structures … the motion detector comprise a trained machine learning network, especially a deep learning network. This machine learning network is preferably a neural network and/or a convolutional network and/or an end-to-end network.--, in line 45, col. 6 to line 16, col. 7;
also see Oliveira Ferreira: e.g., -- [0101] Here, a trained neural network may predict the two ventricular insertion points on a first image frame of a slice or alternatively on all image frames. Then, deformation fields are considered by the trained neural network. These deformation fields are defined as mappings indicating a direction and/or a magnitude of how much each pixel in every image frame moves over time. By thus following or predicting the positions of the two ventricular insertion points along the image frames in time, the movement feature and classification module 33 (in a segmental classification embodiment) may divide the mask into the respective segments.
[0102] Based on the above deformation fields, the movement feature and classification module 33 may compute one or more time series with regard to the mask characteristics. In particular, for different regions of the mask and for at least a pre-determined number of image frames in time, for example a portion or all image frames in a particular slice, a time series of a tracked deformation feature may be considered with regard to one or more of an inner radius of the mask, an outer radius of the mask, an inner curvature of the mask, an outer curvature of the mask, a bloodpool area, and a segment thickness.
[0103] In other words, based on the above deformation fields, the time series tracks a temporal behaviour of a movement feature (e.g., inner radius of the mask, outer radius of the mask, inner curvature of the mask, outer curvature of the mask, bloodpool area, a segment thickness) and extracts this temporal behaviour as the movement feature. This indicates, for example, the amount of movement of a particular segment in a particular anatomic layer of the heart.--, in [0101]-[0103]; and,
-- the deformation direction and/or a deformation magnitude may be determined by considering infinitesimal voxel deformations dx and dy related to the movement of structures present between successive frames in time. Then each image frame may be reduced into a second representation defining the deformation direction and/or the deformation magnitude of the mask over the angle θ, and thus reducing the 4D image data representation of the image frame, I(x, y, z, t), into a second 3D data representation of the image frame, I(θ, z, t) in which a second image intensity now represents the deformation direction and/or the deformation magnitude (of a frame-to-frame motion). This second 3D data representation is illustrated in FIG. 11, indicating for respective angles θ, for respective times t (or correspondingly the frame number), and the number of slices z that the intensity gray value defines a corresponding deformation magnitude or deformation direction. As indicated in FIG. 11, the deformation magnitude or deformation direction indicated based on the infinitesimal voxel deformations dx and dy with regard to the movement of structures present between successive frames in time are different representations with regard to myocardium deformation and motion tracking as compared to the deformation fields defined as mappings indicating a direction and/or a magnitude of how much each pixel in every image frame moves over time.--, in [0114];
further see WANG: e.g., --FIG. 5 is a schematic diagram showing displacement of a point during deformation--, and, --In cardiology, stress is interaction forces acting across surfaces between adjacent regions of muscle, while strain represents the change of shape at any point in the wall between the original reference state and the subsequent deformed state--, in lines 52-65, col. 10; and, --(75) Regional myocardial stress and strain have direct or indirect relationship with cardiac diseases. At present, regional strain distribution measurement is much more practical than the stress measurement. For measuring systolic function, end-diastole is a conventional unstrained reference state. An illustration of the relationship between the initial (initial, reference, and undeformed configurations are interchangeable throughout this section) and deformed configuration is shown in FIG. 5. Suppose a material point at position X:(X.sub.1, X.sub.2, X.sub.3) in the undeformed solid moves to a new position x:(x.sub.1, x.sub.2, x.sub.3) when the solid is loaded. A mapping x=χ(X, t) would describe the motion. The displacement of the material point u is u(t)=x(t)−X (1)
One measurement of the deformation in the reference configuration is the length change of a segment dX at point X: --, in lines 2-050, col. 11; also see: --the mid-wall contours at the reference state were deformed to the following phases according to the motion fields obtained from 3D SinMod and results were compared with the deformed contours obtained from 3D HARP [11]. FIG. 20 shows the workflow for comparison of mid-wall contour deformation using motion field from the 3D SinMod method with the 3D HARP method. FIG. 21 shows mid-wall contours at a mid-ventricular slice for all phases. Traversing from top-left to the bottom-right are phases 1 to 20. The red contours are deformed with motion field from 3D SinMod. The green contours are deformed with motion field from 3D HARP. --, in lines 1-52, col. 21),
determine the correction factor for the motion field using an artificial neural network pre-trained for predicting the correction factor based on a difference between the first set of feature points and the second set of feature points (see Wetzl: e.g., Fig. 4, “motion detector 22” -- the movement data (of the target) is calculated based on an adaptive algorithm trained for calculating movement data on the basis of temporal adjacent images based on the localization data. Such adaptive algorithm could be trained on a plurality of test images with targets, wherein the localization data of the targets are given and the movement of the target is known.--, in lines 35-56, col. 5; and, -- calculating the movement data (of the target), a quantitative motion curve is generated describing the average motion magnitude between temporal adjacent images. …A ground truth for the motion detection algorithm (e.g. a neural network) is computed based on elastic registration between adjacent image time points, i.e. calculating where pixels that belong to the target structure will move from one time point to the next. If the average of this movement is low, this is considered a resting phase. Based on this ground truth, the algorithm is trained to predict this average movement value. So in such method steps no elastic registration is performed, but the trained algorithm provides results characterizing the motion of the target.
(28) According to a method according to an exemplary embodiment, a motion detection is performed by comparing the calculated movement data (of the target) with a predefined threshold movement value.--, in line 58, col. 5 through line 27, col. 6;
-- two or more objects may be localized in the images, wherein two or more landmarks and/or contours are obtained. The motion data is then preferably calculated based on the relative movement between two or more objects…. Thus, it is preferred to track at least one target “globally”. It is particularly preferred that a target is chosen for this “global tracking” that is in a consistent motion state, wherein the relative motions to other structures should be the same. Then it is possible to depict all examined structures … the motion detector comprise a trained machine learning network, especially a deep learning network. This machine learning network is preferably a neural network and/or a convolutional network and/or an end-to-end network.--, in line 45, col. 6 to line 16, col. 7;
{Apparently, Wetz discloses determine “a correction factor”, which is the calculated “movement” and “motion data”, because these detected/calculated, or predicted “movement value” and “motion data” read on “a correction factor” as describing the average motion magnitude between temporal adjacent images, and to depict all examined structures};
also see Oliveira Ferreira: e.g., -- [0101] Here, a trained neural network may predict the two ventricular insertion points on a first image frame of a slice or alternatively on all image frames. Then, deformation fields are considered by the trained neural network. These deformation fields are defined as mappings indicating a direction and/or a magnitude of how much each pixel in every image frame moves over time. By thus following or predicting the positions of the two ventricular insertion points along the image frames in time, the movement feature and classification module 33 (in a segmental classification embodiment) may divide the mask into the respective segments.
[0102] Based on the above deformation fields, the movement feature and classification module 33 may compute one or more time series with regard to the mask characteristics. In particular, for different regions of the mask and for at least a pre-determined number of image frames in time, for example a portion or all image frames in a particular slice, a time series of a tracked defor