DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s submission filed 1/22/2025 has been entered. The claims 2, 16 and 19 have been cancelled. The claims 1, 3-15, 17, 18 and 20 are pending in the current application.
Response to Arguments
Applicant's arguments filed 1/22/2026 have been fully considered but they are not found persuasive.
In Remarks, applicant individually attacked each of the cited references and argued in essence with respect to the claim limitation where the second assistant information includes: one or more quantification information OR visualization information indicating one or more of a characteristic of the lesion OR the finding within the target region, OR a progression of the lesion or the finding within the target region.
Claim limitation is subject to broadest reasonable interpretation consistent with applicant’s specification. Limitation from the specification cannot be imported into the claims.
The cited references meet applicant’s contended claim limitation.
Atsumori teaches at Paragraph 0096 that a place suspected as a lesion is displayed in the region 202 and at Paragraph 0102 that a lesion 431 is highlighted and the lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 of FIG. 14. Moreover, Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
However, applicant speculated that simultaneously highlight lesions caused by two types of diseases does not imply that different types of lesions are classified or distinctly highlighted. Applicant speculated that the different lesions caused by two types of diseases are highlighted with the same annotation.
The attribute data such as a disease name meets the claim one or more of a characteristic of the lesion or the finding within the target region.
Applicant contended that Atsumori’s disclosure at Paragraph 0115 does not imply that different lesion types are classified. This argument is unfounded. Simultaneously highlight lesions caused by two types of diseases by adding attribute data such as a disease name means that two different lesion types (two types of diseases) are classified. Moreover, applicant’s arguments are not commensurate with the broadest scope of the claim invention. The claim invention broadly recites “one or more of quantification information or visualization information”.
Applicant attempted to specifically map the first assistant information and the second assistant information corresponding to the two types of diseases in Atsumori. However, Atsumori also teaches at FIG. 6 first assistant information (212) based on a first threshold and a second assistant information (202) even for the same type of disease within the region 211.
Even if applicant were permitted to specifically map the first assistant information and the second assistant information corresponding to the two types of diseases, applicant ignored the combination of the references cited in the Office Action. First of all, applicant speculated that Atsumori’s highlighting of the two types of diseases does not teach classifying the two types of diseases or distinctly highlighting the two types of diseases. Secondly, applicant ignored the combination of the references in an obviousness type of rejection.
In Remarks, applicant completely ignored Gogin’s specific teaching in relation to the claim invention with general allegation. Applicant’s arguments are unavailing as applicant failed to address the specific teaching of Gogin in combination with Atsumori. Applicant made general allegation with respect to Gogin’s teaching in relation to the claim limitation. Gogin teaches obtaining the characteristic of the lung region by visualizing a first region corresponding to density 1 with the second annotation information and visualizing the second region corresponding to density 2 with the first annotation information. The combination of Gogin and Atmori teaches distinctly highlight two types of diseases with different visualization information.
Gogin teaches at FIG. 6-7 and Paragraph 0081-0090 that generating the second assistant information at FIG. 7 different from the first assistance information of FIG. 6 for the CT images of a patient using the same CT acquisition.
The region corresponding to density 1 is indicated with the second annotation information based on the first annotation information, wherein the diagnostic information of the lung region is visualized based on density 1 and density 2 (e.g., displayed using different colors in Paragraph 0071).
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0087 that the images 710 may similarly comprise regions corresponding to a plurality of densities (e.g., density 1 through density 5), which may be generated and/or identified using the same plurality of range values (and/or other settings) used in generating images 610 of FIG. 6. However, the use of quantification lung reconstruction allows for enhanced representation of regions or areas corresponding to certain densities (e.g., density 1) which may be associated with certain respiratory conditions (e.g., COVID-19).
Moreover, applicant separately attacked Schirman with allegation that Schirman does not teach the contended claim limitation. The examiner cannot concur.
Applicant attacked Shirman’s technique of using a difference image to detect the tumor growth of a tumor. However, Shirman’s display of the tumor region is provided with respect to a medical image, as opposed to applicant’s contention of a difference image. The difference image is merely used to detect to progression of the lesion in the medical image.
Applicant ignored Schirman’s classification of two types of diseases by labeling the two types of diseases or two types of progression of the same disease.
Schirman explicitly teaches a progression of the lesion by indicating the tumor growth (growth types). Moreover, the regions labeled in the difference image corresponds to the regions of the medical image (see Paragraph 0041 and FIG. 2).
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-15, 17, 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Atsumori et al. US-PGPUB No. 2023/0306604 (hereinafter Atsumori) in view of
Deepthi S. et al. US-PGPUB No. 2020/0311877 (hereinafter Deepthi);
Schirman et al. US-PGPUB No. 2019/0188853 (hereinafter Schirman);
Gogin et al. US-PGPUB No. 2022/0346665 (hereinafter Gogin);
Bazan et al. US-PGPUB No. 2023/0330094 (hereinafter Bazan).
Re Claim 1:
Atsumori in view of Gogin/Deepthi/Schirman/Bazan teaches a medical image diagnosis assistant apparatus comprising a processor, wherein the processor is configured to (Atsumori teaches at FIG. 3 and Paragraph 0119 that the described functions may be implemented through software by using a processor such as a CPU 112 to interpret and execute a program for realizing each function and a program may be stored in a storage device such as a memory 111):
acquire information about at least one target region indicating one or more of a lesion, or a finding including a lesion candidate in a medical image (
Atsumori teaches at Paragraph 0028-0029 that an image processing system Z includes a medical image capturing unit M and the image processing apparatus 1 performs lesion highlighting processing of highlighting a lesion in the medical image 201. Atsumori teaches at Paragraph 0096 that a place suspected as a lesion is displayed in the region 202 and at Paragraph 0102 that a lesion 431 is highlighted and the lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 of FIG. 14. Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0083 that the images 610 may comprise regions (e.g., displayed using different colors in the images) corresponding to a plurality of densities (e.g., density 1 through density 5). The regions may be identified (and thus displayed accordingly—e.g., using a color assigned to the corresponding density), based on processing of the image data using a plurality of range values (e.g., intensity (HU) thresholds) which may be used to define and detect these different densities—that is, with range of values (or any other settings, though not shown) for each the densities (density 1 through density 5).
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times.
Deepthi teaches at FIG. 1 and Paragraph 0029-0042 that the MRI system 10 acquires information (MR images) about at least one target region of the subject 16);
acquire distribution information about a distribution of signal intensity values within the target region (
Atsumori teaches at Paragraph 0031 that the computer 100 includes an image acquisition unit 101, a region setting unit 102, an intensity value group ratio distribution calculation unit and an intensity value determination unit 107 and at FIG. 8 and Paragraph 0034 that the intensity value group ratio distribution calculation unit 103 calculates histograms 301 and 302 and histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution.
Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0083 that the images 610 may comprise regions (e.g., displayed using different colors in the images) corresponding to a plurality of densities (e.g., density 1 through density 5). The regions may be identified (and thus displayed accordingly—e.g., using a color assigned to the corresponding density), based on processing of the image data using a plurality of range values (e.g., intensity (HU) thresholds) which may be used to define and detect these different densities—that is, with range of values (or any other settings, though not shown) for each the densities (density 1 through density 5).
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times.
Deepthi teaches at Paragraph [0111] that an intensity distribution of the MR image/map is determined (that is, an intensity distribution of both the foreground and background is determined). In one embodiment, a histogram of pixel intensities may be generated, wherein the x-axis includes the pixel/voxel intensity, and the y-axis includes the number of pixels/voxels at each intensity level.
Deepthi teaches at Paragraph [0112] that a first intensity threshold is set to an intensity which maximizes a between class variance (or equivalently, the first intensity threshold is set to an intensity which minimized the within class variance), of the intensity distribution determined at step 508. The between class variance is calculated for a first and second class, of the previously determined intensity distribution, by dividing the intensity distribution into a first class comprising pixels with intensity less than the selected intensity threshold, and a second class comprising pixels with intensity greater than the selected intensity threshold.
Deepthi teaches at Paragraph 0119 that generating a mask for the foreground includes comparing an intensity of each pixel/voxel of the foreground of the MR image/image against a lower intensity threshold and an upper intensity threshold of the selected intensity range, and in response to the intensity of the pixel/voxel being within the selected intensity range (that is, in response to the intensity of a pixel/voxel being both below an upper intensity threshold and above a lower intensity threshold), setting a mask value corresponding to the pixel/voxel to one, or otherwise setting the mask value corresponding to the pixel/voxel to zero if the intensity of the pixel/voxel is outside of the selected intensity range. In some embodiments, mask values corresponding to the background pixel/voxels are zero.
Deepthi teaches at Paragraph [0126] that FIG. 9 also shows default threshold 908, illustrating the original MR image 902 with a pre-determined intensity threshold (i.e., default threshold) applied, wherein pixels within the original MR image 902 below the intensity threshold are suppressed/masked. As can be seen in FIG. 9, numerous holes occur within the anatomical regions of default threshold 908. Predicted threshold 910 depicts original MR image 902 with a predicted threshold applied (determined as the max intensity value of all background pixels) applied. Otsu threshold 912 depicts original MR image 902 with an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 902). Although predicted threshold 910 and Otsu threshold 912 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 902 is further exacerbated compared to default threshold 908. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto.
Deepthi teaches at Paragraph [0129] that FIG. 10 also shows default threshold 1008, illustrating the original MR image 1002 with a pre-determined intensity threshold applied, wherein pixels within the original MR image 1002 below the intensity threshold are suppressed/masked. As can be seen in FIG. 10, numerous holes occur within the anatomical regions of default threshold 1008. Predicted threshold 1010 and Otsu threshold 1012 likewise depict original MR image 1002 with either a predicted threshold applied (determined as the max value of all background pixels) or an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 1002), respectively. Although predicted threshold 1010 and Otsu threshold 1012 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 1002 is further exacerbated compared to default threshold 1008. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto.
Deepthi teaches at Paragraph 0095 that a first intensity threshold is determined. Method 500 discusses the process of determining the first intensity threshold in more detail. In some embodiments, the intensity threshold is calculated based on an intensity distribution of the received MR image/map according to Otsu's method. In some embodiments, the intensity threshold may be selected based on the intensity of the pixels/voxels classified as background at 404 of method 400. As a more specific example, step 406 may comprise determining a maximum pixel/voxel intensity of the background, and setting the intensity threshold to the determined maximum background intensity.
Deepthi teaches at Paragraph [0096] that a first mask is generated based on the first intensity threshold. In some embodiments, generating the first mask may comprise comparing a pixel/voxel intensity of each background pixel/voxel with the first intensity threshold, and responsive to the intensity being lower than the first intensity threshold, setting a mask value corresponding to the pixel/voxel to zero, or otherwise setting the mask value corresponding to the pixel to one. Mask values corresponding to foreground pixels/voxels are set to one.
Deepthi teaches at Paragraph [0100] that a second mask is generated based on the user selected intensity threshold. The second mask may suppress output of pixels/voxels (e.g., in the background) with an intensity below the user selected intensity threshold);
generate first assistant information based on a first threshold for the signal intensity values within the target region (
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222.
Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0083 that the images 610 may comprise regions (e.g., displayed using different colors in the images) corresponding to a plurality of densities (e.g., density 1 through density 5). The regions may be identified (and thus displayed accordingly—e.g., using a color assigned to the corresponding density), based on processing of the image data using a plurality of range values (e.g., intensity (HU) thresholds) which may be used to define and detect these different densities—that is, with range of values (or any other settings, though not shown) for each the densities (density 1 through density 5).
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times.
Deepthi teaches at Paragraph 0123 that an exemplary MR image 700 of abdomen is shown. Image 700 is an original DWI image without background suppression, which includes a background 702 (the relatively dark peripheral regions of image 700), separated by boundary 706 (the relatively bright boundary), from foreground 704 (the relatively bright region bounded by boundary 706, which corresponds to the imaged anatomical region). Background 702 includes a plurality of pixels, wherein an output/intensity of each of the background pixels has not yet been suppressed. Foreground 704 includes a second plurality of pixels and in Paragraph 0124 that image 800 includes a background 802 separated by boundary 806 and at Paragraph 0127 that applied predicted mask 916 is seen to be in close agreement with the human best effort shown by applied ground truth mask 914, although some pixels on the boundary of the anatomical features are segmented out.
Deepthi teaches at Paragraph [0112] that a first intensity threshold is set to an intensity which maximizes a between class variance (or equivalently, the first intensity threshold is set to an intensity which minimized the within class variance), of the intensity distribution determined at step 508. The between class variance is calculated for a first and second class, of the previously determined intensity distribution, by dividing the intensity distribution into a first class comprising pixels with intensity less than the selected intensity threshold, and a second class comprising pixels with intensity greater than the selected intensity threshold.
Deepthi teaches at Paragraph 0119 that generating a mask for the foreground includes comparing an intensity of each pixel/voxel of the foreground of the MR image/image against a lower intensity threshold and an upper intensity threshold of the selected intensity range, and in response to the intensity of the pixel/voxel being within the selected intensity range (that is, in response to the intensity of a pixel/voxel being both below an upper intensity threshold and above a lower intensity threshold), setting a mask value corresponding to the pixel/voxel to one, or otherwise setting the mask value corresponding to the pixel/voxel to zero if the intensity of the pixel/voxel is outside of the selected intensity range. In some embodiments, mask values corresponding to the background pixel/voxels are zero.
Deepthi teaches at Paragraph [0126] that FIG. 9 also shows default threshold 908, illustrating the original MR image 902 with a pre-determined intensity threshold (i.e., default threshold) applied, wherein pixels within the original MR image 902 below the intensity threshold are suppressed/masked. As can be seen in FIG. 9, numerous holes occur within the anatomical regions of default threshold 908. Predicted threshold 910 depicts original MR image 902 with a predicted threshold applied (determined as the max intensity value of all background pixels) applied. Otsu threshold 912 depicts original MR image 902 with an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 902). Although predicted threshold 910 and Otsu threshold 912 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 902 is further exacerbated compared to default threshold 908. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto.
Deepthi teaches at Paragraph [0129] that FIG. 10 also shows default threshold 1008, illustrating the original MR image 1002 with a pre-determined intensity threshold applied, wherein pixels within the original MR image 1002 below the intensity threshold are suppressed/masked. As can be seen in FIG. 10, numerous holes occur within the anatomical regions of default threshold 1008. Predicted threshold 1010 and Otsu threshold 1012 likewise depict original MR image 1002 with either a predicted threshold applied (determined as the max value of all background pixels) or an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 1002), respectively. Although predicted threshold 1010 and Otsu threshold 1012 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 1002 is further exacerbated compared to default threshold 1008. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto.
Deepthi teaches at Paragraph 0095 that a first intensity threshold is determined. Method 500 discusses the process of determining the first intensity threshold in more detail. In some embodiments, the intensity threshold is calculated based on an intensity distribution of the received MR image/map according to Otsu's method. In some embodiments, the intensity threshold may be selected based on the intensity of the pixels/voxels classified as background at 404 of method 400. As a more specific example, step 406 may comprise determining a maximum pixel/voxel intensity of the background, and setting the intensity threshold to the determined maximum background intensity.
Deepthi teaches at Paragraph [0096] that a first mask is generated based on the first intensity threshold. In some embodiments, generating the first mask may comprise comparing a pixel/voxel intensity of each background pixel/voxel with the first intensity threshold, and responsive to the intensity being lower than the first intensity threshold, setting a mask value corresponding to the pixel/voxel to zero, or otherwise setting the mask value corresponding to the pixel to one. Mask values corresponding to foreground pixels/voxels are set to one.
Deepthi teaches at Paragraph [0100] that a second mask is generated based on the user selected intensity threshold. The second mask may suppress output of pixels/voxels (e.g., in the background) with an intensity below the user selected intensity threshold);
wherein the signal intensity values within the target region are classified into a first interval corresponding to a first state and a second interval corresponding to a second state based on the first threshold (
For example, Atsumori teaches at FIG. 6 and FIG. 14 visualizing the inner region based on a first interval distribution information including the intensity values of the pixels of the target lesion region highlighted and second internal distribution information including the intensity values of the pixels of the non-lesion region included in the second assistant information.
Atsumori teaches at Paragraph 0089-0091 that the intensity values in the inner region 212 are grouped into a first intensity value groups with intensity value of 577 or more of pixels of the lesion region and a second intensity value groups with the intensity value group with intensity value of equal to or less than 576 wherein pixels of the non-lesion region having an intensity value equal to or greater than the intensity value 577 are highlighted while the second intensity value groups are not highlighted based on the intensity value threshold 576.
The first intensity value groups are associated with the first interval [577, 1024] and the second intensity value groups are associated with the second interval [32, 576] based on the intensity value threshold 576.
Atsumori not only teaches highlighting a bright place, but also highlighting a dark place. Atsumori teaches at Paragraph 0113-0114 that the threshold setting unit 106 sets the threshold in the broken line 331 in FIG. 12 and selects an intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than the threshold and the display processing unit 108 highlights a pixel having an intensity value smaller than the value of the selected intensity value group.
Atsumori teaches at Paragraph 0114 that the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group and the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222.
Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Gogin teaches at FIG. 4 that based on the first threshold 977, the signal intensity values are classified into a first interval (density 1) and a second interval (density 2).
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0083 that the images 610 may comprise regions (e.g., displayed using different colors in the images) corresponding to a plurality of densities (e.g., density 1 through density 5). The regions may be identified (and thus displayed accordingly—e.g., using a color assigned to the corresponding density), based on processing of the image data using a plurality of range values (e.g., intensity (HU) thresholds) which may be used to define and detect these different densities—that is, with range of values (or any other settings, though not shown) for each the densities (density 1 through density 5).
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times.
); and
generate second assistant information based on the distribution information and the first assistant information (
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222.
Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Gogin teaches at FIG. 4 that generating the second color for the second intensity interval (density 2) based on the intensity distribution information of the target region and the first color for the density 1, wherein the diagnostic characteristic of the target region is visualized based on density 1 and density 2.
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0083 that the images 610 may comprise regions (e.g., displayed using different colors in the images) corresponding to a plurality of densities (e.g., density 1 through density 5). The regions may be identified (and thus displayed accordingly—e.g., using a color assigned to the corresponding density), based on processing of the image data using a plurality of range values (e.g., intensity (HU) thresholds) which may be used to define and detect these different densities—that is, with range of values (or any other settings, though not shown) for each the densities (density 1 through density 5).
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times.
Deepthi teaches at Paragraph 0123 that an exemplary MR image 700 of abdomen is shown. Image 700 is an original DWI image without background suppression, which includes a background 702 (the relatively dark peripheral regions of image 700), separated by boundary 706 (the relatively bright boundary), from foreground 704 (the relatively bright region bounded by boundary 706, which corresponds to the imaged anatomical region). Background 702 includes a plurality of pixels, wherein an output/intensity of each of the background pixels has not yet been suppressed. Foreground 704 includes a second plurality of pixels and in Paragraph 0124 that image 800 includes a background 802 separated by boundary 806 and at Paragraph 0127 that applied predicted mask 916 is seen to be in close agreement with the human best effort shown by applied ground truth mask 914, although some pixels on the boundary of the anatomical features are segmented out.
Deepthi teaches at FIGS. 9-10 generating a second mask based on the applied predicted mask (first assistant information) based on the first threshold + Otsu threshold 1018 and based on the intensity distribution.
Deepthi teaches at Paragraph [0126] that FIG. 9 also shows default threshold 908, illustrating the original MR image 902 with a pre-determined intensity threshold (i.e., default threshold) applied, wherein pixels within the original MR image 902 below the intensity threshold are suppressed/masked. As can be seen in FIG. 9, numerous holes occur within the anatomical regions of default threshold 908. Predicted threshold 910 depicts original MR image 902 with a predicted threshold applied (determined as the max intensity value of all background pixels) applied. Otsu threshold 912 depicts original MR image 902 with an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 902). Although predicted threshold 910 and Otsu threshold 912 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 902 is further exacerbated compared to default threshold 908. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto.
Deepthi teaches at Paragraph [0129] that FIG. 10 also shows default threshold 1008, illustrating the original MR image 1002 with a pre-determined intensity threshold applied, wherein pixels within the original MR image 1002 below the intensity threshold are suppressed/masked. As can be seen in FIG. 10, numerous holes occur within the anatomical regions of default threshold 1008. Predicted threshold 1010 and Otsu threshold 1012 likewise depict original MR image 1002 with either a predicted threshold applied (determined as the max value of all background pixels) or an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 1002), respectively. Although predicted threshold 1010 and Otsu threshold 1012 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 1002 is further exacerbated compared to default threshold 1008. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto);
wherein a diagnostic characteristic of the target region is visualized based on first interval distribution information corresponding to the first interval of the signal intensity values within a first region and second interval distribution information corresponding to the second interval of the signal intensity values within a second region included in the second assistant information (
For example, Atsumori teaches at FIG. 6 and FIG. 14 visualizing the inner region based on a first interval distribution information including the intensity values of the pixels of the target lesion region highlighted and second internal distribution information including the intensity values of the pixels of the non-lesion region included in the second assistant information.
Atsumori teaches at Paragraph 0089-0091 that the intensity values in the inner region 212 are grouped into a first intensity value groups with intensity value of 577 or more of pixels of the lesion region and a second intensity value groups with the intensity value group with intensity value of equal to or less than 576 wherein pixels of the non-lesion region having an intensity value equal to or greater than the intensity value 577 are highlighted while the second intensity value groups are not highlighted based on the intensity value threshold 576.
The first intensity value groups are associated with the first interval [577, 1024] and the second intensity value groups are associated with the second interval [32, 576] based on the intensity value threshold 576.
Atsumori not only teaches highlighting a bright place, but also highlighting a dark place. Atsumori teaches at Paragraph 0113-0114 that the threshold setting unit 106 sets the threshold in the broken line 331 in FIG. 12 and selects an intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than the threshold and the display processing unit 108 highlights a pixel having an intensity value smaller than the value of the selected intensity value group.
Atsumori teaches at Paragraph 0114 that the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group and the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222.
Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Gogin teaches at FIG. 6-7 and Paragraph 0081-0090 that generating the second assistant information at FIG. 7 different from the first assistance information of FIG. 6 for the CT images of a patient using the same CT acquisition.
The region corresponding to density 1 is indicated with the second annotation information based on the first annotation information, wherein the diagnostic information of the lung region is visualized based on density 1 and density 2.
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0087 that the images 710 may similarly comprise regions corresponding to a plurality of densities (e.g., density 1 through density 5), which may be generated and/or identified using the same plurality of range values (and/or other settings) used in generating images 610 of FIG. 6. However, the use of quantification lung reconstruction allows for enhanced representation of regions or areas corresponding to certain densities (e.g., density 1) which may be associated with certain respiratory conditions (e.g., COVID-19).
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times);
Wherein the second assistant information includes: one or more of quantification information or visualization information indicating one or more of: a characteristics of the lesion or the finding within the target area, or a progression of the lesion or the finding within the target region (
Atsumori teaches at Paragraph 0096 that a place suspected as a lesion is displayed in the region 202 and at Paragraph 0102 that a lesion 431 is highlighted and the lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 of FIG. 14. Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Gogin teaches at FIG. 6-7 and Paragraph 0081-0090 that generating the second assistant information at FIG. 7 different from the first assistance information of FIG. 6 for the CT images of a patient using the same CT acquisition.
The region corresponding to density 1 is indicated with the second annotation information based on the first annotation information, wherein the diagnostic information of the lung region is visualized based on density 1 and density 2.
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0087 that the images 710 may similarly comprise regions corresponding to a plurality of densities (e.g., density 1 through density 5), which may be generated and/or identified using the same plurality of range values (and/or other settings) used in generating images 610 of FIG. 6. However, the use of quantification lung reconstruction allows for enhanced representation of regions or areas corresponding to certain densities (e.g., density 1) which may be associated with certain respiratory conditions (e.g., COVID-19).
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have incorporated Gogin’s labeling both lesion types into Atsumori’s annotating two types of diseases to have distinctly annotating two types of lesions in the same image. One of the ordinary skill in the art would have been motivated to have provided a lesion annotation system for distinctly annotating two types of lesions.
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have incorporated Gogin’s characterization of the images with the precise identification of the lesion regions for different acquisition images and/or Schirman’s lesion/tumor progression represented by type A progression, type B progression and type C progression and/or Deepthi’s method for generating a second mask based on the intensity distribution information and the first mask into Atsumori’s method of generating the target lesion region to have performed segmentation of the target region and the background region for the different acquisition images and to have labelled the lesion region as being classified into type A lesion progression, type B lesion progression and/or type C lesion progression based on the different intensity intervals for the different acquisition images. One of the ordinary skill in the art would have been motivated to have performed image segmentation based on the intensity distribution and the first threshold and the second threshold for the different acquisition images.
Re Claim 3:
The claim 3 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that the second assistant information comprises information for visualizing at least one of first interval distribution information or the second interval distribution information.
Atsumori further teaches the claim limitation that the second assistant information comprises information for visualizing at least one of first interval distribution information or the second interval distribution information (
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
It is noted that the histograms 301 and 302 includes the first interval distribution 608/576 of the histogram 301 and the second interval distribution 608/576 of the histogram 302 within the target lesion region. Moreover, the histogram 301 includes the first interval distribution 576 and the second interval distribution 608.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches that the signal intensity values within the lesion are classified into a first interval (first intensity value group 608/576 of the first histogram 301) corresponding to a first state (the inner region 212) and a second interval (a second intensity value group 608/576 of the second histogram 302) corresponding to a second state (peripheral region 222) based on the set threshold as disclosed in Paragraph 0086.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222).
Re Claim 4:
The claim 4 encompasses the same scope of invention as that of the claim 3 except additional claim limitation that the second assistant information further comprises at least one of a first visualization element representative of the first interval distribution information or a second visualization element representative of the second interval distribution information.
Atsumori further teaches the claim limitation that the second assistant information further comprises at least one of a first visualization element representative of the first interval distribution information or a second visualization element representative of the second interval distribution information (
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
It is noted that the histograms 301 and 302 includes the first interval distribution 608/576 of the histogram 301 and the second interval distribution 608/576 of the histogram 302 within the target lesion region. Moreover, the histogram 301 includes the first interval distribution 576 and the second interval distribution 608.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches that the signal intensity values within the lesion are classified into a first interval (first intensity value group 608/576 of the first histogram 301) corresponding to a first state (the inner region 212) and a second interval (a second intensity value group 608/576 of the second histogram 302) corresponding to a second state (peripheral region 222) based on the set threshold as disclosed in Paragraph 0086.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222).
Re Claim 5:
The claim 5 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that the second assistant information comprises at least one of first interval distribution quantification information corresponding to the first interval of the signal intensity values within the target region or second interval distribution quantification information corresponding to the second interval of the signal intensity values within the target region.
Atsumori further teaches the claim limitation that the second assistant information comprises at least one of first interval distribution quantification information corresponding to the first interval of the signal intensity values within the target region or second interval distribution quantification information corresponding to the second interval of the signal intensity values within the target region (
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
It is noted that the histograms 301 and 302 includes the first interval distribution 608/576 of the histogram 301 and the second interval distribution 608/576 of the histogram 302 within the target lesion region. Moreover, the histogram 301 includes the first interval distribution 576 and the second interval distribution 608.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches that the signal intensity values within the lesion are classified into a first interval (first intensity value group 608/576 of the first histogram 301) corresponding to a first state (the inner region 212) and a second interval (a second intensity value group 608/576 of the second histogram 302) corresponding to a second state (peripheral region 222) based on the set threshold as disclosed in Paragraph 0086.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222).
Re Claim 6:
The claim 6 encompasses the same scope of invention as that of the claim 5 except additional claim limitation that the first interval distribution quantification information comprises at least one of a percentile, maximum value, minimum value, mean value, mode value, or median value of pixels/voxels of signal intensity values within the target region corresponding to the first interval, and wherein the second interval distribution quantification information comprises at least one of a percentile, maximum value, minimum value, mean value, mode value, or median value of pixels/voxels of signal intensity values within the target region corresponding to the second interval.
Atsumori further teaches the claim limitation that the first interval distribution quantification information comprises at least one of a percentile, maximum value, minimum value, mean value, mode value, or median value of pixels/voxels of signal intensity values within the target region corresponding to the first interval, and wherein the second interval distribution quantification information comprises at least one of a percentile, maximum value, minimum value, mean value, mode value, or median value of pixels/voxels of signal intensity values within the target region corresponding to the second interval (
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
It is noted that the histograms 301 and 302 includes the first interval distribution 608/576 of the histogram 301 and the second interval distribution 608/576 of the histogram 302 within the target lesion region. Moreover, the histogram 301 includes the first interval distribution 576 and the second interval distribution 608.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches that the signal intensity values within the lesion are classified into a first interval (first intensity value group 608/576 of the first histogram 301) corresponding to a first state (the inner region 212) and a second interval (a second intensity value group 608/576 of the second histogram 302) corresponding to a second state (peripheral region 222) based on the set threshold as disclosed in Paragraph 0086.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222).
Re Claim 7:
The claim 7 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that the processor is further configured to generate at least one of: first overlay visualization information on the medical image for at least one first sub-region within the target region corresponding to the first interval; or second overlay visualization information on the medical image for at least one second sub-region within the target region corresponding to the second interval.
Atsumori further teaches the claim limitation that the processor is further configured to generate at least one of: first overlay visualization information on the medical image for at least one first sub-region within the target region corresponding to the first interval; or second overlay visualization information on the medical image for at least one second sub-region within the target region corresponding to the second interval (
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
It is noted that the histograms 301 and 302 includes the first interval distribution 608/576 of the histogram 301 and the second interval distribution 608/576 of the histogram 302 within the target lesion region. Moreover, the histogram 301 includes the first interval distribution 576 and the second interval distribution 608.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches that the signal intensity values within the lesion are classified into a first interval (first intensity value group 608/576 of the first histogram 301) corresponding to a first state (the inner region 212) and a second interval (a second intensity value group 608/576 of the second histogram 302) corresponding to a second state (peripheral region 222) based on the set threshold as disclosed in Paragraph 0086.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222).
Re Claim 8:
The claim 8 encompasses the same scope of invention as that of the claim 7 except additional claim limitation that the processor is further configured to: generate the first overlay visualization information when a user input for the first interval associated with the second assistant information is recognized; and generate the second overlay visualization information when a user input for the second interval associated with the second assistant information is recognized.
Atsumori further teaches the claim limitation that the processor is further configured to: generate the first overlay visualization information when a user input for the first interval associated with the second assistant information is recognized; and generate the second overlay visualization information when a user input for the second interval associated with the second assistant information is recognized (
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
It is noted that the histograms 301 and 302 includes the first interval distribution 608/576 of the histogram 301 and the second interval distribution 608/576 of the histogram 302 within the target lesion region. Moreover, the histogram 301 includes the first interval distribution 576 and the second interval distribution 608.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches that the signal intensity values within the lesion are classified into a first interval (first intensity value group 608/576 of the first histogram 301) corresponding to a first state (the inner region 212) and a second interval (a second intensity value group 608/576 of the second histogram 302) corresponding to a second state (peripheral region 222) based on the set threshold as disclosed in Paragraph 0086.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222).
Re Claim 9:
The claim 9 encompasses the same scope of invention as that of the claim 1 except additional claim limitation wherein the first threshold is associated with at least one of a presence/absence, progress, or severity of a disease associated with the target region.
Atsumori further teaches the claim limitation wherein the first threshold is associated with at least one of a presence/absence, progress, or severity of a disease associated with the target region (
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120).
Re Claim 10:
The claim 10 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that information about the target region is segmentation information about a boundary of the target region.
Atsumori further teaches the claim limitation that information about the target region is segmentation information about a boundary of the target region (
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
It is noted that the histograms 301 and 302 includes the first interval distribution 608/576 of the histogram 301 and the second interval distribution 608/576 of the histogram 302 within the target lesion region. Moreover, the histogram 301 includes the first interval distribution 576 and the second interval distribution 608.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches that the signal intensity values within the lesion are classified into a first interval (first intensity value group 608/576 of the first histogram 301) corresponding to a first state (the inner region 212) and a second interval (a second intensity value group 608/576 of the second histogram 302) corresponding to a second state (peripheral region 222) based on the set threshold as disclosed in Paragraph 0086.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222).
Re Claim 11:
The claim 11 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that the target region is a finding region detected in association with a disease or lesion in the medical image.
Atsumori further teaches the claim limitation that the target region is a finding region detected in association with a disease or lesion in the medical image (
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
It is noted that the histograms 301 and 302 includes the first interval distribution 608/576 of the histogram 301 and the second interval distribution 608/576 of the histogram 302 within the target lesion region. Moreover, the histogram 301 includes the first interval distribution 576 and the second interval distribution 608.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches that the signal intensity values within the lesion are classified into a first interval (first intensity value group 608/576 of the first histogram 301) corresponding to a first state (the inner region 212) and a second interval (a second intensity value group 608/576 of the second histogram 302) corresponding to a second state (peripheral region 222) based on the set threshold as disclosed in Paragraph 0086.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222).
Re Claim 12:
The claim 12 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that the target region is a region obtained as a result of segmentation of an anatomical structure in the medical image.
Atsumori further teaches the claim limitation that the target region is a region obtained as a result of segmentation of an anatomical structure in the medical image (
Atsumori teaches at Paragraph 0058 that FIG. 6 illustrates a medical image 201 related to a brain.
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
It is noted that the histograms 301 and 302 includes the first interval distribution 608/576 of the histogram 301 and the second interval distribution 608/576 of the histogram 302 within the target lesion region. Moreover, the histogram 301 includes the first interval distribution 576 and the second interval distribution 608.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches that the signal intensity values within the lesion are classified into a first interval (first intensity value group 608/576 of the first histogram 301) corresponding to a first state (the inner region 212) and a second interval (a second intensity value group 608/576 of the second histogram 302) corresponding to a second state (peripheral region 222) based on the set threshold as disclosed in Paragraph 0086.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222).
Re Claim 13:
The claim 13 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that the processor is further configured to generate the first assistant information further including a second threshold value for the signal intensity values within the target region, and wherein the signal intensity values within the target region are classified into a first interval corresponding to a first state, a second interval corresponding to a second state, and a third interval corresponding to a third state based on the first threshold and the second threshold.
Atsumori further teaches the claim limitation that the processor is further configured to generate the first assistant information further including a second threshold value for the signal intensity values within the target region, and wherein the signal intensity values within the target region are classified into a first interval corresponding to a first state, a second interval corresponding to a second state, and a third interval corresponding to a third state based on the first threshold and the second threshold (
Atsumori teaches at Paragraph 0058 that FIG. 6 illustrates a medical image 201 related to a brain.
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
It is noted that the histograms 301 and 302 includes the first interval distribution 608/576 of the histogram 301 and the second interval distribution 608/576 of the histogram 302 within the target lesion region. Moreover, the histogram 301 includes the first interval distribution 576 and the second interval distribution 608.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches that the signal intensity values within the lesion are classified into a first interval (first intensity value group 608/576 of the first histogram 301) corresponding to a first state (the inner region 212) and a second interval (a second intensity value group 608/576 of the second histogram 302) corresponding to a second state (peripheral region 222) based on the set threshold as disclosed in Paragraph 0086.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222).
Re Claim 14:
The claim 14 encompasses the same scope of invention as that of the claim 1 except additional claim limitation that the distribution information is histogram information corresponding to a distribution of the signal intensity values of pixels/voxels within the target region.
Atsumori further teaches the claim limitation that the distribution information is histogram information corresponding to a distribution of the signal intensity values of pixels/voxels within the target region (
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
It is noted that the histograms 301 and 302 includes the first interval distribution 608/576 of the histogram 301 and the second interval distribution 608/576 of the histogram 302 within the target lesion region. Moreover, the histogram 301 includes the first interval distribution 576 and the second interval distribution 608.
Atsumori teaches at Paragraph 0058 that FIG. 6 illustrates a medical image 201 related to a brain.
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches that the signal intensity values within the lesion are classified into a first interval (first intensity value group 608/576 of the first histogram 301) corresponding to a first state (the inner region 212) and a second interval (a second intensity value group 608/576 of the second histogram 302) corresponding to a second state (peripheral region 222) based on the set threshold as disclosed in Paragraph 0086.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph [0087] Subsequently, the intensity value determination unit 107 extracts an intensity value group having a percentile rank equal to or higher than the threshold. Thus, the intensity value determination unit 107 extracts the intensity value to be highlighted on the basis of the threshold set in the ratio difference rank distribution 330b. In the example illustrated in FIG. 12, since the threshold is “0.95”, the intensity value group whose percentile rank is in the top 5% is extracted. In the example illustrated in FIG. 12, intensity value groups “608” and “576” are extracted as the intensity value groups having a percentile rank equal to or higher than the threshold (indicated by white circles 332 in FIG. 12). Then, the intensity value determination unit 107 selects the smallest intensity value group among the extracted intensity value groups. In the example illustrated in FIG. 12, the intensity value group “576” is selected.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222).
Re Claim 15:
Atsumori in view of Deepthi/Schirman/Gogin/Bazan teaches a medical image diagnosis assistant apparatus comprising a
processor, wherein the processor is configured to (Atsumori teaches at FIG. 3 and Paragraph 0119 that the described functions may be implemented through software by using a processor such as a CPU 112 to interpret and execute a program for realizing each function and a program may be stored in a storage device such as a memory 111):
acquire information about at least one first target region indicating one or more of a lesion or a finding including a lesion candidate in a first medical image acquired at a first time for a subject (Atsumori teaches at Paragraph 0028-0029 that an image processing system Z includes a medical image capturing unit M and the image processing apparatus 1 performs lesion highlighting processing of highlighting a lesion in the medical image 201.
Atsumori teaches at Paragraph 0096 that a place suspected as a lesion is displayed in the region 202 and at Paragraph 0102 that a lesion 431 is highlighted and the lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 of FIG. 14. Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Gogin teaches at FIG. 6-7 and Paragraph 0081-0090 that generating the second assistant information at FIG. 7 different from the first assistance information of FIG. 6 for the CT images of a patient using the same CT acquisition.
The region corresponding to density 1 is indicated with the second annotation information based on the first annotation information, wherein the diagnostic information of the lung region is visualized based on density 1 and density 2.
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0087 that the images 710 may similarly comprise regions corresponding to a plurality of densities (e.g., density 1 through density 5), which may be generated and/or identified using the same plurality of range values (and/or other settings) used in generating images 610 of FIG. 6. However, the use of quantification lung reconstruction allows for enhanced representation of regions or areas corresponding to certain densities (e.g., density 1) which may be associated with certain respiratory conditions (e.g., COVID-19).
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times.
Deepthi teaches at FIG. 1 and Paragraph 0029-0042 that the MRI system 10 acquires information (MR images) about at least one target region of the subject 16);
acquire information about at least one second target region in a second medical image acquired at a second time for the subject (
Atsumori teaches at Paragraph 0096 that a place suspected as a lesion is displayed in the region 202 and at Paragraph 0102 that a lesion 431 is highlighted and the lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 of FIG. 14. Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Gogin teaches at FIG. 6-7 and Paragraph 0081-0090 that generating the second assistant information at FIG. 7 different from the first assistance information of FIG. 6 for the CT images of a patient using the same CT acquisition.
The region corresponding to density 1 is indicated with the second annotation information based on the first annotation information, wherein the diagnostic information of the lung region is visualized based on density 1 and density 2.
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0087 that the images 710 may similarly comprise regions corresponding to a plurality of densities (e.g., density 1 through density 5), which may be generated and/or identified using the same plurality of range values (and/or other settings) used in generating images 610 of FIG. 6. However, the use of quantification lung reconstruction allows for enhanced representation of regions or areas corresponding to certain densities (e.g., density 1) which may be associated with certain respiratory conditions (e.g., COVID-19).
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times.
Deepthi teaches at Paragraph [0135] The above described systems and methods for reducing background noise in MR images may use a trained neural network to segment MR images/maps into foreground and background. A technical effect of classifying a plurality of pixels/voxels comprising an MR image/map as foreground or background using a trained neural network, determining an intensity threshold based on the MR image/map, and applying the intensity threshold to the background pixels/voxels, and not the foreground pixels/voxels, is that a greater degree of background noise reduction may be achieved, while mitigating loss of anatomical regions in the foreground by formation of “holes”. A further technical effect of the above approach is that MR images of substantially any anatomical regions may be segmented using the trained neural network of the subject application, and therefore the above approach may be applied to MR images/maps of anatomical structures generally, without significant alterations to the method workflow or the parameters of the neural network.
Deepthi teaches at Paragraph 0131 that a side-by-side comparison of MR images of abdomen with and without background suppression in a visualization application. The visualization application uses “windowing” approach to visualize MR images captured with higher precisions or higher dynamic ranges of intensity values than standard images, which maps an intensity interval of interest to the dynamic range of the display. The applications set up the range of interval and center of this interval for the mapping. In the original MR image 1102, the range of interval and center of interval are not optimal due to background noise, where a substantial amount of noise from the background pixels may be observed, and some of the boundary of the anatomical features of the image is lost. An example reference for visualization of the anatomical features of original MR image 1102 is shown in LAVA FLEX 1106. LAVA FLEX images may be used for verifying correct anatomical details and anatomical feature boundary. It is to be understood that any suitable reference for verifying correct anatomical details and anatomical feature boundary may be used similarly to the LAVA FLEX images described herein. Noise reduced image 1104, which represents an MR image with background noise selectively reduced according to the methods herein disclosed, shows improved image quality compared to original MR image 1102.
Deepthi teaches at Paragraph [0132] Similar to FIG. 11, FIG. 12 illustrates original MR image 1202, wherein a substantial amount of noise from the background pixels may be observed, and some of the boundary of the anatomical features of the image is lost. An example reference for visualization of the anatomical features of original MR image 1202 is shown in LAVA FLEX 1206. Noise reduced image 1204, which represents original MR image 1202 with background noise selectively reduced according to the methods herein disclosed, shows improved image quality compared to the original MR image 1202.
Atsumori teaches at Paragraph 0026 that the present embodiment can be applied to 2D or 3D images such as CT, MRI, PET, SPECT images and at Paragraph 0111 that segmenting stroke lesions in brain MRI images and at FIG. 15 and Paragraph 0116 that the images taken using the same medical device are displayed, but images taken using different medical devices may be displayed on the same display screen 400 and images taken at different positions of the same patient may be displayed on the same display screen 400.
Atsumori teaches at FIG. 4 and Paragraph 0046 that the image acquisition unit 101 reads a first image 201 stored in the storage device (implying that the image acquisition unit 101 may also read a second image 201 stored in the storage device since the process of FIG. 4 can be repeated).
Atsumori teaches at FIG. 16 and Paragraph 0107 that the processing can be extended to three dimensions by using the same inner line 211 for each of a plurality of 2D images (slice images) adjacent in a z-axis direction and a medical image 201-b adjacent to the medical image 201-a in the z-axis direction and at Paragraph 0110 the lesion within a designated range is highlighted by simple processing that includes comparing intensity distributions of the selected range and a peripheral region of the selected range.
Atsumori teaches at Paragraph 0031 that the computer 100 includes an image acquisition unit 101, a region setting unit 102, an intensity value group ratio distribution calculation unit and an intensity value determination unit 107 and at FIG. 8 and Paragraph 0034 that the intensity value group ratio distribution calculation unit 103 calculates histograms 301 and 302 and histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution);
generate first distribution information, wherein signal intensity values within the first target region are classified into a first interval corresponding to a first state and a second interval corresponding to a second state based on a first threshold of the signal intensity values (
Atsumori teaches at Paragraph 0096 that a place suspected as a lesion is displayed in the region 202 and at Paragraph 0102 that a lesion 431 is highlighted and the lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 of FIG. 14. Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Gogin teaches at FIG. 6-7 and Paragraph 0081-0090 that generating the second assistant information at FIG. 7 different from the first assistance information of FIG. 6 for the CT images of a patient using the same CT acquisition.
The region corresponding to density 1 is indicated with the second annotation information based on the first annotation information, wherein the diagnostic information of the lung region is visualized based on density 1 and density 2.
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0087 that the images 710 may similarly comprise regions corresponding to a plurality of densities (e.g., density 1 through density 5), which may be generated and/or identified using the same plurality of range values (and/or other settings) used in generating images 610 of FIG. 6. However, the use of quantification lung reconstruction allows for enhanced representation of regions or areas corresponding to certain densities (e.g., density 1) which may be associated with certain respiratory conditions (e.g., COVID-19).
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times.
For example, Atsumori teaches at FIG. 6 and FIG. 14 visualizing the inner region based on a first interval distribution information including the intensity values of the pixels of the target lesion region highlighted and second internal distribution information including the intensity values of the pixels of the non-lesion region included in the second assistant information.
Atsumori teaches at Paragraph 0089-0091 that the intensity values in the inner region 212 are grouped into a first intensity value groups with intensity value of 577 or more of pixels of the lesion region and a second intensity value groups with the intensity value group with intensity value of equal to or less than 576 wherein pixels of the non-lesion region having an intensity value equal to or greater than the intensity value 577 are highlighted while the second intensity value groups are not highlighted based on the intensity value threshold 576.
Atsumori teaches at FIG. 6 and FIGS. 13-14 that the intensity values of the inner region 211 are classified into a first interval corresponding to the intervals 224/256 and a second interval corresponding to the interval 672/704 and intensity values of the peripheral region 222 are also classified into a third interval corresponding to the intervals 512/544 and a fourth interval corresponding to the intervals 608/640 where the first threshold is applied to both the inner region and peripheral region); and
generate second distribution information, wherein the signal intensity values within the second target region are classified into a third interval corresponding to the first state and a fourth interval corresponding to the second state based on the first threshold (
Atsumori teaches at Paragraph 0096 that a place suspected as a lesion is displayed in the region 202 and at Paragraph 0102 that a lesion 431 is highlighted and the lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 of FIG. 14. Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Gogin teaches at FIG. 6-7 and Paragraph 0081-0090 that generating the second assistant information at FIG. 7 different from the first assistance information of FIG. 6 for the CT images of a patient using the same CT acquisition.
The region corresponding to density 1 is indicated with the second annotation information based on the first annotation information, wherein the diagnostic information of the lung region is visualized based on density 1 and density 2.
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0087 that the images 710 may similarly comprise regions corresponding to a plurality of densities (e.g., density 1 through density 5), which may be generated and/or identified using the same plurality of range values (and/or other settings) used in generating images 610 of FIG. 6. However, the use of quantification lung reconstruction allows for enhanced representation of regions or areas corresponding to certain densities (e.g., density 1) which may be associated with certain respiratory conditions (e.g., COVID-19).
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times.
Atsumori teaches at FIG. 6 and FIGS. 13-14 that the intensity values of the inner region 211 are classified into a first interval corresponding to the intervals 224/256 and a second interval corresponding to the interval 672/704 and intensity values of the peripheral region 222 are also classified into a third interval corresponding to the intervals 512/544 and a fourth interval corresponding to the intervals 608/640 where the first threshold is applied to both the inner region and peripheral region.
Moreover, Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. Accordingly, the first inner region of the first lesion are classified into the first interval and second interval and the second inner region of the second lesion can be classified into the third interval and fourth interval using the same threshold),
wherein a change between a first diagnostic characteristic of the first target region and a second diagnostic characteristic of the second target region is visualized based on at least one of:
a first change between first interval distribution information corresponding to the signal intensity values of the first interval and third interval distribution information corresponding to the signal intensity values of the third interval; or
a second change between second interval distribution information corresponding to the signal intensity values of the second interval and fourth interval distribution information corresponding to the signal intensity values of the fourth interval,
wherein the first distribution information includes: one or more of quantification information or visualization information indicating one or more of: a characteristics of the lesion or the finding within the target region, or a progression of the lesion or the finding within the first target region and wherein the second distribution information includes: one or more of quantification information or visualization information indicating one or more of: a characteristic of the lesion or the finding within the target region, or a progression of the lesion or the finding within the second target region (
Atsumori teaches at Paragraph 0096 that a place suspected as a lesion is displayed in the region 202 and at Paragraph 0102 that a lesion 431 is highlighted and the lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 of FIG. 14. Atsumori teaches at [0115] By highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Schirman teaches at [0033] Each of the plurality of probability distributions represents a different type of change. A non-limiting example may be a first probability distribution estimated from the observed intensity distribution may represent tumor growth, a second probability distribution may represent a transition zone, and a third probability distribution may represent edema. Having estimated these different probability distributions, intensity intervals may be determined which each represent the different type of change. For example, each intensity interval may be selected as being an interval where the respective probability distribution is larger than other probability distributions, denoting that an intensity falling within the intensity range is most likely to be associated with the type of change represented by the particular probability distribution. Having determined the plurality of intensity intervals, the image data in the difference image may be labeled accordingly, in that suitable metadata may be created. It is noted that the labeling may not need to represent a biological interpretation, e.g., whether it is tumor growth, transition zone or edema, but rather merely represent different labeling, e.g., type A, type B and type C, which allows such biological interpretation to be subsequently assigned, e.g., by a radiologist or an automatic classification algorithm. Thus, it can be distinguished between different types of change if a change has been occurred, i.e. the change can be characterized. This characterization of the change can be regarded as a modelling of different classes of change within a “changed” class.
Gogin teaches at FIG. 6-7 and Paragraph 0081-0090 that generating the second assistant information at FIG. 7 different from the first assistance information of FIG. 6 for the CT images of a patient using the same CT acquisition.
The region corresponding to density 1 is indicated with the second annotation information based on the first annotation information, wherein the diagnostic information of the lung region is visualized based on density 1 and density 2.
Gogin teaches at FIGS. 3-10 and Paragraph 0076 that the use of lung reconstruction may result enhancement in lesion detection, evidenced in increase in region corresponding to density 1 as illustrated in FIG. 4 and at Paragraph 0087 that the images 710 may similarly comprise regions corresponding to a plurality of densities (e.g., density 1 through density 5), which may be generated and/or identified using the same plurality of range values (and/or other settings) used in generating images 610 of FIG. 6. However, the use of quantification lung reconstruction allows for enhanced representation of regions or areas corresponding to certain densities (e.g., density 1) which may be associated with certain respiratory conditions (e.g., COVID-19).
Schirman teaches at Paragraph [0096] FIG. 4 shows a labeled medical image 410 in which a labeling 415 of image data by the system 100 of FIG. 1 is shown in the form of an overlay. The labeled difference image 410 may be generated by the system 100 of FIG. 1 as an output image, e.g., for display to a clinician. The labeling may be performed by determining into which of the plurality of intensity ranges the image data of the difference image falls. Effectively, the pixel or voxel may be labeled to identify to which subpopulation the particular pixel or voxel is estimated to belong. An example of such a labeling is simply type A, type B, type C, etc., or similar neutral labeling. As such, the labels may not directly represent a biological interpretation. Nevertheless, such a biological interpretation may be explicitly or implicitly assigned to the labels, e.g., by a radiologist or an automatic classification algorithm.
Bazan teaches at FIG. 3 and Paragraph 0206 that the tumor growth at days 13, 20 and 30 are quantified.
Having the combined teaching of Bazan and Gogin, it would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have processed images of Bazan acquired for days 13, 20 and 30 by Gogin’s image processing apparatus for displaying the lesion region using a plurality of intensity range values such that the lesion progression can be determined in the same manner as Schirman. One of the ordinary skill in the art would have been motivated to have provided a lesion progression comparison based on the different images acquired at different times.
The change may be caused by enlarging the medical image (or based on the user delineation of the inner region perceived as the potential lesion region) and processing the enlarged medical image to find the target region (the inner region considered as the potential lesion region) including the actual lesion region and the actual non-lesion region.
Atsumori teaches at Paragraph 0059 that the inner region 212 is set as a region that includes the region 202 considered as the lesion and at Paragraph 0110 that a region range that is selected by a user as corresponding to a lesion is received as an input and the lesion within a designated range is highlighted by simple processing that includes comparing intensity distributions of the selected range and a peripheral region of the selected range.
For example, Atsumori teaches at FIG. 6 and FIG. 14 visualizing the inner region based on a first interval distribution information including the intensity values of the pixels of the target lesion region highlighted and second internal distribution information including the intensity values of the pixels of the non-lesion region included in the second assistant information.
Atsumori teaches at Paragraph 0089-0091 that the intensity values in the inner region 212 are grouped into a first intensity value groups with intensity value of 577 or more of pixels of the lesion region and a second intensity value groups with the intensity value group with intensity value of equal to or less than 576 wherein pixels of the non-lesion region having an intensity value equal to or greater than the intensity value 577 are highlighted while the second intensity value groups are not highlighted based on the intensity value threshold 576.
Atsumori teaches at Paragraph 0091 that pixels having an intensity value equal to or greater than the intensity value “577” are highlighted. Atsumori teaches at Paragraph 0098 that the user sets the inner line 211 and the region setting unit 102 sets the outer line 221 on the basis of the inner line 211 and the medical image 201A is displayed as a result of performing the lesion highlighting processing. Atsumori teaches at FIG. 15 that enlarging the medical image from the image 410 to the image 420 results in the change of the first interval distribution and the second interval distribution and/or the second interval distribution and third interval distribution such that a different lesion region is highlighted in the medical image due to the change of the first interval and the second interval and/or the third interval and fourth interval for the different regions.
Atsumori teaches at FIG. 6 and FIGS. 13-14 that the intensity values of the inner region 211 are classified into a first interval corresponding to the intervals 224/256 and a second interval corresponding to the interval 672/704 and intensity values of the peripheral region 222 are also classified into a third interval corresponding to the intervals 512/544 and a fourth interval corresponding to the intervals 608/640 where the first threshold is applied to both the inner region and peripheral region.
Moreover, Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. Accordingly, the first inner region of the first lesion are classified into the first interval and second interval and the second inner region of the second lesion can be classified into the third interval and fourth interval using the same threshold
Atsumori teaches at Paragraph 0031 that the computer 100 includes an image acquisition unit 101, a region setting unit 102, an intensity value group ratio distribution calculation unit and an intensity value determination unit 107 and at FIG. 8 and Paragraph 0034 that the intensity value group ratio distribution calculation unit 103 calculates histograms 301 and 302 and histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution.
Deepthi teaches at Paragraph [0112] that a first intensity threshold is set to an intensity which maximizes a between class variance (or equivalently, the first intensity threshold is set to an intensity which minimized the within class variance), of the intensity distribution determined at step 508. The between class variance is calculated for a first and second class, of the previously determined intensity distribution, by dividing the intensity distribution into a first class comprising pixels with intensity less than the selected intensity threshold, and a second class comprising pixels with intensity greater than the selected intensity threshold.
Deepthi teaches at Paragraph 0119 that generating a mask for the foreground includes comparing an intensity of each pixel/voxel of the foreground of the MR image/image against a lower intensity threshold and an upper intensity threshold of the selected intensity range, and in response to the intensity of the pixel/voxel being within the selected intensity range (that is, in response to the intensity of a pixel/voxel being both below an upper intensity threshold and above a lower intensity threshold), setting a mask value corresponding to the pixel/voxel to one, or otherwise setting the mask value corresponding to the pixel/voxel to zero if the intensity of the pixel/voxel is outside of the selected intensity range. In some embodiments, mask values corresponding to the background pixel/voxels are zero.
Deepthi teaches at Paragraph [0126] that FIG. 9 also shows default threshold 908, illustrating the original MR image 902 with a pre-determined intensity threshold (i.e., default threshold) applied, wherein pixels within the original MR image 902 below the intensity threshold are suppressed/masked. As can be seen in FIG. 9, numerous holes occur within the anatomical regions of default threshold 908. Predicted threshold 910 depicts original MR image 902 with a predicted threshold applied (determined as the max intensity value of all background pixels) applied. Otsu threshold 912 depicts original MR image 902 with an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 902). Although predicted threshold 910 and Otsu threshold 912 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 902 is further exacerbated compared to default threshold 908. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto.
Deepthi teaches at Paragraph [0129] that FIG. 10 also shows default threshold 1008, illustrating the original MR image 1002 with a pre-determined intensity threshold applied, wherein pixels within the original MR image 1002 below the intensity threshold are suppressed/masked. As can be seen in FIG. 10, numerous holes occur within the anatomical regions of default threshold 1008. Predicted threshold 1010 and Otsu threshold 1012 likewise depict original MR image 1002 with either a predicted threshold applied (determined as the max value of all background pixels) or an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 1002), respectively. Although predicted threshold 1010 and Otsu threshold 1012 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 1002 is further exacerbated compared to default threshold 1008. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto.
Deepthi teaches at Paragraph 0095 that a first intensity threshold is determined. Method 500 discusses the process of determining the first intensity threshold in more detail. In some embodiments, the intensity threshold is calculated based on an intensity distribution of the received MR image/map according to Otsu's method. In some embodiments, the intensity threshold may be selected based on the intensity of the pixels/voxels classified as background at 404 of method 400. As a more specific example, step 406 may comprise determining a maximum pixel/voxel intensity of the background, and setting the intensity threshold to the determined maximum background intensity.
Deepthi teaches at Paragraph [0096] that a first mask is generated based on the first intensity threshold. In some embodiments, generating the first mask may comprise comparing a pixel/voxel intensity of each background pixel/voxel with the first intensity threshold, and responsive to the intensity being lower than the first intensity threshold, setting a mask value corresponding to the pixel/voxel to zero, or otherwise setting the mask value corresponding to the pixel to one. Mask values corresponding to foreground pixels/voxels are set to one.
Deepthi teaches at Paragraph [0100] that a second mask is generated based on the user selected intensity threshold. The second mask may suppress output of pixels/voxels (e.g., in the background) with an intensity below the user selected intensity threshold.
Atsumori teaches at Paragraph 0026 that the present embodiment can be applied to 2D or 3D images such as CT, MRI, PET, SPECT images and at Paragraph 0111 that segmenting stroke lesions in brain MRI images and at FIG. 15 and Paragraph 0116 that the images taken using the same medical device are displayed, but images taken using different medical devices may be displayed on the same display screen 400 and images taken at different positions of the same patient may be displayed on the same display screen 400.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at Paragraph 0067 that the intensity value group ratio distribution calculation unit 103 divides the number of pixels of each intensity value group by a total number of pixels of the inner region 212 (the number of pixels constituting the inner region 212). Thus, the intensity value group ratio distribution calculation unit 103 calculates a ratio (referred to as the intensity value group ratio) of pixels having each intensity value group in the inner region 212.
Atsumori teaches at Paragraph 0049 that the intensity value group ratio distribution calculation unit 103 calculates an intensity value group ratio distribution for the inner region 212, and further calculates an intensity value group ratio distribution for the peripheral region 222
Atsumori teaches at Paragraph 0070 that the intensity value group ratio distribution calculation unit 103 divides the number of pixels of each intensity value group by the total number of pixels of the peripheral region 222 (the number of pixels constituting the peripheral region 222). Thus, the intensity value group ratio distribution calculation unit 103 calculates a ratio (referred to as the intensity value group ratio) of pixels having each intensity value group in the peripheral region 222 (S124 of FIG. 7).
Atsumori teaches at FIG. 4 and Paragraph 0046 that the image acquisition unit 101 reads a first image 201 stored in the storage device (implying that the image acquisition unit 101 may also read a second image 201 stored in the storage device since the process of FIG. 4 can be repeated).
Atsumori teaches at FIG. 16 and Paragraph 0107 that the processing can be extended to three dimensions by using the same inner line 211 for each of a plurality of 2D images (slice images) adjacent in a z-axis direction and a medical image 201-b adjacent to the medical image 201-a in the z-axis direction and at Paragraph 0110 the lesion within a designated range is highlighted by simple processing that includes comparing intensity distributions of the selected range and a peripheral region of the selected range.
Atsumori teaches at Paragraph 0031 that the computer 100 includes an image acquisition unit 101, a region setting unit 102, an intensity value group ratio distribution calculation unit and an intensity value determination unit 107 and at FIG. 8 and Paragraph 0034 that the intensity value group ratio distribution calculation unit 103 calculates histograms 301 and 302 and histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution.
Deepthi teaches at Paragraph [0112] that a first intensity threshold is set to an intensity which maximizes a between class variance (or equivalently, the first intensity threshold is set to an intensity which minimized the within class variance), of the intensity distribution determined at step 508. The between class variance is calculated for a first and second class, of the previously determined intensity distribution, by dividing the intensity distribution into a first class comprising pixels with intensity less than the selected intensity threshold, and a second class comprising pixels with intensity greater than the selected intensity threshold.
Deepthi teaches at Paragraph 0119 that generating a mask for the foreground includes comparing an intensity of each pixel/voxel of the foreground of the MR image/image against a lower intensity threshold and an upper intensity threshold of the selected intensity range, and in response to the intensity of the pixel/voxel being within the selected intensity range (that is, in response to the intensity of a pixel/voxel being both below an upper intensity threshold and above a lower intensity threshold), setting a mask value corresponding to the pixel/voxel to one, or otherwise setting the mask value corresponding to the pixel/voxel to zero if the intensity of the pixel/voxel is outside of the selected intensity range. In some embodiments, mask values corresponding to the background pixel/voxels are zero.
Deepthi teaches at Paragraph [0126] that FIG. 9 also shows default threshold 908, illustrating the original MR image 902 with a pre-determined intensity threshold (i.e., default threshold) applied, wherein pixels within the original MR image 902 below the intensity threshold are suppressed/masked. As can be seen in FIG. 9, numerous holes occur within the anatomical regions of default threshold 908. Predicted threshold 910 depicts original MR image 902 with a predicted threshold applied (determined as the max intensity value of all background pixels) applied. Otsu threshold 912 depicts original MR image 902 with an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 902). Although predicted threshold 910 and Otsu threshold 912 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 902 is further exacerbated compared to default threshold 908. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto.
Deepthi teaches at Paragraph [0129] that FIG. 10 also shows default threshold 1008, illustrating the original MR image 1002 with a pre-determined intensity threshold applied, wherein pixels within the original MR image 1002 below the intensity threshold are suppressed/masked. As can be seen in FIG. 10, numerous holes occur within the anatomical regions of default threshold 1008. Predicted threshold 1010 and Otsu threshold 1012 likewise depict original MR image 1002 with either a predicted threshold applied (determined as the max value of all background pixels) or an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 1002), respectively. Although predicted threshold 1010 and Otsu threshold 1012 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 1002 is further exacerbated compared to default threshold 1008. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto.
Deepthi teaches at Paragraph 0095 that a first intensity threshold is determined. Method 500 discusses the process of determining the first intensity threshold in more detail. In some embodiments, the intensity threshold is calculated based on an intensity distribution of the received MR image/map according to Otsu's method. In some embodiments, the intensity threshold may be selected based on the intensity of the pixels/voxels classified as background at 404 of method 400. As a more specific example, step 406 may comprise determining a maximum pixel/voxel intensity of the background, and setting the intensity threshold to the determined maximum background intensity.
Deepthi teaches at Paragraph [0096] that a first mask is generated based on the first intensity threshold. In some embodiments, generating the first mask may comprise comparing a pixel/voxel intensity of each background pixel/voxel with the first intensity threshold, and responsive to the intensity being lower than the first intensity threshold, setting a mask value corresponding to the pixel/voxel to zero, or otherwise setting the mask value corresponding to the pixel to one. Mask values corresponding to foreground pixels/voxels are set to one.
Deepthi teaches at Paragraph [0100] that a second mask is generated based on the user selected intensity threshold. The second mask may suppress output of pixels/voxels (e.g., in the background) with an intensity below the user selected intensity threshold); and
generate visualization information based on the first distribution information and the second distribution information (
Atsumori teaches at Paragraph [0114] that the threshold setting unit 106 may set two thresholds in step S151 of FIG. 4. In this case, the intensity value determination unit 107 selects a first intensity value group having a minimum percentile rank among intensity value groups having a percentile rank equal to or higher than a threshold (for example, “0.95”). Then, the display processing unit 108 highlights a pixel (referred to as a bright pixel) having an intensity value greater than the value of the selected, first intensity value group. The intensity value determination unit 107 also selects a second intensity value group having a maximum percentile rank among intensity value groups having a percentile rank equal to or lower than another threshold (for example, “0.05”). Then, the display processing unit 108 highlights a pixel (referred to as a dark pixel) having an intensity value smaller than the value of the selected, second intensity value group. Then, the display processing unit 108 highlights the bright pixel and the dark pixel on the same medical image 201A. In this way, it is possible to highlight both a bright portion and a dark portion of the medical image 201.
Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. The user may add attribute data such as a disease name to the medical image 201A with highlights and store the medical image in the storage device (storage) 120.
Atsumori teaches at Paragraph 0034 that histograms 301 and 302 are a first intensity value frequency distribution and a second intensity value frequency distribution, which are information related to distribution of the number of pixels having a predetermined intensity value for the inner region 212 and peripheral region 222 that have been set.
Atsumori teaches at FIG. 13 displaying the intensity value groups to include the intensity value distribution information 301 and 302 the intensity value group 576.
Atsumori teaches at FIG. 13 and Paragraph 0089 that the intensity value determination unit 107 selects the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. A broken line 341 in FIG. 13 indicates the intensity value group “576” selected as the minimum intensity value group among the intensity value groups having a percentile rank equal to or higher than the threshold. Then, the intensity value determination unit 107 selects an intensity value (that is, an intensity value of “577” or more) that is larger than a value (“576”) of the selected intensity value group “576” in the inner region 212.
Atsumori teaches at Paragraph [0091] FIG. 14 illustrates the medical image (second image) 201A that is the result of lesion highlighting processing according to the present embodiment. In the medical image 201A, pixels having an intensity value equal to or greater than the intensity value “577” selected in FIG. 13 are highlighted as indicated by reference numeral 351. That is, the display processing unit 108 determines the pixel to be highlighted on the basis of the intensity value determined by the intensity value determination unit 107, and displays the medical image 201A in which said pixel is highlighted on the display device 132.
Atsumori teaches at Paragraph [0102] On the screen 430 and the screen 440, a lesion 431 is highlighted. The lesion 431 that is highlighted is a portion corresponding to pixels denoted by reference numeral 351 in FIG. 14.
Atsumori teaches at FIGS. 13-15 and Paragraph 0064 that the histogram 301 shown using black bars indicates the intensity value group ratio in the inner region and the histogram 302 shown using hatched bars indicates the intensity value group ratio in the peripheral region 222.
Deepthi teaches at Paragraph 0123 that an exemplary MR image 700 of abdomen is shown. Image 700 is an original DWI image without background suppression, which includes a background 702 (the relatively dark peripheral regions of image 700), separated by boundary 706 (the relatively bright boundary), from foreground 704 (the relatively bright region bounded by boundary 706, which corresponds to the imaged anatomical region). Background 702 includes a plurality of pixels, wherein an output/intensity of each of the background pixels has not yet been suppressed. Foreground 704 includes a second plurality of pixels and in Paragraph 0124 that image 800 includes a background 802 separated by boundary 806 and at Paragraph 0127 that applied predicted mask 916 is seen to be in close agreement with the human best effort shown by applied ground truth mask 914, although some pixels on the boundary of the anatomical features are segmented out.
Deepthi teaches at FIGS. 9-10 generating a second mask based on the applied predicted mask (first assistant information) based on the first threshold + Otsu threshold 1018 and based on the intensity distribution.
Deepthi teaches at Paragraph [0126] that FIG. 9 also shows default threshold 908, illustrating the original MR image 902 with a pre-determined intensity threshold (i.e., default threshold) applied, wherein pixels within the original MR image 902 below the intensity threshold are suppressed/masked. As can be seen in FIG. 9, numerous holes occur within the anatomical regions of default threshold 908. Predicted threshold 910 depicts original MR image 902 with a predicted threshold applied (determined as the max intensity value of all background pixels) applied. Otsu threshold 912 depicts original MR image 902 with an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 902). Although predicted threshold 910 and Otsu threshold 912 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 902 is further exacerbated compared to default threshold 908. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto.
Deepthi teaches at Paragraph [0129] that FIG. 10 also shows default threshold 1008, illustrating the original MR image 1002 with a pre-determined intensity threshold applied, wherein pixels within the original MR image 1002 below the intensity threshold are suppressed/masked. As can be seen in FIG. 10, numerous holes occur within the anatomical regions of default threshold 1008. Predicted threshold 1010 and Otsu threshold 1012 likewise depict original MR image 1002 with either a predicted threshold applied (determined as the max value of all background pixels) or an Otsu threshold applied (determined based on Otsu's method using the full intensity distribution of original MR image 1002), respectively. Although predicted threshold 1010 and Otsu threshold 1012 both reduce background noise to a greater extent, in the illustrated example, a degree of information loss from the anatomical regions of original MR image 1002 is further exacerbated compared to default threshold 1008. However, it is to be understood that in other examples, an image having an Otsu threshold applied thereto may have fewer loss of anatomical details than an associated image having a default threshold applied thereto).
It would have been obvious to one of the ordinary skill in the art before the filing date of the instant application to have incorporated Gogin’s characterization of the images with the precise identification of the lesion regions for different acquisition images and/or Schirman’s lesion/tumor progression represented by type A progression, type B progression and type C progression and/or Deepthi’s method for generating a second mask based on the intensity distribution information and the first mask into Atsumori’s method of generating the target lesion region to have performed segmentation of the target region and the background region for the different acquisition images and to have labelled the lesion region as being classified into type A lesion progression, type B lesion progression and/or type C lesion progression based on the different intensity intervals for the different acquisition images. One of the ordinary skill in the art would have been motivated to have performed image segmentation based on the intensity distribution and the first threshold and the second threshold for the different acquisition images.
Re Claim 17:
The claim 17 encompasses the same scope of invention as that of the claim 15 except additional claim limitation that the processor is further configured to generate visualization information visualizing at least one of the first change or the second change.
Atsumori further teaches the claim limitation that the processor is further configured to generate visualization information visualizing at least one of the first change or the second change (
Atsumori teaches at Paragraph 0091 that pixels having an intensity value equal to or greater than the intensity value “577” are highlighted. Atsumori teaches at Paragraph 0098 that the user sets the inner line 211 and the region setting unit 102 sets the outer line 221 on the basis of the inner line 211 and the medical image 201A is displayed as a result of performing the lesion highlighting processing. Atsumori teaches at FIG. 15 that enlarging the medical image from the image 410 to the image 420 results in the change of the first interval distribution and the second interval distribution and/or the second interval distribution and third interval distribution such that a different lesion region is highlighted in the medical image due to the change of the first interval and the second interval and/or the third interval and fourth interval for the different regions.
Atsumori teaches at FIG. 6 and FIGS. 13-14 that the intensity values of the inner region 211 are classified into a first interval corresponding to the intervals 224/256 and a second interval corresponding to the interval 672/704 and intensity values of the peripheral region 222 are also classified into a third interval corresponding to the intervals 512/544 and a fourth interval corresponding to the intervals 608/640 where the first threshold is applied to both the inner region and peripheral region.
Moreover, Atsumori teaches at Paragraph [0115] that by highlighting the bright pixel and the dark pixel on the same medical image 201A, it is possible to simultaneously highlight lesions caused by two types of diseases. Accordingly, the first inner region of the first lesion are classified into the first interval and second interval and the second inner region of the second lesion can be classified into the third interval and fourth interval using the same threshold).
Re Claim 18:
The claim 18 is in parallel with the claim 1 in a method form. The claim 18 is subject to the same rationale of rejection as the claim 1.
Re Claim 20:
The claim 20 encompasses the same scope of invention as that of the claim 18 except additional claim limitation that wherein the generating second assistant comprises generating the second assistant information including at least one of first interval distribution quantification information corresponding to the first interval of the signal intensity values within the target region or second interval distribution quantification information corresponding to the second interval of the signal intensity values within the target region.
The claim 20 is in parallel with the claim 5 in a method form. The claim 20 is subject to the same rationale of rejection as the claim 5.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIN CHENG WANG whose telephone number is (571)272-7665. The examiner can normally be reached Mon-Fri 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at 571-270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JIN CHENG WANG/Primary Examiner, Art Unit 2617
Prosecution Insights