DETAILED ACTION
Response to Arguments
The amendments and arguments filed 12/23/2025 have been entered and made of record.
The Applicant's amendments and arguments filed 12/23/2025 have been considered but are moot in view of the new ground(s) of rejection because the Applicant has amended independent claim 1. Furthermore, Applicant's arguments in view of the amendments filed 12/23/2025 have been fully considered but they are not persuasive:
Re amended claim 1, Applicant asserts (in page 8 of the Arguments of 12/23/2025) that cited references, Vlasimsky, JIANG, Hong and Yi fail to disclose added limitation of wherein the first set of radiomics features includes at least one of: gray-level co-occurrence matrices (GLCM) textures, grey-level run- length matrix (GLRLM) textures, gray level size zone matrix (GLSZM) textures, neighbouring gray tone difference matrix (NGTDM) textures, and gray-level difference matrix (GLDM) textures;
However, the Examiner disagrees, because:
First, Vlasimsky as modified by JIANG disclose determining at least one lung nodule region of the first chest image based on the at least one lung nodule, and classifying the at least one lung nodule region based on a first set of radiomics features of the at least one lung nodule region of the first chest image (see Vlasimsky: e.g., --ML systems are trained to correlate features in x-rays with data obtained from CT scanning. Those features may be imperceptible to a trained human technician. Nevertheless, the ML systems of the invention can correlate them to features in CT scans associated with a chronic lung pathology. In doing so, the ML systems leverage CT scanning data to improve the diagnostic utility of x-ray imaging….The collection of neural networks includes at least a first neural network that analyzes the first image and, and a second neural network that analyzes the subsection of the image, wherein the neural networks of the collection each independently make an inference as to the presence of an abnormality. The system includes an ensemble classifier that reports the presence of an abnormality at a location in the lung using the collection of neural network inferences as inputs…..the system segments the image to select a subsection by performing an object detection operation on the image to create a region proposal for an object detected in the image, and then selects the subsection from within the region proposal. In some embodiments, the second neural network assigns a confidence score to the bounded potential objects. The second neural network may classify potential objects as detected objects using the likelihood score. The second neural network may classify objects by creating a heatmap of bounded potential objects and their corresponding confidence scores and classifies objects using the heatmap.
[0014] In certain aspects, the present invention also includes ML systems trained using data from various sources separated by time and/or geography. These training data can include, for example, chest x-ray images, CT scans, and pathology results. Distributed ML subsystems can be placed at, or connected to, those locations and can update the central ML system.--, in [0010]-[0014], and, --The module may (i) resize a chest x-ray file to produce a first image at a down-sampled resolution, and (ii) place a subsection of the of the chest x-ray file into a second image at an original resolution of the chest x-ray file. The system may further include a first and a second neural network. The first neural network analyzes the first image to output a first set of scores indicating probabilities of nodules at locations in the lung. The second neural network analyzes the second image to output a second set of scores of probabilities of a nodule at a location in the lung. The neural networks may have been trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. The system may also include an ensemble classifier that reports the presence of the nodule in the location in the lung using the first set of scores and the second set of scores as inputs.
[0016] In certain aspects, the system may further also include a feature engineering module that creates features from the first and second sets of scores and provides the features as inputs for the ensemble classifier. The ensemble classifier may be, for example, a machine learning model trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. In certain aspects, the ensemble classifier is a random forest.--, in [0015]-[0016], [0018]-[0019]);
and, above Vlasimsky as modified by JIANG’s disclosures of “radiomics features of the at least one lung nodule region of the first chest image” include – a digital image into a two-dimensional grid or matrix of pixels, which are often regularly sized and dispersed across the matrix/grid.--, in [0055], -- The second neural network can examine the x-ray subsections pixel by pixel to detect minute distinctions grey shading, and thereby detect potential features--, in [0058], and, --“size or shape of a potential lung abnormality, while the “local” output is indicative of more detailed features, or “local” for example, the density and texture of a potential lung abnormality.”,--; in Vlasimsky’s [0061];
Vlasimsky as modified by JIANG do not explicitly disclose the first set of radiomics features includes at least one of: gray-level co-occurrence matrices (GLCM) textures, grey-level run- length matrix (GLRLM) textures, gray level size zone matrix (GLSZM) textures, neighbouring gray tone difference matrix (NGTDM) textures, and gray-level difference matrix (GLDM) textures;
Hong discloses the first set of radiomics features includes at least one of: gray-level co-occurrence matrices (GLCM) textures, grey-level run- length matrix (GLRLM) textures, gray level size zone matrix (GLSZM) textures, neighbouring gray tone difference matrix (NGTDM) textures, and gray-level difference matrix (GLDM) textures {Hong also discloses these radiomics features of the at least one lung nodule region of the first chest image applied in detection and classification} (see HONG: e.g., cited --[15] X. Tang, "Texture information in run-length matrices," IEEE Transactions on Image Processing, Vol. 7, No. 11, pp. 1602-1609,1998.--, in [0015], and,
-- Feature vector extraction and selection (step S204)
In step S204, feature vectors for each image are extracted and selected to find meaningful features for classifying the nodule types from the multi-view images generated in step S202. As shown in Table 1, the feature vectors extracted for each image are 8 histogram feature vectors considering the brightness of the pixel, 12 above thresholding value, 5 percent CT attenuation value, 56 gray-level co-occurrence matrices (GLCM) -based Haralick feature vectors that take into account the brightness and position information of the pixels and the gray level run-length matrix (GLRLM) based on 88 feature vectors [16]. Before extracting the texture feature, the image with brightness range of 3071 ~ 1024 (HU) is normalized to 8 bit value in the range of 0 ~ 255, and the brightness level of image is set to 16 level and 32 level when calculating GLCM feature and GLRLM feature. Normalize and extract features.--, under Feature vector extraction and selection (step S204), in page 5/12 of English version of KR-101927481-B1 provided with this Office Action);
Vlasimsky (as modified by JIANG) and HONG are combinable as they are in the same field of endeavor: lung nodule detection from low-dose chest CT images processing and analysis. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vlasimsky (as modified by JIANG) ’s method using HONG’s teachings by including the first set of radiomics features includes at least one of: gray-level co-occurrence matrices (GLCM) textures, grey-level run- length matrix (GLRLM) textures, gray level size zone matrix (GLSZM) textures, neighbouring gray tone difference matrix (NGTDM) textures, and gray-level difference matrix (GLDM) textures to Vlasimsky (as modified by JIANG)’s set of features {applied in lung nodule detection and classification} in order to provide the multi-view image and to analyze features from various planes of the divided nodal region (see Hong: e.g., in pages 6-8/12 of English version of KR-101927481-B1 provided with this Office Action).
Therefore, claims 1-12 are still not patentably distinguishable over the prior art reference(s). Further discussions are addressed in the prior art rejection section below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9 are rejected under 35 U.S.C. 103 as being unpatentable over Vlasimsky (US 20220180514 A1), in view of JIANG (“Deep Learning Reconstruction Shows Better Lung Nodule Detection for Ultra-Low-Dose Chest CT”, Radiology: Volume 303: Number 1 - April 2022, pages 202-212), and further in view of HONG (KR 101927481 B1).
Re Claim 1, Vlasimsky discloses method of processing a low-dose computed tomography (CT) image (see Vlasimsky: e.g., --[0005] Lung cancer is typically detected through radiographs, such as X-rays and chest computed tomography (CT) scans, when a nodule appears in the lung. … Low-dose CT (LDCT) scans have also been used to screen people at higher risk on an annual basis.--, in [0005], and, --systems and methods for analyzing chronic pulmonary diseases, such as lung cancer, using machine learning (ML) systems that detect lung nodules in chest x-rays. Preferred systems of the invention use at least two neural networks that analyze a chest x-ray. The first neural network analyzes the entire chest x-ray, preferably at a reduced resolution to improve throughput, and provide a “global” analysis of whether the x-ray contains lung nodules. The second neural network analyzes subsections of the x-ray, preferably using object detection or tiling or raster scanning, to provide “local” analyses of whether specific locations in the x-ray contain lung nodules.
[0009] ML systems can be trained using training data that includes chest x-rays, CT scans, and known pathologies to correlate features in chest x-rays with lung nodules. In addition, CT scans can be used to “ground truth” the ML systems' analyses of chest x-rays (i.e., as a check of the ML system's accuracy).--, in [0008]-[0009]), comprising:
receiving a first chest image (see Vlasimsky: e.g., --[0005] Lung cancer is typically detected through radiographs, such as X-rays and chest computed tomography (CT) scans, when a nodule appears in the lung. … Low-dose CT (LDCT) scans have also been used to screen people at higher risk on an annual basis.--, in [0005], and, --systems and methods for analyzing chronic pulmonary diseases, such as lung cancer, using machine learning (ML) systems that detect lung nodules in chest x-rays. Preferred systems of the invention use at least two neural networks that analyze a chest x-ray. The first neural network analyzes the entire chest x-ray, preferably at a reduced resolution to improve throughput, and provide a “global” analysis of whether the x-ray contains lung nodules. The second neural network analyzes subsections of the x-ray, preferably using object detection or tiling or raster scanning, to provide “local” analyses of whether specific locations in the x-ray contain lung nodules.
[0009] ML systems can be trained using training data that includes chest x-rays, CT scans, and known pathologies to correlate features in chest x-rays with lung nodules. In addition, CT scans can be used to “ground truth” the ML systems' analyses of chest x-rays (i.e., as a check of the ML system's accuracy).--, in [0008]-[0009]);
Vlasimsky however does not explicitly disclose the first chest image generated by a low-dose CT method,
Jiang discloses chest image generated by a low-dose CT method (see Jiang: e.g., -- Further radiation dose reduction for low-dose CT screening may help encourage more use of this method. Ultra-low-dose (ULD) CT reduces the dose level to 0.13-0.49 mSv (7-9), still higher than that of chest radiography (0.03-0.1 mSv) (10).--, in pages 202-203);
Vlasimsky and JIANG are combinable as they are in the same field of endeavor: lung nodule detection from low-dose chest CT images processing and analysis using neural networks and deep learning. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vlasimsky’s method using JIANG’s teachings by including chest image generated by a low-dose CT method to Vlasimsky’s chest x-rays, CT scans including Low-dose CT (LDCT) scans have also been used to screen people in order to provide and achieve radiation dose reduction for low-dose CT screening (see JIANG: e.g., in pages 202-203);
Vlasimsky as modified by JIANG further disclose detecting at least one hung nodule in the first chest image (see Vlasimsky: e.g., --[0005] Lung cancer is typically detected through radiographs, such as X-rays and chest computed tomography (CT) scans, when a nodule appears in the lung. … Low-dose CT (LDCT) scans have also been used to screen people at higher risk on an annual basis.--, in [0005], and, --systems and methods for analyzing chronic pulmonary diseases, such as lung cancer, using machine learning (ML) systems that detect lung nodules in chest x-rays. Preferred systems of the invention use at least two neural networks that analyze a chest x-ray. The first neural network analyzes the entire chest x-ray, preferably at a reduced resolution to improve throughput, and provide a “global” analysis of whether the x-ray contains lung nodules. The second neural network analyzes subsections of the x-ray, preferably using object detection or tiling or raster scanning, to provide “local” analyses of whether specific locations in the x-ray contain lung nodules.
[0009] ML systems can be trained using training data that includes chest x-rays, CT scans, and known pathologies to correlate features in chest x-rays with lung nodules. In addition, CT scans can be used to “ground truth” the ML systems' analyses of chest x-rays (i.e., as a check of the ML system's accuracy).--, in [0008]-[0009]);
determining at least one lung nodule region of the first chest image based on the at least one lung nodule (see Vlasimsky: e.g., --[0005] Lung cancer is typically detected through radiographs, such as X-rays and chest computed tomography (CT) scans, when a nodule appears in the lung. … Low-dose CT (LDCT) scans have also been used to screen people at higher risk on an annual basis.--, in [0005], and, --systems and methods for analyzing chronic pulmonary diseases, such as lung cancer, using machine learning (ML) systems that detect lung nodules in chest x-rays. Preferred systems of the invention use at least two neural networks that analyze a chest x-ray. The first neural network analyzes the entire chest x-ray, preferably at a reduced resolution to improve throughput, and provide a “global” analysis of whether the x-ray contains lung nodules. The second neural network analyzes subsections of the x-ray, preferably using object detection or tiling or raster scanning, to provide “local” analyses of whether specific locations in the x-ray contain lung nodules.
[0009] ML systems can be trained using training data that includes chest x-rays, CT scans, and known pathologies to correlate features in chest x-rays with lung nodules. In addition, CT scans can be used to “ground truth” the ML systems' analyses of chest x-rays (i.e., as a check of the ML system's accuracy).--, in [0008]-[0009]; and, --The collection of neural networks includes at least a first neural network that analyzes the first image and, and a second neural network that analyzes the subsection of the image, wherein the neural networks of the collection each independently make an inference as to the presence of an abnormality. The system includes an ensemble classifier that reports the presence of an abnormality at a location in the lung using the collection of neural network inferences as inputs…..the system segments the image to select a subsection by performing an object detection operation on the image to create a region proposal for an object detected in the image, and then selects the subsection from within the region proposal. In some embodiments, the second neural network assigns a confidence score to the bounded potential objects. The second neural network may classify potential objects as detected objects using the likelihood score. The second neural network may classify objects by creating a heatmap of bounded potential objects and their corresponding confidence scores and classifies objects using the heatmap.
[0014] In certain aspects, the present invention also includes ML systems trained using data from various sources separated by time and/or geography. These training data can include, for example, chest x-ray images, CT scans, and pathology results. Distributed ML subsystems can be placed at, or connected to, those locations and can update the central ML system.--, in [0011]-[0014]); and
classifying the at least one lung nodule region based on a first set of radiomics
features of the at least one lung nodule region of the first chest image to obtain a nodule score of the at least one lung nodule in the lung nodule region (see Vlasimsky: e.g., --ML systems are trained to correlate features in x-rays with data obtained from CT scanning. Those features may be imperceptible to a trained human technician. Nevertheless, the ML systems of the invention can correlate them to features in CT scans associated with a chronic lung pathology. In doing so, the ML systems leverage CT scanning data to improve the diagnostic utility of x-ray imaging….The collection of neural networks includes at least a first neural network that analyzes the first image and, and a second neural network that analyzes the subsection of the image, wherein the neural networks of the collection each independently make an inference as to the presence of an abnormality. The system includes an ensemble classifier that reports the presence of an abnormality at a location in the lung using the collection of neural network inferences as inputs…..the system segments the image to select a subsection by performing an object detection operation on the image to create a region proposal for an object detected in the image, and then selects the subsection from within the region proposal. In some embodiments, the second neural network assigns a confidence score to the bounded potential objects. The second neural network may classify potential objects as detected objects using the likelihood score. The second neural network may classify objects by creating a heatmap of bounded potential objects and their corresponding confidence scores and classifies objects using the heatmap.
[0014] In certain aspects, the present invention also includes ML systems trained using data from various sources separated by time and/or geography. These training data can include, for example, chest x-ray images, CT scans, and pathology results. Distributed ML subsystems can be placed at, or connected to, those locations and can update the central ML system.--, in [0010]-[0014], and, --The module may (i) resize a chest x-ray file to produce a first image at a down-sampled resolution, and (ii) place a subsection of the of the chest x-ray file into a second image at an original resolution of the chest x-ray file. The system may further include a first and a second neural network. The first neural network analyzes the first image to output a first set of scores indicating probabilities of nodules at locations in the lung. The second neural network analyzes the second image to output a second set of scores of probabilities of a nodule at a location in the lung. The neural networks may have been trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. The system may also include an ensemble classifier that reports the presence of the nodule in the location in the lung using the first set of scores and the second set of scores as inputs.
[0016] In certain aspects, the system may further also include a feature engineering module that creates features from the first and second sets of scores and provides the features as inputs for the ensemble classifier. The ensemble classifier may be, for example, a machine learning model trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. In certain aspects, the ensemble classifier is a random forest.--, in [0015]-[0016], [0018]-[0019]);
although above Vlasimsky as modified by JIANG’s disclosures of “radiomics features of the at least one lung nodule region of the first chest image” include – a digital image into a two-dimensional grid or matrix of pixels, which are often regularly sized and dispersed across the matrix/grid.--, in [0055], -- The second neural network can examine the x-ray subsections pixel by pixel to detect minute distinctions grey shading, and thereby detect potential features--, in [0058], and, --“size or shape of a potential lung abnormality, while the “local” output is indicative of more detailed features, or “local” for example, the density and texture of a potential lung abnormality.”,--; in Vlasimsky’s [0061];
Vlasimsky as modified by JIANG do not explicitly disclose the first set of radiomics features includes at least one of: gray-level co-occurrence matrices (GLCM) textures, grey-level run- length matrix (GLRLM) textures, gray level size zone matrix (GLSZM) textures, neighbouring gray tone difference matrix (NGTDM) textures, and gray-level difference matrix (GLDM) textures;
Hong discloses the first set of radiomics features includes at least one of: gray-level co-occurrence matrices (GLCM) textures, grey-level run- length matrix (GLRLM) textures, gray level size zone matrix (GLSZM) textures, neighbouring gray tone difference matrix (NGTDM) textures, and gray-level difference matrix (GLDM) textures {Hong also discloses these radiomics features of the at least one lung nodule region of the first chest image applied in detection and classification} (see HONG: e.g., cited --[15] X. Tang, "Texture information in run-length matrices," IEEE Transactions on Image Processing, Vol. 7, No. 11, pp. 1602-1609,1998.--, in [0015], and,
-- Feature vector extraction and selection (step S204)
In step S204, feature vectors for each image are extracted and selected to find meaningful features for classifying the nodule types from the multi-view images generated in step S202. As shown in Table 1, the feature vectors extracted for each image are 8 histogram feature vectors considering the brightness of the pixel, 12 above thresholding value, 5 percent CT attenuation value, 56 gray-level co-occurrence matrices (GLCM) -based Haralick feature vectors that take into account the brightness and position information of the pixels and the gray level run-length matrix (GLRLM) based on 88 feature vectors [16]. Before extracting the texture feature, the image with brightness range of 3071 ~ 1024 (HU) is normalized to 8 bit value in the range of 0 ~ 255, and the brightness level of image is set to 16 level and 32 level when calculating GLCM feature and GLRLM feature. Normalize and extract features.--, under Feature vector extraction and selection (step S204), in page 5/12 of English version of KR-101927481-B1 provided with this Office Action);
Vlasimsky (as modified by JIANG) and HONG are combinable as they are in the same field of endeavor: lung nodule detection from low-dose chest CT images processing and analysis. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vlasimsky (as modified by JIANG) ’s method using HONG’s teachings by including the first set of radiomics features includes at least one of: gray-level co-occurrence matrices (GLCM) textures, grey-level run- length matrix (GLRLM) textures, gray level size zone matrix (GLSZM) textures, neighbouring gray tone difference matrix (NGTDM) textures, and gray-level difference matrix (GLDM) textures to Vlasimsky (as modified by JIANG)’s set of features {applied in lung nodule detection and classification} in order to provide the multi-view image and to analyze features from various planes of the divided nodal region (see Hong: e.g., in pages 6-8/12 of English version of KR-101927481-B1 provided with this Office Action).
Re Claim 2, Vlasimsky as modified by JIANG and HONG further disclose obtaining one or more sections of the first chest image (see Vlasimsky: e.g., --ML systems are trained to correlate features in x-rays with data obtained from CT scanning. Those features may be imperceptible to a trained human technician. Nevertheless, the ML systems of the invention can correlate them to features in CT scans associated with a chronic lung pathology. In doing so, the ML systems leverage CT scanning data to improve the diagnostic utility of x-ray imaging….The collection of neural networks includes at least a first neural network that analyzes the first image and, and a second neural network that analyzes the subsection of the image, wherein the neural networks of the collection each independently make an inference as to the presence of an abnormality. The system includes an ensemble classifier that reports the presence of an abnormality at a location in the lung using the collection of neural network inferences as inputs…..the system segments the image to select a subsection by performing an object detection operation on the image to create a region proposal for an object detected in the image, and then selects the subsection from within the region proposal. In some embodiments, the second neural network assigns a confidence score to the bounded potential objects. The second neural network may classify potential objects as detected objects using the likelihood score. The second neural network may classify objects by creating a heatmap of bounded potential objects and their corresponding confidence scores and classifies objects using the heatmap.
[0014] In certain aspects, the present invention also includes ML systems trained using data from various sources separated by time and/or geography. These training data can include, for example, chest x-ray images, CT scans, and pathology results. Distributed ML subsystems can be placed at, or connected to, those locations and can update the central ML system.--, in [0010]-[0014], and, --The module may (i) resize a chest x-ray file to produce a first image at a down-sampled resolution, and (ii) place a subsection of the of the chest x-ray file into a second image at an original resolution of the chest x-ray file. The system may further include a first and a second neural network. The first neural network analyzes the first image to output a first set of scores indicating probabilities of nodules at locations in the lung. The second neural network analyzes the second image to output a second set of scores of probabilities of a nodule at a location in the lung. The neural networks may have been trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. The system may also include an ensemble classifier that reports the presence of the nodule in the location in the lung using the first set of scores and the second set of scores as inputs.
[0016] In certain aspects, the system may further also include a feature engineering module that creates features from the first and second sets of scores and provides the features as inputs for the ensemble classifier. The ensemble classifier may be, for example, a machine learning model trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. In certain aspects, the ensemble classifier is a random forest.--, in [0015]-[0016], [0018]-[0019]);
detecting the at least one lung nodule in the first chest image based on the one or more sections of the first chest image (see Vlasimsky: e.g., --ML systems are trained to correlate features in x-rays with data obtained from CT scanning. Those features may be imperceptible to a trained human technician. Nevertheless, the ML systems of the invention can correlate them to features in CT scans associated with a chronic lung pathology. In doing so, the ML systems leverage CT scanning data to improve the diagnostic utility of x-ray imaging….The collection of neural networks includes at least a first neural network that analyzes the first image and, and a second neural network that analyzes the subsection of the image, wherein the neural networks of the collection each independently make an inference as to the presence of an abnormality. The system includes an ensemble classifier that reports the presence of an abnormality at a location in the lung using the collection of neural network inferences as inputs…..the system segments the image to select a subsection by performing an object detection operation on the image to create a region proposal for an object detected in the image, and then selects the subsection from within the region proposal. In some embodiments, the second neural network assigns a confidence score to the bounded potential objects. The second neural network may classify potential objects as detected objects using the likelihood score. The second neural network may classify objects by creating a heatmap of bounded potential objects and their corresponding confidence scores and classifies objects using the heatmap.
[0014] In certain aspects, the present invention also includes ML systems trained using data from various sources separated by time and/or geography. These training data can include, for example, chest x-ray images, CT scans, and pathology results. Distributed ML subsystems can be placed at, or connected to, those locations and can update the central ML system.--, in [0010]-[0014], and, --The module may (i) resize a chest x-ray file to produce a first image at a down-sampled resolution, and (ii) place a subsection of the of the chest x-ray file into a second image at an original resolution of the chest x-ray file. The system may further include a first and a second neural network. The first neural network analyzes the first image to output a first set of scores indicating probabilities of nodules at locations in the lung. The second neural network analyzes the second image to output a second set of scores of probabilities of a nodule at a location in the lung. The neural networks may have been trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. The system may also include an ensemble classifier that reports the presence of the nodule in the location in the lung using the first set of scores and the second set of scores as inputs.
[0016] In certain aspects, the system may further also include a feature engineering module that creates features from the first and second sets of scores and provides the features as inputs for the ensemble classifier. The ensemble classifier may be, for example, a machine learning model trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. In certain aspects, the ensemble classifier is a random forest.--, in [0015]-[0016], [0018]-[0019]).
Re Claim 3, although Vlasimsky as modified by JIANG further disclose detecting the at least one lung nodule in the first chest image based on the one or more sections of the first chest image (see Vlasimsky: e.g., --[0008] The invention provides systems and methods for analyzing chronic pulmonary diseases, such as lung cancer, using machine learning (ML) systems that detect lung nodules in chest x-rays. Preferred systems of the invention use at least two neural networks that analyze a chest x-ray. The first neural network analyzes the entire chest x-ray, preferably at a reduced resolution to improve throughput, and provide a “global” analysis of whether the x-ray contains lung nodules. The second neural network analyzes subsections of the x-ray, preferably using object detection or tiling or raster scanning, to provide “local” analyses of whether specific locations in the x-ray contain lung nodules.--, in [0008], [0011]-[0013], and [0015]-[0017]);
Vlasimsky as modified by JIANG however still do not explicitly disclose that the one or more sections of the first chest image include sections along at least one of: a sagittal plane, a coronal plane, an axial plane, a first plane inclined 30 degrees from the coronal plane to the sagittal plane, a second plane inclined 30 degrees from the coronal plane to the axial plane, a third plane inclined 15 degrees from the sagittal plane to the coronal plane, or a fourth plane inclined 15 degrees from the sagittal plane to the axial plane,
Hong discloses the one or more sections of the first chest image include sections along at least one of: a sagittal plane, a coronal plane, an axial plane, a first plane inclined 30 degrees from the coronal plane to the sagittal plane, a second plane inclined 30 degrees from the coronal plane to the axial plane, a third plane inclined 15 degrees from the sagittal plane to the coronal plane, or a fourth plane inclined 15 degrees from the sagittal plane to the axial plane (see HONG: e.g., --, an MVI image is generated in addition to an axial view image in order to use more information of the divided nodule area. As shown in FIG. 4, the multi-view image is composed of 6 planes considering three x-axis, x-axis, z-axis and y-axis from the upper left corner to the lower right corner
in consideration of the top surface, coronal plane, sagittal plane and xyz axis. consist of. A cube with center point (center point; c. P.) Is created at the center of gravity of the liver glass nodule. When each plane is represented by an angle with respect to a plane composed of two axes such as xy-z (0 ?) and another axis, in FIG. 4, the three 2D planes on the upper side are xy-z (0 ?) (0 ?) and sagittal planes can be represented
as yz-x (0 ?), and the six lower 2.5D planes can be represented as yz-x (45 ?) and yz-x 135 ?), zx-y (45 ?), zx-y (135 ?), xy-z (45 ?) and xy-z (135 ?).
Thus, the nodule characteristics can be analyzed by considering the influence of the shape and position of the liver nodule by dividing the liver nodule region from the pulmonary parenchyma region and generating the multi-view image from the divided liver nodule.--, in page 6/12 of English version of KR-101927481-B1 provided with this Office Action; and, -- In order to evaluate the classification method of low-solid-content free lung nodule by multi-view images and texture analysis on chest CT images according to one embodiment of the present invention, 10 pure pure glass nodules and 10 intermixed nodules A total of 20 chest CT nodules were obtained.--, under Experiments and results, in pages 7-8/12 of English version of KR-101927481-B1 provided with this Office Action);
Vlasimsky (as modified by JIANG) and HONG are combinable as they are in the same field of endeavor: lung nodule detection from low-dose chest CT images processing and analysis. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vlasimsky (as modified by JIANG) ’s method using HONG’s teachings by including the one or more sections of the first chest image include sections along at least one of: a sagittal plane, a coronal plane, an axial plane, a first plane inclined 30 degrees from the coronal plane to the sagittal plane, a second plane inclined 30 degrees from the coronal plane to the axial plane, a third plane inclined 15 degrees from the sagittal plane to the coronal plane, or a fourth plane inclined 15 degrees from the sagittal plane to the axial plane to Vlasimsky (as modified by JIANG)’s one or more sections of chest x-rays, CT scans including Low-dose CT (LDCT) scans have also been used to screen people {in lung nodule detection and classification} in order to provide the multi-view image and to analyze features from various planes of the divided nodal region (see Hong: e.g., in pages 6-8/12 of English version of KR-101927481-B1 provided with this Office Action).
Re Claim 4, Vlasimsky as modified by JIANG and HONG further disclose wherein determining the at least one hing nodule region comprises:
obtaining a boundary of each of the at least one lung nodule region (see Vlasimsky: e.g., --ML systems are trained to correlate features in x-rays with data obtained from CT scanning. Those features may be imperceptible to a trained human technician. Nevertheless, the ML systems of the invention can correlate them to features in CT scans associated with a chronic lung pathology. In doing so, the ML systems leverage CT scanning data to improve the diagnostic utility of x-ray imaging….The collection of neural networks includes at least a first neural network that analyzes the first image and, and a second neural network that analyzes the subsection of the image, wherein the neural networks of the collection each independently make an inference as to the presence of an abnormality. The system includes an ensemble classifier that reports the presence of an abnormality at a location in the lung using the collection of neural network inferences as inputs…..the system segments the image to select a subsection by performing an object detection operation on the image to create a region proposal for an object detected in the image, and then selects the subsection from within the region proposal. In some embodiments, the second neural network assigns a confidence score to the bounded potential objects. The second neural network may classify potential objects as detected objects using the likelihood score. The second neural network may classify objects by creating a heatmap of bounded potential objects and their corresponding confidence scores and classifies objects using the heatmap.
[0014] In certain aspects, the present invention also includes ML systems trained using data from various sources separated by time and/or geography. These training data can include, for example, chest x-ray images, CT scans, and pathology results. Distributed ML subsystems can be placed at, or connected to, those locations and can update the central ML system.--, in [0010]-[0014], and, --The module may (i) resize a chest x-ray file to produce a first image at a down-sampled resolution, and (ii) place a subsection of the of the chest x-ray file into a second image at an original resolution of the chest x-ray file. The system may further include a first and a second neural network. The first neural network analyzes the first image to output a first set of scores indicating probabilities of nodules at locations in the lung. The second neural network analyzes the second image to output a second set of scores of probabilities of a nodule at a location in the lung. The neural networks may have been trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. The system may also include an ensemble classifier that reports the presence of the nodule in the location in the lung using the first set of scores and the second set of scores as inputs.
[0016] In certain aspects, the system may further also include a feature engineering module that creates features from the first and second sets of scores and provides the features as inputs for the ensemble classifier. The ensemble classifier may be, for example, a machine learning model trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. In certain aspects, the ensemble classifier is a random forest.--, in [0015]-[0016], [0018]-[0019]); and
calculating a size of each of the at least one lung nodule region based on the boundary of the corresponding lang nodule (see Vlasimsky: e.g., -- the third neural network provides an “intermediate” output. This output may be a set of scores indicative of, for example, certain anatomical features such as ribs and/or larger lung abnormalities. By using such a third neural network, lung abnormalities that were identified using the second neural network as separate, individual abnormalities can be resolved into a single large abnormality. In certain aspects, the third neural network may provide an “intermediate” output indicative of the entire size or shape of a potential lung abnormality, while the “local” output is indicative of more detailed features, or “local” for example, the density and texture of a potential lung abnormality.--, in [0061]; also see:
--ML systems are trained to correlate features in x-rays with data obtained from CT scanning. Those features may be imperceptible to a trained human technician. Nevertheless, the ML systems of the invention can correlate them to features in CT scans associated with a chronic lung pathology. In doing so, the ML systems leverage CT scanning data to improve the diagnostic utility of x-ray imaging….The collection of neural networks includes at least a first neural network that analyzes the first image and, and a second neural network that analyzes the subsection of the image, wherein the neural networks of the collection each independently make an inference as to the presence of an abnormality. The system includes an ensemble classifier that reports the presence of an abnormality at a location in the lung using the collection of neural network inferences as inputs…..the system segments the image to select a subsection by performing an object detection operation on the image to create a region proposal for an object detected in the image, and then selects the subsection from within the region proposal. In some embodiments, the second neural network assigns a confidence score to the bounded potential objects. The second neural network may classify potential objects as detected objects using the likelihood score. The second neural network may classify objects by creating a heatmap of bounded potential objects and their corresponding confidence scores and classifies objects using the heatmap.
[0014] In certain aspects, the present invention also includes ML systems trained using data from various sources separated by time and/or geography. These training data can include, for example, chest x-ray images, CT scans, and pathology results. Distributed ML subsystems can be placed at, or connected to, those locations and can update the central ML system.--, in [0010]-[0014], and, --The module may (i) resize a chest x-ray file to produce a first image at a down-sampled resolution, and (ii) place a subsection of the of the chest x-ray file into a second image at an original resolution of the chest x-ray file. The system may further include a first and a second neural network. The first neural network analyzes the first image to output a first set of scores indicating probabilities of nodules at locations in the lung. The second neural network analyzes the second image to output a second set of scores of probabilities of a nodule at a location in the lung. The neural networks may have been trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. The system may also include an ensemble classifier that reports the presence of the nodule in the location in the lung using the first set of scores and the second set of scores as inputs.
[0016] In certain aspects, the system may further also include a feature engineering module that creates features from the first and second sets of scores and provides the features as inputs for the ensemble classifier. The ensemble classifier may be, for example, a machine learning model trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. In certain aspects, the ensemble classifier is a random forest.--, in [0015]-[0016], [0018]-[0019]).
.
Re Claim 5, Vlasimsky as modified by JIANG and HONG further disclose wherein classifying the at least one lung nodule region comprises: determining a texture type of each of the at least one lung nodule region based on the first set of radiomics features (see Vlasimsky: e.g., -- the third neural network provides an “intermediate” output. This output may be a set of scores indicative of, for example, certain anatomical features such as ribs and/or larger lung abnormalities. By using such a third neural network, lung abnormalities that were identified using the second neural network as separate, individual abnormalities can be resolved into a single large abnormality. In certain aspects, the third neural network may provide an “intermediate” output indicative of the entire size or shape of a potential lung abnormality, while the “local” output is indicative of more detailed features, or “local” for example, the density and texture of a potential lung abnormality.--, in [0061]; also see:
--ML systems are trained to correlate features in x-rays with data obtained from CT scanning. Those features may be imperceptible to a trained human technician. Nevertheless, the ML systems of the invention can correlate them to features in CT scans associated with a chronic lung pathology. In doing so, the ML systems leverage CT scanning data to improve the diagnostic utility of x-ray imaging….The collection of neural networks includes at least a first neural network that analyzes the first image and, and a second neural network that analyzes the subsection of the image, wherein the neural networks of the collection each independently make an inference as to the presence of an abnormality. The system includes an ensemble classifier that reports the presence of an abnormality at a location in the lung using the collection of neural network inferences as inputs…..the system segments the image to select a subsection by performing an object detection operation on the image to create a region proposal for an object detected in the image, and then selects the subsection from within the region proposal. In some embodiments, the second neural network assigns a confidence score to the bounded potential objects. The second neural network may classify potential objects as detected objects using the likelihood score. The second neural network may classify objects by creating a heatmap of bounded potential objects and their corresponding confidence scores and classifies objects using the heatmap.
[0014] In certain aspects, the present invention also includes ML systems trained using data from various sources separated by time and/or geography. These training data can include, for example, chest x-ray images, CT scans, and pathology results. Distributed ML subsystems can be placed at, or connected to, those locations and can update the central ML system.--, in [0010]-[0014], and, --The module may (i) resize a chest x-ray file to produce a first image at a down-sampled resolution, and (ii) place a subsection of the of the chest x-ray file into a second image at an original resolution of the chest x-ray file. The system may further include a first and a second neural network. The first neural network analyzes the first image to output a first set of scores indicating probabilities of nodules at locations in the lung. The second neural network analyzes the second image to output a second set of scores of probabilities of a nodule at a location in the lung. The neural networks may have been trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. The system may also include an ensemble classifier that reports the presence of the nodule in the location in the lung using the first set of scores and the second set of scores as inputs.
[0016] In certain aspects, the system may further also include a feature engineering module that creates features from the first and second sets of scores and provides the features as inputs for the ensemble classifier. The ensemble classifier may be, for example, a machine learning model trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. In certain aspects, the ensemble classifier is a random forest.--, in [0015]-[0016], [0018]-[0019]);
determining a margin type of each of the at least one lung nodule in the lung nodule region based on the first set of radiomics features, and determining the nodule score of the at least one lung nodule region based on the sizes, the texture types, the margin types of the at least one hung nodule region (see Vlasimsky: e.g., -- the third neural network provides an “intermediate” output. This output may be a set of scores indicative of, for example, certain anatomical features such as ribs and/or larger lung abnormalities. By using such a third neural network, lung abnormalities that were identified using the second neural network as separate, individual abnormalities can be resolved into a single large abnormality. In certain aspects, the third neural network may provide an “intermediate” output indicative of the entire size or shape of a potential lung abnormality, while the “local” output is indicative of more detailed features, or “local” for example, the density and texture of a potential lung abnormality.--, in [0061]; also see:
--ML systems are trained to correlate features in x-rays with data obtained from CT scanning. Those features may be imperceptible to a trained human technician. Nevertheless, the ML systems of the invention can correlate them to features in CT scans associated with a chronic lung pathology. In doing so, the ML systems leverage CT scanning data to improve the diagnostic utility of x-ray imaging….The collection of neural networks includes at least a first neural network that analyzes the first image and, and a second neural network that analyzes the subsection of the image, wherein the neural networks of the collection each independently make an inference as to the presence of an abnormality. The system includes an ensemble classifier that reports the presence of an abnormality at a location in the lung using the collection of neural network inferences as inputs…..the system segments the image to select a subsection by performing an object detection operation on the image to create a region proposal for an object detected in the image, and then selects the subsection from within the region proposal. In some embodiments, the second neural network assigns a confidence score to the bounded potential objects. The second neural network may classify potential objects as detected objects using the likelihood score. The second neural network may classify objects by creating a heatmap of bounded potential objects and their corresponding confidence scores and classifies objects using the heatmap.
[0014] In certain aspects, the present invention also includes ML systems trained using data from various sources separated by time and/or geography. These training data can include, for example, chest x-ray images, CT scans, and pathology results. Distributed ML subsystems can be placed at, or connected to, those locations and can update the central ML system.--, in [0010]-[0014], and, --The module may (i) resize a chest x-ray file to produce a first image at a down-sampled resolution, and (ii) place a subsection of the of the chest x-ray file into a second image at an original resolution of the chest x-ray file. The system may further include a first and a second neural network. The first neural network analyzes the first image to output a first set of scores indicating probabilities of nodules at locations in the lung. The second neural network analyzes the second image to output a second set of scores of probabilities of a nodule at a location in the lung. The neural networks may have been trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. The system may also include an ensemble classifier that reports the presence of the nodule in the location in the lung using the first set of scores and the second set of scores as inputs.
[0016] In certain aspects, the system may further also include a feature engineering module that creates features from the first and second sets of scores and provides the features as inputs for the ensemble classifier. The ensemble classifier may be, for example, a machine learning model trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. In certain aspects, the ensemble classifier is a random forest.--, in [0015]-[0016], [0018]-[0019]).
Re Claim 6, Vlasimsky as modified by JIANG and HONG further disclose wherein the margin type includes sharp circumscribed, lobulated, indistinct, and speculated, the texture type includes solid, sub-solid, and ground glass opacity (see Vlasimsky: e.g., -- the method also includes characterizing an identified lung nodule with the machine learning system. Characterizing may include, for example, classifying a nodule as a tumor, benign and/or malignant, and/or assessing or predicting nodule progression, volumetric sizing, nodule etiology, nodule histology, and/or a treatment response. A classified tumor may by analyzed using the system using Response Evaluation Criteria in Solid Tumor guidelines.--, in [0031]; -- [0061] In certain aspects, the third neural network provides an “intermediate” output. This output may be a set of scores indicative of, for example, certain anatomical features such as ribs and/or larger lung abnormalities. By using such a third neural network, lung abnormalities that were identified using the second neural network as separate, individual abnormalities can be resolved into a single large abnormality. In certain aspects, the third neural network may provide an “intermediate” output indicative of the entire size or shape of a potential lung abnormality, while the “local” output is indicative of more detailed features, or “local” for example, the density and texture of a potential lung abnormality.--, in [0061]. and, -- [0155] Deep learning is part of a broader family of machine learning methods based on learning representations of data. An observation (e.g., an image) can be represented in many ways such as a vector of intensity values per pixel, or in a more abstract way as a set of edges, regions of particular shape, etc. Those features are represented at nodes in the network. Preferably, each feature is structured as a feature vector, a multi-dimensional vector of numerical features that represent some object. The feature provides a numerical representation of objects, since such representations facilitate processing and statistical analysis. Feature vectors are similar to the vectors of explanatory variables used in statistical procedures such as linear regression. Feature vectors are often combined with weights using a dot product in order to construct a linear predictor function that is used to determine a score for making a prediction.--, in [0155]; also see JIANG: e.g., -- imaging features of histologically confirmed malignant nodules. Malignancy-related features included lobulated shapes, spiculated margins, pleural tags, and air bronchograms (22).--, in page 204, 209; further see: -- the long diameter and volume of nodules were over- or underestimated in ULD CT with CECT as a reference, which was also observed in some previous studies of ULD CT (24) and low-dose CT (25,26). In our study, DUR-H slightly underestimated the long diameter and volume of subsolid nodules but overestimated the solid and calcified nodules. Due to the partial volume effect, there is a transition zone between high attenuation (nodules) and low-attenuation (pulmonary parenchyma) objects on CT images, which is important for accurate volumetry (27). Since the measured values were determined by the segmentation algorithm, the transition zone around solid or calcified nodules would lead to overestimation. However, subsolid nodules (especially pure ground-glass nodules) have blurred margins and low attenuation and are therefore indistinguishable from lung parenchyma, which would reduce the accuracy of nodule segmentation, resulting in underestimation. Subsequently, we observed malignancy-related imaging features at different ULD CT reconstruction sequences. Among them, DUR images had the most abundant subtle features, and the overall malignant feature detection rate was 81.5%.--, in page 210).
Re Claim 7, Vlasimsky as modified by JIANG and HONG further disclose determining a location of the at least one lung nodule (see Vlasimsky: e.g., -- the third neural network provides an “intermediate” output. This output may be a set of scores indicative of, for example, certain anatomical features such as ribs and/or larger lung abnormalities. By using such a third neural network, lung abnormalities that were identified using the second neural network as separate, individual abnormalities can be resolved into a single large abnormality. In certain aspects, the third neural network may provide an “intermediate” output indicative of the entire size or shape of a potential lung abnormality, while the “local” output is indicative of more detailed features, or “local” for example, the density and texture of a potential lung abnormality.--, in [0061]; also see:
--ML systems are trained to correlate features in x-rays with data obtained from CT scanning. Those features may be imperceptible to a trained human technician. Nevertheless, the ML systems of the invention can correlate them to features in CT scans associated with a chronic lung pathology. In doing so, the ML systems leverage CT scanning data to improve the diagnostic utility of x-ray imaging….The collection of neural networks includes at least a first neural network that analyzes the first image and, and a second neural network that analyzes the subsection of the image, wherein the neural networks of the collection each independently make an inference as to the presence of an abnormality. The system includes an ensemble classifier that reports the presence of an abnormality at a location in the lung using the collection of neural network inferences as inputs…..the system segments the image to select a subsection by performing an object detection operation on the image to create a region proposal for an object detected in the image, and then selects the subsection from within the region proposal. In some embodiments, the second neural network assigns a confidence score to the bounded potential objects. The second neural network may classify potential objects as detected objects using the likelihood score. The second neural network may classify objects by creating a heatmap of bounded potential objects and their corresponding confidence scores and classifies objects using the heatmap.
[0014] In certain aspects, the present invention also includes ML systems trained using data from various sources separated by time and/or geography. These training data can include, for example, chest x-ray images, CT scans, and pathology results. Distributed ML subsystems can be placed at, or connected to, those locations and can update the central ML system.--, in [0010]-[0014], and, --The module may (i) resize a chest x-ray file to produce a first image at a down-sampled resolution, and (ii) place a subsection of the of the chest x-ray file into a second image at an original resolution of the chest x-ray file. The system may further include a first and a second neural network. The first neural network analyzes the first image to output a first set of scores indicating probabilities of nodules at locations in the lung. The second neural network analyzes the second image to output a second set of scores of probabilities of a nodule at a location in the lung. The neural networks may have been trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. The system may also include an ensemble classifier that reports the presence of the nodule in the location in the lung using the first set of scores and the second set of scores as inputs.
[0016] In certain aspects, the system may further also include a feature engineering module that creates features from the first and second sets of scores and provides the features as inputs for the ensemble classifier. The ensemble classifier may be, for example, a machine learning model trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. In certain aspects, the ensemble classifier is a random forest.--, in [0015]-[0016], [0018]-[0019]; also see JIANG: e.g., -- imaging features of histologically confirmed malignant nodules. Malignancy-related features included lobulated shapes, spiculated margins, pleural tags, and air bronchograms (22).--, in page 204, 209; further see: -- the long diameter and volume of nodules were over- or underestimated in ULD CT with CECT as a reference, which was also observed in some previous studies of ULD CT (24) and low-dose CT (25,26). In our study, DUR-H slightly underestimated the long diameter and volume of subsolid nodules but overestimated the solid and calcified nodules. Due to the partial volume effect, there is a transition zone between high attenuation (nodules) and low-attenuation (pulmonary parenchyma) objects on CT images, which is important for accurate volumetry (27). Since the measured values were determined by the segmentation algorithm, the transition zone around solid or calcified nodules would lead to overestimation. However, subsolid nodules (especially pure ground-glass nodules) have blurred margins and low attenuation and are therefore indistinguishable from lung parenchyma, which would reduce the accuracy of nodule segmentation, resulting in underestimation. Subsequently, we observed malignancy-related imaging features at different ULD CT reconstruction sequences. Among them, DUR images had the most abundant subtle features, and the overall malignant feature detection rate was 81.5%.--, in page 210).
Re Claim 8, Vlasimsky as modified by JIANG and HONG further disclose wherein the location of the at least one lung nodule includes a right upper lobe, a right middle lobe, a right lower lobe, a left upper lobe, a left lower lobe, and a lingular lobe (see Vlasimsky: Fig. 3A, also see JIANG: e.g., Fig. 4, and Fig. 6).
Re Claim 9, Vlasimsky as modified by JIANG and HONG further disclose wherein classifying the at least one lung nodule region is based on the first set of radiomics features and a first set of slice features of the at least one lung nodule region of the first chest image (see Vlasimsky: e.g., --[0005] Lung cancer is typically detected through radiographs, such as X-rays and chest computed tomography (CT) scans, when a nodule appears in the lung. … Low-dose CT (LDCT) scans have also been used to screen people at higher risk on an annual basis.--, in [0005], and, --systems and methods for analyzing chronic pulmonary diseases, such as lung cancer, using machine learning (ML) systems that detect lung nodules in chest x-rays. Preferred systems of the invention use at least two neural networks that analyze a chest x-ray. The first neural network analyzes the entire chest x-ray, preferably at a reduced resolution to improve throughput, and provide a “global” analysis of whether the x-ray contains lung nodules. The second neural network analyzes subsections of the x-ray, preferably using object detection or tiling or raster scanning, to provide “local” analyses of whether specific locations in the x-ray contain lung nodules.
[0009] ML systems can be trained using training data that includes chest x-rays, CT scans, and known pathologies to correlate features in chest x-rays with lung nodules. In addition, CT scans can be used to “ground truth” the ML systems' analyses of chest x-rays (i.e., as a check of the ML system's accuracy).--, in [0008]-[0009]; also see HONG: e.g., --Experiments and results:
In order to evaluate the classification method of low-solid-content free lung nodule by multi-view images and texture analysis on chest CT images according to one embodiment of the present invention, 10 pure pure glass nodules and 10 intermixed nodules A total of 20 chest CT nodules were obtained. Twenty nodules with a diameter of less than 5 mm were selected by a specialist. The images were taken by SIEMENS Sensation 16 CT Scanner and Philips Brilliance 64 CT Scanner in Seoul National University Hospital. The kernels used in the SIEMENS CT Scanner were B30f, B50f, B60f and the kernel used in the Philips CT Scanner Is YC. The image resolution is 512 ? 512, the pixel size is 0.50 ? 0.50 mm to 0.76 ? 0.76 mm, and the slice interval is 1 mm.--, in pages 6-8/12 of English version of KR-101927481-B1 provided with this Office Action).
Claims 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Vlasimsky as modified by JIANG and HONG, and further in view of YI (US 20220172826 A1).
Re Claim 10, although Vlasimsky as modified by JIANG and HONG disclose extracting a region in the first chest image by using a U-Net model (see Vlasimsky: e.g., -- [0103] FIG. 13 shows that preprocessing may include data validation 1303, data augmentation 1305, an image extractor 1307, image quality validation 1309, and a lung segmentation U-Net model 1311.--, in [0103], and [0110]);
Vlasimsky as modified by JIANG and HONG however still do not explicitly disclose extracting a heart region in the first chest image, determining a coronary artery calcification (CAC) score of the heart region by an transferred Efficient Net model,
YI discloses extracting a heart region in the first chest image, determining a coronary artery calcification (CAC) score of the heart region by an transferred Efficient Net model (see YI: e.g., -- [0003] Currently, medical images such as computed tomography (CT) images are widely used to analyze lesions and use analysis results for diagnosis. For example, chest CT images are frequently used for reading because they allow readers to observe abnormalities in parts of the human body such as the lungs, the bronchi, and the heart.--, in [0003], -- for the lungs, a lung nodule as well as chronic obstructive pulmonary disease (COPD) may be diagnosed, emphysema may be diagnosed, and/or chronic bronchitis and/or an airway-related disease may also be diagnosed. In addition, coronary artery calcification (CAC) scoring may be analyzed in a chest CT image in addition to lung disease…. [0013] In medical image reading assistance using an artificial neural network, each finding must include diagnostic assistant information obtained by quantifying probability or confidence. Since it is not possible to provide all the findings to a user, the findings are filtered by applying a predetermined threshold, and only passed findings are provided to the user. --, in [0011]-[0013], and, Fig. 2, -- generate the follow-up information between first findings on the first medical image and second findings on the second medical image by comparing the locations of the first findings and the locations of the second findings. In this case, the at least one processor 210 may include the first artificial neural network 230 so that the first artificial neural network 230 can infer whether the first findings and the second findings are follow-up matched with each other. In other words, the at least one processor 210 may control the first artificial neural network 230 so that the first artificial neural network 230 identifies a first subset of second findings corresponding to the first findings among the second findings on the second medical image.--, in [0056]-[0058], and, -- [0093] FIG. 6 shows an example of a medical image reading assistant workflow according to an embodiment of the present invention, which illustrates an example of a medical image reading assistant screenshot for lung cancer screening (LCS).
[0094] The lung cancer screening shown in FIG. 6 is used to detect lung nodules after reconstruction when executed in a low-dose or ultralow-dose CT. Although various conventional techniques have been proposed for this, it is most important to distinguish between a normal organ and a lesion, and thus an important goal is to obtain an area having a brightness value different from a surrounding brightness value.--, in [0093]-[0094]);
Vlasimsky (as modified by JIANG and HONG) and YI are combinable as they are in the same field of endeavor: lung nodule detection from low-dose chest CT images processing and analysis. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Vlasimsky (as modified by JIANG and HONG) ’s method using YI’s teachings by including extracting a heart region in the first chest image, determining a coronary artery calcification (CAC) score of the heart region by an transferred Efficient Net model to Vlasimsky (as modified by JIANG and HONG)’s chest x-rays, CT scans including Low-dose CT (LDCT) scans, and U-Net and extraction have also been used to screen people in order to analyze coronary artery calcification (CAC) scoring in a chest CT image in addition to lung disease (see YI: e.g., in [0003], [0011], and [0094]).
Re Claim 11, Vlasimsky as modified by JIANG, Hong and YI further disclose providing a treatment recommendation based on the CAC score (see YI: e.g., -- [0003] Currently, medical images such as computed tomography (CT) images are widely used to analyze lesions and use analysis results for diagnosis. For example, chest CT images are frequently used for reading because they allow readers to observe abnormalities in parts of the human body such as the lungs, the bronchi, and the heart.--, in [0003], -- for the lungs, a lung nodule as well as chronic obstructive pulmonary disease (COPD) may be diagnosed, emphysema may be diagnosed, and/or chronic bronchitis and/or an airway-related disease may also be diagnosed. In addition, coronary artery calcification (CAC) scoring may be analyzed in a chest CT image in addition to lung disease…. [0013] In medical image reading assistance using an artificial neural network, each finding must include diagnostic assistant information obtained by quantifying probability or confidence. Since it is not possible to provide all the findings to a user, the findings are filtered by applying a predetermined threshold, and only passed findings are provided to the user. --, in [0011]-[0013], and, Fig. 2, -- generate the follow-up information between first findings on the first medical image and second findings on the second medical image by comparing the locations of the first findings and the locations of the second findings. In this case, the at least one processor 210 may include the first artificial neural network 230 so that the first artificial neural network 230 can infer whether the first findings and the second findings are follow-up matched with each other. In other words, the at least one processor 210 may control the first artificial neural network 230 so that the first artificial neural network 230 identifies a first subset of second findings corresponding to the first findings among the second findings on the second medical image.--, in [0056]-[0058], and, -- [0093] FIG. 6 shows an example of a medical image reading assistant workflow according to an embodiment of the present invention, which illustrates an example of a medical image reading assistant screenshot for lung cancer screening (LCS).
[0094] The lung cancer screening shown in FIG. 6 is used to detect lung nodules after reconstruction when executed in a low-dose or ultralow-dose CT. Although various conventional techniques have been proposed for this, it is most important to distinguish between a normal organ and a lesion, and thus an important goal is to obtain an area having a brightness value different from a surrounding brightness value.--, in [0093]-[0094]).
Re Claim 12, Vlasimsky as modified by JIANG, HONG and YI further disclose wherein the transferred Efficient Net model is trained from a pre-trained model for heart full-dose reference CT images and a low-dose reference CT image captured from a same region (see Vlasimsky: e.g., --[0005] Lung cancer is typically detected through radiographs, such as X-rays and chest computed tomography (CT) scans, when a nodule appears in the lung. … Low-dose CT (LDCT) scans have also been used to screen people at higher risk on an annual basis.--, in [0005], and, --systems and methods for analyzing chronic pulmonary diseases, such as lung cancer, using machine learning (ML) systems that detect lung nodules in chest x-rays. Preferred systems of the invention use at least two neural networks that analyze a chest x-ray. The first neural network analyzes the entire chest x-ray, preferably at a reduced resolution to improve throughput, and provide a “global” analysis of whether the x-ray contains lung nodules. The second neural network analyzes subsections of the x-ray, preferably using object detection or tiling or raster scanning, to provide “local” analyses of whether specific locations in the x-ray contain lung nodules.
[0009] ML systems can be trained using training data that includes chest x-rays, CT scans, and known pathologies to correlate features in chest x-rays with lung nodules. In addition, CT scans can be used to “ground truth” the ML systems' analyses of chest x-rays (i.e., as a check of the ML system's accuracy).--, in [0008]-[0009], and, -- the invention provides systems for detecting lung abnormalities. Systems of the disclosure may include an image pre-processing module that resizes a chest x-ray image to produce a first image at a down-sampled or up-sampled resolution and segments the image into at least one subsection of the image that represents an organ of a body and a collection of neural networks, trained using a plurality of network architectures. The collection of neural networks includes at least a first neural network that analyzes the first image and, and a second neural network that analyzes the subsection of the image, wherein the neural networks of the collection each independently make an inference as to the presence of an abnormality. The system includes an ensemble classifier that reports the presence of an abnormality at a location in the lung using the collection of neural network inferences as inputs. Each neural network of the collection may be, for example, one of Faster R-CNN, Inception-Resnet, DenseNet, or NasNet (e.g., preferably with no two being the same). In some embodiments, the system first analyzes the image for quality and positioning accuracy and is operable to reject an image from further processing and provide a real-time notification instructing a technician to acquire another image. For example, the system may apply one or more exclusion criteria to the image, which optionally include patient age, over exposure, under exposure, or content of image metadata. Optionally, the pre-processing module (i) checks for image quality and positioning, (ii) standardizes image brightness and contrast, and/or (iii) standardizes the image across a plurality of images acquisition devices.
[0012] In certain embodiments, the system includes a feature engineering module that creates features from the collections of neural networks and provides the features as inputs for the ensemble classifier. The ensemble classifier may use averaging, logistic regression, a generalized linear model, or a random forest algorithm.
[0013] In preferred embodiments, the subsection (for the “local” image analysis) is at an original first resolution of the chest x-ray image and the first image (for the “global” image analysis) is at a lower second resolution than the original resolution. The collection of neural networks may include a third neural network. In some embodiments, the system parses the image into one or more segments (e.g., as adjacent “tiles” or overlapping pieces from a raster) where each image segment may include an image of intermediate resolution. The third neural network may analyze the image segments at the intermediate resolution to make an inference as to the presence of an abnormality. In some embodiments, the system segments the image to select a subsection by performing an object detection operation on the image to create a region proposal for an object detected in the image, and then selects the subsection from within the region proposal. In some embodiments, the second neural network assigns a confidence score to the bounded potential objects. The second neural network may classify potential objects as detected objects using the likelihood score. The second neural network may classify objects by creating a heatmap of bounded potential objects and their corresponding confidence scores and classifies objects using the heatmap.
[0014] In certain aspects, the present invention also includes ML systems trained using data from various sources separated by time and/or geography. These training data can include, for example, chest x-ray images, CT scans, and pathology results. Distributed ML subsystems can be placed at, or connected to, those locations and can update the central ML system. In certain embodiments, distributed systems include computer hardware with machine learning systems stored therein, in which the hardware is shipped (e.g., by overland freight and/or by air) to the clinical sites (e.g., hospitals or research institutions) where the data are located. Additionally or alternatively, the distributed ML systems may be connected (e.g., transiently, e.g., for a few hours or days) to the data at those clinical sites. A federated learning model can be used to update the ML system using data analyzed by the subsystems. By using such an arrangement, the ML systems of the invention can be trained using data from distributed sources, while ensuring that confidential patient data do not leave a hospital or other research institution. Moreover, in certain aspects, the ML systems or subsystems can preprocess data to eliminate biases or artifacts attributable to different instruments, e.g., CT scanners from different manufacturers.
[0015] In certain aspects, the present invention provides a system for detecting lung abnormalities in a subject. The system may include an image pre-processing module. The module may (i) resize a chest x-ray file to produce a first image at a down-sampled resolution, and (ii) place a subsection of the of the chest x-ray file into a second image at an original resolution of the chest x-ray file. The system may further include a first and a second neural network. The first neural network analyzes the first image to output a first set of scores indicating probabilities of nodules at locations in the lung. The second neural network analyzes the second image to output a second set of scores of probabilities of a nodule at a location in the lung. The neural networks may have been trained using chest x-ray images, lung CT scans, lung PET-CT scans, and/or clinical outcome data. The system may also include an ensemble classifier that reports the presence of the nodule in the location in the lung using the first set of scores and the second set of scores as inputs.--, in [0011]-[0015]).
Conclusion
Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEI WEN YANG whose telephone number is (571)270-5670. The examiner can normally be reached on 8:00 - 5:00 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEI WEN YANG/Primary Examiner, Art Unit 2662