DETAILED ACTION
Claims 1, 2, 4-8, 10 and 13-15 are currently pending.
Response to Arguments
Applicant’s arguments with respect to claims 1, 2, 4-8, 10 and 13-15 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4-8, 10 and 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Huo et al. US Publication 2020/0098108 (hereafter “Huo”), Pereira et al. US Publication 2015/0018664 (hereafter “Pereira”) and Kalafut et al. US Publication 2021/0098127 (hereafter “Kalafut”).
Referring to claims 1, 13 and 14, Huo discloses a diagnosis support device comprising:
a processor (paragraph 86, As shown in FIG. 2, the computing device 200 may include a processor 210); and
a memory connected to or built in the processor (paragraph 86, As shown in FIG. 2, the computing device 200 may include a storage 220), wherein the processor is configured to:
acquire a medical image (paragraph 123, In 541, the processing device 400a (e.g., the acquisition module 412) may obtain a target image of the target object);
extract a plurality of anatomical regions of an organ from the medical image by inputting the medical image into a segmentation model (paragraph 127, In 543, the processing device 400a (e.g., the segmentation module 414) may segment a target region from the target image);
input images of the plurality of anatomical regions to a plurality of feature amount derivation models prepared for each of the plurality of anatomical regions, and output a plurality of feature amounts for each of the plurality of anatomical regions from the feature amount derivation models (paragraph 132, In 545, the processing device 400a (e.g., the determination module 416) may determine a morphological characteristic value of the target region in the target image);
input the plurality of feature amounts which are output for each of the plurality of anatomical regions to a disease opinion derivation model, and output a disease opinion from the disease opinion derivation model (paragraph 142, In 549, the processing device 400a (e.g., the assessment module 418) may assess the condition of the organ or tissue of the target object);
wherein each feature amount derivation model to which an image of a corresponding anatomical region is input and from which a feature amount set is output (paragraph 114, The morphological characteristic determination module 408 may be configured to determine a morphological characteristic value of a sub-region of an organ or tissue. In some embodiments, the morphological characteristic value of a sub-region may be determined using at least one morphometry technique. Exemplary morphometry techniques may include a voxel-based morphometry technique, a tensor-based morphometry technique, a deformation-based morphometry technique, or the like) (paragraph 133, The morphological characteristic value of the target region may include a volume of an organ or tissue, a volume of the target region, a density of the target region, a thickness of the target region, a surface area of the target region, a width of the target region, a deformation (size and/or orientation) of the target region, etc);
input disease-related information related to a disease to the disease opinion derivation model in addition to the plurality of feature amounts (paragraph 139, In some embodiments, the sample objects may have one or more disease labels. In some embodiments, different sample objects may have different disease labels),
derive a first contribution which represents a degree of contribution to output of the opinion for each of the anatomical regions (paragraph 143-144, the processing device 400a may determine rankings of the morphological characteristic value of the target region among the portion of the plurality of morphological characteristic values, and assess the condition of the organ or tissue of the target object based on the rankings); and
present the opinion and a derivation result of the first contribution for each of the anatomical regions (paragraph 224, FIGS. 15 and 16 are schematic diagrams of exemplary medical image processing application interfaces according to some embodiments of the present disclosure) (paragraph 143, the processing device 400a may determine a first ranking of the morphological characteristic value of the target region among the portion of the plurality of morphological characteristic values, and assess the condition of the organ or tissue of the target object based on the first ranking);
derive a second contribution which represents a degree of contribution to output of the opinion for each of the items (paragraph 144, the processing device 400a may determine a second ranking of the morphological characteristic values of the sample regions in the plurality of sample images based on age of the sample object in each of the plurality of sample images when each sample image is acquired); and
present a derivation result of the second contribution for each of the items (paragraph 144, The processing device 400a may assess the condition of the organ or tissue of the target object based on the ranking),
wherein the organ is a brain and the disease is dementia (paragraph 124, Exemplary CNS disorders may include a Alzheimer's Disease (AD), a Idiopathic Parkinson's disease, a Mild Cognitive Impairment (MCI), a Vascular Dementia (VaD), a Cerebral Amyloid Angiopathy (CAA), a Frontotemporal Lobar Degeneration (FTLD), a Dementia with Lewy Bodies (DLB), a Progressive Supranuclear Palsy (PSP), a Multiple System Atrophy (MSA), a Creutzfeldt-Jakob Disease (CJD), a Traumatic Brain Injury, or the like).
While Huo discloses deriving contributions for each of the anatomical regions, Huo does not disclose expressly training each feature derivation model, labeling anatomical regions and deriving the degree of contribution of each of the anatomical regions to the disease opinion.
Pereira discloses wherein each feature amount derivation model is a trained machine learning model (paragraph 62, The classification modules are trained offline prior to diagnosis of an unknown patient using training data from patients with known TBI diagnoses. The training data includes the structural MRI, rs-fMRI series, dMRI, and phenotype data for a group of patients, as well as a doctor's diagnosis of each patient);
extracting a plurality of anatomical regions of an organ from the medical image by attaching a label to each anatomical region by inputting the medical image into a segmentation model (paragraph 43-44, The ROIs are defined by a digital brain atlas that is defined in the template space and uniquely maps which voxels in the rs-fMRI data (transformed to the template space) belong to each of M distinct brain regions. In some embodiments, the Automated Anatomical Labeling (AAL) atlas, which defines 116 brain regions, may be used as the brain atlas);
input disease-related information related to a disease to the disease opinion derivation model in addition to the plurality of feature amounts (page 12-13, claims 1 and 4 include determining a TBI diagnosis for the patient based on at least the network features and the connectivity features using a trained classifier, wherein the determining is further based on genetic features of the patient);
deriving, for each of the anatomical regions, a first contribution which represents a degree of contribution of each of the anatomical regions to output of the opinion that is output from the disease opinion model (paragraph 98, The contribution map in FIG. 4B shows the contribution of each ROI by coloring each voxel with the value for the ROI to which it belong);
derive a second contribution which represents a degree of contribution to output of the opinion for each of the items (paragraph 78, all of the types of classifiers support the calculation of a measure of feature influence. For example, the feature weights in a linear classifier may be used as the measure of feature influence. The feature influence measure is used to rank the features that had more influence on the classifier decision and output this information).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to train each feature derivation model, label anatomical regions and derive the degree of contribution of each of the anatomical regions to the disease opinion. The motivation for doing so would have been to improve the performance of the model and to provide the user with an explanation of what damage contributed to the prediction to be understand the diagnosis.
Huo and Pereira do not disclose expressly inputting one or more items selected from a dementia test score, a spinal fluid test or a blood test.
Kalafut discloses wherein the disease-related information includes one or more items selected from a score of a dementia test, a test result of a spinal fluid test, or a test result of a blood test (paragraph 34, The information related to a state of a patient can include, for example, clinical symptomology data, heart-rate data, blood pressure data and/or other patient data);
Before the effective filing date of the claimed invention, it would have obvious to input blood test information to predict disease. The motivation for doing so would have been to improve the accuracy of disease prediction. Therefore, it would have been obvious to combine Pereira and Kalafut with Huo to obtain the invention as specified in claims 1, 13 and 14.
Referring to claim 2, Huo discloses wherein the processor is configured to:
present the derivation result in descending order of the first contribution (paragraph 182, The morphological characteristic values of the first target region in the target image and the first sample regions in the portion of the plurality of sample images may be ranked according to a ranking rule (e.g., in a descending order or ascending order)).
Referring to claim 4, Huo discloses wherein the disease-related information includes a plurality of items, and the processor is configured to:
derive a second contribution which represents a degree of contribution to output of the opinion for each of the items (paragraph 144, the processing device 400a may determine a second ranking of the morphological characteristic values of the sample regions in the plurality of sample images based on age of the sample object in each of the plurality of sample images when each sample image is acquired); and
present a derivation result of the second contribution for each of the items (paragraph 144, The processing device 400a may assess the condition of the organ or tissue of the target object based on the ranking).
Referring to claim 5, Huo discloses wherein the feature amount derivation model includes at least one of an auto-encoder, a single-task convolutional neural network for class determination, or a multi-task convolutional neural network for class determination (paragraph 153, The initial artificial intelligence model may include an initial deep learning model such as an initial CNN model (e.g., an initial 3D CNN model), an initial deep CNN (DCNN) model, an initial Fully Convolutional Network (FCN) model, an initial Recurrent Neural Network (RNN) model, an initial U-Net model, an initial V-Net model, etc).
Referring to claim 6, Huo discloses the processor is configured to:
input an image of one anatomical region of the anatomical regions to the plurality of feature amount derivation models, and output the feature amounts from each of the plurality of feature amount derivation models (paragraph 133, the processing device 400a may determine the morphological characteristic value of the target region using one or more morphometry techniques. Exemplary morphometry techniques may include a voxel-based morphometry technique, a tensor-based morphometry technique, a deformation-based morphometry technique, or the like, or any combination thereof).
Referring to claim 7, Huo discloses wherein the disease opinion derivation model is configured by any one method of a neural network, a support vector machine, or boosting (paragraph 153, The initial artificial intelligence model may include an initial deep learning model such as an initial CNN model (e.g., an initial 3D CNN model), an initial deep CNN (DCNN) model, an initial Fully Convolutional Network (FCN) model, an initial Recurrent Neural Network (RNN) model, an initial U-Net model, an initial V-Net model, etc).
Referring to claim 8, Huo discloses the processor is configured to:
perform normalization processing of matching the acquired medical image with a reference medical image prior to extraction of the anatomical regions (paragraph 112, Exemplary image segmentation algorithms may include a template matching algorithm).
Referring to claim 10, Huo discloses wherein the plurality of anatomical regions include at least one of a hippocampus or a temporal lobe (paragraph 113, Exemplary brain sub-regions may include the whole brain, the grey matter, the white matter, the amygdala, the putamen, the hippocampus, the globus pallidus, the thalamus, the anterior cingulate cortex, the middle cingulate cortex, the posterior cingulate cortex, the insula, the superior temporal gyrus, the middle temporal gyrus, the temporal pole, etc).
Referring to claim 15, Huo discloses a dementia diagnosis support method causing a computer that includes a processor and a memory connected to or built in the processor to execute a process comprising:
acquiring a medical image in which a brain appears (paragraph 123, In 541, the processing device 400a (e.g., the acquisition module 412) may obtain a target image of the target object);
extracting a plurality of anatomical regions of an organ from the medical image by inputting the medical image into a segmentation model (paragraph 127, In 543, the processing device 400a (e.g., the segmentation module 414) may segment a target region from the target image);
inputting images of the plurality of anatomical regions to a plurality of feature amount derivation models prepared for each of the plurality of anatomical regions, and output a plurality of feature amounts for each of the plurality of anatomical regions from the feature amount derivation models (paragraph 132, In 545, the processing device 400a (e.g., the determination module 416) may determine a morphological characteristic value of the target region in the target image);
wherein each feature amount derivation model to which an image of a corresponding anatomical region is input and from which a feature amount set is output (paragraph 114, The morphological characteristic determination module 408 may be configured to determine a morphological characteristic value of a sub-region of an organ or tissue. In some embodiments, the morphological characteristic value of a sub-region may be determined using at least one morphometry technique. Exemplary morphometry techniques may include a voxel-based morphometry technique, a tensor-based morphometry technique, a deformation-based morphometry technique, or the like) (paragraph 133, The morphological characteristic value of the target region may include a volume of an organ or tissue, a volume of the target region, a density of the target region, a thickness of the target region, a surface area of the target region, a width of the target region, a deformation (size and/or orientation) of the target region, etc);
inputting the plurality of feature amounts which are output for each of the plurality of anatomical regions to a dementia opinion derivation model, and outputting a dementia opinion from the dementia opinion derivation model (paragraph 142, In 549, the processing device 400a (e.g., the assessment module 418) may assess the condition of the organ or tissue of the target object);
inputting disease-related information related to the dementia to the dementia opinion derivation model in addition to the plurality of feature amounts (paragraph 139, In some embodiments, the sample objects may have one or more disease labels. In some embodiments, different sample objects may have different disease labels),
deriving a first contribution which represents a degree of contribution to output of the opinion for each of the anatomical regions (paragraph 143-144, the processing device 400a may determine rankings of the morphological characteristic value of the target region among the portion of the plurality of morphological characteristic values, and assess the condition of the organ or tissue of the target object based on the rankings); and
presenting the opinion and a derivation result of the first contribution for each of the anatomical regions (paragraph 224, FIGS. 15 and 16 are schematic diagrams of exemplary medical image processing application interfaces according to some embodiments of the present disclosure) (paragraph 143, the processing device 400a may determine a first ranking of the morphological characteristic value of the target region among the portion of the plurality of morphological characteristic values, and assess the condition of the organ or tissue of the target object based on the first ranking);
deriving a second contribution which represents a degree of contribution to output of the opinion for each of the items (paragraph 144, the processing device 400a may determine a second ranking of the morphological characteristic values of the sample regions in the plurality of sample images based on age of the sample object in each of the plurality of sample images when each sample image is acquired); and
presenting a derivation result of the second contribution for each of the items (paragraph 144, The processing device 400a may assess the condition of the organ or tissue of the target object based on the ranking),
While Huo discloses deriving contributions for each of the anatomical regions, Huo does not disclose expressly training each feature derivation model, labeling anatomical regions and deriving the degree of contribution of each of the anatomical regions to the disease opinion.
Pereira discloses wherein each feature amount derivation model is a trained machine learning model (paragraph 62, The classification modules are trained offline prior to diagnosis of an unknown patient using training data from patients with known TBI diagnoses. The training data includes the structural MRI, rs-fMRI series, dMRI, and phenotype data for a group of patients, as well as a doctor's diagnosis of each patient);
extracting a plurality of anatomical regions of an organ from the medical image by attaching a label to each anatomical region by inputting the medical image into a segmentation model (paragraph 43-44, The ROIs are defined by a digital brain atlas that is defined in the template space and uniquely maps which voxels in the rs-fMRI data (transformed to the template space) belong to each of M distinct brain regions. In some embodiments, the Automated Anatomical Labeling (AAL) atlas, which defines 116 brain regions, may be used as the brain atlas);
inputting disease-related information related to a dementia to the dementia opinion derivation model in addition to the plurality of feature amounts (page 12-13, claims 1 and 4 include determining a TBI diagnosis for the patient based on at least the network features and the connectivity features using a trained classifier, wherein the determining is further based on genetic features of the patient);
deriving, for each of the anatomical regions, a first contribution which represents a degree of contribution of each of the anatomical regions to output of the opinion that is output from the dementia opinion model (paragraph 98, The contribution map in FIG. 4B shows the contribution of each ROI by coloring each voxel with the value for the ROI to which it belong);
deriving a second contribution which represents a degree of contribution to output of the opinion for each of the items (paragraph 78, all of the types of classifiers support the calculation of a measure of feature influence. For example, the feature weights in a linear classifier may be used as the measure of feature influence. The feature influence measure is used to rank the features that had more influence on the classifier decision and output this information).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to train each feature derivation model, label anatomical regions and derive the degree of contribution of each of the anatomical regions to the disease opinion. The motivation for doing so would have been to improve the performance of the model and to provide the user with an explanation of what damage contributed to the prediction to be understand the diagnosis.
Huo and Pereira do not disclose expressly inputting one or more items selected from a dementia test score, a spinal fluid test or a blood test.
Kalafut discloses wherein the disease-related information includes one or more items selected from a score of a dementia test, a test result of a spinal fluid test, or a test result of a blood test (paragraph 34, The information related to a state of a patient can include, for example, clinical symptomology data, heart-rate data, blood pressure data and/or other patient data);
Before the effective filing date of the claimed invention, it would have obvious to input blood test information to predict disease. The motivation for doing so would have been to improve the accuracy of disease prediction. Therefore, it would have been obvious to combine Pereira and Kalafut with Huo to obtain the invention as specified in claim 15.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER K HUNTSINGER whose telephone number is (571)272-7435. The examiner can normally be reached Monday - Friday 8:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Q Tieu can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER K HUNTSINGER/ Primary Examiner, Art Unit 2682
