DETAILED ACTION
Claims 1-15 are currently pending.
Response to Arguments
Applicant's arguments filed 4/15/26 have been fully considered but they are not persuasive.
The Applicant argues on pages 9 and 10 of the response in essence that: According to the present application, the "feature amount set" is derived by directly inputting an anatomical region image into a trained machine learning model (e.g., a convolutional neural network or an auto-encoder) prepared specifically for that anatomical region. Through convolution and pooling processes, the model outputs a massive number of feature amounts (e.g., several tens to hundreds of thousands) that comprehensively represent complex shape and texture features of that specific anatomical region image, such as the degree of atrophy or a decrease in blood flow metabolism. In contrast, Pereira extracts a single vector time series representing rs-fMRI activity by averaging the time courses within each Region of Interest (ROI) (paragraph [0043]). The features evaluated in Pereira represent the aggregated connectivity or correlation between pairs of ROIs (ROI-ROI connections) (paragraphs [0097] and [0098]). Therefore, Pereira does not disclose or suggest the claimed feature of inputting an "image" of an anatomical region into a specific "trained machine learning model" prepared for that region to output a plurality of comprehensive image- based "feature amounts" for that specific region.
Huo discloses that morphological characteristic determination module 408 determines a morphological characteristic value (i.e. feature amounts) of a sub-region of an organ or tissue (i.e. an inputted image) (paragraph 114). The morphological characteristic value of the target region may include a volume of an organ or tissue, a volume of the target region, a density of the target region, a thickness of the target region, a surface area of the target region, a width of the target region, a deformation (size and/or orientation) of the target region, etc (paragraph 133). While Huo discloses that the segmentation model is a trained machine learning model, Huo does not disclose expressly that the feature amount derivation model is a trained machine learning model. Pereira discloses that the classification modules are trained to extract anatomical, connectivity features, and/or network features (paragraph 62).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-15 are rejected under 35 U.S.C. 103 as being unpatentable over Huo et al. US Publication 2020/0098108 (hereafter “Huo”) and Pereira et al. US Publication 2015/0018664 (hereafter “Pereira”).
Referring to claims 1, 13 and 14, Huo discloses a diagnosis support device comprising:
a processor (paragraph 86, As shown in FIG. 2, the computing device 200 may include a processor 210); and
a memory connected to or built in the processor (paragraph 86, As shown in FIG. 2, the computing device 200 may include a storage 220), wherein the processor is configured to:
acquire a medical image (paragraph 123, In 541, the processing device 400a (e.g., the acquisition module 412) may obtain a target image of the target object);
extract a plurality of anatomical regions of an organ from the medical image by inputting the medical image into a segmentation model (paragraph 127, In 543, the processing device 400a (e.g., the segmentation module 414) may segment a target region from the target image);
input images of the plurality of anatomical regions to a plurality of feature amount derivation models prepared for each of the plurality of anatomical regions, and output a plurality of feature amounts for each of the plurality of anatomical regions from the feature amount derivation models (paragraph 132, In 545, the processing device 400a (e.g., the determination module 416) may determine a morphological characteristic value of the target region in the target image);
input the plurality of feature amounts which are output for each of the plurality of anatomical regions to a disease opinion derivation model, and output a disease opinion from the disease opinion derivation model (paragraph 142, In 549, the processing device 400a (e.g., the assessment module 418) may assess the condition of the organ or tissue of the target object),
wherein the each feature amount derivation model to which an image of a corresponding anatomical region is input and from which a feature amount set is output (paragraph 114, The morphological characteristic determination module 408 may be configured to determine a morphological characteristic value of a sub-region of an organ or tissue. In some embodiments, the morphological characteristic value of a sub-region may be determined using at least one morphometry technique. Exemplary morphometry techniques may include a voxel-based morphometry technique, a tensor-based morphometry technique, a deformation-based morphometry technique, or the like) (paragraph 133, The morphological characteristic value of the target region may include a volume of an organ or tissue, a volume of the target region, a density of the target region, a thickness of the target region, a surface area of the target region, a width of the target region, a deformation (size and/or orientation) of the target region, etc);
derive a first contribution which represents a degree of contribution to output of the opinion for each of the anatomical regions (paragraph 143-144, the processing device 400a may determine rankings of the morphological characteristic value of the target region among the portion of the plurality of morphological characteristic values, and assess the condition of the organ or tissue of the target object based on the rankings); and
present the opinion and a derivation result of the first contribution for each of the anatomical regions (paragraph 224, FIGS. 15 and 16 are schematic diagrams of exemplary medical image processing application interfaces according to some embodiments of the present disclosure) (paragraph 143, the processing device 400a may determine a first ranking of the morphological characteristic value of the target region among the portion of the plurality of morphological characteristic values, and assess the condition of the organ or tissue of the target object based on the first ranking).
While Huo discloses deriving contributions for each of the anatomical regions, Huo does not disclose expressly wherein each feature amount derivation model is a trained, attaching a label to each anatomical region or deriving contributions which represents a degree of contribution of each of the anatomical regions to output of the opinion that is output from the disease opinion mode.
Pereira discloses wherein each feature amount derivation model is a trained machine learning model (paragraph 62, The classification modules are trained offline prior to diagnosis of an unknown patient using training data from patients with known TBI diagnoses. The training data includes the structural MRI, rs-fMRI series, dMRI, and phenotype data for a group of patients, as well as a doctor's diagnosis of each patient);
extracting a plurality of anatomical regions of an organ from the medical image by attaching a label to each anatomical region by inputting the medical image into a segmentation model (paragraph 43-44, The ROIs are defined by a digital brain atlas that is defined in the template space and uniquely maps which voxels in the rs-fMRI data (transformed to the template space) belong to each of M distinct brain regions. In some embodiments, the Automated Anatomical Labeling (AAL) atlas, which defines 116 brain regions, may be used as the brain atlas);
deriving, for each of the anatomical regions, a first contribution which represents a degree of contribution of each of the anatomical regions to output of the opinion that is output from the disease opinion model (paragraph 98, The contribution map in FIG. 4B shows the contribution of each ROI by coloring each voxel with the value for the ROI to which it belong).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to train each feature derivation model, label anatomical regions and derive the degree of contribution of each of the anatomical regions to the disease opinion. The motivation for doing so would have been to improve the performance of the model and to provide the user with an explanation of what damage contributed to the prediction to be understand the diagnosis. Therefore, it would have been obvious to combine Pereira with Huo to obtain the invention as specified in claims 1, 13 and 14.
Referring to claim 2, Huo discloses wherein the processor is configured to:
present the derivation result in descending order of the first contribution (paragraph 182, The morphological characteristic values of the first target region in the target image and the first sample regions in the portion of the plurality of sample images may be ranked according to a ranking rule (e.g., in a descending order or ascending order)).
Referring to claim 3, Huo discloses wherein the processor is configured to:
input disease-related information related to the disease to the disease opinion derivation model in addition to the plurality of feature amounts (paragraph 139, In some embodiments, the sample objects may have one or more disease labels. In some embodiments, different sample objects may have different disease labels).
Referring to claim 4, Huo discloses wherein the disease-related information includes a plurality of items, and the processor is configured to:
derive a second contribution which represents a degree of contribution to output of the opinion for each of the items (paragraph 144, the processing device 400a may determine a second ranking of the morphological characteristic values of the sample regions in the plurality of sample images based on age of the sample object in each of the plurality of sample images when each sample image is acquired); and
present a derivation result of the second contribution for each of the items (paragraph 144, The processing device 400a may assess the condition of the organ or tissue of the target object based on the ranking).
Referring to claim 5, Huo discloses wherein the feature amount derivation model includes at least one of an auto-encoder, a single-task convolutional neural network for class determination, or a multi-task convolutional neural network for class determination (paragraph 153, The initial artificial intelligence model may include an initial deep learning model such as an initial CNN model (e.g., an initial 3D CNN model), an initial deep CNN (DCNN) model, an initial Fully Convolutional Network (FCN) model, an initial Recurrent Neural Network (RNN) model, an initial U-Net model, an initial V-Net model, etc).
Referring to claim 6, Huo discloses the processor is configured to:
input an image of one anatomical region of the anatomical regions to the plurality of different feature amount derivation models, and output the feature amounts from each of the plurality of feature amount derivation models (paragraph 133, the processing device 400a may determine the morphological characteristic value of the target region using one or more morphometry techniques. Exemplary morphometry techniques may include a voxel-based morphometry technique, a tensor-based morphometry technique, a deformation-based morphometry technique, or the like, or any combination thereof).
Referring to claim 7, Huo discloses wherein the disease opinion derivation model is configured by any one method of a neural network, a support vector machine, or boosting (paragraph 153, The initial artificial intelligence model may include an initial deep learning model such as an initial CNN model (e.g., an initial 3D CNN model), an initial deep CNN (DCNN) model, an initial Fully Convolutional Network (FCN) model, an initial Recurrent Neural Network (RNN) model, an initial U-Net model, an initial V-Net model, etc).
Referring to claim 8, Huo discloses the processor is configured to:
perform normalization processing of matching the acquired medical image with a reference medical image prior to extraction of the anatomical regions (paragraph 112, Exemplary image segmentation algorithms may include a template matching algorithm).
Referring to claim 9, Huo discloses wherein the organ is a brain and the disease is dementia (paragraph 124, Exemplary CNS disorders may include a Alzheimer's Disease (AD), a Idiopathic Parkinson's disease, a Mild Cognitive Impairment (MCI), a Vascular Dementia (VaD), a Cerebral Amyloid Angiopathy (CAA), a Frontotemporal Lobar Degeneration (FTLD), a Dementia with Lewy Bodies (DLB), a Progressive Supranuclear Palsy (PSP), a Multiple System Atrophy (MSA), a Creutzfeldt-Jakob Disease (CJD), a Traumatic Brain Injury, or the like).
Referring to claim 10, Huo discloses wherein the plurality of anatomical regions include at least one of a hippocampus or a temporal lobe (paragraph 113, Exemplary brain sub-regions may include the whole brain, the grey matter, the white matter, the amygdala, the putamen, the hippocampus, the globus pallidus, the thalamus, the anterior cingulate cortex, the middle cingulate cortex, the posterior cingulate cortex, the insula, the superior temporal gyrus, the middle temporal gyrus, the temporal pole, etc).
Referring to claim 11, Huo discloses wherein the processor is configured to:
input disease-related information related to the disease to the disease opinion derivation model in addition to the plurality of feature amounts,
wherein the disease-related information includes at least one of a volume of the anatomical region, a score of a dementia test, a test result of a genetic test, a test result of a spinal fluid test, or a test result of a blood test (paragraph 133, The morphological characteristic value of the target region may include a volume of an organ or tissue, a volume of the target region, a density of the target region, a thickness of the target region, a surface area of the target region, a width of the target region, a deformation (size and/or orientation) of the target region, etc).
Referring to claim 12, Huo discloses wherein the disease-related information includes a plurality of items, and
the processor is configured to:
input disease-related information related to the disease to the disease opinion derivation model in addition to the plurality of feature amounts (paragraph 133, The morphological characteristic value of the target region may include a volume of an organ or tissue, a volume of the target region, a density of the target region, a thickness of the target region, a surface area of the target region, a width of the target region, a deformation (size and/or orientation) of the target region, etc),
derive a second contribution which represents a degree of contribution to output of the opinion for each of the items (paragraph 144, the processing device 400a may determine a second ranking of the morphological characteristic values of the sample regions in the plurality of sample images based on age of the sample object in each of the plurality of sample images when each sample image is acquired); and
present a derivation result of the second contribution for each of the items (paragraph 144, The processing device 400a may assess the condition of the organ or tissue of the target object based on the ranking),
wherein the disease-related information includes at least one of a volume of the anatomical region, a score of a dementia test, a test result of a genetic test, a test result of a spinal fluid test, or a test result of a blood test (paragraph 133, The morphological characteristic value of the target region may include a volume of an organ or tissue, a volume of the target region, a density of the target region, a thickness of the target region, a surface area of the target region, a width of the target region, a deformation (size and/or orientation) of the target region, etc).
Referring to claim 15, Huo discloses a dementia diagnosis support method causing a computer that includes a processor and a memory connected to or built in the processor to execute a process comprising:
acquire a medical image in which a brain appears (paragraph 123, In 541, the processing device 400a (e.g., the acquisition module 412) may obtain a target image of the target object);
extracting a plurality of anatomical regions of an organ from the medical image by inputting the medical image into a segmentation model (paragraph 127, In 543, the processing device 400a (e.g., the segmentation module 414) may segment a target region from the target image);
inputting images of the plurality of anatomical regions to a plurality of feature amount derivation models prepared for each of the plurality of anatomical regions, and output a plurality of feature amounts for each of the plurality of anatomical regions from the feature amount derivation models (paragraph 132, In 545, the processing device 400a (e.g., the determination module 416) may determine a morphological characteristic value of the target region in the target image);
wherein the each feature amount derivation model to which an image of a corresponding anatomical region is input and from which a feature amount set is output (paragraph 114, The morphological characteristic determination module 408 may be configured to determine a morphological characteristic value of a sub-region of an organ or tissue. In some embodiments, the morphological characteristic value of a sub-region may be determined using at least one morphometry technique. Exemplary morphometry techniques may include a voxel-based morphometry technique, a tensor-based morphometry technique, a deformation-based morphometry technique, or the like) (paragraph 133, The morphological characteristic value of the target region may include a volume of an organ or tissue, a volume of the target region, a density of the target region, a thickness of the target region, a surface area of the target region, a width of the target region, a deformation (size and/or orientation) of the target region, etc);
inputting the plurality of feature amounts which are output for each of the plurality of anatomical regions to a disease opinion derivation model, and output a disease opinion from the disease opinion derivation model (paragraph 142, In 549, the processing device 400a (e.g., the assessment module 418) may assess the condition of the organ or tissue of the target object);
deriving a first contribution which represents a degree of contribution to output of the opinion for each of the anatomical regions (paragraph 143-144, the processing device 400a may determine rankings of the morphological characteristic value of the target region among the portion of the plurality of morphological characteristic values, and assess the condition of the organ or tissue of the target object based on the rankings); and
presenting the opinion and a derivation result of the first contribution for each of the anatomical regions (paragraph 224, FIGS. 15 and 16 are schematic diagrams of exemplary medical image processing application interfaces according to some embodiments of the present disclosure) (paragraph 143, the processing device 400a may determine a first ranking of the morphological characteristic value of the target region among the portion of the plurality of morphological characteristic values, and assess the condition of the organ or tissue of the target object based on the first ranking).
While Huo discloses deriving contributions for each of the anatomical regions, Huo does not disclose expressly wherein each feature amount derivation model is a trained, attaching a label to each anatomical region or deriving contributions which represents a degree of contribution of each of the anatomical regions to output of the opinion that is output from the disease opinion mode.
Pereira discloses wherein each feature amount derivation model is a trained machine learning model (paragraph 62, The classification modules are trained offline prior to diagnosis of an unknown patient using training data from patients with known TBI diagnoses. The training data includes the structural MRI, rs-fMRI series, dMRI, and phenotype data for a group of patients, as well as a doctor's diagnosis of each patient);
extracting a plurality of anatomical regions of an organ from the medical image by attaching a label to each anatomical region by inputting the medical image into a segmentation model (paragraph 43-44, The ROIs are defined by a digital brain atlas that is defined in the template space and uniquely maps which voxels in the rs-fMRI data (transformed to the template space) belong to each of M distinct brain regions. In some embodiments, the Automated Anatomical Labeling (AAL) atlas, which defines 116 brain regions, may be used as the brain atlas);
deriving, for each of the anatomical regions, a first contribution which represents a degree of contribution of each of the anatomical regions to output of the opinion that is output from the disease opinion model (paragraph 98, The contribution map in FIG. 4B shows the contribution of each ROI by coloring each voxel with the value for the ROI to which it belong).
Before the effective filing date of the claimed invention, it would have obvious to a person of ordinary skill in the art to train each feature derivation model, label anatomical regions and derive the degree of contribution of each of the anatomical regions to the disease opinion. The motivation for doing so would have been to provide the user with an explanation of what damage contributed to the prediction to be understand the diagnosis. Therefore, it would have been obvious to combine Pereira with Huo to obtain the invention as specified in claim 15.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER K HUNTSINGER whose telephone number is (571)272-7435. The examiner can normally be reached Monday - Friday 8:30 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Q Tieu can be reached at 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER K HUNTSINGER/Primary Examiner, Art Unit 2682
Prosecution Insights