DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-4, 6-8, and 10-18 are pending. Claims 5 and 9 are canceled.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 4/19/24 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
The abstract of the disclosure is objected to because in lines 1-2, “identified microbleeds voxels” should read –identified microbleed voxels– and in line 2, “removing identified microbleed voxels” should read –removing the identified microbleed voxels–. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
The disclosure is objected to because of the following informalities:
In Para. 0011, line 1, “microbleeds segmentation” should read –microbleed segmentation–.
In Para. 0041, line 1, the “7” in “Fig. 7” should be bolded for consistency.
In Para. 0130, line 4, the line/dash after “T2” should be removed.
Appropriate correction is required.
The disclosure is objected to because it contains embedded hyperlinks and/or other forms of browser-executable code. Applicant is required to delete the embedded hyperlinks and/or other forms of browser-executable code; references to websites should be limited to the top-level domain name without any prefix such as http:// or other browser-executable code. See MPEP § 608.01.
Claim Objections
Claim 1 is objected to because of the following informalities: In lines 8-9, “removing identified microbleed voxels” should read –removing the identified microbleed voxels–. Appropriate correction is required.
Claim 6 is objected to because of the following informalities: In line 3, “images” should read –images. – A period is missing at the end of the claim. Appropriate correction is required.
Claim 18 is objected to because of the following informalities:
In line 6, “brain image” should read –the brain image.
In line 10, “removing identified microbleed voxels” should read –removing the identified microbleed voxels–.
Appropriate correction is required.
Applicant is advised that should claim 1 be found allowable, claims 2, 4, and 10 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Applicant is advised that should claim 3 be found allowable, claim 12 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Applicant is advised that should claim 6 be found allowable, claim 13 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Applicant is advised that should claim 7 be found allowable, claim 14 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Applicant is advised that should claim 8 be found allowable, claim 15 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-4, 6-8, and 10-18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "microbleed voxels" in lines 5 and 13. It is unclear and indefinite if these microbleed voxels are the same as those previously recited in the claim (i.e., in the providing limitation).
Claim 2 recites the limitations "false positives", “identified microbleed voxels”, “comparison”, “image characteristic”, and “one or more surrounding non-microbleed voxels.” It is unclear and indefinite if these limitations are the same as that in claim 1. The examiner interprets that they are the same since the limitations are a duplicate of claim 1.
Claim 3 depends on claim 2 and is therefore also rejected under 112(b).
Claim 4 recites the limitations "relevant image characteristic”, “cerebral spinal fluid”, “gray matter/white matter”, “first training set of brain images”, “microbleed voxels”, “other brain tissue voxels”, “second set of microbleed brain images”, and “one or more experts”. It is unclear and indefinite if these limitations are the same as that in claim 1. The examiner interprets that they are the same since the limitations are a duplicate of claim 1.
Claims 6-8 depends on claim 4 and are therefore also rejected under 112(b).
Claim 10 recites the limitations "relevant image characteristic”, “cerebral spinal fluid”, “gray mater/white matter”, “first training set of brain images”, “microbleed voxels”, “other brain tissue voxels”, “second set of microbleed brain images”, and “one or more experts”. It is unclear and indefinite if these limitations are the same as that in claim 1. The examiner interprets that they are the same since the limitations are a duplicate of claim 1.
Claim 11 recites the limitation “said image characteristic” in lines 1-2. It is unclear and indefinite which image characteristic is being referred to (i.e., the “image characteristic for distinguishing cerebral spinal fluid from gray matter/white matter” or the “image characteristic of said identified microbleed voxels and of one or more surrounding non-microbleed voxels”).
Claims 12-15 depend on claim 10 and are therefore also rejected under 112(b).
Claim 16 recites the limitation "therapy to a based on” in line 6. It is unclear and indefinite what the therapy is being administered to, as there is a word missing between “a” and “based on”. Therefore, the scope of the claim is indefinite.
Claim 17 recites the limitation "said modifying" in line 2. There is insufficient antecedent basis for this limitation in the claim. Only “amyloid-modifying therapy”, not a modifying step, is previously recited in the claim. Claim 17 also recites the limitation “identified microbleed voxels” in lines 2-3. It is unclear and indefinite which identified microbleed voxels are being referred to (i.e., the identified microbleed voxels related to amyloid-related microbleeds in claim 16 or the identified microbleed voxels in claim 1).
Claim 18 recites the limitation "microbleed voxels" in lines 7 and 15-16. It is unclear and indefinite if these microbleed voxels are the same as those previously recited in the claim (i.e., in the providing limitation).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6-7, 10-14, and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over “Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks” by Dou et al. (hereinafter “Dou”) in view of “Automated detection of cerebral microbleeds on T2*-weighted MRI” by Chesebro et al. (hereinafter “Chesebro”) and further in view of “Automated detection of cerebral microbleeds in patients with traumatic brain injury” by Heuvel et al. (hereinafter “Heuvel”).
Regarding claim 1, Dou teaches, A method of processing a brain image to determine presence of microbleeds, the method comprising (Abstract: “Cerebral microbleeds (CMBs) are small haemorrhages nearby blood vessels. They have been recognized as important diagnostic biomarkers for many cerebrovascular diseases and cognitive dysfunctions… In this paper, we propose a novel automatic method to detect CMBs from magnetic resonance (MR) images by exploiting the 3D convolutional neural network (CNN)”):
providing a classifier trained to recognize microbleed voxels (Pg. 1184, II. Methodology: “In the screening stage, the 3D FCN model takes a whole volumetric data as input and directly outputs a 3D score volume. Each value on the 3D score volume represents the probability of CMB at a corresponding voxel of the input volume”; Fig. 2);
receiving a brain image of a subject (Fig. 2: testing volume; Abstract: “In this paper, we propose a novel automatic method to detect CMBs from magnetic resonance (MR) images by exploiting the 3D convolutional neural network (CNN)”);
and identifying microbleed voxels in said brain image using said classifier (Pg. 1184, II. Methodology: “In the screening stage, the 3D FCN model takes a whole volumetric data as input and directly outputs a 3D score volume. Each value on the 3D score volume represents the probability of CMB at a corresponding voxel of the input volume”; Fig. 2: 3D FCN Screening);
wherein false positives in said identified microbleed voxels are reduced by at least one of: removing identified microbleed voxels based on a comparison between an image (Pg. 1184, II. Methodology: “Subsequently, in the discrimination stage, we further remove false positive candidates by applying a 3D CNN discrimination model to distinguish true CMBs from challenging mimics with high-level feature representations”; Fig. 2: 3D CNN Discrimination; Pg. 1188, Discrimination Stage: “We first found that a number of false positives were produced in the first stage with a training block size of 16
×
16
×
16. By enlarging the block size, richer contextual information within larger surrounding neighborhood can provide additional clues to better distinguish CMBs from their mimics”);
Dou does not expressly disclose the following limitations: removing identified microbleed voxels based on a comparison between an image characteristic; and said classifier being first trained on a relevant image characteristic for distinguishing cerebral spinal fluid from gray matter/white matter using a first training set of brain images and being further trained to distinguish microbleed voxels from other brain tissue voxels using a second set of microbleed brain images segmented by one or more experts.
However, Chesebro teaches, removing identified microbleed voxels based on a comparison between an image characteristic (Pg. 3, Detection of potential microbleed regions of interest: “This definition was used because a neighborhood of this size both ensures the entire microbleed artifact will be captured for analysis in the next stage and also standardizes the selected ROIs, making geometric features more comparable”; Pg. 4, 3D geometric filtering: “To assist in the removal of false positive locations, we used the geometric information contained in each ROI identified in the previous step. We selected a priori four characteristics of the ROI as having the potential to differentiate between true and false positive locations: the 3D image entropy of the ROI, the 2D image entropy of the maximum intensity projection of the ROI, and the volume and compactness of the central blob in each ROI as identified via Frangi filtering. In an image, each pixel i has a probability pi of being a given intensity, measured as the fraction of all pixels in the image at that intensity…True and false positive ROI entropies are illustrated in Fig. 4”);
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine removing identified microbleed voxels based on a comparison between an image characteristic as taught by Chesebro with the method of Dou in order to differentiate between true and false positive locations (Chesebro, Pg. 4). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately.
The combination of Dou and Chesebro does not expressly disclose the following limitation: and said classifier being first trained on a relevant image characteristic for distinguishing cerebral spinal fluid from gray matter/white matter using a first training set of brain images and being further trained to distinguish microbleed voxels from other brain tissue voxels using a second set of microbleed brain images segmented by one or more experts.
However, Heuvel teaches, and said classifier being first trained on a relevant image characteristic for distinguishing cerebral spinal fluid from gray matter/white matter using a first training set of brain images and being further trained to distinguish microbleed voxels from other brain tissue voxels using a second set of microbleed brain images segmented by one or more experts (Fig. 3: annotations and brain mask are input into the voxel classifier; Pg. 243, 2.1.2. Annotations: “only one trained expert manually annotated the CMBs in all 33 patients…and less hypointense lesions”; Pg. 243, 2.2.1. Brain mask: “brain mask defines which voxels belong to the brain and which voxels belong to the skull and air surrounding the brain. The brain mask was made in three steps. Firstly, the gray matter, white matter, and spinal fluid were segmented into three probability maps using SPM12b…as the final SWI brain mask”).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine a classifier being trained on an image characteristic for distinguishing cerebral spinal fluid from gray matter/white matter and being trained to distinguish microbleed voxels from other brain tissue voxels using microbleed brain images segmented by an expert as taught by Heuvel with the combined method of Dou and Chesebro in order to accurately detect CMBs (Heuvel, Abstract). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 1.
Regarding claim 2, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 1.
The combination of Dou, Chesebro, and Heuvel further teaches, The method of claim 1 (see claim 1 above), wherein false positives in said identified microbleed voxels are reduced by removing identified microbleed voxels based on a comparison between an image characteristic of said identified microbleed voxels and of one or more surrounding non-microbleed voxels (Dou, Pg. 1184, II. Methodology: “Subsequently, in the discrimination stage, we further remove false positive candidates by applying a 3D CNN discrimination model to distinguish true CMBs from challenging mimics with high-level feature representations”; Dou, Fig. 2: 3D CNN Discrimination; Dou, Pg. 1188, Discrimination Stage: “We first found that a number of false positives were produced in the first stage with a training block size of 16
×
16
×
16. By enlarging the block size, richer contextual information within larger surrounding neighborhood can provide additional clues to better distinguish CMBs from their mimics”; Chesebro, Pg. 3, Detection of potential microbleed regions of interest: “This definition was used because a neighborhood of this size both ensures the entire microbleed artifact will be captured for analysis in the next stage and also standardizes the selected ROIs, making geometric features more comparable”; Chesebro, Pg. 4, 3D geometric filtering: “To assist in the removal of false positive locations, we used the geometric information contained in each ROI identified in the previous step. We selected a priori four characteristics of the ROI as having the potential to differentiate between true and false positive locations: the 3D image entropy of the ROI, the 2D image entropy of the maximum intensity projection of the ROI, and the volume and compactness of the central blob in each ROI as identified via Frangi filtering. In an image, each pixel i has a probability pi of being a given intensity, measured as the fraction of all pixels in the image at that intensity…True and false positive ROI entropies are illustrated in Fig. 4”).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 1 apply to claim 2 and are incorporated herein by reference. Thus, the method recited in claim 2 is met by Dou, Chesebro, and Heuvel.
Regarding claim 3, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 2.
The combination of Dou, Chesebro, and Heuvel further teaches, The method of claim 2 (see claim 2 above), wherein said brain image is preprocessed for intensity non-uniformity correction and/or linear intensity standardization (Heuvel, Fig. 3: bias field correction; Heuvel, Pg. 243, 2.2.3. Bias field correction: “The performance of the automated detection system is degraded by inhomogeneities caused by the bias field. For this reason the T1 scan was bias field corrected, using FSL FAST…to correct the bias field in these scans; Note: bias field correction is an intensity non-uniformity correction).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 2 apply to claim 3 and are incorporated herein by reference. Thus, the method recited in claim 3 is met by Dou, Chesebro, and Heuvel.
Regarding claim 4, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 2.
The combination of Dou, Chesebro, and Heuvel further teaches, The method of claim 2 (see claim 2 above), wherein said providing a classifier comprises: training said classifier on a relevant image characteristic for distinguishing cerebral spinal fluid from gray matter/white matter using a first training set of brain images (Heuvel, Fig. 3: brain mask is input into the voxel classifier; Heuvel, Pg. 243, 2.2.1. Brain mask: “brain mask defines which voxels belong to the brain and which voxels belong to the skull and air surrounding the brain. The brain mask was made in three steps. Firstly, the gray matter, white matter, and spinal fluid were segmented into three probability maps using SPM12b…as the final SWI brain mask”);
further training said classifier to distinguish microbleed voxels from other brain tissue voxels using a second set of microbleed brain images segmented by one or more experts (Heuvel, Fig. 3: annotations are input into the voxel classifier; Heuvel, Pg. 243, 2.1.2. Annotations: “only one trained expert manually annotated the CMBs in all 33 patients… and less hypointense lesions”).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 2 apply to claim 4 and are incorporated herein by reference. Thus, the method recited in claim 4 is met by Dou, Chesebro, and Heuvel.
Regarding claim 6, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 4.
The combination of Dou, Chesebro, and Heuvel further teaches, The method of claim 4 (see claim 4 above), wherein said training said classifier on a relevant image characteristic using a first training set of brain images comprises using patches of voxels taken from training images (Heuvel, Fig. 3: brain mask is input into the voxel classifier; Heuvel, Pg. 243, 2.2.1. Brain mask: “brain mask defines which voxels belong to the brain and which voxels belong to the skull and air surrounding the brain. The brain mask was made in three steps. Firstly, the gray matter, white matter, and spinal fluid were segmented into three probability maps using SPM12b…as the final SWI brain mask”; Heuvel, Pg. 243, 2.3.1. Initial candidate detection: “Not all the voxels in the brain mask were used to train the classifier… Only local minima of the SWI scan (in a 3
×
3
×
3 voxel neighborhood), that have an intensity below the mean intensity of the voxels inside the brain are selected for training”; Note: the Examiner interprets the 3
×
3
×
3 voxel neighborhood as a voxel patch).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 4 apply to claim 6 and are incorporated herein by reference. Thus, the method recited in claim 6 is met by Dou, Chesebro, and Heuvel.
Regarding claim 7, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 4.
The combination of Dou, Chesebro, and Heuvel further teaches, The method of claim 4 (see claim 4 above), wherein said further training comprises using patches of voxels from said second set of microbleed brain images in different rotational positions (Heuvel: As shown in Figs. 1 and 2, scans are taken from different view/orientations, such as axial, sagittal, and coronal views; Heuvel: Pg. 246: “For every detected location an overview of the entire brain in axial direction was shown to determine its location in the brain. A zoomed-in version of the location was visible in axial, coronal, and sagittal direction, to be able to analyze the shape of the detected object in 3D”; Heuvel, Fig. 3: annotations are input into the voxel classifier; Pg. 243, 2.1.2. Annotations: “Manually annotating CMBs in TBI patients is a very time consuming task. Therefore, only one trained expert manually annotated the CMBs in all 33 patients…and less hypointense lesions”; Heuvel, Pg. 243, 2.1.1. Patient data: for every TBI patient and healthy subject, there was a SWI scan with a voxel size of 0.98
×
0.98
×
1 mm3 and a T1 MP-RAGE scan with a voxel size of 1 mm isotropic; Note: the Examiner interprets the voxel size as a voxel patch and the different brain views/orientations as rotational positions).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 4 apply to claim 7 and are incorporated herein by reference. Thus, the method recited in claim 7 is met by Dou, Chesebro, and Heuvel.
Regarding claim 10, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 1.
The combination of Dou, Chesebro, and Heuvel further teaches, The method of claim 1 (see claim 1 above), wherein said classifier being first trained on a relevant image characteristic for distinguishing cerebral spinal fluid from gray matter/white matter using a first training set of brain images and being further trained to distinguish microbleed voxels from other brain tissue voxels using a second set of microbleed brain images segmented by one or more experts (Heuvel, Fig. 3: annotations and brain mask are input into the voxel classifier; Heuvel, Pg. 243, 2.1.2. Annotations: “only one trained expert manually annotated the CMBs in all 33 patients…and less hypointense lesions”; Heuvel, Pg. 243, 2.2.1. Brain mask: “brain mask defines which voxels belong to the brain and which voxels belong to the skull and air surrounding the brain. The brain mask was made in three steps. Firstly, the gray matter, white matter, and spinal fluid were segmented into three probability maps using SPM12b…as the final SWI brain mask”).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 1 apply to claim 10 and are incorporated herein by reference. Thus, the method recited in claim 10 is met by Dou, Chesebro, and Heuvel.
Regarding claim 11, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 10.
The combination of Dou, Chesebro, and Heuvel further teaches, The method of claim 10 (see claim 10 above), wherein said image characteristic is an intensity value (Chesebro, Pg. 3, Detection of potential microbleed regions of interest: “This definition was used because a neighborhood of this size both ensures the entire microbleed artifact will be captured for analysis in the next stage and also standardizes the selected ROIs, making geometric features more comparable”; Chesebro, Pg. 4, 3D geometric filtering: “To assist in the removal of false positive locations, we used the geometric information contained in each ROI identified in the previous step. We selected a priori four characteristics of the ROI as having the potential to differentiate between true and false positive locations: the 3D image entropy of the ROI, the 2D image entropy of the maximum intensity projection of the ROI, and the volume and compactness of the central blob in each ROI as identified via Frangi filtering. In an image, each pixel i has a probability pi of being a given intensity, measured as the fraction of all pixels in the image at that intensity…True and false positive ROI entropies are illustrated in Fig. 4”).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 10 apply to claim 11 and are incorporated herein by reference. Thus, the method recited in claim 11 is met by Dou, Chesebro, and Heuvel.
Regarding claim 12, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 10.
The combination of Dou, Chesebro, and Heuvel further teaches, The method of 10 (see claim 10 above), wherein said brain image is preprocessed for intensity non-uniformity correction and/or linear intensity standardization (Heuvel, Fig. 3: bias field correction; Heuvel, Pg. 243, 2.2.3. Bias field correction: “The performance of the automated detection system is degraded by inhomogeneities caused by the bias field. For this reason the T1 scan was bias field corrected, using FSL FAST…to correct the bias field in these scans; Note: bias field correction is an intensity non-uniformity correction).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 10 apply to claim 12 and are incorporated herein by reference. Thus, the method recited in claim 12 is met by Dou, Chesebro, and Heuvel.
Regarding claim 13, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 10.
The combination of Dou, Chesebro, and Heuvel further teaches, The method of claim 10 (see claim 10 above), wherein said training said classifier on a relevant image characteristic using a first training set of brain images comprises using patches of voxels taken from training images (Heuvel, Fig. 3: brain mask is input into the voxel classifier; Heuvel, Pg. 243, 2.2.1. Brain mask: “brain mask defines which voxels belong to the brain and which voxels belong to the skull and air surrounding the brain. The brain mask was made in three steps. Firstly, the gray matter, white matter, and spinal fluid were segmented into three probability maps using SPM12b…as the final SWI brain mask”; Heuvel, Pg. 243, 2.3.1. Initial candidate detection: “Not all the voxels in the brain mask were used to train the classifier… Only local minima of the SWI scan (in a 3
×
3
×
3 voxel neighborhood), that have an intensity below the mean intensity of the voxels inside the brain are selected for training”; Note: the Examiner interprets the 3
×
3
×
3 voxel neighborhood as a voxel patch).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 10 apply to claim 13 and are incorporated herein by reference. Thus, the method recited in claim 13 is met by Dou, Chesebro, and Heuvel.
Regarding claim 14, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 13.
The combination of Dou, Chesebro, and Heuvel further teaches, The method of claim 13 (see claim 13 above), wherein said further training comprises using patches of voxels from said second set of microbleed brain images in different rotational positions (Heuvel: As shown in Figs. 1 and 2, scans are taken from different view/orientations, such as axial, sagittal, and coronal views; Heuvel: Pg. 246: “For every detected location an overview of the entire brain in axial direction was shown to determine its location in the brain. A zoomed-in version of the location was visible in axial, coronal, and sagittal direction, to be able to analyze the shape of the detected object in 3D”; Heuvel, Fig. 3: annotations are input into the voxel classifier; Pg. 243, 2.1.2. Annotations: “Manually annotating CMBs in TBI patients is a very time consuming task. Therefore, only one trained expert manually annotated the CMBs in all 33 patients…and less hypointense lesions”; Heuvel, Pg. 243, 2.1.1. Patient data: for every TBI patient and healthy subject, there was a SWI scan with a voxel size of 0.98
×
0.98
×
1 mm3 and a T1 MP-RAGE scan with a voxel size of 1 mm isotropic; Note: the Examiner interprets the voxel size as a voxel patch and the different brain views/orientations as rotational positions).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 13 apply to claim 14 and are incorporated herein by reference. Thus, the method recited in claim 14 is met by Dou, Chesebro, and Heuvel.
Regarding claim 16, the combination of the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 1.
The combination of Dou, Chesebro, and Heuvel further teaches, A method of treating a patient using amyloid-modifying therapy, the method comprising (Dou, Pg. 1182, I. Introduction: “The existence of CMBs and their distribution patterns have been recognized as important diagnostic biomarkers of cerebrovascular diseases. For example, the lobar distribution of CMBs suggests probable cerebral amyloid angiopathy…reliable detection of the presence and number of CMBs is crucial for cerebral diagnosis and may guide physicians in determining which drugs to use for necessary treatment, such as stroke prevention”; Dou, Pg. 1193: “The proposed automatic CMB detection framework has great significance in clinical practice. The CMB distribution patterns have been proven to be associated with many cerebrovascular diseases and cognitive dysfunction. For example, the lobar distribution of CMBs suggests probable cerebral amyloid angiopathy…patients with lobar CMBs have an increased risk for stroke and stroke-related mortality, indicating that these patients should be treated with the utmost care”; Chesebro, Pg. 1: “Lobar distributions of cerebral microbleeds are considered markers of cerebral amyloid angiopathy, and are a prominent feature of Alzheimer’s disease (AD). In addition to signaling vascular forms of amyloid pathology, particularly in AD, microbleeds have emerged as a pernicious side effect of antiamyloid treatments, so-called amyloid related imaging abnormalities related to hemosiderin deposits (ARIAH), a necessary and important consideration in the enrollment of participants into AD therapeutic trials. Although microbleeds can be present asymptomatically, early detection can be crucial in estimating risk for later cerebrovascular disease and cognitive decline”):
a. obtaining a brain image of said patient (Dou, Fig. 2: testing volume; Dou, Abstract: “In this paper, we propose a novel automatic method to detect CMBs from magnetic resonance (MR) images by exploiting the 3D convolutional neural network (CNN)”);
b. processing said brain image of said patient using the method of claim 1 (Dou, Abstract: “Cerebral microbleeds (CMBs) are small haemorrhages nearby blood vessels. They have been recognized as important diagnostic biomarkers for many cerebrovascular diseases and cognitive dysfunctions… In this paper, we propose a novel automatic method to detect CMBs from magnetic resonance (MR) images by exploiting the 3D convolutional neural network (CNN)”; see claim 1 above);
and c. administering said therapy to a based on said identifying microbleed voxels related to amyloid-related microbleeds (Dou, Pg. 1184, II. Methodology: “In the screening stage, the 3D FCN model takes a whole volumetric data as input and directly outputs a 3D score volume. Each value on the 3D score volume represents the probability of CMB at a corresponding voxel of the input volume”; Dou, Fig. 2: 3D FCN Screening; Dou, Pg. 1182, I. Introduction: “The existence of CMBs and their distribution patterns have been recognized as important diagnostic biomarkers of cerebrovascular diseases. For example, the lobar distribution of CMBs suggests probable cerebral amyloid angiopathy…reliable detection of the presence and number of CMBs is crucial for cerebral diagnosis and may guide physicians in determining which drugs to use for necessary treatment, such as stroke prevention”; Dou, Pg. 1193: “The proposed automatic CMB detection framework has great significance in clinical practice. The CMB distribution patterns have been proven to be associated with many cerebrovascular diseases and cognitive dysfunction. For example, the lobar distribution of CMBs suggests probable cerebral amyloid angiopathy…patients with lobar CMBs have an increased risk for stroke and stroke-related mortality, indicating that these patients should be treated with the utmost care”; Chesebro, Pg. 1: “Lobar distributions of cerebral microbleeds are considered markers of cerebral amyloid angiopathy, and are a prominent feature of Alzheimer’s disease (AD). In addition to signaling vascular forms of amyloid pathology, particularly in AD, microbleeds have emerged as a pernicious side effect of antiamyloid treatments, so-called amyloid related imaging abnormalities related to hemosiderin deposits (ARIAH), a necessary and important consideration in the enrollment of participants into AD therapeutic trials. Although microbleeds can be present asymptomatically, early detection can be crucial in estimating risk for later cerebrovascular disease and cognitive decline”).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 1 apply to claim 16 and are incorporated herein by reference. Thus, the method recited in claim 16 is met by Dou, Chesebro, and Heuvel.
Regarding claim 17, the combination of the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 16.
The combination of Dou, Chesebro, and Heuvel further teaches, The method of claim 16 (see claim 16 above), wherein steps (a) through (c) are repeated over time and said modifying comprises comparing changes in identified microbleed voxels over time (Dou, Pg. 1188, A. Dataset and Preprocessing: SWI images are acquired at a repetition time of 17 ms; Dou, Pg. 1188, A. Dataset and Preprocessing: data of multiple subject were used; Chesebro, Pg. 2: MRI images were obtained with different repetition times; Chesebro: Pg. 7: microbleed positive participants had a follow-up MRI scan and formed the longitudinal sample, and there were longitudinal scans in which it was confirmed that the locations detected at multiple timepoints were microbleeds”; Chesebro, Pg. 10: “We also demonstrated, in a small subset of participants, that the automated algorithm exhibits higher sensitivity in longitudinal identification of potential microbleed locations than visual ratings. Longitudinal reliability in identifying persistent artifacts is critical to future assessments of microbleeds, as the longitudinal stability of microbleed artifacts remains unknown, and accurate identification on multiple scans would help to elucidate the degree to which they change in appearance over time”; see claim 16 above for mapping of steps (a) through (c)).
The proposed combination as well as the motivation for combining the Dou, Chesebro, and Heuvel references presented in the rejection of claim 16 apply to claim 17 and are incorporated herein by reference. Thus, the method recited in claim 17 is met by Dou, Chesebro, and Heuvel.
Regarding claim 18, Dou teaches, An apparatus for processing a brain image to determine presence of microbleeds, the apparatus comprising (Abstract: “Cerebral microbleeds (CMBs) are small haemorrhages nearby blood vessels. They have been recognized as important diagnostic biomarkers for many cerebrovascular diseases and cognitive dysfunctions… In this paper, we propose a novel automatic method to detect CMBs from magnetic resonance (MR) images by exploiting the 3D convolutional neural network (CNN)”; Pg. 1192, E. System Implementation: “We implemented the proposed framework based on Theano library using dual Intel Xeon(R) processors E5–2650 2.6 GHz and a GPU of NVIDIA GeForce GTX TITAN Z”; Pg. 1183: effectiveness of the 3D CNN lies in the computational cost and memory requirement):
a processor and non-volatile memory storing processor instructions that when executed by the processor perform (Pg. 1192, E. System Implementation: “We implemented the proposed framework based on Theano library using dual Intel Xeon(R) processors E5–2650 2.6 GHz and a GPU of NVIDIA GeForce GTX TITAN Z”; Pg. 1183: effectiveness of the 3D CNN lies in the computational cost and memory requirement; Note: processors have memory):
providing a classifier trained to recognize microbleed voxels (Pg. 1184, II. Methodology: “In the screening stage, the 3D FCN model takes a whole volumetric data as input and directly outputs a 3D score volume. Each value on the 3D score volume represents the probability of CMB at a corresponding voxel of the input volume”; Fig. 2);
receiving a brain image of a subject (Fig. 2: testing volume; Abstract: “In this paper, we propose a novel automatic method to detect CMBs from magnetic resonance (MR) images by exploiting the 3D convolutional neural network (CNN)”);
and identifying microbleed voxels in said brain image using said classifier (Pg. 1184, II. Methodology: “In the screening stage, the 3D FCN model takes a whole volumetric data as input and directly outputs a 3D score volume. Each value on the 3D score volume represents the probability of CMB at a corresponding voxel of the input volume”; Fig. 2: 3D FCN Screening);
wherein false positives in said identified microbleed voxels are reduced by at least one of: removing identified microbleed voxels based on a comparison between an image (Pg. 1184, II. Methodology: “Subsequently, in the discrimination stage, we further remove false positive candidates by applying a 3D CNN discrimination model to distinguish true CMBs from challenging mimics with high-level feature representations”; Fig. 2: 3D CNN Discrimination; Pg. 1188, Discrimination Stage: “We first found that a number of false positives were produced in the first stage with a training block size of 16
×
16
×
16. By enlarging the block size, richer contextual information within larger surrounding neighborhood can provide additional clues to better distinguish CMBs from their mimics”);
Dou does not expressly disclose the following limitations: removing identified microbleed voxels based on a comparison between an image characteristic; and said classifier being first trained on a relevant image characteristic for distinguishing cerebral spinal fluid from gray matter/white matter using a first training set of brain images and being further trained to distinguish microbleed voxels from other brain tissue voxels using a second set of microbleed brain images segmented by one or more experts.
However, Chesebro teaches, removing identified microbleed voxels based on a comparison between an image characteristic (Pg. 3, Detection of potential microbleed regions of interest: “This definition was used because a neighborhood of this size both ensures the entire microbleed artifact will be captured for analysis in the next stage and also standardizes the selected ROIs, making geometric features more comparable”; Pg. 4, 3D geometric filtering: “To assist in the removal of false positive locations, we used the geometric information contained in each ROI identified in the previous step. We selected a priori four characteristics of the ROI as having the potential to differentiate between true and false positive locations: the 3D image entropy of the ROI, the 2D image entropy of the maximum intensity projection of the ROI, and the volume and compactness of the central blob in each ROI as identified via Frangi filtering. In an image, each pixel i has a probability pi of being a given intensity, measured as the fraction of all pixels in the image at that intensity…True and false positive ROI entropies are illustrated in Fig. 4”);
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine removing identified microbleed voxels based on a comparison between an image characteristic as taught by Chesebro with the method of Dou in order to differentiate between true and false positive locations (Chesebro, Pg. 4). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately.
The combination of Dou and Chesebro does not expressly disclose the following limitation: and said classifier being first trained on a relevant image characteristic for distinguishing cerebral spinal fluid from gray matter/white matter using a first training set of brain images and being further trained to distinguish microbleed voxels from other brain tissue voxels using a second set of microbleed brain images segmented by one or more experts.
However, Heuvel teaches, and said classifier being first trained on a relevant image characteristic for distinguishing cerebral spinal fluid from gray matter/white matter using a first training set of brain images and being further trained to distinguish microbleed voxels from other brain tissue voxels using a second set of microbleed brain images segmented by one or more experts (Fig. 3: annotations and brain mask are input into the voxel classifier; Pg. 243, 2.1.2. Annotations: “only one trained expert manually annotated the CMBs in all 33 patients…and less hypointense lesions”; Pg. 243, 2.2.1. Brain mask: “brain mask defines which voxels belong to the brain and which voxels belong to the skull and air surrounding the brain. The brain mask was made in three steps. Firstly, the gray matter, white matter, and spinal fluid were segmented into three probability maps using SPM12b…as the final SWI brain mask”).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine a classifier being trained on an image characteristic for distinguishing cerebral spinal fluid from gray matter/white matter and being trained to distinguish microbleed voxels from other brain tissue voxels using microbleed brain images segmented by an expert as taught by Heuvel with the combined method of Dou and Chesebro in order to accurately detect CMBs (Heuvel, Abstract). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 18.
Claims 8 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over “Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks” by Dou et al. (hereinafter “Dou”) in view of “Automated detection of cerebral microbleeds on T2*-weighted MRI” by Chesebro et al. (hereinafter “Chesebro”) and further in view of “Automated detection of cerebral microbleeds in patients with traumatic brain injury” by Heuvel et al. (hereinafter “Heuvel”) and “Detecting cerebral microbleeds with transfer learning” by Hong et al. (hereinafter “Hong”).
Regarding claim 8, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 4.
Dou, Chesebro, and Heuvel does not expressly disclose the following limitation: wherein said classifier trained to recognize microbleed voxels has an accuracy greater than 95%.
However, Hong teaches, wherein said classifier trained to recognize microbleed voxels has an accuracy greater than 95% (Abstract: “we proposed a method based on ResNet-50 for exploring the possibility of further improving the accuracy of CMBs detection in this study…and an accuracy of 97.46
±
0.524%”; Pg. 1128, 4.2 Prediction results of different ways of transfer learning).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine a classifier recognizing microbleed voxels with an accuracy greater than 95% as taught by Hong with the combined method of Dou, Chesebro, and Heuvel in order to detect CMBs accurately and reliably for diagnosing and researching some cerebrovascular diseases and cognitive dysfunctions (Hong, Abstract). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 8.
Regarding claim 15, the combination of Dou, Chesebro, and Heuvel teaches the limitations as explained above in claim 10.
Dou, Chesebro, and Heuvel does not expressly disclose the following limitation: wherein said classifier trained to recognize microbleed voxels has an accuracy greater than 95%.
However, Hong teaches, wherein said classifier trained to recognize microbleed voxels has an accuracy greater than 95% (Abstract: “we proposed a method based on ResNet-50 for exploring the possibility of further improving the accuracy of CMBs detection in this study…and an accuracy of 97.46
±
0.524%”; Pg. 1128, 4.2 Prediction results of different ways of transfer learning).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine a classifier recognizing microbleed voxels with an accuracy greater than 95% as taught by Hong with the combined method of Dou, Chesebro, and Heuvel in order to detect CMBs accurately and reliably for diagnosing and researching some cerebrovascular diseases and cognitive dysfunctions (Hong, Abstract). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 15.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
“MRI Segmentation and Classification of Human Brain Using Deep Learning for Diagnosis of Alzheimer’s Disease: A Survey” by Yamanakkanavar et al.
Chen et al. (US 2017/0147908 A1)
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daniella M. DiGuglielmo whose telephone number is (571)272-0183. The examiner can normally be reached Monday - Friday 8:00 AM - 4:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Daniella M. DiGuglielmo/Examiner, Art Unit 2666
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666