DETAILED ACTIONS
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim this application being a Continuation of Application No. PCT/US2022/044850, filed on September 27, 2022, and Provision priority of Application No. 63/249,974 filed on September 9, 2021.
Information Disclosure Statement
The information disclosure statement (“IDS”) filed on 07/11/2024 was reviewed and the listed references were noted.
Drawings
The 7-page drawings have been considered and placed on record in the file.
Status of Claims
Claims 1-20 are pending.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 15-16 and 18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Guo et al., (US 2021/0150671 A1, published May 20, 2021), hereinafter referred to as Guo.
Claim 15
Guo discloses a computer-implemented method for enhancing image quality (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”) and anomaly detection (Guo, [0089], “brain extraction using brain masks (e.g., binary maps) was completed using PCNN3D”, [0309], “3-D Pulse-Coupled Neural Networks”), [0129], “For scans of tumor subjects, tumor masks can be generated in addition to the brain masks using the Fuzzy-C-Means segmentation”) the method comprising:
(a) obtaining a multi-contrast image of a subject (Guo, [0008], “The MRI information can include (i) a low-dosage Gd MRI scan(s), or (ii) a Gd-free MRI scan(s). A Gd contrast can be generated in the Gd enhanced map(s) using a T2-weighted MRI image of the portion(s).”), wherein the multi-contrast image comprises an image of a first contrast acquired with a reduced dose of contrast agent (Guo, [0089], “The raw scans from the Bruker scanner were converted to NIfTI format, and for each subject, rigid-body spatial normalization was used to align the Pre scan 105, the Low scan 110, and the High scan 115.”, [0091], “registration of the pre-contrast and the post-contrast T1-weighted scans”);
(b) providing a deep learning network model (Guo, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image.”, Fig. 26A-26D) comprising a multi-contrast branched architecture (Guo, the example network shown in Fig. 26A-26D shows multi-contrast branched architecture); and
PNG
media_image1.png
435
600
media_image1.png
Greyscale
(c) taking the multi-contrast image (Guo, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image”) and the anomaly mask as input to a second deep network model (Guo, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image.”, [0089], “ground truth tumor masks were generated in addition to the brain masks using the FCM (e.g., Fuzzy-C-Means) Clustering Based Segmentation”, the tumor mask is used as a ground truth input to train the deep learning network model, [0157], “In order to accurately capture the high variance present within brain tumors, the exemplary DeepContrast model was trained with a large-scale brain tumor MM dataset, which again was able to generate contrast predictions that were similar to the ground truth”, [0178], “For scans of tumor subjects, the CBV maps and brain masks were derived using the same methods as descripted in the healthy mouse brain study, and tumor masks were generated in addition to the brain masks using the Fuzzy-C-Means segmentation. (See, e.g., Reference 104). 6 GBM subjects were added to the training set while 3 GBM subjects replaced the original testing set of the Healthy Mouse Brain Model.”) to generate a predicted image with improved quality (Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250. All the layers can include a residual connection 250; the decoding layers 230-250 can be implemented with attention mechanism. Various Max-pooling can be performed (e.g., with a factor of 2) as well as upsampling 255 also with a factor of 2. Additionally, Conv2D+Batch Normalization (“BN)+Rectified Linear Unit (“ReLu”) layers 260 can be included. The exemplary network can take both Pre and Low as the two-channel input or only Pre as the single-channel input. Estimated 2D CBV maps with brain extraction can be the output.”, [0112], “Both the Pre+Low and the Pre image can predict the tumor region with enhanced contrast and similar space feature by ResAttU-Net.”)
Claim 16
Guo discloses the computer-implemented method of claim 15 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the multi-contrast branched architecture comprises a first branch configured to process the image of the first contrast and an image of a second contrast (Guo, Fig. 26A-26D, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image”).
Claim 18
Guo discloses the computer-implemented method of claim 16 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the multi-contrast branched architecture (Guo, Fig. 26A-26D) comprises a second branch to process the image of the first contrast (Guo, Fig. 26A-26D, [0095], “exemplary network can take both Pre and Low as the two-channel input”) and the anomaly mask (Guo, [0089], “ground truth tumor masks were generated in addition to the brain masks using the FCM (e.g., Fuzzy-C-Means) Clustering Based Segmentation”, the tumor mask is used as a ground truth input to train the deep learning network model, [0157], “In order to accurately capture the high variance present within brain tumors, the exemplary DeepContrast model was trained with a large-scale brain tumor MM dataset, which again was able to generate contrast predictions that were similar to the ground truth”, [0178], “For scans of tumor subjects, the CBV maps and brain masks were derived using the same methods as descripted in the healthy mouse brain study, and tumor masks were generated in addition to the brain masks using the Fuzzy-C-Means segmentation. (See, e.g., Reference 104). 6 GBM subjects were added to the training set while 3 GBM subjects replaced the original testing set of the Healthy Mouse Brain Model.”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 5, 7-12, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Kim et al., "Unsupervised anomaly detection in MR images using multicontrast information" (May 2021), hereinafter referred to as Kim.
Claim 1
Guo discloses a computer-implemented method for enhancing image quality (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”) and anomaly detection (Guo, [0089], “brain extraction using brain masks (e.g., binary maps) was completed using PCNN3D”, [0309], “3-D Pulse-Coupled Neural Networks”), [0129], “For scans of tumor subjects, tumor masks can be generated in addition to the brain masks using the Fuzzy-C-Means segmentation”) the method comprising:
(a) obtaining a multi-contrast image of a subject (Guo, [0008], “The MRI information can include (i) a low-dosage Gd MRI scan(s), or (ii) a Gd-free MRI scan(s). A Gd contrast can be generated in the Gd enhanced map(s) using a T2-weighted MRI image of the portion(s).”), wherein the multi-contrast image comprises an image of a first contrast acquired with a reduced dose of contrast agent (Guo, [0089], “The raw scans from the Bruker scanner were converted to NIfTI format, and for each subject, rigid-body spatial normalization was used to align the Pre scan 105, the Low scan 110, and the High scan 115.”, [0091], “registration of the pre-contrast and the post-contrast T1-weighted scans”);
(b) generating an anomaly mask using a first deep learning network ([0089], “brain extraction using brain masks (e.g., binary maps) was completed using PCNN3D”, [0309], “3-D Pulse-Coupled Neural Networks”), [0129], “For scans of tumor subjects, tumor masks can be generated in addition to the brain masks using the Fuzzy-C-Means segmentation”); and
(c) taking the multi-contrast image (Guo, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image”) and the anomaly mask as input to a second deep network model (Guo, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image.”, [0089], “ground truth tumor masks were generated in addition to the brain masks using the FCM (e.g., Fuzzy-C-Means) Clustering Based Segmentation”, the tumor mask is used as a ground truth input to train the deep learning network model, [0157], “In order to accurately capture the high variance present within brain tumors, the exemplary DeepContrast model was trained with a large-scale brain tumor MM dataset, which again was able to generate contrast predictions that were similar to the ground truth”, [0178], “For scans of tumor subjects, the CBV maps and brain masks were derived using the same methods as descripted in the healthy mouse brain study, and tumor masks were generated in addition to the brain masks using the Fuzzy-C-Means segmentation. (See, e.g., Reference 104). 6 GBM subjects were added to the training set while 3 GBM subjects replaced the original testing set of the Healthy Mouse Brain Model.”) to generate a predicted image with improved quality (Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250. All the layers can include a residual connection 250; the decoding layers 230-250 can be implemented with attention mechanism. Various Max-pooling can be performed (e.g., with a factor of 2) as well as upsampling 255 also with a factor of 2. Additionally, Conv2D+Batch Normalization (“BN)+Rectified Linear Unit (“ReLu”) layers 260 can be included. The exemplary network can take both Pre and Low as the two-channel input or only Pre as the single-channel input. Estimated 2D CBV maps with brain extraction can be the output.”, [0112], “Both the Pre+Low and the Pre image can predict the tumor region with enhanced contrast and similar space feature by ResAttU-Net.”).
Guo does not explicitly disclose generating an anomaly mask using a first deep learning network.
However, Kim teaches generating an anomaly mask using a first deep learning network (Kim, Fig. 2).
PNG
media_image2.png
376
680
media_image2.png
Greyscale
Guo and Kim are both considered to be analogous to the claimed invention because they are in the same field of tumor/anomaly detection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Kim of generating an anomaly mask using a first deep learning network. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been for accurate diagnosis in MRI.
Claim 2
The combination of Guo in view of Kim discloses the computer-implemented method of claim 1 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the multi-contrast image is acquired using a magnetic resonance (MR) device (Guo, [0002], “The present disclosure relates generally to magnetic resonance imaging (“MRI”), and more specifically, to exemplary embodiments of an exemplary system, method and computer-accessible medium for the reduction of the dosage of Gd-based contrast agent in MRI.”, [0089], “The raw scans from the Bruker scanner were converted to NIfTI format, and for each subject, rigid-body spatial normalization was used to align the Pre scan 105, the Low scan 110, and the High scan 115.”).
Claim 3
The combination of Guo in view of Kim discloses the computer-implemented method of claim 1 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the first deep learning network is trained using unsupervised anomaly detection scheme (Kim, Abstract, “an unsupervised algorithm is proposed for pixel-level anomaly detection in multicontrast MRI”).
The proposed combination as well as the motivation for combining the Guo and Kim references presented in the rejection of Claim 1, apply to Claim 3 and are incorporated herein by reference. Thus, the method recited in Claim 3 is met by Guo and Kim.
Claim 5
The combination of Guo in view of Kim discloses the computer-implemented method of claim 1 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the multi-contrast image comprises an image of a second contrast (Guo, [0089], “The raw scans from the Bruker scanner were converted to NIfTI format, and for each subject, rigid-body spatial normalization was used to align the Pre scan 105, the Low scan 110, and the High scan 115.”) that is processed by the first deep learning network for generating the anomaly mask (Guo, [0089], “brain extraction using brain masks (e.g., binary maps) was completed using PCNN3D”, [0309], “3-D Pulse-Coupled Neural Networks”), [0129], “For scans of tumor subjects, tumor masks can be generated in addition to the brain masks using the Fuzzy-C-Means segmentation”, Kim teaches using a deep learning network for the tumor or anomaly segmentation).
The proposed combination as well as the motivation for combining the Guo and Kim references presented in the rejection of Claim 1, apply to Claim 5 and are incorporated herein by reference. Thus, the method recited in Claim 5 is met by Guo and Kim.
Claim 7
The combination of Guo in view of Kim discloses the computer-implemented method of claim 1 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the second deep network model comprises multiple branches (Guo, [0185], All five model variants developed in the exemplary studies, as shown in FIG. 20, share the common residual attention U-Net (“RAU-Net”) architecture. (See e.g., FIGS. 26A-26D, Fig. 26A-26D shows different branches for the deep learning network model used to enhance the MRI).
Claim 8
The combination of Guo in view of Kim discloses the computer-implemented method of claim 7 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein an input to at least one of the multiple branches comprises the image of the first contrast and an image of a different contrast (Guo, Fig. 26A-26D, [0095], “exemplary network can take both Pre and Low as the two-channel input”).
Claim 9
The combination of Guo in view of Kim discloses the computer-implemented method of claim 7 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein an input to at least one of the multiple branches comprises the image of the first contrast (Guo, Fig. 26A-26D, [0095], “exemplary network can take both Pre and Low as the two-channel input”) and the anomaly mask generated in (b) (Guo, [0089], “ground truth tumor masks were generated in addition to the brain masks using the FCM (e.g., Fuzzy-C-Means) Clustering Based Segmentation”, the tumor mask is used as a ground truth input to train the deep learning network model, [0157], “In order to accurately capture the high variance present within brain tumors, the exemplary DeepContrast model was trained with a large-scale brain tumor MM dataset, which again was able to generate contrast predictions that were similar to the ground truth”, [0178], “For scans of tumor subjects, the CBV maps and brain masks were derived using the same methods as descripted in the healthy mouse brain study, and tumor masks were generated in addition to the brain masks using the Fuzzy-C-Means segmentation. (See, e.g., Reference 104). 6 GBM subjects were added to the training set while 3 GBM subjects replaced the original testing set of the Healthy Mouse Brain Model.”).
Claim 10
The combination of Guo in view of Kim discloses the computer-implemented method of claim 7 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein an input to each of the multiple branches comprises at least the image of the first contrast (Guo, Fig. 26A-26D, the first contrast image is used as an input for every branch after preprocessing it combined with the second contrast image).
Claim 11
The combination of Guo in view of Kim discloses the computer-implemented method of claim 7 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the predicted image with improved quality is generated based on multiple predictions generated by the multiple branches (Guo, Fig. 26A-26D, [0185], “All five model variants developed in the exemplary studies, as shown in FIG. 20, share the common residual attention U-Net (“RAU-Net”) architecture. (See e.g., FIGS. 26A-26D). Model inputs can be the non-contrast MRI scans, while the outputs can be the corresponding predicted gadolinium contrast.”).
Claim 12
The combination of Guo in view of Kim discloses the computer-implemented method of claim 11 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the anomaly mask (Guo, [0089], “brain extraction using brain masks (e.g., binary maps) was completed using PCNN3D”, [0309], “3-D Pulse-Coupled Neural Networks”), [0129], “For scans of tumor subjects, tumor masks can be generated in addition to the brain masks using the Fuzzy-C-Means segmentation”, Pinaya teaches using a deep learning network for the tumor or anomaly segmentation) is further utilized as an attention mechanism for training the second deep learning network model (Guo, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250. All the layers can include a residual connection 250; the decoding layers 230-250 can be implemented with attention mechanism.”, [0186], “while the attention gates learn to differentially enhance or suppress specific regions in the feature maps so that the downstream outcomes better suit the desired task”).
Claim 14 is rejected for similar reasons as those described in claim 1. The additional elements in Claim 14 (Guo and Kim) discloses includes: a non-transitory computer-readable storage medium (Guo, [0203], “As shown in FIG. 36, for example a computer-accessible medium 3615 (e.g., as described herein above, a storage device such as a hard disk, floppy disk, memory stick, CD-ROM, RAM, ROM, etc., or a collection thereof) can be provided (e.g., in communication with the processing arrangement 3605)”) including instructions (Guo, Fig. 26A-26D) that, when executed by one or more processors (Guo, [0202], “computer/processor 3610 that can include, for example one or more microprocessors, and use instructions stored on a computer-accessible medium (e.g., RAM, ROM, hard drive, or other storage device)”), cause the one or more processors to perform operations (Guo, [0202], “computer/processor 3610 that can include, for example one or more microprocessors, and use instructions stored on a computer-accessible medium (e.g., RAM, ROM, hard drive, or other storage device)”). The proposed combination as well as the motivation for combining the Guo and Kim references presented in the rejection of Claim 1, apply to Claim 14 and are incorporated herein by reference. Thus, the medium recited in Claim 14 is met by Guo and Kim.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Kim in further view of Pinaya et al., "Unsupervised Brain Anomaly Detection and Segmentation with Transformers” (February 2021), hereinafter referred to as Pinaya.
Claim 4
The combination of Guo in view of Pinaya discloses the computer-implemented method of claim 3 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”).
The combination of Guo in view of Kim does not explicitly disclose wherein the first deep learning network comprises a variational autoencoder (VAE) model trained only on images without anomaly.
However, Pinaya teaches wherein the first deep learning network comprises a variational autoencoder (VAE) model trained only on images without anomaly (Pinaya, Abstract, “Here we combine the latent representation of vector quantised variational autoencoders with an ensemble of autoregressive transformers to enable unsupervised anomaly detection and segmentation defined by deviation from healthy brain imaging data, achievable at low computational cost, within relative modest data regimes”, Section 4.3, “We selected the 15,000 subjects, and their respective FLAIR images, with the lowest white matter hyperintensities volume, as provided by UKB, to train our models, as these subjects were the most radiologically normal.”).
Guo, Kim, and Pinaya are all considered to be analogous to the claimed invention because they are in the same field of tumor/anomaly detection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Pinaya wherein the first deep learning network comprises a variational autoencoder (VAE) model trained only on images without anomaly. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been for better performance.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Kim in further view of Zaharchuk et al., (US 2021/0241458 A1, published August 5, 2021), hereinafter referred to as Zaharchuk.
Claim 6
The combination of Guo in view of Pinaya discloses the computer-implemented method of claim 5 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”).
The combination of Guo in view of Pinaya does not explicitly disclose wherein the image of the first contrast is T1-weighted image and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI).
However, Zaharchuk teaches wherein the image of the first contrast is T1-weighted image (Zaharchuk, Fig. 3, [0026], “high-resolution T1-weighted IR-FSPGR pre-contrast images, post-contrast images with 10% low-dose and 100% full-dose of gadobenate dimeglumine (0.01 and 0.1 mmol/kg, respectively) full-dose images are acquired.”) and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI) (Zaharchuk, [0036], “The co-registered and normalized non-contrast (zero-dose) and low-dose images are loaded from DICOM files and input to the trained network “, [0050], “The zero dose images in this case include multiple MR images acquired using different sequences (e.g., T1w, T2w, FLAIR, DWI) that show different appearance/intensity of different tissues”)).
Guo and Zaharchuk are both considered to be analogous to the claimed invention because they are in the same field of MRI enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Zaharchuk wherein the image of the first contrast is T1-weighted image and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI). Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to show different appearance/intensity of different tissues (Zaharchuk, [0050]).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Kim in further view of Hespen et al., “An anomaly detection approach to identify chronic brain infarcts on MRI” (April 2021), hereinafter referred to as Hespen.
Claim 13
The combination of Guo in view of Kim discloses the computer-implemented method of claim 11 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”).
The combination of Guo in view of Kim does not explicitly disclose further comprising displaying the predicted image overlaid with the anomaly mask.
However, Hespen teaches further comprising displaying the predicted image overlaid with the anomaly mask (Guo teaches generating a contrast-enhance image, Hespen teaches overlaying the anomaly mask over as MRI image, Fig. 5, “Anomaly score overlay maps on transversal image slices (T2-FLAIR images)”).
Guo, Kim, and Hespen are all considered to be analogous to the claimed invention because they are in the same field of tumor/anomaly detection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo and Kim to incorporate the teachings of Hespen of further comprising displaying the predicted image overlaid with the anomaly mask. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to better observed the anomalous regions of the brain.
Claims 17 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Zaharchuk.
Claim 17
Guo discloses the computer-implemented method of claim 16 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”).
Guo does not explicitly disclose wherein the image of the first contrast is T1-weighted image and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI).
However, Zaharchuk teaches wherein the image of the first contrast is T1-weighted image (Zaharchuk, Fig. 3, [0026], “high-resolution T1-weighted IR-FSPGR pre-contrast images, post-contrast images with 10% low-dose and 100% full-dose of gadobenate dimeglumine (0.01 and 0.1 mmol/kg, respectively) full-dose images are acquired.”) and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI) (Zaharchuk, [0036], “The co-registered and normalized non-contrast (zero-dose) and low-dose images are loaded from DICOM files and input to the trained network “, [0050], “The zero dose images in this case include multiple MR images acquired using different sequences (e.g., T1w, T2w, FLAIR, DWI) that show different appearance/intensity of different tissues”)).
Guo and Zaharchuk are both considered to be analogous to the claimed invention because they are in the same field of MRI enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Zaharchuk wherein the image of the first contrast is T1-weighted image and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI). Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to show different appearance/intensity of different tissues (Zaharchuk, [0050]).
Claim 19
Guo discloses the computer-implemented method of claim 15 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”).
Guo does not explicitly disclose wherein the multi-contrast branched architecture comprises at least three branches.
However, Zaharchuk teaches wherein the multi-contrast branched architecture comprises at least three branches (Zaharchuk, Fig. 3, three branches for three different scans).
Guo and Zaharchuk are both considered to be analogous to the claimed invention because they are in the same field of MRI enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Zaharchuk wherein the multi-contrast branched architecture comprises at least three branches. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to show different appearance/intensity of different tissues (Zaharchuk, [0050]).
Claim 20
The combination of Guo in view of Zaharchuk discloses the computer-implemented method of claim 19 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the predicted image with improved quality is generated based on multiple predictions generated by the at least three branches (Zaharchuk, Fig.3).
The proposed combination as well as the motivation for combining the Guo and Zaharchuk references presented in the rejection of Claim 19, apply to Claim 20 and are incorporated herein by reference. Thus, the method recited in Claim 20 is met by Guo and Zaharchuk.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DENISE G ALFONSO whose telephone number is (571)272-1360. The examiner can normally be reached Monday - Friday 7:30 - 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DENISE G ALFONSO/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662