DETAILED ACTIONS
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 are pending.
Response to Amendment
The amendment filed 07/31/2026 has been entered in full.
Response to Arguments
Applicant’s arguments with respect to claims 1, 14, and 15 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 5, 7-12, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Guo et al., (US 2021/0150671 A1), hereinafter referred to as Guo, in view of Kim et al., "Unsupervised anomaly detection in MR images using multicontrast information" (May 2021), hereinafter referred to as Kim, in view of Tamir et al., "A generic deep learning model for reduced gadolinium dose in contrast- enhanced brain MRI" (March 2021), hereinafter referred to as Tamir, (reference falls within the grace period, but is not the "same" inventor due to the addition of Jonathan Tamir, Soren Christensen, Greg Zaharchuk, and Tao Zhang).
Claim 1
Guo discloses a computer-implemented method for enhancing image quality (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”) and anomaly detection (Guo, [0089], “brain extraction using brain masks (e.g., binary maps) was completed using PCNN3D”, [0309], “3-D Pulse-Coupled Neural Networks”), [0129], “For scans of tumor subjects, tumor masks can be generated in addition to the brain masks using the Fuzzy-C-Means segmentation”) the method comprising:
(a) obtaining a multi-contrast image of a subject (Guo, [0008], “The MRI information can include (i) a low-dosage Gd MRI scan(s), or (ii) a Gd-free MRI scan(s). A Gd contrast can be generated in the Gd enhanced map(s) using a T2-weighted MRI image of the portion(s).”), wherein the multi-contrast image comprises an image of a first contrast acquired with a reduced dose of contrast agent (Guo, [0089], “The raw scans from the Bruker scanner were converted to NIfTI format, and for each subject, rigid-body spatial normalization was used to align the Pre scan 105, the Low scan 110, and the High scan 115.”, [0091], “registration of the pre-contrast and the post-contrast T1-weighted scans”);
(b) generating a mask using a first deep learning network ([0089], “brain extraction using brain masks (e.g., binary maps) was completed using PCNN3D”, [0309], “3-D Pulse-Coupled Neural Networks”), [0129], “For scans of tumor subjects, tumor masks can be generated in addition to the brain masks using the Fuzzy-C-Means segmentation”); and
(c) taking the multi-contrast image (Guo, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image”) and the mask as input to a second deep network model (Guo, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image.”, [0089], “ground truth tumor masks were generated in addition to the brain masks using the FCM (e.g., Fuzzy-C-Means) Clustering Based Segmentation”, the tumor mask is used as a ground truth input to train the deep learning network model, [0157], “In order to accurately capture the high variance present within brain tumors, the exemplary DeepContrast model was trained with a large-scale brain tumor MM dataset, which again was able to generate contrast predictions that were similar to the ground truth”, [0178], “For scans of tumor subjects, the CBV maps and brain masks were derived using the same methods as descripted in the healthy mouse brain study, and tumor masks were generated in addition to the brain masks using the Fuzzy-C-Means segmentation. (See, e.g., Reference 104). 6 GBM subjects were added to the training set while 3 GBM subjects replaced the original testing set of the Healthy Mouse Brain Model.”) to generate a predicted image with improved quality (Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250. All the layers can include a residual connection 250; the decoding layers 230-250 can be implemented with attention mechanism. Various Max-pooling can be performed (e.g., with a factor of 2) as well as upsampling 255 also with a factor of 2. Additionally, Conv2D+Batch Normalization (“BN)+Rectified Linear Unit (“ReLu”) layers 260 can be included. The exemplary network can take both Pre and Low as the two-channel input or only Pre as the single-channel input. Estimated 2D CBV maps with brain extraction can be the output.”, [0112], “Both the Pre+Low and the Pre image can predict the tumor region with enhanced contrast and similar space feature by ResAttU-Net.”).
Guo does not explicitly disclose generating an anomaly mask using a first deep learning network.
However, Kim teaches generating an anomaly mask using a first deep learning network (Kim, Fig. 2).
PNG
media_image1.png
376
680
media_image1.png
Greyscale
Guo and Kim are both considered to be analogous to the claimed invention because they are in the same field of tumor/anomaly detection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Kim of generating an anomaly mask using a first deep learning network. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been for accurate diagnosis in MRI.
Guo does not explicitly disclose generating an anomaly mask using a first deep learning network based at least in part on the multi-contrast image, and taking the multi contrast image and the anomaly mask generated as input to a second deep network model to generated a predicted image with improved quality, wherein the anomaly mask is utilized by the second deep network model focuses on one or more anomalous regions indicated by the anomaly mask when generating the predicted image.
However, Tamir teaches generating an anomaly mask using a first deep learning network (Tamir, Fig. 2, Section 2.6, “To make the network pay more attention to the actual enhancement regions, we weighted the L1 loss with an enhancement mask.”) based at least in part on the multi-contrast image (Tamir, Fig. 1), and taking the multi contrast image (Tamir, Fig. 1) and the anomaly mask generated (Tamir, Section 2.6, “To make the network pay more attention to the actual enhancement regions, we weighted the L1 loss with an enhancement mask.”) as input to a second deep network model to generated a predicted image with improved quality (Tamir, Fig. 3) wherein the anomaly mask is utilized by the second deep network model focuses on one or more anomalous regions indicated by the anomaly mask when generating the predicted image (Tamir, Section 2.6, “Even after signal normalization and scaling, the difference between the low- dose and pre- contrast images had some enhancement- like noise perturbations. To make the network pay more attention to the actual enhancement regions, we weighted the L1 loss with an enhancement mask. The enhancement mask was computed as the difference between the low- dose and pre- contrast images. The difference was computed after applying the brain mask and was normalized between 0 and 1. The enhancement mask can be considered as a normalized smooth version of the contrast uptake. Similar technique has been used in Ref. [46] where the loss function is focused only the vicinity of the tumor area).
Guo, Kim and Tamir are both considered to be analogous to the claimed invention because they are in the same field of tumor/anomaly detection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Tamir of generating an anomaly mask using a first deep learning network based at least in part on the multi-contrast image, and taking the multi contrast image and the anomaly mask generated as input to a second deep network model to generated a predicted image with improved quality, wherein the anomaly mask is utilized by the second deep network model focuses on one or more anomalous regions indicated by the anomaly mask when generating the predicted image. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to make the network pay more attention to the actual enhancement regions (Tamir, Section 2.6).
Guo and Zaharchuk are both considered to be analogous to the claimed invention because they are in the same field of tumor/anomaly detection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Tamir of generating an anomaly mask using a first deep learning network based at least in part on the multi-contrast image, and taking the multi contrast image and the anomaly mask generated as input to a second deep network model to generated a predicted image with improved quality, wherein the anomaly mask is utilized by the second deep network model focuses on one or more anomalous regions indicated by the anomaly mask when generating the predicted image. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to make the network pay more attention to the actual enhancement regions (Tamir, Section 2.6).
Claim 2
The combination of Guo in view of Kim in further view of Tamir discloses the computer-implemented method of claim 1 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the multi-contrast image is acquired using a magnetic resonance (MR) device (Guo, [0002], “The present disclosure relates generally to magnetic resonance imaging (“MRI”), and more specifically, to exemplary embodiments of an exemplary system, method and computer-accessible medium for the reduction of the dosage of Gd-based contrast agent in MRI.”, [0089], “The raw scans from the Bruker scanner were converted to NIfTI format, and for each subject, rigid-body spatial normalization was used to align the Pre scan 105, the Low scan 110, and the High scan 115.”).
Claim 3
The combination of Guo in view of Kim in further view of Tamir discloses the computer-implemented method of claim 1 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the first deep learning network is trained using unsupervised anomaly detection scheme (Kim, Abstract, “an unsupervised algorithm is proposed for pixel-level anomaly detection in multicontrast MRI”).
The proposed combination as well as the motivation for combining the Guo and Kim references presented in the rejection of Claim 1, apply to Claim 3 and are incorporated herein by reference. Thus, the method recited in Claim 3 is met by Guo and Kim.
Claim 5
The combination of Guo in view of Kim in further view of Tamir discloses the computer-implemented method of claim 1 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the multi-contrast image comprises an image of a second contrast (Guo, [0089], “The raw scans from the Bruker scanner were converted to NIfTI format, and for each subject, rigid-body spatial normalization was used to align the Pre scan 105, the Low scan 110, and the High scan 115.”) that is processed by the first deep learning network for generating the anomaly mask (Guo, [0089], “brain extraction using brain masks (e.g., binary maps) was completed using PCNN3D”, [0309], “3-D Pulse-Coupled Neural Networks”), [0129], “For scans of tumor subjects, tumor masks can be generated in addition to the brain masks using the Fuzzy-C-Means segmentation”, Kim teaches using a deep learning network for the tumor or anomaly segmentation).
The proposed combination as well as the motivation for combining the Guo and Kim references presented in the rejection of Claim 1, apply to Claim 5 and are incorporated herein by reference. Thus, the method recited in Claim 5 is met by Guo and Kim.
Claim 7
The combination of Guo in view of Kim in further view of Tamir discloses the computer-implemented method of claim 1 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the second deep network model comprises multiple branches (Guo, [0185], All five model variants developed in the exemplary studies, as shown in FIG. 20, share the common residual attention U-Net (“RAU-Net”) architecture. (See e.g., FIGS. 26A-26D, Fig. 26A-26D shows different branches for the deep learning network model used to enhance the MRI).
Claim 8
The combination of Guo in view of Kim in further view of Tamir discloses the computer-implemented method of claim 7 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein an input to at least one of the multiple branches comprises the image of the first contrast and an image of a different contrast (Guo, Fig. 26A-26D, [0095], “exemplary network can take both Pre and Low as the two-channel input”).
Claim 9
The combination of Guo in view of Kim in further view of Tamir discloses the computer-implemented method of claim 7 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein an input to at least one of the multiple branches comprises the image of the first contrast (Guo, Fig. 26A-26D, [0095], “exemplary network can take both Pre and Low as the two-channel input”) and the anomaly mask generated in (b) (Guo, [0089], “ground truth tumor masks were generated in addition to the brain masks using the FCM (e.g., Fuzzy-C-Means) Clustering Based Segmentation”, the tumor mask is used as a ground truth input to train the deep learning network model, [0157], “In order to accurately capture the high variance present within brain tumors, the exemplary DeepContrast model was trained with a large-scale brain tumor MM dataset, which again was able to generate contrast predictions that were similar to the ground truth”, [0178], “For scans of tumor subjects, the CBV maps and brain masks were derived using the same methods as descripted in the healthy mouse brain study, and tumor masks were generated in addition to the brain masks using the Fuzzy-C-Means segmentation. (See, e.g., Reference 104). 6 GBM subjects were added to the training set while 3 GBM subjects replaced the original testing set of the Healthy Mouse Brain Model.”).
Claim 10
The combination of Guo in view of Kim in further view of Tamir discloses the computer-implemented method of claim 7 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein an input to each of the multiple branches comprises at least the image of the first contrast (Guo, Fig. 26A-26D, the first contrast image is used as an input for every branch after preprocessing it combined with the second contrast image).
Claim 11
The combination of Guo in view of Kim in further view of Tamir discloses the computer-implemented method of claim 7 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the predicted image with improved quality is generated based on multiple predictions generated by the multiple branches (Guo, Fig. 26A-26D, [0185], “All five model variants developed in the exemplary studies, as shown in FIG. 20, share the common residual attention U-Net (“RAU-Net”) architecture. (See e.g., FIGS. 26A-26D). Model inputs can be the non-contrast MRI scans, while the outputs can be the corresponding predicted gadolinium contrast.”).
Claim 12
The combination of Guo in view of Kim in further view of Tamir discloses the computer-implemented method of claim 11 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the anomaly mask (Guo, [0089], “brain extraction using brain masks (e.g., binary maps) was completed using PCNN3D”, [0309], “3-D Pulse-Coupled Neural Networks”), [0129], “For scans of tumor subjects, tumor masks can be generated in addition to the brain masks using the Fuzzy-C-Means segmentation”, Pinaya teaches using a deep learning network for the tumor or anomaly segmentation) is further utilized as an attention mechanism for training the second deep learning network model (Guo, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250. All the layers can include a residual connection 250; the decoding layers 230-250 can be implemented with attention mechanism.”, [0186], “while the attention gates learn to differentially enhance or suppress specific regions in the feature maps so that the downstream outcomes better suit the desired task”).
Claim 14 is rejected for similar reasons as those described in claim 1. The additional elements in Claim 14 (Guo, Kim, and Tamir) discloses includes: a non-transitory computer-readable storage medium (Guo, [0203], “As shown in FIG. 36, for example a computer-accessible medium 3615 (e.g., as described herein above, a storage device such as a hard disk, floppy disk, memory stick, CD-ROM, RAM, ROM, etc., or a collection thereof) can be provided (e.g., in communication with the processing arrangement 3605)”) including instructions (Guo, Fig. 26A-26D) that, when executed by one or more processors (Guo, [0202], “computer/processor 3610 that can include, for example one or more microprocessors, and use instructions stored on a computer-accessible medium (e.g., RAM, ROM, hard drive, or other storage device)”), cause the one or more processors to perform operations (Guo, [0202], “computer/processor 3610 that can include, for example one or more microprocessors, and use instructions stored on a computer-accessible medium (e.g., RAM, ROM, hard drive, or other storage device)”). The proposed combination as well as the motivation for combining the Guo, Kim, and Tamir references presented in the rejection of Claim 1, apply to Claim 14 and are incorporated herein by reference. Thus, the medium recited in Claim 14 is met by Guo ,Kim, and Tamir.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Kim in further view of Tamir in further view of Pinaya et al., "Unsupervised Brain Anomaly Detection and Segmentation with Transformers” (February 2021), hereinafter referred to as Pinaya.
Claim 4
The combination of Guo in view of Kim in further view of Tamir discloses the computer-implemented method of claim 3 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”).
The combination of Guo in view of Kim in further view of Tamir does not explicitly disclose wherein the first deep learning network comprises a variational autoencoder (VAE) model trained only on images without anomaly.
However, Pinaya teaches wherein the first deep learning network comprises a variational autoencoder (VAE) model trained only on images without anomaly (Pinaya, Abstract, “Here we combine the latent representation of vector quantised variational autoencoders with an ensemble of autoregressive transformers to enable unsupervised anomaly detection and segmentation defined by deviation from healthy brain imaging data, achievable at low computational cost, within relative modest data regimes”, Section 4.3, “We selected the 15,000 subjects, and their respective FLAIR images, with the lowest white matter hyperintensities volume, as provided by UKB, to train our models, as these subjects were the most radiologically normal.”).
Guo, Kim, Tamir, and Pinaya are all considered to be analogous to the claimed invention because they are in the same field of tumor/anomaly detection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Pinaya wherein the first deep learning network comprises a variational autoencoder (VAE) model trained only on images without anomaly. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been for better performance.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Kim in further view of Tamir in further view of Zaharchuk et al., (US 2021/0241458 A1, published August 5, 2021), hereinafter referred to as Zaharchuk.
Claim 6
The combination of Guo in view of Kim in further view of Tamir in view of Pinaya discloses the computer-implemented method of claim 5 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”).
The combination of Guo in view of Kim in further view of Tamir in further view of Pinaya does not explicitly disclose wherein the image of the first contrast is T1-weighted image and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI).
However, Zaharchuk teaches wherein the image of the first contrast is T1-weighted image (Zaharchuk, Fig. 3, [0026], “high-resolution T1-weighted IR-FSPGR pre-contrast images, post-contrast images with 10% low-dose and 100% full-dose of gadobenate dimeglumine (0.01 and 0.1 mmol/kg, respectively) full-dose images are acquired.”) and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI) (Zaharchuk, [0036], “The co-registered and normalized non-contrast (zero-dose) and low-dose images are loaded from DICOM files and input to the trained network “, [0050], “The zero dose images in this case include multiple MR images acquired using different sequences (e.g., T1w, T2w, FLAIR, DWI) that show different appearance/intensity of different tissues”)).
Guo and Zaharchuk are both considered to be analogous to the claimed invention because they are in the same field of MRI enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Zaharchuk wherein the image of the first contrast is T1-weighted image and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI). Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to show different appearance/intensity of different tissues (Zaharchuk, [0050]).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Kim in further view of Tamir in further view of Hespen et al., “An anomaly detection approach to identify chronic brain infarcts on MRI” (April 2021), hereinafter referred to as Hespen.
Claim 13
The combination of Guo in view of Kim in further view of Tamir discloses the computer-implemented method of claim 11 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”).
The combination of Guo in view of Kim in further view of Tamir does not explicitly disclose further comprising displaying the predicted image overlaid with the anomaly mask.
However, Hespen teaches further comprising displaying the predicted image overlaid with the anomaly mask (Guo teaches generating a contrast-enhance image, Hespen teaches overlaying the anomaly mask over as MRI image, Fig. 5, “Anomaly score overlay maps on transversal image slices (T2-FLAIR images)”).
Guo, Kim, Tamir, and Hespen are all considered to be analogous to the claimed invention because they are in the same field of tumor/anomaly detection. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo and Kim to incorporate the teachings of Hespen of further comprising displaying the predicted image overlaid with the anomaly mask. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to better observed the anomalous regions of the brain.
Claims 15-20 are rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Zaharchuk.
Claim 15
Guo discloses a computer-implemented method for enhancing image quality (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”) and anomaly detection (Guo, [0089], “brain extraction using brain masks (e.g., binary maps) was completed using PCNN3D”, [0309], “3-D Pulse-Coupled Neural Networks”), [0129], “For scans of tumor subjects, tumor masks can be generated in addition to the brain masks using the Fuzzy-C-Means segmentation”) the method comprising:
(a) obtaining a multi-contrast image of a subject (Guo, [0008], “The MRI information can include (i) a low-dosage Gd MRI scan(s), or (ii) a Gd-free MRI scan(s). A Gd contrast can be generated in the Gd enhanced map(s) using a T2-weighted MRI image of the portion(s).”), wherein the multi-contrast image comprises an image of a first contrast acquired with a reduced dose of contrast agent (Guo, [0089], “The raw scans from the Bruker scanner were converted to NIfTI format, and for each subject, rigid-body spatial normalization was used to align the Pre scan 105, the Low scan 110, and the High scan 115.”, [0091], “registration of the pre-contrast and the post-contrast T1-weighted scans”);
(b) providing a deep learning network model (Guo, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image.”, Fig. 26A-26D) comprising a multi-contrast branched architecture (Guo, the example network shown in Fig. 26A-26D shows multi-contrast branched architecture) comprising a plurality of branches, wherein each branch of the plurality of branches is configured to process a different combination of contrasts (Guo, Fig. 26A-26D, each branch has different contrast); and
PNG
media_image2.png
435
600
media_image2.png
Greyscale
(c) taking the multi-contrast image (Guo, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image”) and a mask as input to a second deep network model (Guo, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image.”, [0089], “ground truth tumor masks were generated in addition to the brain masks using the FCM (e.g., Fuzzy-C-Means) Clustering Based Segmentation”, the tumor mask is used as a ground truth input to train the deep learning network model, [0157], “In order to accurately capture the high variance present within brain tumors, the exemplary DeepContrast model was trained with a large-scale brain tumor MM dataset, which again was able to generate contrast predictions that were similar to the ground truth”, [0178], “For scans of tumor subjects, the CBV maps and brain masks were derived using the same methods as descripted in the healthy mouse brain study, and tumor masks were generated in addition to the brain masks using the Fuzzy-C-Means segmentation. (See, e.g., Reference 104). 6 GBM subjects were added to the training set while 3 GBM subjects replaced the original testing set of the Healthy Mouse Brain Model.”) to generate a predicted image with improved quality (Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250. All the layers can include a residual connection 250; the decoding layers 230-250 can be implemented with attention mechanism. Various Max-pooling can be performed (e.g., with a factor of 2) as well as upsampling 255 also with a factor of 2. Additionally, Conv2D+Batch Normalization (“BN)+Rectified Linear Unit (“ReLu”) layers 260 can be included. The exemplary network can take both Pre and Low as the two-channel input or only Pre as the single-channel input. Estimated 2D CBV maps with brain extraction can be the output.”, [0112], “Both the Pre+Low and the Pre image can predict the tumor region with enhanced contrast and similar space feature by ResAttU-Net.”).
Guo does not explicitly disclose an anomaly mask generated using an unsupervised anomaly detection scheme as input to the deep network model to generate a predicted image with improved quality, wherein the anomaly mask is provided as an input to a branch of the plurality of branches, and wherein the predicted image with improved quality is generated based on multiple predictions generated by the plurality of branches.
However, Zaharchuk teaches an anomaly mask generated using an unsupervised anomaly detection scheme (Zaharchuk Fig. 3 non-enhancing mask, [0028], “Normalization is used to remove bias in the images obtained at different dosage levels, which is performed using a mask 314 of non-enhancing tissues. Specifically, to remove the systematic differences between different signal intensity levels in non-enhancing regions (such as scalp fat), co-registration and signal normalization based on average voxel value within a mask is performed. Alternatively, the normalization can be based on max/min intensity of the images or certain percentiles. Also, the normalization can be based on matching the distribution of intensity to certain predefined or standard distribution. Mask can be applied to calculate the average, max, min, percentile or the distribution to get better normalization performance to match different set of images.” as input to the deep network model to generate a predicted image with improved quality (Zaharchuk, Fig. 3, synthesized full-dose CE-MRI), wherein the anomaly mask is provided as an input to a branch of the plurality of branches (Zaharchuk, Fig. 3, the mask is input along with the three different contrast image that are on different branches), and wherein the predicted image with improved quality is generated based on multiple predictions generated by the plurality of branches (Zaharchuk, Fig. 3).
Guo and Zaharchuk are both considered to be analogous to the claimed invention because they are in the same field of MRI enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Zaharchuk of an anomaly mask generated using an unsupervised anomaly detection scheme as input to the deep network model to generate a predicted image with improved quality, wherein the anomaly mask is provided as an input to a branch of the plurality of branches, and wherein the predicted image with improved quality is generated based on multiple predictions generated by the plurality of branches. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to show different appearance/intensity of different tissues (Zaharchuk, [0050]).
Claim 16
The combination of Guo in view of Zaharchuk discloses the computer-implemented method of claim 15 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the multi-contrast branched architecture comprises a first branch configured to process the image of the first contrast and an image of a second contrast (Guo, Fig. 26A-26D, [0094], “The performance of the U-Net with only attention units (“AttU-Net”), U-Net with only residual units (“ResU-Net”) and the U-Net with both residual and attention unit (“ResAttU-Net”) were analyzed with the input of Pre+Low image”).
Claim 17
Guo discloses the computer-implemented method of claim 16 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”).
Guo does not explicitly disclose wherein the image of the first contrast is T1-weighted image and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI).
However, Zaharchuk teaches wherein the image of the first contrast is T1-weighted image (Zaharchuk, Fig. 3, [0026], “high-resolution T1-weighted IR-FSPGR pre-contrast images, post-contrast images with 10% low-dose and 100% full-dose of gadobenate dimeglumine (0.01 and 0.1 mmol/kg, respectively) full-dose images are acquired.”) and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI) (Zaharchuk, [0036], “The co-registered and normalized non-contrast (zero-dose) and low-dose images are loaded from DICOM files and input to the trained network “, [0050], “The zero dose images in this case include multiple MR images acquired using different sequences (e.g., T1w, T2w, FLAIR, DWI) that show different appearance/intensity of different tissues”)).
Guo and Zaharchuk are both considered to be analogous to the claimed invention because they are in the same field of MRI enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Zaharchuk wherein the image of the first contrast is T1-weighted image and the image of the second contrast is selected from the group consisting of T2-weighted image, fluid attenuated inversion recovery (FLAIR), proton density (PD), and diffusion weighted (DWI). Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to show different appearance/intensity of different tissues (Zaharchuk, [0050]).
Claim 18
The combination of Guo in view of Zaharchuk discloses the computer-implemented method of claim 16 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the multi-contrast branched architecture (Guo, Fig. 26A-26D) comprises a second branch to process the image of the first contrast (Guo, Fig. 26A-26D, [0095], “exemplary network can take both Pre and Low as the two-channel input”) and the anomaly mask (Guo, [0089], “ground truth tumor masks were generated in addition to the brain masks using the FCM (e.g., Fuzzy-C-Means) Clustering Based Segmentation”, the tumor mask is used as a ground truth input to train the deep learning network model, [0157], “In order to accurately capture the high variance present within brain tumors, the exemplary DeepContrast model was trained with a large-scale brain tumor MM dataset, which again was able to generate contrast predictions that were similar to the ground truth”, [0178], “For scans of tumor subjects, the CBV maps and brain masks were derived using the same methods as descripted in the healthy mouse brain study, and tumor masks were generated in addition to the brain masks using the Fuzzy-C-Means segmentation. (See, e.g., Reference 104). 6 GBM subjects were added to the training set while 3 GBM subjects replaced the original testing set of the Healthy Mouse Brain Model.”).
Claim 19
Guo discloses the computer-implemented method of claim 15 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”).
Guo does not explicitly disclose wherein the multi-contrast branched architecture comprises at least three branches.
However, Zaharchuk teaches wherein the multi-contrast branched architecture comprises at least three branches (Zaharchuk, Fig. 3, three branches for three different scans).
Guo and Zaharchuk are both considered to be analogous to the claimed invention because they are in the same field of MRI enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method as taught by Guo to incorporate the teachings of Zaharchuk wherein the multi-contrast branched architecture comprises at least three branches. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to show different appearance/intensity of different tissues (Zaharchuk, [0050]).
Claim 20
The combination of Guo in view of Zaharchuk discloses the computer-implemented method of claim 19 (Guo, Abstract, “method, and computer-accessible medium for generating a gadolinium (“Gd”) enhanced map(s) of a portion(s) of a patient(s)”, “The Gd enhanced map(s) can be a full dosage Gd enhanced map. The full dosage Gd enhanced map(s) can be a full dosage Gd enhanced cerebral blood volume map(s). The machine learning procedure can be a convolutional neural network.”, [0095], “As shown in FIG. 2, the exemplary architecture of ResAttU-Net can include of 5 encoding layers 205, 210, 215, 220, and 225 and 5 decoding layers 230, 235, 240, 245, and 250”), wherein the predicted image with improved quality is generated based on multiple predictions generated by the at least three branches (Zaharchuk, Fig.3).
The proposed combination as well as the motivation for combining the Guo and Zaharchuk references presented in the rejection of Claim 19, apply to Claim 20 and are incorporated herein by reference. Thus, the method recited in Claim 20 is met by Guo and Zaharchuk.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DENISE G ALFONSO whose telephone number is (571)272-1360. The examiner can normally be reached Monday - Friday 7:30 - 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DENISE G ALFONSO/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662