DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/07/2026 has been entered.
Claim Status
Claims 1-20 are pending for examination in the application filed 04/22/2026. Claims 1, 8, and 17 have been amended.
Priority
Acknowledgement is made of Applicant’s claim to priority of provisional application 63/376,529, filing date 09/21/2022.
Response to Arguments and Amendments
Applicant's arguments filed 04/22/2026 have been fully considered but they are not persuasive. Applicant argues on pages 8-9 of the Remarks that because Chen fails to teach the limitation of “the physics-guided data augmentation and network consistency concept configured to allow for all acquired data to be utilized for data consistency purposes and calculation of a loss function”, then Chen cannot possibly teach the newly added limitation of “further wherein the physics-guided data augmentation is based, at least in part, on the coil sensitivity maps”. Examiner disagrees. Hu teaches the limitation of “the physics-guided data augmentation and network consistency concept configured to allow for all acquired data to be utilized for data consistency purposes and calculation of a loss function”, as described on page 7 of the Final Rejection filed 03/03/2026. Chen teaches wherein the network structure includes physics-guided data augmentation and a network consistency concept, further wherein the physics-guided data augmentation is based, at least in part, on the coil sensitivity maps ([0029] FIG. 5 illustrates an example process 500 for training a neural network (e.g., an instance of the ANN 104 of FIG. 1 and/or ANN 304 of FIG. 3) to perform the multi-slice MRI data processing operations described herein. The training may be performed using data collected from practical MRI procedures (e.g., under-sampled multi-slice MRI data acquired using an SMS technique), and/or computer-simulated or computer-augmented MRI data. [0022] In examples, the ANN 304 may further include a data consistency (DC) checker 310 (e.g., as a layer of the ANN 304) that is configured to check and/or improve the fidelity of the MRI data predicted by the ANN 304. For example, the DC checker 310 may be configured to receive the MRI data produced by the Siamese network (e.g., denoised first and second intermediate MRI images respectively predicted by the sub-networks 308a and 308b based on the input), process the data (e.g., the first and second intermediate MRI images) to derive corresponding MRI (e.g., k-space data). [0005] In examples, the first under-sampled MRI data comprised in the SMS dataset may include MRI data that are acquired using a first set of one or more coils. The second under-sampled MRI data comprised in the SMS dataset may include MRI data acquired using a second set of one or more coils. In these examples, respective coil sensitivity maps associated with the first set of one or more coils and the second set of one or more coils may be determined and used to estimate the k-space data described above). Please see below for the complete updated 35 U.S.C. 103 rejections.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 3 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 3 recites the limitation "further comprising both a stop-gradient and an additional denoise block, wherein, optionally, the additional denoise block utilizes an additional Unet”. There is insufficient antecedent basis for this limitation in the claim. Claim 1, which claim 3 depends from, does not include a denoise block or a U-net, which makes reference to an additional denoise block and an additional U-net indefinite. Please clarify.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 6, 8, and 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Chen (US20230135995A1) in view of Hu (Hu, C., Li, C., Wang, H., Liu, Q., Zheng, H., Wang, S. (2021). Self-supervised Learning for MRI Reconstruction with a Parallel Network Training Framework. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12906. Springer, Cham).
Regarding claim 1, Chen teaches system to process images to improve the quality of MRI images ([0002] Described herein are systems, methods, and instrumentalities associated with reconstructing magnetic resonance imaging (MRI) images based on a simultaneous multi-slice (e.g., two or more) dataset comprising under-sampled MRI data (e.g., MRI imagery or k-space data). [0001] The collection of k-space data may be a slow process and, as such, under-sampling may be applied to accelerate the operation. The under-sampled k-space data may then be reconstructed (e.g., into an MRI image) to obtain results having a similar quality as a fully-sampled dataset (e.g., a fully-sample MRI image)), comprising:
an MRI ([0014] FIG. 1 is a block diagram illustrating an example system 100 for processing a simultaneous multi-slice (SMS) dataset 102 collected by a magnetic resonance imaging (MRI) device (e.g., an MRI scanner)); a processor (processor 602); and a memory, enabled to store data in electronic communication with the processor ([0034] The mass storage device 608 may include one or more magnetic disks such as one or more internal hard disks, one or more removable disks, one or more magneto-optical disks, one or more CD-ROM or DVD-ROM disks, etc., on which instructions and/or data may be stored to facilitate the operation of the processor 602),
wherein the memory is able to receive image data of a dynamic scene from the MRI ([0034] The mass storage device 608 may include one or more magnetic disks such as one or more internal hard disks, one or more removable disks, one or more magneto-optical disks, one or more CD-ROM or DVD-ROM disks, etc., on which instructions and/or data may be stored to facilitate the operation of the processor 602. [0014] The SMS dataset 102 may also include imagery data (e.g., one or more MRI images) that visually depicts the anatomical structure based on the k-space data collected by the MRI device. These images may include a single static image or multiple dynamic images (e.g., multi-contrast images) that may be derived, for example, by applying a Fourier transform (e.g., inverse fast Fourier transform (FFT)) to the collected k-space data),
and the processor is able to utilize a model based on a physics-guided Siamese network structure ([0018] Each of the multiple sub-networks may be trained to process a corresponding MRI slice included in the SMS dataset 302 and, together, the multiple sub-networks may be capable of learning (e.g., identifying) the similarities and/or dissimilarities of the different MRI slices included in the SMS dataset 302 and denoise (e.g., remove artifacts from) the SMS dataset 302 based on the learned (e.g., identified) similarities and/or dissimilarities. The example in FIG. 3 shows that the sub-networks (e.g., 308a and 308b) may be configured to form a Siamese neural network)
utilizing an encoding matrix with coil sensitivity maps and an undersampling mask that is converted to an intermediate fully sampled model deep learning that is then passed into a re-undersampling process and then reconstructed into unsampled k-space through a second model deep learning process ([0005] In examples, the first under-sampled MRI data comprised in the SMS dataset may include MRI data that are acquired using a first set of one or more coils. The second under-sampled MRI data comprised in the SMS dataset may include MRI data acquired using a second set of one or more coils. In these examples, respective coil sensitivity maps associated with the first set of one or more coils and the second set of one or more coils may be determined and used to estimate the k-space data described above. [0027] Once obtained, the coil sensitivity maps associated with the coils may be applied (e.g., by the ANN 304 and/or the DC checker 310) along with the Fourier transforms to reconstruct the multi-slice MRI data. For instance, MRI data (e.g., MRI images) associated with the multiple coils may be multiplied with corresponding complex conjugates of the coil sensitivity maps and then summed together to obtain coil-combined MRI images that may then be provided to the sub-networks 308a, 308b for denoising. Fig. 4 408: perform data consistency check and reconstruct k-space associated with the multi-slice MRI data based on the intermediate MRI image. [0004] In examples, the ANN described herein may further comprise a data consistency (DC) component configured to estimate k-space data based on a first intermediate image generated by the ANN using the first under-sampled MRI data and a second intermediate image generated by the ANN using the second under-sampled MRI data. The first reconstructed MRI image and the second reconstructed MRI image may then be generated by applying an inverse Fourier transform (e.g., a 3D fast Fourier transform (FFT)) to the estimated k-space data. In examples, prior to applying the inverse Fourier transform to the estimated k-space data, at least a portion of the estimated k-space data may be replaced with a corresponding portion of the SMS dataset);
wherein the network structure includes physics-guided data augmentation and a network consistency concept, further wherein the physics-guided data augmentation is based, at least in part, on the coil sensitivity maps ([0029] FIG. 5 illustrates an example process 500 for training a neural network (e.g., an instance of the ANN 104 of FIG. 1 and/or ANN 304 of FIG. 3) to perform the multi-slice MRI data processing operations described herein. The training may be performed using data collected from practical MRI procedures (e.g., under-sampled multi-slice MRI data acquired using an SMS technique), and/or computer-simulated or computer-augmented MRI data. [0022] In examples, the ANN 304 may further include a data consistency (DC) checker 310 (e.g., as a layer of the ANN 304) that is configured to check and/or improve the fidelity of the MRI data predicted by the ANN 304. For example, the DC checker 310 may be configured to receive the MRI data produced by the Siamese network (e.g., denoised first and second intermediate MRI images respectively predicted by the sub-networks 308a and 308b based on the input), process the data (e.g., the first and second intermediate MRI images) to derive corresponding MRI (e.g., k-space data). [0005] In examples, the first under-sampled MRI data comprised in the SMS dataset may include MRI data that are acquired using a first set of one or more coils. The second under-sampled MRI data comprised in the SMS dataset may include MRI data acquired using a second set of one or more coils. In these examples, respective coil sensitivity maps associated with the first set of one or more coils and the second set of one or more coils may be determined and used to estimate the k-space data described above).
Chen does not explicitly teach a self-supervised learning with self-supervised regularization model; the physics-guided data augmentation and network consistency concept configured to allow for all acquired data to be utilized for data consistency purposes and calculation of a loss function.
Hu, in the same field of endeavor of self-supervised learning for MRI, teaches a self-supervised learning with self-supervised regularization model ([Abstract] To address this issue, we propose a novel self supervised learning method. [2.1 Mathematical Model of CS-MRI Reconstruction] x is the desired image, y is the undersampled k-space measurement, A denotes the encoding matrix which include Fourier transform F and sampling matrix P, R(x) denotes the utilized regularization, and λ is the regularization parameter. The purpose of MRI reconstruction is to recover the desired image x from its measurement y);
the physics-guided data augmentation and network consistency concept configured to allow for all acquired data to be utilized for data consistency purposes and calculation of a loss function ([Abstract] Specifically, during model optimization, two subsets are constructed by randomly selecting part of k-space data from the undersampled data and then fed into two parallel reconstruction networks to perform information recovery. Two reconstruction losses are defined on all the scanned data points to enhance the network’s capability of recovering the frequency information. Meanwhile, to constrain the learned unscanned data points of the network, a difference loss is designed to enforce consistency between the two parallel networks).
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of Hu to use a self-supervised learning with self -supervised regularization model because "Image reconstruction from undersampled k-space data plays an important role in accelerating the acquisition of MR data, and a lot of deep learning-based methods have been exploited recently. Despite the achieved inspiring results, the optimization of these methods commonly relies on the fully-sampled reference data, which are time-consuming and difficult to collect…Experimental results demonstrate that the proposed self-supervised method can achieve competitive reconstruction performance compared to the corresponding supervised learning method at high acceleration rates (4 and 8)" [Abstract] and to configure the physics-guided data augmentation and network consistency concept to allow for all acquired data to be utilized for data consistency purposes and calculation of a loss function because "it is difficult to obtain fully-sampled data in many scenarios due to physiological constraints or physical constraints. Recently, a self-supervised learning method (self-supervised learning via data undersampling, SSDU) was proposed specifically to solve the issue, where the undersampled data is split into two disjoint sets. One is treated as the input and the other is used to define the loss. Despite the impressive reconstruction performance achieved, there are two important issues. First, the two sets need to be split with caution. When the second set does not contain enough data, the training process becomes unstable. Second, since no constraint is imposed on the unscanned data points, there is no guarantee that the final outputs are the expected high-quality images and high uncertainties exist" [Introduction].
Regarding claim 2, Chen and Hu teach the system of claim 1. Chen teaches a denoise block to control noise from undersampling, wherein, optionally, the denoise block utilizes Unet operating on the unsampled k-space through a second model deep learning process ([0022] For example, the DC checker 310 may be configured to receive the MRI data produced by the Siamese network (e.g., denoised first and second intermediate MRI images respectively predicted by the sub-networks 308a and 308b based on the input), process the data (e.g., the first and second intermediate MRI images) to derive corresponding MRI (e.g., k-space data), and obtain respective MRI images (e.g., disentangled MRI images 306a and 306b) corresponding to the multiple slices of the SMS dataset 302 based on the derived MRI data (e.g., by applying an inverse Fourier transform such as an inverse FFT to the derived MRI data)).
Regarding claim 6, Chen and Hu teach the system of claim 1. Chen further teaches wherein the re-undersampling includes an intermediate multicoil k-space followed by random undersampling followed by generating a coil combined image ([0005] In examples, the first under-sampled MRI data comprised in the SMS dataset may include MRI data that are acquired using a first set of one or more coils. The second under-sampled MRI data comprised in the SMS dataset may include MRI data acquired using a second set of one or more coils. In these examples, respective coil sensitivity maps associated with the first set of one or more coils and the second set of one or more coils may be determined and used to estimate the k-space data described above. [0027] Once obtained, the coil sensitivity maps associated with the coils may be applied (e.g., by the ANN 304 and/or the DC checker 310) along with the Fourier transforms to reconstruct the multi-slice MRI data. For instance, MRI data (e.g., MRI images) associated with the multiple coils may be multiplied with corresponding complex conjugates of the coil sensitivity maps and then summed together to obtain coil-combined MRI images that may then be provided to the sub-networks 308a, 308b for denoising. [0014] Such under-sampled k-space data may be characterized by a Cartesian or non-Cartesian trajectory, and/or may be collected using a uniform, random, or pseudo-random under-sampling technique).
Regarding claim 8, Chen teaches a system to process images, for single band and/or multiband acceleration, to improve the quality of MRI images ([0002] Described herein are systems, methods, and instrumentalities associated with reconstructing magnetic resonance imaging (MRI) images based on a simultaneous multi-slice (e.g., two or more) dataset comprising under-sampled MRI data (e.g., MRI imagery or k-space data). [0001] The collection of k-space data may be a slow process and, as such, under-sampling may be applied to accelerate the operation. The under-sampled k-space data may then be reconstructed (e.g., into an MRI image) to obtain results having a similar quality as a fully-sampled dataset (e.g., a fully-sample MRI image)), comprising:
an MRI ([0014] FIG. 1 is a block diagram illustrating an example system 100 for processing a simultaneous multi-slice (SMS) dataset 102 collected by a magnetic resonance imaging (MRI) device (e.g., an MRI scanner)); a processor (processor 602); and a memory, enabled to store data in electronic communication with the processor ([0034] The mass storage device 608 may include one or more magnetic disks such as one or more internal hard disks, one or more removable disks, one or more magneto-optical disks, one or more CD-ROM or DVD-ROM disks, etc., on which instructions and/or data may be stored to facilitate the operation of the processor 602),
wherein the memory is able to receive image data of a dynamic scene from the MRI ([0034] The mass storage device 608 may include one or more magnetic disks such as one or more internal hard disks, one or more removable disks, one or more magneto-optical disks, one or more CD-ROM or DVD-ROM disks, etc., on which instructions and/or data may be stored to facilitate the operation of the processor 602. [0014] The SMS dataset 102 may also include imagery data (e.g., one or more MRI images) that visually depicts the anatomical structure based on the k-space data collected by the MRI device. These images may include a single static image or multiple dynamic images (e.g., multi-contrast images) that may be derived, for example, by applying a Fourier transform (e.g., inverse fast Fourier transform (FFT)) to the collected k-space data),
and the processor is able to utilize a model based on a physics-guided Siamese network structure ([0018] Each of the multiple sub-networks may be trained to process a corresponding MRI slice included in the SMS dataset 302 and, together, the multiple sub-networks may be capable of learning (e.g., identifying) the similarities and/or dissimilarities of the different MRI slices included in the SMS dataset 302 and denoise (e.g., remove artifacts from) the SMS dataset 302 based on the learned (e.g., identified) similarities and/or dissimilarities. The example in FIG. 3 shows that the sub-networks (e.g., 308a and 308b) may be configured to form a Siamese neural network)
utilizing an encoding matrix with coil sensitivity maps and an undersampling mask that is converted to a first model deep learning block that communicates with a re-undersampling block and then with a plurality of physics guided subnets ([0005] In examples, the first under-sampled MRI data comprised in the SMS dataset may include MRI data that are acquired using a first set of one or more coils. The second under-sampled MRI data comprised in the SMS dataset may include MRI data acquired using a second set of one or more coils. In these examples, respective coil sensitivity maps associated with the first set of one or more coils and the second set of one or more coils may be determined and used to estimate the k-space data described above. [0027] Once obtained, the coil sensitivity maps associated with the coils may be applied (e.g., by the ANN 304 and/or the DC checker 310) along with the Fourier transforms to reconstruct the multi-slice MRI data. For instance, MRI data (e.g., MRI images) associated with the multiple coils may be multiplied with corresponding complex conjugates of the coil sensitivity maps and then summed together to obtain coil-combined MRI images that may then be provided to the sub-networks 308a, 308b for denoising. [0004] In examples, the ANN described herein may further comprise a data consistency (DC) component configured to estimate k-space data based on a first intermediate image generated by the ANN using the first under-sampled MRI data and a second intermediate image generated by the ANN using the second under-sampled MRI data. The first reconstructed MRI image and the second reconstructed MRI image may then be generated by applying an inverse Fourier transform (e.g., a 3D fast Fourier transform (FFT)) to the estimated k-space data. In examples, prior to applying the inverse Fourier transform to the estimated k-space data, at least a portion of the estimated k-space data may be replaced with a corresponding portion of the SMS dataset);
wherein the network structure includes physics-guided data augmentation and a network consistency concept, further wherein the physics-guided data augmentation is based, at least in part, on the coil sensitivity maps ([0029] FIG. 5 illustrates an example process 500 for training a neural network (e.g., an instance of the ANN 104 of FIG. 1 and/or ANN 304 of FIG. 3) to perform the multi-slice MRI data processing operations described herein. The training may be performed using data collected from practical MRI procedures (e.g., under-sampled multi-slice MRI data acquired using an SMS technique), and/or computer-simulated or computer-augmented MRI data. [0022] In examples, the ANN 304 may further include a data consistency (DC) checker 310 (e.g., as a layer of the ANN 304) that is configured to check and/or improve the fidelity of the MRI data predicted by the ANN 304. For example, the DC checker 310 may be configured to receive the MRI data produced by the Siamese network (e.g., denoised first and second intermediate MRI images respectively predicted by the sub-networks 308a and 308b based on the input), process the data (e.g., the first and second intermediate MRI images) to derive corresponding MRI (e.g., k-space data). [0005] In examples, the first under-sampled MRI data comprised in the SMS dataset may include MRI data that are acquired using a first set of one or more coils. The second under-sampled MRI data comprised in the SMS dataset may include MRI data acquired using a second set of one or more coils. In these examples, respective coil sensitivity maps associated with the first set of one or more coils and the second set of one or more coils may be determined and used to estimate the k-space data described above).
Chen does not explicitly teach a self-supervised learning with self-supervised regularization model; the physics-guided data augmentation and network consistency concept configured to allow for all acquired data to be utilized for data consistency purposes and calculation of a loss function.
Hu, in the same field of endeavor of self-supervised learning for MRI, teaches a self-supervised learning with self-supervised regularization model ([Abstract] To address this issue, we propose a novel self supervised learning method. [2.1 Mathematical Model of CS-MRI Reconstruction] x is the desired image, y is the undersampled k-space measurement, A denotes the encoding matrix which include Fourier transform F and sampling matrix P, R(x) denotes the utilized regularization, and λ is the regularization parameter. The purpose of MRI reconstruction is to recover the desired image x from its measurement y);
the physics-guided data augmentation and network consistency concept configured to allow for all acquired data to be utilized for data consistency purposes and calculation of a loss function ([Abstract] Specifically, during model optimization, two subsets are constructed by randomly selecting part of k-space data from the undersampled data and then fed into two parallel reconstruction networks to perform information recovery. Two reconstruction losses are defined on all the scanned data points to enhance the network’s capability of recovering the frequency information. Meanwhile, to constrain the learned unscanned data points of the network, a difference loss is designed to enforce consistency between the two parallel networks).
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of Hu to use a self-supervised learning with self -supervised regularization model because "Image reconstruction from undersampled k-space data plays an important role in accelerating the acquisition of MR data, and a lot of deep learning-based methods have been exploited recently. Despite the achieved inspiring results, the optimization of these methods commonly relies on the fully-sampled reference data, which are time-consuming and difficult to collect…Experimental results demonstrate that the proposed self-supervised method can achieve competitive reconstruction performance compared to the corresponding supervised learning method at high acceleration rates (4 and 8)" [Abstract] and to configure the physics-guided data augmentation and network consistency concept to allow for all acquired data to be utilized for data consistency purposes and calculation of a loss function because "it is difficult to obtain fully-sampled data in many scenarios due to physiological constraints or physical constraints. Recently, a self-supervised learning method (self-supervised learning via data undersampling, SSDU) was proposed specifically to solve the issue, where the undersampled data is split into two disjoint sets. One is treated as the input and the other is used to define the loss. Despite the impressive reconstruction performance achieved, there are two important issues. First, the two sets need to be split with caution. When the second set does not contain enough data, the training process becomes unstable. Second, since no constraint is imposed on the unscanned data points, there is no guarantee that the final outputs are the expected high-quality images and high uncertainties exist" [Introduction].
Regarding claim 15, Chen and Hu teach the system of claim 8. Chen further teaches wherein the re-undersampling includes an intermediate multicoil k-space followed by random undersampling that utilizes a design comparable to the original undersampling mask followed by generating a coil combined image ([0005] In examples, the first under-sampled MRI data comprised in the SMS dataset may include MRI data that are acquired using a first set of one or more coils. The second under-sampled MRI data comprised in the SMS dataset may include MRI data acquired using a second set of one or more coils. In these examples, respective coil sensitivity maps associated with the first set of one or more coils and the second set of one or more coils may be determined and used to estimate the k-space data described above. [0027] Once obtained, the coil sensitivity maps associated with the coils may be applied (e.g., by the ANN 304 and/or the DC checker 310) along with the Fourier transforms to reconstruct the multi-slice MRI data. For instance, MRI data (e.g., MRI images) associated with the multiple coils may be multiplied with corresponding complex conjugates of the coil sensitivity maps and then summed together to obtain coil-combined MRI images that may then be provided to the sub-networks 308a, 308b for denoising. [0014] Such under-sampled k-space data may be characterized by a Cartesian or non-Cartesian trajectory, and/or may be collected using a uniform, random, or pseudo-random under-sampling technique).
Regarding claim 16, Chen and Hu teach the system of claim 8. Chen further teaches a physics-guided Siamese network, which includes physics-guided data augmentation, and a physics-guided network consistency concept that is included in a loss function ([0018] Each of the multiple sub-networks may be trained to process a corresponding MRI slice included in the SMS dataset 302 and, together, the multiple sub-networks may be capable of learning (e.g., identifying) the similarities and/or dissimilarities of the different MRI slices included in the SMS dataset 302 and denoise (e.g., remove artifacts from) the SMS dataset 302 based on the learned (e.g., identified) similarities and/or dissimilarities. The example in FIG. 3 shows that the sub-networks (e.g., 308a and 308b) may be configured to form a Siamese neural network. [0029] FIG. 5 illustrates an example process 500 for training a neural network (e.g., an instance of the ANN 104 of FIG. 1 and/or ANN 304 of FIG. 3) to perform the multi-slice MRI data processing operations described herein. The training may be performed using data collected from practical MRI procedures (e.g., under-sampled multi-slice MRI data acquired using an SMS technique), and/or computer-simulated or computer-augmented MRI data. [0002] The training may further include determining a combined training loss (e.g., such as an average loss, a triplet loss, etc.) by jointly considering a first training loss associated with the first estimated MRI image and a second training loss associated with the second estimated MRI image, and adjusting parameters of the instance of the ANN based on a gradient descent of the combined training loss).
Chen does not explicitly teach the self- supervised learning with self-supervised regularization model.
Hu, in the same field of endeavor of self-supervised learning for MRI, teaches the self-supervised learning with self-supervised regularization model ([Abstract] To address this issue, we propose a novel self supervised learning method. [2.1 Mathematical Model of CS-MRI Reconstruction] x is the desired image, y is the undersampled k-space measurement, A denotes the encoding matrix which include Fourier transform F and sampling matrix P, R(x) denotes the utilized regularization, and λ is the regularization parameter. The purpose of MRI reconstruction is to recover the desired image x from its measurement y);
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of Hu to use a self-supervised learning with self -supervised regularization model because "Image reconstruction from undersampled k-space data plays an important role in accelerating the acquisition of MR data, and a lot of deep learning-based methods have been exploited recently. Despite the achieved inspiring results, the optimization of these methods commonly relies on the fully-sampled reference data, which are time-consuming and difficult to collect…Experimental results demonstrate that the proposed self-supervised method can achieve competitive reconstruction performance compared to the corresponding supervised learning method at high acceleration rates (4 and 8)" [Abstract].
Regarding claim 17, Chen teaches a method for processing images, that are single and/or multiband, to improve the quality of MRI images ([0002] Described herein are systems, methods, and instrumentalities associated with reconstructing magnetic resonance imaging (MRI) images based on a simultaneous multi-slice (e.g., two or more) dataset comprising under-sampled MRI data (e.g., MRI imagery or k-space data). [0001] The collection of k-space data may be a slow process and, as such, under-sampling may be applied to accelerate the operation. The under-sampled k-space data may then be reconstructed (e.g., into an MRI image) to obtain results having a similar quality as a fully-sampled dataset (e.g., a fully-sample MRI image)), comprising:
utilizing a processor in electronic communication with a memory ([0034] The mass storage device 608 may include one or more magnetic disks such as one or more internal hard disks, one or more removable disks, one or more magneto-optical disks, one or more CD-ROM or DVD-ROM disks, etc., on which instructions and/or data may be stored to facilitate the operation of the processor 602), , wherein the memory is able to receive image data of an image from an MRI ([0014] The SMS dataset 102 may also include imagery data (e.g., one or more MRI images) that visually depicts the anatomical structure based on the k-space data collected by the MRI device. These images may include a single static image or multiple dynamic images (e.g., multi-contrast images) that may be derived, for example, by applying a Fourier transform (e.g., inverse fast Fourier transform (FFT)) to the collected k-space data),
and utilizing the processor with a model based on a physics-guided Siamese network structure ([0018] Each of the multiple sub-networks may be trained to process a corresponding MRI slice included in the SMS dataset 302 and, together, the multiple sub-networks may be capable of learning (e.g., identifying) the similarities and/or dissimilarities of the different MRI slices included in the SMS dataset 302 and denoise (e.g., remove artifacts from) the SMS dataset 302 based on the learned (e.g., identified) similarities and/or dissimilarities. The example in FIG. 3 shows that the sub-networks (e.g., 308a and 308b) may be configured to form a Siamese neural network)
utilizing an encoding matrix with coil sensitivity maps and an undersampling mask that is converted to a first model deep learning block that communicates with a re-undersampling block and then with a plurality of physics guided subnets ([0005] In examples, the first under-sampled MRI data comprised in the SMS dataset may include MRI data that are acquired using a first set of one or more coils. The second under-sampled MRI data comprised in the SMS dataset may include MRI data acquired using a second set of one or more coils. In these examples, respective coil sensitivity maps associated with the first set of one or more coils and the second set of one or more coils may be determined and used to estimate the k-space data described above. [0027] Once obtained, the coil sensitivity maps associated with the coils may be applied (e.g., by the ANN 304 and/or the DC checker 310) along with the Fourier transforms to reconstruct the multi-slice MRI data. For instance, MRI data (e.g., MRI images) associated with the multiple coils may be multiplied with corresponding complex conjugates of the coil sensitivity maps and then summed together to obtain coil-combined MRI images that may then be provided to the sub-networks 308a, 308b for denoising. [0004] In examples, the ANN described herein may further comprise a data consistency (DC) component configured to estimate k-space data based on a first intermediate image generated by the ANN using the first under-sampled MRI data and a second intermediate image generated by the ANN using the second under-sampled MRI data. The first reconstructed MRI image and the second reconstructed MRI image may then be generated by applying an inverse Fourier transform (e.g., a 3D fast Fourier transform (FFT)) to the estimated k-space data. In examples, prior to applying the inverse Fourier transform to the estimated k-space data, at least a portion of the estimated k-space data may be replaced with a corresponding portion of the SMS dataset);
wherein the network structure includes physics-guided data augmentation and a network consistency concept, further wherein the physics-guided data augmentation is based, at least in part, on the coil sensitivity maps ([0029] FIG. 5 illustrates an example process 500 for training a neural network (e.g., an instance of the ANN 104 of FIG. 1 and/or ANN 304 of FIG. 3) to perform the multi-slice MRI data processing operations described herein. The training may be performed using data collected from practical MRI procedures (e.g., under-sampled multi-slice MRI data acquired using an SMS technique), and/or computer-simulated or computer-augmented MRI data. [0022] In examples, the ANN 304 may further include a data consistency (DC) checker 310 (e.g., as a layer of the ANN 304) that is configured to check and/or improve the fidelity of the MRI data predicted by the ANN 304. For example, the DC checker 310 may be configured to receive the MRI data produced by the Siamese network (e.g., denoised first and second intermediate MRI images respectively predicted by the sub-networks 308a and 308b based on the input), process the data (e.g., the first and second intermediate MRI images) to derive corresponding MRI (e.g., k-space data). [0005] In examples, the first under-sampled MRI data comprised in the SMS dataset may include MRI data that are acquired using a first set of one or more coils. The second under-sampled MRI data comprised in the SMS dataset may include MRI data acquired using a second set of one or more coils. In these examples, respective coil sensitivity maps associated with the first set of one or more coils and the second set of one or more coils may be determined and used to estimate the k-space data described above).
Chen does not explicitly teach a self-supervised learning with self-supervised regularization model; the physics-guided data augmentation and network consistency concept configured to allow for all acquired data to be utilized for data consistency purposes and calculation of a loss function.
Hu, in the same field of endeavor of self-supervised learning for MRI, teaches a self-supervised learning with self-supervised regularization model ([Abstract] To address this issue, we propose a novel self supervised learning method. [2.1 Mathematical Model of CS-MRI Reconstruction] x is the desired image, y is the undersampled k-space measurement, A denotes the encoding matrix which include Fourier transform F and sampling matrix P, R(x) denotes the utilized regularization, and λ is the regularization parameter. The purpose of MRI reconstruction is to recover the desired image x from its measurement y);
the physics-guided data augmentation and network consistency concept configured to allow for all acquired data to be utilized for data consistency purposes and calculation of a loss function ([Abstract] Specifically, during model optimization, two subsets are constructed by randomly selecting part of k-space data from the undersampled data and then fed into two parallel reconstruction networks to perform information recovery. Two reconstruction losses are defined on all the scanned data points to enhance the network’s capability of recovering the frequency information. Meanwhile, to constrain the learned unscanned data points of the network, a difference loss is designed to enforce consistency between the two parallel networks).
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the method of Chen with the teachings of Hu to use a self-supervised learning with self -supervised regularization model because "Image reconstruction from undersampled k-space data plays an important role in accelerating the acquisition of MR data, and a lot of deep learning-based methods have been exploited recently. Despite the achieved inspiring results, the optimization of these methods commonly relies on the fully-sampled reference data, which are time-consuming and difficult to collect…Experimental results demonstrate that the proposed self-supervised method can achieve competitive reconstruction performance compared to the corresponding supervised learning method at high acceleration rates (4 and 8)" [Abstract] and to configure the physics-guided data augmentation and network consistency concept to allow for all acquired data to be utilized for data consistency purposes and calculation of a loss function because "it is difficult to obtain fully-sampled data in many scenarios due to physiological constraints or physical constraints. Recently, a self-supervised learning method (self-supervised learning via data undersampling, SSDU) was proposed specifically to solve the issue, where the undersampled data is split into two disjoint sets. One is treated as the input and the other is used to define the loss. Despite the impressive reconstruction performance achieved, there are two important issues. First, the two sets need to be split with caution. When the second set does not contain enough data, the training process becomes unstable. Second, since no constraint is imposed on the unscanned data points, there is no guarantee that the final outputs are the expected high-quality images and high uncertainties exist" [Introduction].
Claims 4-5 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Hu and Sandino (US20220375141A1).
Regarding claim 4, Chen and Hu teach the system of claim 1. Chen does not explicitly teach wherein the process utilizes deep learning priors.
Sandino, in the same field of endeavor of MRI reconstruction, teaches wherein the process utilizes deep learning priors.
PNG
media_image1.png
327
591
media_image1.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of Sandino to use deep learning priors for "processing the spatial basis functions and temporal basis functions to produce reconstructed spatial basis functions and reconstructed temporal basis functions, wherein the processing iteratively applies conjugate gradient and convolutional neural network updates using 2D or 3D spatial and 1D temporal networks; and decompressing the reconstructed spatial basis functions and reconstructed temporal basis functions to produce a reconstructed MRI image having one or more temporal dimensions and two or more spatial dimensions" [0013].
Regarding claim 5, Chen, Hu, and Sandino teach the system of claim 4. Chen further teaches the system to process images, that are single band and/or multiband, to improve the quality of MRI images ([0002] Described herein are systems, methods, and instrumentalities associated with reconstructing magnetic resonance imaging (MRI) images based on a simultaneous multi-slice (e.g., two or more) dataset comprising under-sampled MRI data (e.g., MRI imagery or k-space data).
Chen does not explicitly teach wherein the deep learning priors include a physics-guided network that uses a ResNet structure with a predetermined number of residual blocks and is unrolled for a predetermined number of iterations.
Sandino, in the same field of endeavor of MRI reconstruction, teaches wherein the deep learning priors include a physics-guided network that uses a ResNet structure with a predetermined number of residual blocks and is unrolled for a predetermined number of iterations (See Fig. 2A-2B. [0042] Embodiments of the invention may use different neural network architectures to reconstruct basis functions. These network architectures can include residual networks (ResNets). [0032] We compare three different UNN methods with respect to reconstruction speed and standard image quality metrics (PSNR, SSIM): [0033] 1. MoDL [11]: Unrolled half quadratic splitting network with five outer-loop iterations containing 2D+time residual networks [12], 64 features/convolution. [0034] Ten inner-loop (CG) iterations are used to perform each model inversion. This network acts on the raw data, and does not perform any sort of compression. [0035] 2. DSLR: Unrolled AltMin-PGD network with five iterations containing 2D spatial and 1D temporal residual networks, 256 features/convolution, 8 basis functions, overlapping blocks of size 16×16. [0036] 3. DSLR+: Unrolled AltMin-CG network with five outer-loop iterations containing 2D spatial and 1D temporal residual networks, 256 features/convolution, 8 basis functions, overlapping blocks of size 16×16. Ten inner-loop (CG) iterations are used to perform each model inversion).
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of Sandino to use deep learning priors including a physics guided network that uses ResNet structure with a predetermined number of residual blocks and is unrolled for a predetermined number of iterations because "These complex-valued basis functions are iteratively processed by DSLR+ network 214 composed of K iterations, each iteration 216 containing alternating conjugate gradient (CG) updates 218, 220 and CNN updates 222, 224…At the end of the network, the final basis functions Lb(K) 226 and Rb(K) 228 are combined to form the output images 230" [0030] and "for integrating MRI physics-based modeling to constrain the compressed representation of the data during reconstruction (also known as data consistency)" [0008].
Regarding claim 7, Chen and Hu teach the system of claim 1. Chen further teaches wherein the intermediate fully sampled model deep learning includes a series of iterations for image reconstruction followed by data consistency analysis ([0031] For example, it may be determined that the training termination criteria are satisfied if the combined training loss is below a predetermined threshold, if changes in the respective combined training losses between two training iterations (e.g., between consecutive training iterations) are below a predetermined threshold, etc. [0004] In examples, the ANN described herein may further comprise a data consistency (DC) component configured to estimate k-space data based on a first intermediate image generated by the ANN using the first under-sampled MRI data and a second intermediate image generated by the ANN using the second under-sampled MRI data).
Chen does not explicitly teach includes a series of iterations, each including ResNet for deep residual learning for image reconstruction.
Sandino, in the same field of endeavor of MRI reconstruction, teaches includes a series of iterations, each including ResNet for deep residual learning for image reconstruction ([0030] FIGS. 2A and 2B show a DSLR+ network architecture according to an embodiment of the invention…For simplicity, 2D and 1D residual networks (ResNets) 222, 224 comprised of 6 convolutions each are used to model the proximal updates. At the end of the network, the final basis functions L.sub.b.sup.(K) 226 and R.sub.b.sup.(K) 228 are combined to form the output images 230).
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of Sandino to use ResNet for image reconstruction "for integrating MRI physics-based modeling to constrain the compressed representation of the data during reconstruction (also known as data consistency)" [0008].
Claims 9-12 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Hu and He (Chen, X., & He, K. (2021). Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 15750-15758)).
Regarding claim 9, Chen and Hu teach the system of claim 8. Chen does not explicitly teach wherein the first model deep learning block includes a stop-gradient.
He, in the same field of endeavor of Siamese networks, teaches, wherein the first model deep learning block includes a stop-gradient.
PNG
media_image2.png
439
571
media_image2.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of He to use a stop-gradient because "collapsing solutions do exist, but a stop-gradient operation (Figure 1) is critical to prevent such solutions" [pg. 15750 para. 4].
Regarding claim 10, Chen and Hu teach the system of claim 8. Chen further teaches wherein the plurality of physics-guided subnets includes one block with backpropagation ([0031] If the determination at 512 is that the training termination criteria are not satisfied, the neural network may at 516 adjust its parameters by backpropagating the training loss (e.g., based on a gradient descent associated with the training loss) through the neural network).
Chen does not explicitly teach the remainder of physics-guided subnets include a stop-gradient with shared weights with only one subnet updating weights during backpropagation and the other subnets using stop-gradient to prevent collapsing.
He, in the same field of endeavor of Siamese networks, teaches the remainder of physics-guided subnets include a stop-gradient with shared weights with only one subnet updating weights during backpropagation and the other subnets using stop-gradient to prevent collapsing (See Fig. 1. [pg. 15751 para. 8] Our empirical study challenges the necessity of the momentum encoder for preventing collapsing. We discover that the stop-gradient operation is critical. This discovery can be obscured with the usage of a momentum encoder, which is always accompanied with stop-gradient (as it is not updated by its parameters’ gradients)).
PNG
media_image3.png
707
656
media_image3.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of He to use a stop-gradient because "collapsing solutions do exist, but a stop-gradient operation (Figure 1) is critical to prevent such solutions" [pg. 15750 para. 4].
Regarding claim 11, Chen, Hu, and He teach the system of claim 10. Chen further teaches wherein the physics-guided subnet that includes one block with backpropagation includes a second model deep learning block connected to a denoise block ([0022] For example, the DC checker 310 may be configured to receive the MRI data produced by the Siamese network (e.g., denoised first and second intermediate MRI images respectively predicted by the sub-networks 308a and 308b based on the input), process the data (e.g., the first and second intermediate MRI images) to derive corresponding MRI (e.g., k-space data), and obtain respective MRI images (e.g., disentangled MRI images 306a and 306b) corresponding to the multiple slices of the SMS dataset 302 based on the derived MRI data (e.g., by applying an inverse Fourier transform such as an inverse FFT to the derived MRI data)).
Regarding claim 12, Chen, Hu, and He teach the system of claim 11. Chen further teaches wherein the plurality of physics-guided subnets includes a third model deep learning block connected to a denoise block ([0018] In examples, the ANN 304 may include multiple (e.g., two or more) sub-networks (e.g., 308a and 308b shown in FIG. 3) having identical or substantially similar structures (e.g., in terms of the number of layers, types of layers, number of feature maps or vectors generated by each network, etc.) and/or identical or substantially similar operating parameters (e.g., weights associated with the kernels or filters of each network). Each of the multiple sub-networks may be trained to process a corresponding MRI slice included in the SMS dataset 302 and, together, the multiple sub-networks may be capable of learning (e.g., identifying) the similarities and/or dissimilarities of the different MRI slices included in the SMS dataset 302 and denoise (e.g., remove artifacts from) the SMS dataset 302 based on the learned (e.g., identified) similarities and/or dissimilarities).
Chen does not explicitly teach a stop gradient.
He, in the same field of endeavor of Siamese networks, teaches a stop gradient (Fig. 1).
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of He to use a stop-gradient because "collapsing solutions do exist, but a stop-gradient operation (Figure 1) is critical to prevent such solutions" [pg. 15750 para. 4].
Regarding claim 18, Chen and Hu teach the method of claim 17. Chen further teaches wherein the plurality of physics-guided subnets includes one block with backpropagation ([0031] If the determination at 512 is that the training termination criteria are not satisfied, the neural network may at 516 adjust its parameters by backpropagating the training loss (e.g., based on a gradient descent associated with the training loss) through the neural network).
Chen does not explicitly teach the remainder with stop-gradient with shared weights with only one subnet updating weights during backpropagation and the other subnets using stop-gradient to prevent collapsing.
He, in the same field of endeavor of Siamese networks, teaches the remainder with stop-gradient with shared weights with only one subnet updating weights during backpropagation and the other subnets using stop-gradient to prevent collapsing (See Fig. 1. [pg. 15751 para. 8] Our empirical study challenges the necessity of the momentum encoder for preventing collapsing. We discover that the stop-gradient operation is critical. This discovery can be obscured with the usage of a momentum encoder, which is always accompanied with stop-gradient (as it is not updated by its parameters’ gradients)).
PNG
media_image3.png
707
656
media_image3.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the method of Chen with the teachings of He to use a stop-gradient because "collapsing solutions do exist, but a stop-gradient operation (Figure 1) is critical to prevent such solutions" [pg. 15750 para. 4].
Regarding claim 19, Chen, Hu, and He teach the method of claim 18. Chen further teaches wherein the physics-guided subnet that includes one block with backpropagation includes a second model deep learning block connected to a denoise block ([0022] For example, the DC checker 310 may be configured to receive the MRI data produced by the Siamese network (e.g., denoised first and second intermediate MRI images respectively predicted by the sub-networks 308a and 308b based on the input), process the data (e.g., the first and second intermediate MRI images) to derive corresponding MRI (e.g., k-space data), and obtain respective MRI images (e.g., disentangled MRI images 306a and 306b) corresponding to the multiple slices of the SMS dataset 302 based on the derived MRI data (e.g., by applying an inverse Fourier transform such as an inverse FFT to the derived MRI data)).
and the plurality of physics guided subnets with stop gradient includes a third model deep learning block connected to a denoise block ([0018] In examples, the ANN 304 may include multiple (e.g., two or more) sub-networks (e.g., 308a and 308b shown in FIG. 3) having identical or substantially similar structures (e.g., in terms of the number of layers, types of layers, number of feature maps or vectors generated by each network, etc.) and/or identical or substantially similar operating parameters (e.g., weights associated with the kernels or filters of each network). Each of the multiple sub-networks may be trained to process a corresponding MRI slice included in the SMS dataset 302 and, together, the multiple sub-networks may be capable of learning (e.g., identifying) the similarities and/or dissimilarities of the different MRI slices included in the SMS dataset 302 and denoise (e.g., remove artifacts from) the SMS dataset 302 based on the learned (e.g., identified) similarities and/or dissimilarities).
Chen does not explicitly teach a stop gradient.
He, in the same field of endeavor of Siamese networks, teaches a stop gradient (Fig. 1).
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the method of Chen with the teachings of He to use a stop-gradient because "collapsing solutions do exist, but a stop-gradient operation (Figure 1) is critical to prevent such solutions" [pg. 15750 para. 4].
Claims 13 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Hu, He, and Sandino.
Regarding claim 13, Chen, Hu, and He teach the system of claim 12. Chen further teaches wherein the first model deep learning block, the second model deep learning block, and the third model deep learning block includes a series of iterations including data consistency block for image reconstruction followed by data consistency analysis ([0031] For example, it may be determined that the training termination criteria are satisfied if the combined training loss is below a predetermined threshold, if changes in the respective combined training losses between two training iterations (e.g., between consecutive training iterations) are below a predetermined threshold, etc. [0004] In examples, the ANN described herein may further comprise a data consistency (DC) component configured to estimate k-space data based on a first intermediate image generated by the ANN using the first under-sampled MRI data and a second intermediate image generated by the ANN using the second under-sampled MRI data).
Chen does not explicitly teach includes a series of iterations each including an unrolled network including a ResNet for deep residual learning for image reconstruction.
Sandino, in the same field of endeavor of MRI reconstruction, teaches includes a series of iterations each including an unrolled network including a ResNet for deep residual learning for image reconstruction ([0030] FIGS. 2A and 2B show a DSLR+ network architecture according to an embodiment of the invention…For simplicity, 2D and 1D residual networks (ResNets) 222, 224 comprised of 6 convolutions each are used to model the proximal updates. At the end of the network, the final basis functions L.sub.b.sup.(K) 226 and R.sub.b.sup.(K) 228 are combined to form the output images 230. [0035] 2. DSLR: Unrolled AltMin-PGD network with five iterations containing 2D spatial and 1D temporal residual networks, 256 features/convolution, 8 basis functions, overlapping blocks of size 16×16).
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of Sandino to use ResNet for image reconstruction "for integrating MRI physics-based modeling to constrain the compressed representation of the data during reconstruction (also known as data consistency)" [0008].
Regarding claim 20, Chen, Hu, and He teach the method of claim 19. Chen further teaches wherein the first model deep learning block, the second model deep learning block, and the third model deep learning block includes a series of iterations for image reconstruction followed by data consistency analysis ([0031] For example, it may be determined that the training termination criteria are satisfied if the combined training loss is below a predetermined threshold, if changes in the respective combined training losses between two training iterations (e.g., between consecutive training iterations) are below a predetermined threshold, etc. [0004] In examples, the ANN described herein may further comprise a data consistency (DC) component configured to estimate k-space data based on a first intermediate image generated by the ANN using the first under-sampled MRI data and a second intermediate image generated by the ANN using the second under-sampled MRI data).
Chen does not explicitly teach includes a series of iterations each including ResNet for deep residual learning for image reconstruction.
Sandino, in the same field of endeavor of MRI reconstruction, teaches includes a series of iterations, each including ResNet for deep residual learning for image reconstruction ([0030] FIGS. 2A and 2B show a DSLR+ network architecture according to an embodiment of the invention…For simplicity, 2D and 1D residual networks (ResNets) 222, 224 comprised of 6 convolutions each are used to model the proximal updates. At the end of the network, the final basis functions L.sub.b.sup.(K) 226 and R.sub.b.sup.(K) 228 are combined to form the output images 230).
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the method of Chen with the teachings of Sandino to use ResNet for image reconstruction "for integrating MRI physics-based modeling to constrain the compressed representation of the data during reconstruction (also known as data consistency)" [0008].
Claims 3 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Hu, He, and Mailhe (US11422217B2).
Regarding claim 3, Chen and Hu teach the system of claim 1. Chen does not explicitly teach further comprising both a stop-gradient and an additional Unet, wherein optionally, the additional denoise block utilizes an additional Unet.
He, in the same field of endeavor of Siamese networks, teaches a stop-gradient (Fig. 1).
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of He to use a stop-gradient because "collapsing solutions do exist, but a stop-gradient operation (Figure 1) is critical to prevent such solutions" [pg. 15750 para. 4].
Mailhe, in the same field of endeavor of MRI acceleration, teaches an additional Unet, wherein optionally, the additional denoise block utilizes an additional Unet ([col. 8 ln. 21-29] Alternatively, a different regularizer (i.e., generator of PGAN) is provided for each iteration. Different PGANs are trained for different iterations in the reconstruction. Each generator and/or PGAN may have the same architecture, but each is separately learned so that different values of the learnable parameters may be provided for different iterations of the reconstruction. Each generator for each reconstruction iteration is progressively trained, such as training separate image-to-image networks. [col. 10 ln. 37-39] The generator 301 is an image-to-image network which receives an input image 300 and outputs an image 328. Any image-to-image network may be used, such as a U-net.
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of Mailhe "so that different values of the learnable parameters may be provided for different iterations of the reconstruction" [Mailhe col. 8 ln 25-27].
Regarding claim 14, Chen, Hu, and He teach the system of claim 12. Chen does not explicitly teach wherein the denoise block includes controlling noise from undersampling with Unet operating on the unsampled k-space through a model deep learning process.
Mailhe, in the same field of endeavor of MRI acceleration, teaches wherein the denoise block includes controlling noise from undersampling with Unet operating on the unsampled k-space through a model deep learning process ([col. 10 ln. 26-39] In one embodiment, the GAN being progressively trained is an image-to-image network trained to act as a regularizer in the reconstruction. PGAN is adapted into the image-to-image neural network architecture. FIG. 3 shows an example. FIG. 3 shows a GAN formed by the generator 301 and the discriminator 330. The generator 301 receives the image 300 (e.g., data representing the patient in the object or image domain) and outputs a denoised or regularized image 328. The discriminator 330 determines whether the image 328 is estimated (i.e., made up by the generator 301) or is an actual image without noise or artifact. The generator 301 is an image-to-image network which receives an input image 300 and outputs an image 328. Any image-to-image network may be used, such as a U-net. [col. 6 ln. 47-56] A is the MRI model to connect the image to MRI-space (k-space), which can involve a combination of an under-sampling matrix U, a Fourier transform F, and sensitivity maps S. T represents a sparsifying (shrinkage) transform. λ is a regularization parameter. The first term of the right side of equation 1 represents the image (2D or 3D spatial distribution or representation) fit to the acquired data, and the second term of the right side is a term added for denoising by reduction of artifacts (e.g., aliasing) due to under sampling).
Therefore, it would have been obvious to a person of ordinary skill in the art before the time of filing to modify the system of Chen with the teachings of Mailhe to use a denoise block "for reconstruction of a magnetic resonance (MR) image in an MR system" [Mailhe col. 1 ln. 65-66].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jacqueline R Zak whose telephone number is (571)272-4077. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACQUELINE R ZAK/Examiner, Art Unit 2666
/Molly Wilburn/Primary Examiner, Art Unit 2666
Prosecution Insights