DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, 7-8, 10, 13, and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao “SMORE: A Self-Supervised Anti-Aliasing and Super-Resolution Algorithm for MRI Using Deep Learning” (included in the IDS) (hereinafter “Zhao”) in view of Yatsuo US20230103170 (hereinafter “Yatsuo”).
Regarding claim 1, Zhao teaches a system for super-resolution of magnetic resonance (MR) images, the system comprising (see page 807 Section II, the algorithm Synthetic Multi-Orientation Resolution Enhancement (SMORE) which uses a super-resolution network for use of MRI):
an input for receiving a two-dimensional (2D) multi-slice MR dataset of a subject (see page 810 col 1 and algorithm 3, the use of low-resolution image data for SMORE(2D) [MRI acquired in 2D protocols, page 807 Section II paragraph 2] which includes MR data with multiple slice [see page 805 section I bullet 2]. The LR input images can be of subjects as shown in Figure 4);
a pre-processing module coupled to the input and configured to generate a convolved input from the received 2D multi-slice MR dataset by applying slice-profile convolution to the received 2D multi-slice MR dataset (see page 810 subsection 1 and algorithm 3 step 2, the image is blurred in the x-axis using a 1D Gaussian profile to act as the MRI slice selection profile to create training data [the blurring in the x-axis using a Gaussian profile to create data is interpreted to be equivalent to generating convolved input]);
a through-plane super-resolution neural network coupled to the pre-processing module and configured to generate a through-plane super-resolution imaging volume based on the convolved input (see page 810 subsection 2 and algorithm 3, the applying of the trained super-resolution network to the coronal slices or x-z plane [convolved input] producing through plane super-resolution images [see page 807 paragraph 2]); and
PNG
media_image1.png
184
450
media_image1.png
Greyscale
a post-processing module coupled to the through-plane super-resolution neural network and configured to generate a three-dimensional (3D) isotropic super-resolution imaging volume by applying (see page 814 paragraph 1, the use of SMORE(2D) to output an up sampled isotropic volume [3D isotropic super-resolution imaging volume]. Step 4 of Algorithm 3 using Fourier Burst Accumulation (FBA) which is an alternative to deconvolution for image deblurring but not explicitly therefore Zhao does not explicitly teach applying slice-profile deconvolution to the through-plane super-resolution imaging volume and a secondary reference will teach obviousness).
Zhao does not explicitly teach generate an imaging volume by applying slice-profile deconvolution to the imaging volume.
Yatsuo teaches generate an imaging volume by applying slice-profile deconvolution to the through-plane imaging volume (see paragraph 0071-0072, the super-resolution processing unit performs the deconvolution calculation on the point spread function (PSF) and the MRI image to generate a higher-resolution image. At imaging the slice plane is in a direction orthogonal to the imaging cross-section [through-plane] in accordance with the imaging pulse sequence [the inclusion of multiple slices]).
Zhao and Yatsuo are analogous art because they are from the same field of endeavor of an apparatus for super-resolution processing applied to a 2D MRI of a subject to improve resolution.
Before the effective filling date of the invention, it would have been obvious to one of ordinary skill in the art to modify Zhao to apply the deconvolution to the image as taught by Wu. The motivation for doing so would have been to increase the resolution of the MRI image (Yatsuo, paragraph 0068).
Regarding claim 4, Zhao and Yatsuo teach the system according to claim 1.
Zhao teaches applying slice-profile convolution to the received 2D multi-slice MR dataset reformats the 2D multi-slice MR dataset to an orthogonal plane (see algorithm 3, step2 includes the rotating of the image. See Figure 14, SMORE(2D) is performed on the orthogonal direction).
Regarding claim 7, Zhao and Yatsuo teaches the system according to claim 1.
Zhao teaches the through-plane super-resolution neural network is trained using a training input dataset generated by applying slice-profile downsampling to a 2D multi-slice MR training dataset (see page 810 subsection 1 and 2, creating training data with the introduction of aliasing which includes down sampling based on the slices and then training the SSR with the aliased images).
Regarding claim 8, Zhao and Yatsuo teaches the system according to claim 7.
Zhao teaches the training input dataset is a low-resolution training input dataset (see page 810 subsection 1 and 2, the creation of training data with low resolution (LR) and the training of the SSR with the LR images).
Claims 10, 13, and 16-17 are analogous method to the apparatus of claim 1, 4, and 7-8, respectively, thus are analyzed and rejected similar to claims 10, 13, and 16-17.
Claims 2, 3, 11, 12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Yatsuo in view of Zhao “Applications of a Deep Learning method for anti-aliasing and super-resolution in MRI” (included in the IDS) (hereinafter “Shao”).
Regarding claim 2, Zhao and Yatsuo teaches the system according to claim 1.
Zhao nor Yatsuo teach he 2D multi-slice MR dataset is one of a turbo spin-echo (TSE) dataset or a fast spin-echo (FSE) dataset.
Shao teaches the 2D multi-slice MR dataset is one of a turbo spin-echo (TSE) dataset or a fast spin-echo (FSE) dataset (see section 2.4, the MR images of the subject were performed using Turbo Spin Echo sequence. The dataset includes stacks of images [multi-slice] with resolution 256 x 256 [two-dimensional].
Zhao, Yatsuo, and Shao are analogous art because they are from the same field of endeavor of an apparatus for super-resolution processing applied to a 2D MRI of a subject to improve resolution.
Before the effective filling date of the invention, it would have been obvious to one of ordinary skill in the art to modify Zhao and Yatsuo to use a 2D multi-slice MR dataset is one of TSE as taught by Shao. The motivation for doing so would have been Zhao references Shao on page 807 paragraph 2 as an application of SMORE and appears to have the same first author of Can Zhao.
Regarding claim 3, Zhao in view of Yatsuo in view of Shao teach the system according to claim 2.
Zhao teaches the 2D multi-slice MR dataset is one of a Ti-, T2-, or proton density weighted dataset (see section 3 subsection A, the use of T2-weighted images of subjects in the 2D MRI protocol).
Regarding claim 19, Zhao in view of Yatsuo in view of Shao teaches the system according to claim 16.
Zhao nor Yatsuo teach the 2D multi-slice MR training dataset is one of a turbo spin-echo (TSE) or fast spin-echo (FSE) dataset.
Shao teaches the 2D multi-slice MR training dataset is one of a turbo spin-echo (TSE) or fast spin-echo (FSE) dataset (see section 2.1, the SMORE(2D) trains on LR data from the high-resolution data input [a T2-weight Turbo Spin Echo is input as an experiment thus training the algorithm on TSE data, see section 2.4])
Claims 11-12 are analogous method to the apparatus of claims 2-3, respectively, thus are analyzed and rejected similar to claims 2-3.
Claims 5, 9, 14, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Yatsuo in view of Borisch US20210041517 (Included in the IDS) (hereinafter “Borisch”).
Regarding claim 5, Zhao and Yatsuo teach the system according to claim 1.
Zhao teaches the convolved input comprises a convolved (see page 812 subsection 2 and algorithm 3, r is equal a number from 2 to 6 [r is the ratio of through-plane resolution (c) to in-plane resolution (a), r=c/a, therefore in cases where the image dimension are fixed changing r changes the thickness of the slice relative in-plane resolution], thus in the case where r is 6 (thicker slices, less slice) there is a difference of 3 times (6/2=3) the slices as when r is 2 (thinner slices, more slices) [interpreted as the images, which are used to create training data, have at least 3 slices when r is 2]. However, Zhao does not explicitly teach the input comprises a center slice and two adjacent slices so an additional source will teach obviousness).
Zhao nor Yatsuo explicitly teach the input including a center slice and two adjacent slices.
Borisch teaches including a center slice and two adjacent slices (see paragraph 0095, the acquisition of multi-slice MRI where abutting slices are acquired, which includes slices adjacent to another slice).
Zhao, Yatsuo, and Borisch are analogous art because they are from the same field of endeavor of an apparatus for the processing of multi-slice 2D MRIs in the through-plane direction to obtain a higher resolution image.
Before the effective filling date of the invention, it would have been obvious to one of ordinary skill in the art to modify Zhao and Yatsuo to input abutting slices as taught by Borisch. The motivation for doing so would have been to generate a 3D image with the target resolution using the overlapping from the consecutive images (Borisch, paragraph 0037).
Regarding claim 9, Zhao and Yatsuo teaches the system according to claim 7.
Zhao teaches the training input dataset comprises (see page 812 subsection 2 and algorithm 3, r is equal a number from 2 to 6 [r is the ratio of through-plane resolution (c) to in-plane resolution (a), r=c/a, therefore in cases where the image dimension are fixed changing r changes the thickness of the slice relative in-plane resolution], thus in the case where r is 6 (thicker slices, less slice) there is a difference of 3 times (6/2=3) the slices as when r is 2 (thinner slices, more slices) [interpreted as the images, which are used to create training data, have at least 3 slices when r is 2]. This training extraction is performed on the entire image, thus in the case there is 3 slices they would be consecutive. However, Zhao does not explicitly teach that the input dataset comprises three consecutive inputs so an additional reference will teach obviousness).
Zhao nor Yatsuo explicitly teach the input dataset comprises three consecutive images.
Borisch teaches the input dataset comprises three consecutive images (see paragraph 0036-0037, the use of consecutive MRI slices [the imaging of more than 3 slices in multiple passes to be combined and presented in order, see paragraph 0095] to generate a 3D image with the target resolution.
Claims 14 and 18 are analogous method to the apparatus of claims 5 and 9, respectively, thus are analyzed and rejected similar to claims 5 and 9.
Claims 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao in view of Yatsuo in view of Mahapatra “Image Super Resolution Using Generative Adversarial Networks and Local Saliency Maps for Retinal Image Analysis (hereinafter “Mahapatra”).
Regarding claim 6, Zhao and Yatsuo teaches the system according to claim 1.
Zhao teaches the through-plane super-resolution neural network is a (see section IV paragraph 3, SMORE uses an enhanced deep residual network for single image super-resolution (EDSR) as the network for the super-resolution network for the use on through plane images [page 807 paragraph 2]).
Zhao nor Yatsuo teach the super-resolution neural network is a generative adversarial network.
Mahapatra teaches the super-resolution neural network is a generative adversarial network (see page 384 section 3, the use of Generative Adversarial Networks for use in image super resolution methods).
Zhao, Yatsuo, and Mahapatra are analogous art because they are from the same field of endeavor of an apparatus for super-resolution processing applied to medical images of a subject to improve resolution.
Before the effective filling date of the invention, it would have been obvious to one of ordinary skill in the art to modify Zhao and Yatsuo to use a generative adversarial network as taught by Mahapatra. The motivation for doing so would have been use the generative adversarial network to generate solutions that are very similar to real images to therefore encourage perceptually superior solutions (Mahapatra, page 384 Section 3).
Claim 15 is analogous method to the apparatus of claim 6, thus is analyzed and rejected similar to claim 6.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMILY R. HAUK whose telephone number is (571)272-5966. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMILY R HAUK/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights