Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 01/28/2026 has been entered.
Response to Arguments
In pg 11, second full paragraph, Applicant argues that Zhong does not teach applying spatial attention between features of the first (Hi Res) and the second (Low Res) image. However, Zhong ch III equation (6) and surrounding paragraph clearly teaches applying the spatial attention MMAF to inputs Fiy and Fid. Where in the preceding paragraph, equations (4) and (5) respectively define parameters Fiy and Fid as features of the Hi-Res (“first”) image and Low-Res (“second”) image. Without further analysis, at minimum, the function MMAF spatial-attention function is applied to both inputs, satisfying the claimed requirement “applying spatial attention between features of the 1st and 2nd image”. It is not apparent that the term “between” requires a more specific relationship between the two images.
In pg 11, third full paragraph, Applicant argues that the He reference does not apply the claimed degrading specifically to the second (Low-Res) image. However, He reference incorporated by reference from Zhong, and therefore the manner Zhong is incorporating it, also applies in the way He should be understood in this context. Zhong ch III equations (1)-(4) describe the function Sigma, which is specifically applied both to the Hi-Res features (Fy, equations 1,3 and 5) and Low-Res features (Fd, equations 2 and 4), therefore clearly applies to the Low-Res image. Furthermore, the function Sigma is further described as the PReLu activation function in the He reference – see description of equation (1) in Zhong, and citation [38] in Zhong’s References section. Importantly, the He reference describes the PReLu function training on page 1031, right column, paragraph labeled “Training”: there are at least two different steps that are described as random – the random jittering of scale s, and random color altering. Consequently, Zhong’s convolution does include steps that can be reasonably characterized as random degrading.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6-9, 13-15, 18, 20-22, 25, 27, 29-30 are rejected under 35 U.S.C. 103 as being unpatentable over Zhong (“High-Resolution Depth Maps Imaging via Attention-Based Hierarchical Multi-Modal Fusion” IEEE 2021) and further by He (“Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification” IEEE 2015) incorporated by reference within Zhong, see p 2021, right col 1st par, reference [38]) in view of Lyu (“Multi-Contrast Super-Resolution MRI Through a Progressive Network” Arxiv 2019)
As for claim 1, Zhong teaches
One or more processors, comprising:
circuitry to use one or more neural networks (ch III par 1 “multi-modal fusion network”) to perform multi-sequence alignment of [..] images (Zhong Fig 2-3, ch III par 3, “Module of guidance image downsampling”, the guidance image Y is downsampled to match resolution of DLR) by at least ()
combining one or more first features of an object in a first image of the object with one or more second features in a second lower-resolution image of the object based, at least in part, on applying non-local spatial attention between the one or more first features of the first image and the one or more second features of the second lower-resolution image. (Fig 2, Fig 3, ch III par 2, pars “Module of feature extraction” and “Module of multi-modal attention-based fusion”, combining the high-resolution guidance image Y and the low-resolution depth map D using a multi-modal attention based fusion method; additionally please see Response to Arguments above)
wherein the one or more neural networks are trained based, at least_in part, on one or more randomly degraded versions of the second lower-resolution image (He, incorporated by reference within Zhong, pg 1031 right col, paragraph labeled “Training”, discusses randomly sampling scale s and performing random color altering; and further discussed in “Response to Arguments” above)
Zhong does not specifically teach, Lyu however teaches
[super-resolution of] medical images (Lyu, ch IIIA Experiments/Datasets, teaches applying the super-resolution methods to MRI images)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the Super-Resolution method of Zhong by including MRI images of Lyu, as both pertain to the art of generating super-resolution images. The motivation to do so would have been, to expand the categories of images to which a super-resolution method may be applied.
As for independent claims 9, 15, 21, 27, please see discussion of claim 1 above.
As for claim 13, 6, the combination of Zhong and Lyu teaches
the one or more processors are to generate a degraded version of the second lower-resolution image (Fig 2, LR image undergoes multiple convolution operations), and
perform a comparison that includes using a loss function based, at least in part, on a downsampled version of one or more images generated by the one or more neural networks (ch III eq 10, loss function invokes entire network architecture thus including the convolution operations).
As for claims 14, 7, 8, 18, 20, 22, 29, 30, the combination of Zhong and Lyu teaches
the training or more processors are to use a set of training data, (ch VI various training datasets are discussed, e.g. subchapters A.(1)-(2) )
generate a randomly degraded version of the second lower-resolution image (ch III eq 10, loss function invokes entire network architecture thus including convolution operations in Fig 2), and
train one or more attention layers of the one or more neural networks to apply non-local spatial attention from the one or more first features to the one or more second features based, at least in part, on using the randomly degraded version of the second lower- resolution image as an input to the one or more neural networks (ch III eq 10, loss function invokes entire network architecture thus including the MMAF module of Fig 3).
As for claim 25, the combination of Zhong and Lyu teaches
the one or more processors are also to at least train the one or more neural networks to generate one or more images of the object that correspond to the second lower-resolution image, at a same resolution of the first image (Fig 2, the SR output is the same size image as the HR Guidance input image)
B. Claims 2, 4-5, 10-12, 16, 17, 19, 23-24, 26, 28, 31-32 rejected under 35 U.S.C. 103 as being unpatentable over Zhong in view of Lyu
As for claims 10, 16, 17, 32, Zhong does not teach, Lyu however teaches
the first image is a first magnetic resonance imaging (MRI) image generated using T1 weighting (Lyu, Fig 1a, ch IIA par 2, ch IIIA par 2, using T1 weighted images for HR images),
the second lower-resolution image is a second MRI image generated using T2 weighting (Lyu ch IIB down-sampling T2 images for LR images) , and
the one or more processors are to use the one or more neural networks to generate images that correspond to the second lower-resolution image of the object, at a higher resolution than the second lower-resolution image (Lyu Fig 1 the T2 SR output corresponds to the T2 input image)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the image super-resolution method of Zhong by including MRI images of Lyu, as both pertain to the art of training neural networks to perform super-resolution of images. The motivation to do so would have been to create higher-resolution MRI images.
As for claims 28, 5, 23, Zhong does not teach, Lyu however teaches
the object is an anatomical structure, the second lower-resolution image is a magnetic resonance imaging (MRI) image (Lyu, as above), a computed tomography (CT) image, or a positron emission tomography (PET) image, and
the one or more generated images correspond to the second lower-resolution image, at a higher resolution than the second lower-resolution image (as above)
As for claims 19, 4, 11, 24 Zhong does not teach, Lyu however teaches
training the one or more neural networks further includes a second comparison of the one or more generated images of the object with the first image of the object, and using a weighted loss function based, at least in part, on the first comparison and the second comparison (Lyu ch D, (1)-(4) teaches four different Loss components)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the image super-resolution method of Zhong by including the compound loss function of Lyu, as both pertain to the art of training neural networks to perform super-resolution of images. The motivation to do so would have been, Zhong teaches a loss function L, however does not provide implementation details for this function. Lyu provides relevant details for a loss function used in a similar scenario.
As for claim 26, Zhong does not teach, Lyu however teaches
the training includes using a loss function based, at least in part, on a structural similarity index measure (Lyu ch IIID(4) Texture Matching Loss), a mean squared error loss (ch IID(2)), and a normalized mutual information value (ch IIID (3) Perceptual Loss or IIID(1) Adversarial Loss)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the image super-resolution method of Zhong by including the compound loss function of Lyu, as both pertain to the art of training neural networks to perform super-resolution of images. The motivation to do so would have been, Zhong teaches a loss function L, however does not provide implementation details for this function. Lyu provides relevant details for a loss function used in a similar scenario.
As for claims 31, 12, 2, please see discussion of claims 10, 19 and 26 above. Cited references and motivations to combine apply analogously.
B. Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Zhong in view of Lyu in further view of MRI Basics (https://case.edu/med/neurology/NR/MRI%20Basics.htm 2016)
As for claim 3, the combination of Zhong and Lyu teaches
the object is an anatomical structure, the first image is a first magnetic resonance imaging (MRI) image, and the second lower-resolution image is a second MRI image (Lyu see discussion of claim 10 above)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the image super-resolution method of Zhong by including MRI images of Lyu, as both pertain to the art of training neural networks to perform super-resolution of images. The motivation to do so would have been to create higher-resolution MRI images.
the combination of Zhong and Lyu doesn’t specifically teach, MRI Basics however teaches
generated based, at least in part, using at least one of a different time to echo or a different repetition time than used to generate the first MRI image (MRI Basics, par “Physics of MRI” explanation of TE (Time to Echo), and pg 2 table “Most common MRI sequences and their Approximate TR and TE times”)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention, to modify the combination of Zhong and Lyu by further including the MRI specifications of MRI Basics, as both pertain to capturing MRI images. The motivation to do so would have been, Lyu teaches T1 and T2 MRI image capture specifications, but does not specify how they differ. MRI Basics provides substantially more detail on T1 and T2 MRI image capture.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARK ROZ whose telephone number is (571)270-3382. The examiner can normally be reached on 9AM-5PM M-F.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARK ROZ/
Primary Examiner, Art Unit 2669
Prosecution Insights