DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The disclosure is objected to because of the following informalities:
In paragraph [005] line 9, “training one network per application can result is a complex set of networks that may be too complex to deploy” appears as if it should recite “training one network per application can result [[is]] in a complex set of networks that may be too complex to deploy.”
Paragraph [0014] is missing a semicolon at the end.
Paragraph [0037] is missing a period at the end.
Appropriate correction is required.
Claim Objections
Claims 7, 10, 13, 17, and 20 are objected to because of the following informalities:
In Claim 7 line 2 and Claim 17 line 2, “and” should be “or.”
In Claim 10 line 1, “configured perform” should read “configured to perform.”
In Claim 13 line 2, “meta-parameters a multilayer” should read “meta-parameters to a multilayer.”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claims contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. The specification describes a specific architecture for the second machine-learning (ML) network consisting of an iterative convolutional neural network (CNN) + data consistency (DC) (see Figures 5 and 6). However, claim 1 has been drafted to encompass any type of machine-learning network for the second ML network. The description does not reasonably convey to one of ordinary skill in the art that the inventor had possession of the full scope of the claimed invention, including generalized ML architectures beyond the CNN + DC structure. Furthermore, the claim broadly recites “meta-parameters related to the first medical image data” without specifying how those meta-parameters relate to the first image data. The specification describes several examples of meta-parameters such as T1, T2, FLAIR, field-strength, and acceleration parameters (see Figures 4 and 5 and paragraphs 0036, 0038, and 0050), all of which correspond to parameters involved in acquiring the first image data. The disclosure does not describe meta-parameters outside that acquisition context and therefore fails to demonstrate that the inventor was in possession of the generalized concept of “meta-parameters related to the first medical image data” as presently claimed. Accordingly, claims 11 and 20 are rejected for containing identical subject matter to claim 1. Furthermore, claims 2-10 and 11-19 depend from claims 1 and 11, respectively, and are rejected for the same reasons set forth for claims 1 and 11.
Claims 1-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, because the specification, while being enabling for the particular disclosed embodiment in which both ML networks are jointly trained, does not reasonably provide enablement for the full scope of the claims. The specification does not enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the invention commensurate in scope with these claims. The specification enables an embodiment in which a first ML network learns tuning parameters related to CNN intermediate layers and a regularization parameter for a second ML network that is trained jointly with the first ML network. However, claim 1 is drafted broadly to encompass obtaining and applying “tuning parameters” between arbitrary ML networks without limitation to the specific CNN + DC architecture disclosed. The specification does not teach how a person of ordinary skill in the art could cause the first ML network to learn tuning parameters applicable to any second ML network architecture or weight set without undue experimentation. Moreover, the disclosure explicitly states that no other type of training than joint supervised training is envisioned, providing no guidance or examples for other types of training schemes such as independent, sequential, or unsupervised training. Accordingly, claims 11 and 20 are rejected for containing identical subject matter to Claim 1. Furthermore, claims 2-10 and 11-19 depend from claims 1 and 11, respectively, and are rejected for the same reasons set forth for claims 1 and 11.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 1 recites “an apparatus for reconstructing or filtering medical image data,” but it is unclear how the claimed features, particularly the first ML network and the second ML network, relate to one another and to the medical image data. The claim does not delineate the specific roles or interrelationship of the two networks (e.g., whether the second ML network is trained, or whether it performs reconstruction, filtering, or another function). Furthermore, the claim does not clarify how “tuning parameters” are applied from the first ML network to the second ML network, whether as weights, scaling functions, or another form of modulation, rendering the scope of the claim unclear. Therefore, one of ordinary skill in the art would not be reasonably apprised of the metes and bounds of the claimed invention. Accordingly, claims 11 and 20 are rejected for containing identical subject matter to Claim 1. Furthermore, claims 2-10 and 11-19 depend from claims 1 and 11, respectively, and are rejected for the same reasons set forth for claims 1 and 11.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 3, 4, 7, 9, 10 11, 13, 14, 17, 19, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ding et. al (“Deep Learning With Adaptive Hyper-Parameters for Low-Dose CT Image Reconstruction”).
Regarding Claim 1, Ding teaches an apparatus for reconstructing or filtering medical image data (Ding: Fig. 3 (shown below)), the apparatus comprising:
processing circuitry configured to (Ding: Section III. Method, Implementation;
“For the proposed NN method, training is performed with PyTorch [54] framework on a NVIDIA Titan GPU.”)
receive first medical image data and meta-parameters related to the first medical image data (Ding: Section II. Measurement Model and Problem Formulation;
“It is noted that the noise level varies for different target images. Given a target image x, its noise level is controlled by Ii, i.e., the measure data is corrupted with noise which becomes larger when dose level Ii decreases.”
apply the received meta-parameters to inputs of a first trained machine-learning (ML) network to obtain, from outputs of the first trained ML network, tuning parameters of a second ML network different from the first ML network (Ding: Figs. 1 (shown below) and 11 (shown below);
PNG
media_image1.png
251
646
media_image1.png
Greyscale
PNG
media_image2.png
433
623
media_image2.png
Greyscale
Section I. Introduction: “Inversion block reconstructs an image using both the measurement and the estimate from the previous stage, whose hyper-parameters are predicted by a multi-layer perception neural network (MLP).”
Section V. Results, Visual Comparison of Some Examples: “It can be seen that the predicted hyper-parameter adapts well with inputs of different noise levels, where a lower-dose-level input requires
larger hyper-parameter to incorporate more prior knowledge, and vice versa.”)
apply the received first medical image data to inputs of the second ML network, as tuned by the obtained tuning parameters output from the first ML network, to obtain, from outputs of the second ML network, second medical image data; and (Ding: Figs. 2 (shown below), 3 (shown below);
PNG
media_image3.png
232
582
media_image3.png
Greyscale
PNG
media_image4.png
308
1195
media_image4.png
Greyscale
Section I. Introduction: “De-noising block removes the artifacts of the estimate passed from the inverse block by a convolutional neural network (CNN).”)
output the second medical image data (Ding: Figs. 5 (shown below), 7 (shown below);
PNG
media_image5.png
441
1203
media_image5.png
Greyscale
PNG
media_image6.png
447
1171
media_image6.png
Greyscale
Section III. Method, Denoising Block With CNN-Based Adaptive Prior: “Thus, the proposed CNN take the estimate xk as the input and output a denoised version x˜k.”
Regarding Claim 3, Ding teaches the apparatus of claim 1, wherein the processing circuitry is further configured to apply the received meta-parameters to the first ML network, which is a multilayer perceptron (MLP) network (Ding: Fig. 1 (shown above)).
Regarding Claim 4, Ding teaches the apparatus of claim 1, wherein the processing circuitry is further configured to apply the received first image data to the second ML network, which is a convolutional neural network (CNN) (Ding: Fig. 2 (shown above)).
Regarding Claim 7, Ding teaches the apparatus of claim 1, wherein the first image data is one of magnetic resonance imaging (MRI) data, computed tomography (CT) data, and positron emission tomography (PET) data as shown above in Claim 1 (Ding: Section II. Measurement Model and Problem Formulation (shown above)).
Regarding Claim 9, Ding teaches the apparatus of claim 1, wherein the processing circuitry is further configured to jointly train the first ML network and the second ML network using training data and a single loss function (Ding: Section III. Method, The Overall Architecture of the Proposed Method).
PNG
media_image7.png
649
656
media_image7.png
Greyscale
Regarding Claim 10, Ding teaches the apparatus of claim 1, wherein the second ML network is configured perform inference of one of a reconstruction function and a filtering function based on the input first medical image data as shown in Claim 1 (Ding: Section III. Method, Denoising Block With CNN-Based Adaptive Prior (shown above)).
Regarding Claim 11, Ding teaches all of the limitations of Claim 1 above because Claim 11 recites a method comprising steps that correspond in substance to the functions of the apparatus of Claim 1.
Regarding Claim 13, Ding teaches the method of claim 11, and additional limitations are met as in the consideration of Claim 3 above.
Regarding Claim 14, Ding teaches the method of claim 11, and additional limitations are met as in the consideration of Claim 4 above.
Regarding Claim 17, Ding teaches the method of claim 11, and additional limitations are met as in the consideration of Claim 7 above.
Regarding Claim 19, Ding teaches the method of claim 11, and additional limitations are met as in the consideration of Claim 9 above.
Regarding Claim 20, Ding teaches all of the limitations of Claim 1 above because Claim 20 recites a computer-readable medium comprising instructions causing a processor to perform substantially the same functions as those of the apparatus of Claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Ding et. al in view of Bian et. al (“An Optimization-Based Meta-Learning Model for MRI Reconstruction with Diverse Dataset”).
Regarding Claim 2, Ding teaches the apparatus of claim 1, but fails to teach that the first medical image data received by the processing circuitry is magnetic-resonance k-space data and the second medical data output by the second ML network is a magnetic-resonance image.
However, Bian teaches a meta-learning MRI reconstruction framework in which the reconstruction network receives under sampled k-space measurements (y) as input and outputs reconstructed MR images (Bian: Eqs. 1 (shown below) and 11 (shown below) and Figs. 2 (shown below) and 3 (shown below)). Bian further teaches that the framework uses a bilevel learning strategy wherein a first (meta) network learns task-specific regularization parameters (i.e., hyperparameters) for a second (reconstruction) network, conditioned on MRI-specific acquisition parameters (e.g., under-sampling patterns, under-sampling ratios, and scanning parameters), which they treat as task meta-parameters guiding adaptation. Both Ding and Bian solve the analogous problem of improving reconstruction quality through adaptive network learning.
PNG
media_image8.png
127
734
media_image8.png
Greyscale
PNG
media_image9.png
110
745
media_image9.png
Greyscale
PNG
media_image10.png
517
697
media_image10.png
Greyscale
PNG
media_image11.png
538
728
media_image11.png
Greyscale
Thus, it would have been obvious to one of ordinary skill in the art, prior to filing, to adapt Ding’s multi-network architecture to the MRI domain as taught by Bian, substituting MRI k-space data for the input medical data and configuring the second network to output a magnetic-resonance image. Bian explicitly teaches that applying deep-learning frameworks to MRI reconstruction from k-space accelerates acquisition and improves image quality through adaptive learning across varying scan parameters. The motivation to combine arises directly from Bian’s disclosure that meta-learning and hyperparameter adaptation improve generalization across acquisition conditions in MRI reconstruction, thereby yielding predictable results in accordance with known deep-learning MRI-reconstruction pipelines.
Regarding Claim 12, Ding teaches the method of claim 11, and additional limitations are met by the combination of Ding in view of Bain, as discussed with respect to Claim 2 above.
Claims 5, 6, 15, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Ding et. al in view of Baik et. al (“Meta-Learning with Adaptive Hyperparameters”).
Regarding Claim 5, Ding teaches the apparatus of claim 1, but fails to teach that the tuning parameters output by the first ML network include feature scaling vectors that scale output vectors of corresponding intermediate layers of the second ML network, and the processing circuitry is further configured to modify the second ML network using the feature scaling vectors output from the first ML network.
However, Baik teaches a meta-learner (hyperparameter-generator network gφ) configured to output adaptive, layer-wise scaling hyperparameters αi,j and βi,j for a base network fθ (Baik: Fig. 2 (shown below)). These hyperparameters act as element-wise scaling factors applied to each layer’s parameters or gradients (i.e., feature scaling vectors). Baik further explains that these hyperparameters are generated for each intermediate layer and applied element-wise (Hadamard product) to scale the corresponding parameters or gradients of fθ, thereby modifying the base network’s layer outputs during training (Baik: Eq. 4 (shown below)). The scaling operation shown in Eq. 4 updates the weights of the second network (base network fθ), effectively modifying it using the first network’s (hyperparameter-generator network gφ) output. Baik further notes that this adaptive scaling mechanism enables efficient layer-specific modulation and improves convergence stability and generalization on unseen tasks. Both Ding and Baik address the analogous problem of improving neural network stability and performance during training.
PNG
media_image12.png
263
574
media_image12.png
Greyscale
PNG
media_image13.png
137
581
media_image13.png
Greyscale
Thus, it would have been obvious to one of ordinary skill in the art, prior to filing, to incorporate Baik’s layer-wise scaling mechanism into Ding’s two-network reconstruction system. The motivation is explicitly provided by Baik, who teaches that generating and applying adaptive scaling hyperparameters across layers enhances model performance and reduces over-fitting by allowing fine-grained control of network updates. Accordingly, it would have been obvious to combine Ding’s architecture with Baik’s layer-specific scaling hyperparameters to create a predictable and straightforward modification yielding the well-known benefit of more stable and adaptive training across different medical imaging conditions.
Regarding Claim 6, Ding teaches the apparatus of claim 1, but fails to teach that the tuning parameters output by the first ML network include a regularization parameter in a loss function used in the second ML network.
However, Baik teaches a meta-learning framework where a first neural network (gφ) generates adaptive hyperparameters, including regularization parameter β, that are applied within the loss function of a second neural network fθ (Baik: Fig. 2 (shown above), Eq. 3-5 (shown below)). Both Ding and Baik address the analogous problem of improving neural network stability and performance during training.
PNG
media_image14.png
82
561
media_image14.png
Greyscale
PNG
media_image15.png
67
547
media_image15.png
Greyscale
PNG
media_image16.png
54
546
media_image16.png
Greyscale
Thus, it would have been obvious to a person of ordinary skill in the art, prior to filing, to incorporate such an adaptive regularization scheme into Ding’s multi-network architecture to improve training efficiency and generalization by automatically optimizing the regularization term during training. Baik explicitly teaches that the proposed adaptive regularization method can “achieve better training and generalization… owing to per-step adaptive regularization and learning rates” and be “explicitly trained to achieve generalization on unseen examples” (Baik: Sec. 3.2-3.3). Baik further explains that dynamically generated regularization parameters enable a model to balance stability and adaptability, yielding faster convergence and better performance on diverse datasets. Accordingly, it would have been obvious to combine Ding’s architecture with Baik’s adaptive regularization framework, motivated by Baik’s teaching that such adaptive regularization improves robustness, prevents overfitting, and enhances generalization of the network’s learning process.
Regarding Claim 15, Ding teaches the method of claim 11, and additional limitations are met by the combination of Ding in view of Baik, as discussed with respect to Claim 5 above.
Regarding Claim 16, Ding teaches the method of claim 11, and additional limitations are met by the combination of Ding in view of Baik, as discussed with respect to Claim 6 above.
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Ding et. al in view of Bian et. al, further in view of Mostapha et. al (U.S Patent No. 11,783,485).
Regarding Claim 8, Ding teaches the apparatus of claim 1, but fails to teach that the first medical image data is magnetic-resonance image data and the meta-parameters include a T1 parameter, a T2 parameter, a fluid-attenuated inversion recovery (FLAIR) parameter, a field strength parameter, and an acceleration parameter.
However, Bian teaches a meta-learning MRI reconstruction framework that uses a bilevel learning strategy wherein a first (meta) network learns task-specific regularization parameters (i.e., hyperparameters) for a second (reconstruction) network. Bian explicitly discloses that they “leverage this feature of meta-learning for network training where the MRI training data are acquired by using different under-sampling patterns (e.g., Cartesian mask, Radial mask, Poisson mask), under-sampling ratios, and different settings of the scanning parameters, which result in different levels of contrast (e.g., T1-weighted, T2-weighted, proton-density (PD), and Flair)” (Bian: Introduction). In this context, under-sampling ratios directly correspond to acceleration parameters, as acceleration factors are the reciprocal of under-sampling ratios.
Bian fails to teach that the meta-parameters include a field strength parameter.
However, Mostapha teaches a deep-learning MRI segmentation architecture that explicitly integrates MRI image data and protocol meta-data comprising “sequence parameters (e.g., TR, TE, TI, flip angel, field strength, acquisition plane), geometrical image information (e.g., resolution, field of view), scanner information (e.g., manufacturer, scanner model, coil type), and/or prior intensity statistics of each tissue type (e.g., mean intensity of different types of tissue (e.g., white matter, gray matter, and cerebral spinal fluid))” (Mostapha: paragraph 18, Figs. 3 (shown below) and 4 (shown below)). The meta-data input is provided to the network alongside the MRI image data to condition the network’s processing and improve generalization across scanners and imaging protocols.
PNG
media_image17.png
1023
904
media_image17.png
Greyscale
Thus, it would have been obvious to a person of ordinary skill in the art, prior to filing, to incorporate the specific meta-parameters disclosed by Bian and Mostapha into Ding’s architecture. Bian explicitly teaches that using heterogeneous MRI data sets with various under-sampling patterns, ratios, and acquisition settings creates a “robust and generalizable image reconstruction method” (Bian: Introduction). Moreover, Mostapha explicitly teaches that incorporating MRI protocol information such as T1, T2, FLAIR, and field strength into the network input “allows the machine-learned network to operate well for different protocols used on different patients by different scanners” (Mostapha: paragraph 12). Thus, both Bian and Mostapha teach that using meta-parameters as network input improves model robustness and segmentation/reconstruction accuracy under varying acquisition conditions. Combining Ding’s architecture with the specific meta-parameters disclosed by Bian and Mostapha would have been a predictable design choice yielding the well-known benefit of improved cross-scanner generalization and consistent image quality, leading to enhanced model performance across heterogenous MRI acquisition settings.
Regarding Claim 18, Ding teaches the method of claim 11, and additional limitations are met by the combination of Ding in view of Bain and further in view of Mostapha, as discussed with respect to Claim 8 above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM ADU-JAMFI whose telephone number is (571) 272-9298. The examiner can normally be reached M-T 8:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILLIAM ADU-JAMFI/Examiner, Art Unit 2677 /ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677