DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 12 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xing et al. (“Path Tracing Denoising Based on SURE Adaptive Sampling and Neural Network”) in view of Tohme et al. (US 20240144441 A1).
Regarding claims 1, 10, and 14, Xing et al. disclose a method for training a system for de-noising images (abstract), the system comprising an input-interface and a plurality of trainable bilateral filters configured to filter an image provided by the input interface (cross-bilateral filter, Fig. 1, part III1, training network is a process of updating weights between two layers of network to minimize the loss function value, ɵm,i is the output of our network, which represents the optimal filter parameters, part B2c), the method comprising, and magnetic resonance imaging system for de-noising an image, comprising, and non-transitory storage medium associated with a system comprising an input-interface and a plurality of trainable bilateral filters configured to filter an image provided by the input interface, the non-transitory storage medium having instructions thereon that, when executed by a processor, cause the processor to train the system for de-noising images by: providing a plurality of training images as an input to the system (training set images, part B3); and training the plurality of trainable bilateral filters based on the plurality of training images, the plurality of noise maps, and a calculation of analytical gradients of a loss function with respect to filter parameters of the system, wherein the loss function is based on Stein's unbiased risk estimator (SURE) (
PNG
media_image1.png
808
475
media_image1.png
Greyscale
, part IIIA1, The “backpropagate'' of general MLPs network updates the weights through the results of the network. And our “backpropagate'' is according to the results of the filter. So, this implies that the filter must be differentiable with respect to its filter parameters. Fortunately, our utilized cross-bilateral filter is differentiable, part IIIB2a, 2) The error between the computed and desired outputs is used to determine the effect of each weight on the output error. This requires taking the derivative of the error with respect to each weight. Besides, the activation functions also need to be differentiable. The two steps are performed for all the data used in training set and the error gradient of each weight is accumulated. 3) The weights are updated according to the accumulated error gradient, part IIIB2c).
Xing et al. do not disclose providing a plurality of noise maps indicating a standard deviation of noise for each pixel of one of the plurality of training images. With respect to claim 10 in particular, Xing et al. do not disclose a magnetic resonance scanner.
Tohme et al. teach an input-interface configured to filter an image provided by the input interface (“Additionally, the computing device 216 provides commands and parameters to one or more of the DAS 214, the x-ray controller 210, and the gantry motor controller 212 for controlling system operations such as data acquisition and/or processing. In certain embodiments, the computing device 216 controls system operations based on operator input. The computing device 216 receives the operator input, for example, including commands and/or scanning parameters via an operator console 220 operatively coupled to the computing device 216”, [0045], In one embodiment, the display 232 allows the operator to evaluate the imaged anatomy. The display 232 may also allow the operator to select a volume of interest (VOI) and/or request patient information, for example, via a graphical user interface (GUI) for a subsequent scan or processing, [0052], DL network denoisers 304 are formed as either blind denoisers (no additional information provided to the denoiser 304 other than the input noisy image 314 and reference image 324) and non-blind denoisers (includes a noisy image 314, reference image 324 and image/noise map 350 (FIG. 6) as input to denoiser 304, [0057]) providing a plurality of noise maps indicating a standard deviation of noise for each pixel of one of the plurality of training images (“With regard to the generation of the noise map 350, there are many ways to generate a noise map 350, but the key is to be consistent in the manner of generation of the noise map 350. One exemplary manner in which to generate the noise map 350 for a particular combination of a noisy image 314 and corresponding reference image 324 is to compute the variance or standard deviation of the noise 312 that is added to the clean image 308 to form the noisy image 314. For example, if a count-independent Gaussian noise method is employed to generate the noise 312, Gaussian noise can be added with a variance V to the clean image 308. The noise map 350 will then be a uniform image with the value V. However, this method only works for training the network 304 where it is known exactly how much noise 312 is added to form the noisy image 314/reference image 324 combination. After training of the network 304 is completed, to subsequently denoise a noisy clinical x-ray image, it is required to estimate the amount of noise in the clinical image in order to form a noise map 350 tailored for that image. Again, there are several ways to provide the estimation, such as using a noise model to estimate the relationship between signal level and noise. The '775 patent uses such a noise model, though it is not required in all applications. For example, an alternative method can revolve around empirically finding the correct noise map value that will completely denoise a featureless noisy image at a given signal count level into a flat image at that same count level. This relationship between signal count level and noise map is characterized by several system and acquisition parameters such as the dose level, source energy, pixel size, detector type, attenuation object medium, calibration values, image display medium, among others, to form a noise map model. Furthermore, the fraction amount 319 can also have an impact on the generation of the noise map. One could account for a high fraction 319 that leads to a reference image 324 that includes high noise levels with a noise map with reduced intensity to signal the denoiser to tune down the denoising strength as to match the noise level in the reference image 324 after denoising the noisy image 314. That intensity reduction of the noise map could take the form of simple relationships such as (1—fraction) or more complex relationships. Alternative to a noise-dependent noise map, one could employ a noise-independent noise map in which the desired denoising intensity between 0 and 1 is passed along with the noisy and clean image during training. In such a case, a noise map of 0 would signify “no denoising”, while 1 would signify “full denoising”. Similarly to noise-dependent noise maps, the noise-independent noise map could account for different values of fraction 319 by reducing its intensity as the fraction value 319 increases during the generation of the noisy 314/reference 324/noise map 350 set”, [0062]).
With respect to claim 10 in particular, Tohme et al. also teach a magnetic resonance scanner (operator to specify the commands and/or scanning parameters, [0045], The DL network denoiser 304 utilizes a method of noise reduction that employs denoising with residual noise to suppress noise and improve noise texture in various imaging applications such as X-ray imaging (in single energy, dual energy, tomography, and image pasting modes), fluoroscopy, mammography, as well as other imaging modalities such as CT (Computed Tomography), PET (Positron Emission Tomography), ultrasound and MRI (Magnetic Resonance Imaging) among others, [0054]).
Xing et al. and Tohme et al. are in the same art of denoising images (Xing et al., abstract; Tohme et al., abstract). The combination of Tohme et al. with Xing et al. enables the use of a noise map indicating a standard deviation of noise. It would have been obvious at the time of filing to combine the noise maps of Tohme et al. with the invention of Xing et al. as this was known at the time of filing, the combination would have predictable results, and as Tohme et al. indicate, “By leaving selected amounts of noise in the digital images, the denoiser can be tuned to improve image attributes and texture,” (abstract), “The DL network denoiser 304 utilizes a method of noise reduction that employs denoising with residual noise to suppress noise and improve noise texture in various imaging applications such as X-ray imaging (in single energy, dual energy, tomography, and image pasting modes), fluoroscopy, mammography, as well as other imaging modalities such as CT (Computed Tomography), PET (Positron Emission Tomography), ultrasound and MRI (Magnetic Resonance Imaging) among others,” ([0054]), and “The use of residual noise 320 in the reference image 324 leads to improved algorithms employed by DL network denoisers 304 which allow for a more tunable denoising, while also enabling the use of the same method 300, 400 to achieve various levels of denoising strength (e.g., Low, Medium, High, etc.) by tuning the fraction 318 without resorting to injecting a portion of the noise back after denoising,” ([0057]) demonstrating an improvement to the customization of the noise removal and allowing the invention to be used in multiple commercial medical applications.
Xing et al. and Tohme et al. do not specify there is more than one bilateral filter. It would have been obvious to one having ordinary skill in the art at the time the invention was made to have multiple filters, since it has been held that mere duplication of the essential working parts of a device involves only routine skill in the art. St. Regis Paper Co. v. Bemis Co., 193 USPQ 8. As Xing et al. indicate more than one weight (part IIIA1) this would also be in effect a different bilateral filter each time depending on which weight is used. It is noted that as in later claims it is specified how multiple bilateral filters are connected this would no longer be just a duplication of parts, and those claims were objected to as allowable.
Regarding claims 12 and 15, Xing et al. and Tohme et al. disclose the system according to claim 10 and non-transitory storage medium of claim 14. Xing et al. further indicate the controller is configured to: provide a further image; filter the image with the plurality of trainable bilateral filters; and output a filtered image (test set, part IVE) [test set indicates at least one image beyond the training image is input to the bilateral filter).
Claim(s) 2-4 and 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xing et al. (“Path Tracing Denoising Based on SURE Adaptive Sampling and Neural Network”) and Tohme et al. (US 20240144441 A1) as applied to claim 1 above, further in view of Metzler et al. (IDS: “Unsupervised Learning with Stein’s Unbiased Risk Estimator”).
Regarding claim 2, Xing et al. and Tohme et al. disclose the training method according to claim 1. Xing et al. and Tohme et al. do not disclose the loss function comprises a norm of a difference between an input image y and an output image f(y).
Metzler et al. teach a loss function comprises a norm of a difference between an input image y and an output image f(y) (
PNG
media_image2.png
198
738
media_image2.png
Greyscale
).
Xing et al. and Tohme et al. and Metzler et al. are in the same art of denoising images (Xing et al., abstract; Tohme et al., abstract; Metzler et al., abstract). The combination of Metzler et al. with Xing et al. and Tohme et al. enables the use of a norm of a difference. It would have been obvious at the time of filing to combine the norm of a difference of Metzler et al. with the invention of Xing et al. and Tohme et al. as this was known at the time of filing, the combination would have predictable results, and as Metzler et al. indicate, “We have made three distinct contributions. First we showed that SURE can be used to denoise an image using a CNN without any training data. Second, we demonstrated that SURE can be used to train a CNN denoiser using only noisy training data. Third, we showed that SURE can be used to
train a neural network, using only noisy measurements, to solve the compressive sensing problem. In the context of imaging, our work suggests a new hands-off approach to reconstruct images. Using SURE, one could toss a sensor into a novel imaging environment and have the sensor itself figure out and then apply the appropriate prior to reconstruct images. In the context of machine learning, our work suggests that divergence may be an overlooked proxy
for variance in an estimator. Thus, while SURE is applicable in only fairly specific circumstances,
penalizing divergence could be applied more broadly as a tool to help attack overfitting” (part 8) demonstrating an improvement by not requiring ground truth data which will simplify the invention and the hands off aspect indicates the improvement to the ease of use of the combination of applications.
Regarding claim 3, Xing et al. and Tohme et al. and Metzler et al. disclose the training method according to claim 2. Metzler et al. further teach the loss function comprises a Euclidean norm in the form of a mean squared error |f(y)−y|2 (
PNG
media_image2.png
198
738
media_image2.png
Greyscale
).
Regarding claim 4, Xing et al. and Tohme et al. and Metzler et al. disclose the training method according to claim 3. Metzler et al. further teach the loss function is based on a physics-driven noise model and incorporates noise maps in the form of:
PNG
media_image3.png
261
558
media_image3.png
Greyscale
(
PNG
media_image4.png
488
730
media_image4.png
Greyscale
PNG
media_image5.png
342
736
media_image5.png
Greyscale
, part 2).
Regarding claim 9, Xing et al. and Tohme et al. disclose the training method according to claim 1. Xing et al. and Tohme et al. do not disclose each one of the plurality of training images is not connected to ground-truth data.
Metzler et al. teach each one of the plurality of training images is not connected to ground-truth data (Learning from unlabeled and noisy data is one of the grand challenges of machine learning. As such, it has seen a flurry of research with new ideas proposed continuously. In this work, we revisit a classical idea: Stein’s Unbiased Risk Estimator (SURE). We show that, in the context of image recovery, SURE and its generalizations can be used to train convolutional neural networks (CNNs) for a range of image denoising and recovery problems without any ground truth data, abstract, Two concurrent and independently developed works overlap with some of our contributions. Specifically, [10] demonstrates that SURE can be used to train CNN based denoisers without ground truth data and [11] demonstrates that SURE can be used to train CNNs for compressive sensing using only noisy compressive measurements, part 1, The
main novelty of this section is that we do not have training labels (i.e., ground truth images). Instead and unlike conventional learning approaches in compressive sensing (CS), we train the recovery algorithm using only noisy undersampled linear measurements, part 5).
Xing et al. and Tohme et al. and Metzler et al. are in the same art of denoising images (Xing et al., abstract; Tohme et al., abstract; Metzler et al., abstract). The combination of Metzler et al. with Xing et al. and Tohme et al. enables calculation without needing ground truth data. It would have been obvious at the time of filing to combine the calculation of Metzler et al. with the invention of Xing et al. and Tohme et al. as this was known at the time of filing, the combination would have predictable results, and as Metzler et al. indicate, “We have made three distinct contributions. First we showed that SURE can be used to denoise an image using a CNN without any training data. Second, we demonstrated that SURE can be used to train a CNN denoiser using only noisy training data. Third, we showed that SURE can be used to
train a neural network, using only noisy measurements, to solve the compressive sensing problem. In the context of imaging, our work suggests a new hands-off approach to reconstruct images. Using SURE, one could toss a sensor into a novel imaging environment and have the sensor itself figure out and then apply the appropriate prior to reconstruct images. In the context of machine learning, our work suggests that divergence may be an overlooked proxy
for variance in an estimator. Thus, while SURE is applicable in only fairly specific circumstances,
penalizing divergence could be applied more broadly as a tool to help attack overfitting” (part 8) demonstrating an improvement by not requiring ground truth data which will simplify the invention and the hands off aspect indicates the improvement to the ease of use of the combination of applications.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xing et al. (“Path Tracing Denoising Based on SURE Adaptive Sampling and Neural Network”) and Tohme et al. (US 20240144441 A1) and Metzler et al. (IDS: “Unsupervised Learning with Stein’s Unbiased Risk Estimator”) as applied to claim 4 above, further in view of Anand et al. (IDS: “Wavelet domain non-linear filtering for MRI denoising”).
Regarding claim 5, Xing et al. and Tohme et al. and Metzler et al. disclose the training method according to claim 4. Xing et al. and Tohme et al. and Metzler et al. do not disclose the plurality of training images comprise MRI images calculated by a reconstruction algorithm from k-space data, and wherein the plurality of noise maps are calculated by propagating a noise distribution through the reconstruction algorithm based upon an initial noise associated with a noise adjustment scan.
Anand et al. teach the plurality of training images comprise MRI images calculated by a reconstruction algorithm from k-space data, and wherein the plurality of noise maps are calculated by propagating a noise distribution through the reconstruction algorithm based upon an initial noise associated with a noise adjustment scan (
PNG
media_image6.png
690
466
media_image6.png
Greyscale
, part 2.2).
Xing et al. and Tohme et al. and Anand et al. are in the same art of denoising images (Xing et al., abstract; Tohme et al., abstract; Anand et al., abstract). The combination of Anand et al. with Xing et al. and Tohme et al. enables the use of k-space data and initial noise. It would have been obvious at the time of filing to combine the k-space data of Anand et al. with the invention of Xing et al. and Tohme et al. as this was known at the time of filing, the combination would have predictable results, and as Anand et al. indicate, “Denoising is done in the square magnitude domain, where the noise tends to be signal independent and is additive. The proposed method has been adapted specifically to Rician noise. The visual and the diagnostic quality of the denoised image is well preserved” (abstract), thereby suggesting an image quality improvement when inventions are combined.
Claim(s) 13 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Xing et al. (“Path Tracing Denoising Based on SURE Adaptive Sampling and Neural Network”) and Tohme et al. (US 20240144441 A1) as applied to claim 1 above, further in view of Aberet et al. (US 20230298162 A1).
Regarding claim 13 and 16, Xing et al. and Tohme et al. disclose the magnetic resonance imaging system according to claim 12 and non-transitory storage medium of claim 15. Xing et al. and Tohme et al. do not disclose the controller is configured to: provide an image dataset comprising a plurality of images; and averaging the plurality of images to generate the further image after performing a phase correction.
Aberet et al. teach a controller is configured to: provide an image dataset comprising a plurality of images; and averaging the plurality of images to generate the further image after performing a phase correction (In one embodiment, the phase correction is applied 406 to each of the ground truth representations 400, and the loss 300 used in the training is based on the aggregation (e.g., complex-valued averaging 410) of the images 408 across repetitions. The aggregation of the ground truth representations 408 formed after the phase correction is applied 406 is an average or another combination in a complex-value domain. After applying the phase correction 406 to each individual target repetition, the target repetitions are averaged in the complex-value domain, and this averaged target is used in the loss calculation. In the deepset approach represented in FIGS. 3 and 4, the network outputs 304 after phase correction 412 are averaged or otherwise combined 320 in the complex-values domain, as done for the target combination 410. The loss 330 is then calculated as a complex-value loss or difference between complex values, [0078]).
Xing et al. and Tohme et al. and Aberet et al. are in the same art of denoising images (Xing et al., abstract; Tohme et al., abstract; Aberet et al., abstract). The combination of Aberet et al. with Xing et al. and Tohme et al. enables phase correction of images. It would have been obvious at the time of filing to combine the phase corection of Aberet et al. with the invention of Xing et al. and Tohme et al. as this was known at the time of filing, the combination would have predictable results, and as Aberet et al. indicate, “The phase correction, based on the phase map, is applied to the ground truth and/or the output of the model in training. The resulting machine-learned model may better reconstruct an image as a result of having been trained using phase correction” (abstract) thereby providing a quality improvement to the output image when the inventions are combined.
Allowable Subject Matter
Claims 6-8 and 11 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following art is cited as relevant:
“Retinex by Two Bilateral Filters”:
PNG
media_image7.png
850
922
media_image7.png
Greyscale
“Deep Cascade Residual Networks (DCRNs): Optimizing an Encoder–Decoder Convolutional Neural Network for Low-Dose CT Imaging”:
PNG
media_image8.png
508
802
media_image8.png
Greyscale
PNG
media_image9.png
244
404
media_image9.png
Greyscale
PNG
media_image10.png
194
420
media_image10.png
Greyscale
PNG
media_image11.png
170
412
media_image11.png
Greyscale
“Bilateral Filter: Graph Spectral Interpretation and Extensions”:
PNG
media_image12.png
206
898
media_image12.png
Greyscale
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M ENTEZARI HAUSMANN whose telephone number is (571)270-5084. The examiner can normally be reached 10-7 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent M Rudolph can be reached at (571) 272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE M ENTEZARI HAUSMANN/Primary Examiner, Art Unit 2671