DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 12 is objected to because of the following informalities: Claim 12, line 5, recites “…prediction. Rejection…”, there is an extra period (.) in the claim. Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-7 and 9-15 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Maier-Hein et al., US 2022/0012874.
Regarding claim 1, Maier-Hein discloses a medical system (para 0001; a method and system for augmented imaging of tissue using multispectral information) comprising:
a memory configured to store machine executable instructions (para 0276; The machine learning module 32 may be a software module installed on the computing device 30. The computing device 30 can be a PC or workstation and preferably has a graphics processing unit (GPU) suitable for rapid processing of the machine learning based processes), wherein the memory further stores a trainable machine learning module trained using training data descriptive of a training data distribution to output a reconstructed medical image in response to receiving measured medical image data as input (para 0025-0030 and 0101; said method is for generating said one or more augmented images of tissue of a patient; an autoencoder is a neural network that is trained by unsupervised learning, and which is trained to learn reconstructions that are close to its original input, wherein the input is a medical image; the machine learning model generating the augmented images and the autoencoder that is used for out-of-distribution (OoD) detection correspond to the trainable machine learning module), wherein the memory further stores an out-of-distribution estimation module configured for outputting an out-of-distribution score in response to receiving the measured medical image data, wherein the out-of-distribution score is descriptive of a probability that the measured medical image is within the training data distribution (para 0020; an “out-of-distribution” (OoD) detection algorithm which is also referred to as “out of domain” detection algorithm in the art, is applied to said at least part of said one or more spectral images, or an image derived from said multispectral image), wherein the memory further stores an in-distribution accuracy estimation module configured for outputting an in-distribution accuracy score descriptive of a probability that the reconstructed medical image is accurate (para 0025-0030 and 0392; the uncertainty estimation is carried out, based on a full posterior probability distribution corresponds to the probability that the result of the regression is accurate; in the case of a regressor that generates the augmented images, the accuracy of the augmented images is estimated);
a computational system, wherein execution of the machine executable instructions causes the computational system to:
receive the measured medical image data (fig. 22; para 0392-0394; a multispectral image);
determine the out-of-distribution score and the in-distribution accuracy score consecutively in an order determined a sequence, wherein the out-of-distribution score is determined by inputting the measured medical image data into the out-of-distribution estimation module, wherein the in-distribution accuracy score is determined by inputting the measured medical image data into the in-distribution accuracy estimation module (fig. 22; para 0392-0394; out-of-distribution (OoD) detection and uncertainty estimation (i.e., in-distribution));
detect a rejection of the measured medical image data using the out-of-distribution score and/or the in-distribution accuracy score during execution of the sequence (fig. 22; para 0392-0394; The second line of FIG. 22 shows a case where the spectrum deviates significantly from spectra that had been considered during training, such that the OoD detection determines the spectrum to lie “out of domain” or “out of distribution”. In this case, the regression cannot give meaningful results, and is therefore not carried out; Finally, in the third line of FIG. 22, the spectrum was found to be sufficiently close to the training data, but the posterior probability distribution established in the regression has two peaks, indicating that the result is ambiguous. For situations like this, it is advantageous that a full posterior probability distribution is established rather than just a point estimate for each tissue parameter; OoD is executed before regression and uncertainty estimation (i.e., in-distribution) is executed after regression); and
provide a warning signal if the rejection of the measured medical image data is detected (para 0102; said OoD detection is carried out to determine the closeness of the multispectral image or part of said multispectral image to said given training data set, or to a related training data set, and if the closeness is found to be insufficient, the functional tissue parameter is not determined or is marked as unreliable).
Regarding claim 2, the medical system of claim 1, Maier-Hein further discloses wherein execution of the machine executable instructions further causes the computational system to provide the reconstructed medical image by at least partially inputting the measured medical image data into the trainable machine learning module after completion of the sequence (fig. 22; para 0392-0394).
Regarding claim 3, the medical system of claim 1, Maier-Hein further discloses wherein the memory further stores an anomaly detection estimation module configured to output an anomaly estimation score in response to receiving the measured medical image data, wherein the anomaly estimation score is descriptive of a probability that the measured medical image is anomalous in comparison to the training data distribution, wherein execution of the machine executable instructions further causes the computational system to determine the anomaly estimation score consecutively with the out-of-distribution score and the in-distribution accuracy score in the order determined by the sequence, wherein the anomaly estimation score is determined by inputting the measured medical image data into the anomaly detection estimation module, and wherein execution of the machine executable instructions further causes the computational system to detect a rejection of the measured medical image data using the anomaly estimation score during execution of the sequence (para 0101).
Regarding claim 4, the medical system of claim 3, Maier-Hein further discloses wherein the sequence is predetermined, wherein the measured medical image data is input into the anomaly detection estimation module before the measured medical image data is input into the out-of-distribution estimation module, and wherein the measured medical image data is input into the out-of-distribution estimation module before the measured medical image data is input into the in-distribution accuracy estimation module (para 0101).
Regarding claim 5, the medical system of claim 3, Maier-Hein further discloses wherein the anomaly detection estimation module comprises at least one of the following: an autoencoder trained with samples from the training data distribution, wherein the anomaly estimation score is provided as a measure of difference between an input and an output of the autoencoder; or a density based algorithm configured using predetermined features (para 0101).
Regarding claim 6, the medical system of claim 1, Maier-Hein further discloses wherein the memory further contains an image classifier neural network trained to determine the sequence in response to receiving the measured medical image data as input, wherein execution of the machine executable instructions further causes the computational system to: determine the sequence by inputting the measured medical image into the image classifier neural network (para 0152 and 0433).
Regarding claim 7, the medical system of claim 1, Maier-Hein further discloses wherein the medical system further comprises a medical imaging system, wherein execution of the machine executable instructions further causes the processor to control the medical imaging system to acquire the measured medical image data (para 0050).
Regarding claim 9, the medical system of claim 1, Maier-Hein further discloses wherein the warning signal causes at least one of the following: a reacquisition of the measured medical image data, a display of the warning signal on a display, appending metadata descriptive of the warning signal and/or the measured medical image data to the reconstructed medical image, or abort use of the trainable machine learning module and selecting an alternative reconstruction algorithm to reconstruct the reconstructed medical image (para 0102).
Regarding claim 10, the medical imaging system of claim 1, Maier-Hein further discloses wherein measured medical image data is formatted in image space, and wherein the trainable machine learning module is formatted as an image processing module (para 0031).
Regarding claim 11, the medical system of claim 1, Maier-Hein further discloses wherein the measured medical image data comprises at least one of the following: wherein the measured medical image data comprises medical imaging system measurements; or wherein the measured medical image data comprises medical imaging system measurements and image space data (para 0031).
Regarding claim 12, the medical system of claim 1, Maier-Hein further discloses wherein the out-of-distribution estimation module is implemented using at least one of the following: computing the output of several trained neural networks and compute the variance of the prediction. Rejection is performed if the variance is higher than a given threshold; a density-based rejection algorithm based on predetermined features to perform out-of-distribution estimation; or a statistical characterization of hidden layer neural activation (para 0093-0096).
Regarding claim 13, the medical system of claim 1, Maier-Hein further discloses wherein the trainable machine learning module is configured to output multiple versions of the reconstructed medical image using different random initializations, wherein the in-distribution accuracy score is determined using a statistical comparison between the multiple versions (para 0093-0096).
Regarding claim 14, this claim recites substantially the same limitations that are performed by claim 1 above, and it is rejected for the same reasons.
Regarding claim 15, this claim recites substantially the same limitations that are performed by claim 1 above, and it is rejected for the same reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Maier-Hein et al., US 2022/0012874 in view of Bunn, US 2024/0215945.
Regarding claim 8, the medical system of claim 7, Maier-Hein discloses wherein the medical imaging system (fig. 2; para 0274).
Maier-Hein discloses claim 8 as enumerated above, but Maier-Hein does not explicitly disclose at least one of the following: a magnetic resonance imaging system, a computed tomography system, a positron emission tomography system, a single photon emission tomography system, an ultrasound system, an X-ray system, or a digital fluoroscope system as claimed.
However, Bunn discloses a neural network is trained on a large dataset of medical images or raw data from various imaging modalities, such as ultrasound, MRI, CT, and X-ray, which are labeled with ground truth diagnoses of a wide range of medical conditions (Abstract).
Therefore, taking the combined disclosures of Maier-Hein and Bunn as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a neural network is trained on a large dataset of medical images or raw data from various imaging modalities, such as ultrasound, MRI, CT, and X-ray, which are labeled with ground truth diagnoses of a wide range of medical conditions as taught by Bunn into the invention of Maier-Hein for the benefit of early detection and prediction of diseases, objective symptom measurement, treatment optimization, surgical planning, and real-time monitoring (Bunn: para 0002).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ren et al., US 2022/0253747 discloses systems and method to perform improved detection of out-of-distribution (OOD) inputs.
Feng et al., US 2021/0374524 discloses methods and systems for detecting out-of-distribution (ODD) data.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAN D HUYNH whose telephone number is (571)270-1937. The examiner can normally be reached 8AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN D HUYNH/Primary Examiner, Art Unit 2665