DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-12 are pending.
Information Disclosure Statement
The Information Disclosure Statement filed on 01/26/2024 is in compliance with the provisions of 37 CFR 1.97 and have been considered. An initialed copy of the Form 1449 is enclosed herewith.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, and 9-12 are rejected under 35 U.S.C. 103 as being unpatentable over Liao et al., US 2017/0337682 in view of Kearney et al., US 2022/0180447 further in view of Dewey et al., "A Disentangled Latent Space for Cross-Site MRI Harmonization" MICCAI 2020, LNCS 12267, pp. 720-729, 2020.
Regarding claim 1, Liao discloses an MR image processing method that repeats, by a computing device (training images are obtained and/or generated by acquiring MRI images by MRI scanner from the same patient at different times, paragraph 41), a unit transformation method N number of times (repeating the steps ‘N” or predetermined number of time such that steps are repeated a large number of times to generate a large number of training datasets, paragraphs 139, 55, 59), wherein, the unit transformation method in nth iteration comprises:
generating, by the computing device, a corrected image corrected from a prepared nth image using a predetermined generative model (when it is determined that the transformation parameters have converged to a correct pose/image where a predetermined maximum number of iterations have been performed, wherein MRI simulator or machine learning based simulators (e.g., training a generative adversarial network (GAN) can be used to simulate an imaging modality from another one, the GAN being trained on paired images, paragraphs 69, 42);
calculating, by the computing device, a differential value of a distance between the corrected image and a source image (distance metrics are computed to compare the known dense correspondences for the image pairs selected from pool B in each training dataset to dense correspondences artificially generated by concatenating the dense correspondences between corrected images, they represent the real correspondences between the images, the concatenated dense correspondence between two images should be the same as the real dense correspondence using neural network, paragraphs 142-143,);
Liao fails to explicitly disclose generating a corrected image corrected using a predetermined reversible generative model; and generating, by the computing device, a harmonized image by subtracting differential value from the corrected image; wherein, the generating the corrected image comprises: generating a predetermined array by inputting the nth image into the reversible generative model in a forward direction of the reversible generative model; and generating the corrected image by inputting a scaled array in a reverse direction of the reversible generative model, the scaled array being obtained by scaling a value of each element of the generated array by a predetermined scaling factor (1-α) (0<α<1).
However, Kearney teaches generating a corrected image corrected using a predetermined reversible generative model and a differential value between the corrected image and a source image (generator 618 transforms the synthetic image 622 into a synthetic source domain image 624, paragraph 174, which is again input to the generator 618 to obtain another synthetic source domain image 624, paragraph 176, wherein source input image and synthetic target image are generated by the illustrated reverse GAN network (generator 618 and discriminator 620) and may be used in spatial congruence encouraged by evaluating L1 loss (loss function LF3) at Step 5 and evaluating L1 loss (loss function LF6) at Step 9, paragraphs 179, 182);
generating by inputting nth image into the reversible generative model in a forward direction of the reversible generative model (generator 618 transforms the synthetic image 622 into a synthetic source domain image 624, paragraph 174, which is again input to the generator 618 to obtain another synthetic source domain image 624, paragraph 176, wherein, the illustrated reverse GAN network (generator 618 and discriminator 620) may be used in combination with the illustrated forward GAN network (generator 612 and discriminator 614), paragraph 182);
and generating the corrected image by inputting a scaled array in a reverse direction of the reversible generative model, the scaled array being obtained by scaling a value of each element of the generated array by a predetermined scaling factor (1-α) (0<α<1) (generator 618 transforms the synthetic image 622 into a synthetic source domain image 624, by performing comparison of the synthetic target domain image 622 and the target domain image 606 that was input to the generator 618, which promotes spatial congruence between the source input image 604 and synthetic target image 622 by working as reverse GAN network, wherein, GAN disclosed herein, the generator 712 may include seven multi-scale stage deep encoder-decoder generators, such as using the approach described above with respect to the generator 512. For the machine learning model 710, the output channels of the generator 712 may be passed through a 1×1 convolutional layer as for the generator 512. However, the 1×1 convolution layer may further include a sigmoidal activation function to produce tooth labels. The generator 712 may likewise have stages of a different size than the generator 512, e.g., an input stage of 256×256 with down sampling by a factor of two between stages, paragraphs 174-183, 190-192).
Liao and Kearney are combinable because they both are in the same field of endeavor dealing with neural networks to perform image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao to incorporate the teachings of Kearney for having effective machine learning where the desired inputs of a training data entry are processed to obtain predictions which are compared to the desired outputs to obtain an efficient loss function as taught by Kearney at paragraph 849.
Combination of Liao with Kearney fails to explicitly teach generating, by computing device, a harmonized image by subtracting differential value from the corrected image; wherein, the generating the corrected image comprises: generating a predetermined array by inputting the nth image into generative model.
However, Dewey teaches generating, by computing device (Phillips 3T scanners, page 77, 2nd paragraph), a harmonized image by subtracting differential value from the corrected image (theta values from the reconstructed images were deducted to create a harmonized image, page 727, 2nd paragraph, page 726, 2nd paragraph); wherein, the generating the corrected image (reconstructed image, page 725, 2nd paragraph) comprises: generating a predetermined array by inputting nth image into the generative model (estimation of reconstruction loss concerning reconstruction of images from input images where difference in predetermined theta array values are computed number of times by machine learning models, page 721, 3rd paragraphs and pages 725-726).
Liao and Kearney are combinable with Dewey because they all are in the same field of endeavor dealing with neural networks to perform image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao and Kearney to incorporate the teachings of Dewey to provide effective MRI image harmonization without the need of extensive paired data between different sites as taught by Dewey at abstract, page 720.
Regarding claim 2, Combination of Liao with Kearney and Dewey further teaches wherein the reversible generative model is a prior model (Kearney, GAN generator 612 is a forward network while GAN 618 is a reverse/secondary (former) network, paragraphs 169, 182).
Liao and Kearney are combinable because they both are in the same field of endeavor dealing with neural networks to perform image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao to incorporate the teachings of Kearney for having effective machine learning where the desired inputs of a training data entry are processed to obtain predictions which are compared to the desired outputs to obtain an efficient loss function as taught by Kearney at paragraph 849.
Regarding claim 3, Combination of Liao with Kearney and Dewey further teaches wherein the nth image (xn) input to the reversible generative model (fθ) in the nth iteration of the unit transformation method is the harmonized image (xn) (Dewey, theta values from the reconstructed images were deducted to create a harmonized image, page 727, 2nd paragraph, page 726, 2nd paragraph) generated in (n-1)th iteration of the unit transformation method (n=2, ..., N) (Kearney, generator 618 transforms the synthetic image 622 into a synthetic source domain image 624, by performing comparison of the synthetic target domain image 622 and the target domain image 606 that was input to the generator 618, which promotes spatial congruence between the source input image 604 and synthetic target image 622 by working as reverse GAN network, paragraphs 174-183 and method may be repeated any number of times, paragraph 829).
Liao and Kearney are combinable with Dewey because they all are in the same field of endeavor dealing with neural networks to perform image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao and Kearney to incorporate the teachings of Dewey to provide effective MRI image harmonization without the need of extensive paired data between different sites as taught by Dewey at abstract, page 720.
Regarding claim 4, Combination of Liao with Kearney and Dewey further teaches wherein, the reversible generative model (fθ) is learned using only images belonging to a first domain, in a first iteration (n=1) of the unit transformation method , a first image (x1) input to the reversible generative model (fθ) is either a random image or one of images belonging to the first domain, or a representative image of images belonging to the first domain (Kearney, training algorithm 602 takes as inputs images 604 from a source domain (first imaging modality, e.g., a distorted image domain) and images 606 from a target domain (second imaging modality, e.g., a non-distorted image domain or domain that is less distorted than the first domain). The images 604 and 606 are unpaired in some embodiments, meaning the images 606 are not transformed versions of the images 504 or paired such that an image 604 has a corresponding image 606 visualizing the same patient's anatomy. Instead, the images 506 may be selected from a repository of images and used to assess the transformation of the images 604 using the machine learning model 610 and images are inputted into reversible GAN 618, paragraphs 168, 169, 179-183 is a random image such as random forests machine learning method or the random mask 1206 includes one or more random sequences of characters, each random sequence being placed either randomly on the image 1204, paragraphs 260, 265).
Liao and Kearney are combinable because they both are in the same field of endeavor dealing with neural networks to perform image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao to incorporate the teachings of Kearney for having effective machine learning where the desired inputs of a training data entry are processed to obtain predictions which are compared to the desired outputs to obtain an efficient loss function as taught by Kearney at paragraph 849.
Regarding claim 5, Combination of Liao with Kearney and Dewey further teaches wherein, the reversible generative model (fθ) is learned using only images belonging to a first domain, and the source image (xs) is an image belonging to a second domain different from the first domain (Kearney, source and target domains are not spatially aligned such as reverse GAN is learned using first domain, wherein source input images are belonging to different second domain, 168, 169, 179-183).
Liao and Kearney are combinable because they both are in the same field of endeavor dealing with neural networks to perform image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao to incorporate the teachings of Kearney for having effective machine learning where the desired inputs of a training data entry are processed to obtain predictions which are compared to the desired outputs to obtain an efficient loss function as taught by Kearney at paragraph 849.
Regarding claim 9, Combination of Liao with Kearney and Dewey further teaches wherein, first domain is a domain composed of images output by a first MR scanner, and second domain is a domain composed of images output by a second MR scanner (Dewey, images belonging to first site (first domain) and images belonging to different site (second domain) are acquired by two different Phillips 3T MR scanners, see abstract, page 722, 2nd paragraph and pages 726-727 mentioning analyzing theta values from all sites to create a harmonized image).
Liao and Kearney are combinable with Dewey because they all are in the same field of endeavor dealing with neural networks to perform image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao and Kearney to incorporate the teachings of Dewey to provide effective MRI image harmonization without the need of extensive paired data between different sites as taught by Dewey at abstract, page 720.
Regarding claim 10, Combination of Liao with Kearney and Dewey further teaches wherein, the reversible generative model (fθ) is learned using only images belonging to a first domain, a method for learning the reversible generative model (fθ) comprises: inputting a selected image belonging to the first domain into the reversible generative model in the forward direction of the reversible generative model to generate an array from the reversible generative model; and changing parameters of the reversible generative model to reduce a difference between distributions of images belonging to the first domain and the generated array (Kearney, training algorithm 602 takes as inputs images 604 from a source domain (first imaging modality, e.g., a distorted image domain) and images 606 from a target domain (second imaging modality, e.g., a non-distorted image domain or domain that is less distorted than the first domain). The images 604 and 606 are unpaired in some embodiments, meaning the images 606 are not transformed versions of the images 504 or paired such that an image 604 has a corresponding image 606 visualizing the same patient's anatomy. Instead, the images 506 may be selected from a repository of images and used to assess the transformation of the images 604 using the machine learning model 610 and images are inputted into reversible GAN 618, wherein, generator 618 transforms the synthetic image 622 into a synthetic source domain image 624, paragraph 174, which is again input to the generator 618 to obtain another synthetic source domain image 624, paragraph 176, wherein, the illustrated reverse GAN network (generator 618 and discriminator 620) may be used in combination with the illustrated forward GAN network (generator 612 and discriminator 614), and generator 618 transforms the synthetic image 622 into a synthetic source domain image 624, by performing comparison of the synthetic target domain image 622 and the target domain image 606 that was input to the generator 618, which promotes spatial congruence between the source input image 604 and synthetic target image 622 by working as reverse GAN network, paragraphs 168-183 and method may be repeated any number of times, paragraph 829).
Liao and Kearney are combinable because they both are in the same field of endeavor dealing with neural networks to perform image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao to incorporate the teachings of Kearney for having effective machine learning where the desired inputs of a training data entry are processed to obtain predictions which are compared to the desired outputs to obtain an efficient loss function as taught by Kearney at paragraph 849.
Regarding claim 11, Combination of Liao with Kearney and Dewey further teaches an MR image processing method, comprising: generating, by an MRI scanner (Liao, training images are obtained and/or generated by acquiring MRI images by MRI scanner from the same patient at different times, paragraph 41), a source image (xs) (Kearney, input image, paragraph 182); and repeating, by a computing device (Liao, repeating the steps ‘N” or predetermined number of time such that steps are repeated a large number of times to generate a large number of training datasets, paragraphs 139, 55, 59)…a unit transformation method N number of times, the unit transformation method being a method to transform the generated source image (xs) into a harmonized image (Dewey, theta values from the reconstructed images were deducted to create a harmonized image, page 727, 2nd paragraph, page 726, 2nd paragraph), wherein, the unit transformation method in nth iteration comprises: Rest of claim recites similar features as claim 1 and thus is rejected on the same rationale.
Regarding claim 12, Combination of Liao with Kearney and Dewey further teaches an MR image processing system, comprising: an MRI scanner (Liao, training images are obtained and/or generated by acquiring MRI images by MRI scanner from the same patient at different times, paragraph 41); and a computing device, wherein, the MRI scanner is configured to scan a scan object to generate a source image (xs) (Liao, training images are obtained and/or generated by acquiring MRI images by MRI scanner from the same patient at different times, paragraph 41), the computing device is configured to repeat a unit transformation method N number of times (Liao, repeating the steps ‘N” or predetermined number of time such that steps are repeated a large number of times to generate a large number of training datasets, paragraphs 139, 55, 59), the unit transformation method being a method to transform the generated source image (xs) into a harmonized image, in nth iteration (n=1, ..., N) of the unit transformation method (Dewey, theta values from the reconstructed images were deducted to create a harmonized image, page 727, 2nd paragraph, page 726, 2nd paragraph). Rest of claim recites similar features as claim 1 and thus is rejected on the same rationale.
Claims 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Liao et al., US 2017/0337682 in view of Kearney et al., US 2022/0180447 further in view of Dewey et al., "A Disentangled Latent Space for Cross-Site MRI Harmonization" MICCAI 2020, LNCS 12267, pp. 720-729, 2020 as applied in claim 1 above and further in view of Sofka et al., US 2025/0148601.
Regarding claim 6, Combination of Liao with Kearney and Dewey further teaches inputting, by the computing device, a harmonized image (xN+1) (Dewey, theta values from the reconstructed images were deducted to create a harmonized image, page 727, 2nd paragraph, page 726, 2nd paragraph) generated by the unit transformation method performed by the nth iteration into a predetermined network learned using images belonging to the first domain (Kearney, training algorithm 602 takes as inputs images 604 from a source domain (first imaging modality, e.g., a distorted image domain) and images 606 from a target domain (second imaging modality, e.g., a non-distorted image domain or domain that is less distorted than the first domain, generator 618 transforms the synthetic image 622 into a synthetic source domain image 624, by performing comparison of the synthetic target domain image 622 and the target domain image 606 that was input to the generator 618, which promotes spatial congruence between the source input image 604 and synthetic target image 622 by working as reverse GAN network, paragraphs 174-183); and obtaining, by the computing device, an estimate from the network according to the inputted harmonized image (xN+1) (Dewey, estimation of reconstruction loss concerning reconstruction of images from input images where difference in predetermined theta array values are computed number of times by machine learning models, page 721, 3rd paragraphs and pages 725-726).
Liao and Kearney are combinable with Dewey because they all are in the same field of endeavor dealing with neural networks to perform image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao and Kearney to incorporate the teachings of Dewey to provide effective MRI image harmonization without the need of extensive paired data between different sites as taught by Dewey at abstract, page 720.
Combination of Liao with Kearney and Dewey fails to explicitly teach inputting image, by computing device, into a predetermined estimation network learned using images; and obtaining, by the computing device, an estimate from the estimation network according to the inputted image.
However, Sofka teaches inputting image, by computing device, into a predetermined estimation network learned using images; and obtaining, by the computing device, an estimate from the estimation network according to the inputted image (computing device 104 can initialize or place at least one seed within the input image (e.g., brain MRI image) at a desired location (e.g., such as the selected location at operation 212). The seed can correspond to a starting point for lesion growth (e.g., growing the lesion mask). In some cases, the seed can represent the location from which the abnormality starts to grow. The number of seeds can be adjusted/updated according to the desired number of lesion locations/areas to be grown. In some cases, the seed can be represented by an intensity or a contrast of a pixel or voxel within the image, paragraph 89).
Liao, Kearney, and Dewey are combinable with Sofka because they all are in the same field of endeavor dealing with neural networks to perform medical image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao, Kearney and Dewey to incorporate the teachings of Sofka to be able to implement effective multi-shot magnetic resonance (MR) imaging as taught by Sofka at paragraph 2.
Regarding claim 7, Combination of Liao with Kearney, Dewey and Sofka further teaches wherein, the source image is a first MRI image of a brain, and the estimation network is a network that outputs a location of a tumor (Sofka, the computing device 104 can simulate various types of abnormalities for detection, such as but not limited to stroke, hemorrhage, tumor, paragraphs 115, 140) among the first MRI image of the brain when the first MRI image of the brain is input to the estimation network (Sofka, computing device 104 can initialize or place at least one seed within the input image (e.g., brain MRI image) at a desired location (e.g., such as the selected location at operation 212). The seed can correspond to a starting point for lesion growth (e.g., growing the lesion mask). In some cases, the seed can represent the location from which the abnormality starts to grow. The number of seeds can be adjusted/updated according to the desired number of lesion locations/areas to be grown. In some cases, the seed can be represented by an intensity or a contrast of a pixel or voxel within the image, paragraph 89).
Liao, Kearney, and Dewey are combinable with Sofka because they all are in the same field of endeavor dealing with neural networks to perform medical image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao, Kearney and Dewey to incorporate the teachings of Sofka to be able to implement effective multi-shot magnetic resonance (MR) imaging as taught by Sofka at paragraph 2.
Regarding claim 8, Combination of Liao with Kearney, Dewey and Sofka further teaches wherein, the source image is an MRI image of a human body (Sofka, MRI systems may be utilized to generate images of the inside of the human body such as MRI system can be utilized to generate images of the tissue of the subject associated with at least one body part, paragraphs 3, 64), and the estimation network is a network that outputs a location of a lesion in the MRI image of the human body when the MRI image of the human body is input to the estimation network (Sofka, computing device 104 can initialize or place at least one seed within the input image (e.g., brain MRI image) at a desired location (e.g., such as the selected location at operation 212). The seed can correspond to a starting point for lesion growth (e.g., growing the lesion mask). In some cases, the seed can represent the location from which the abnormality starts to grow. The number of seeds can be adjusted/updated according to the desired number of lesion locations/areas to be grown. In some cases, the seed can be represented by an intensity or a contrast of a pixel or voxel within the image, paragraph 89).
Liao, Kearney, and Dewey are combinable with Sofka because they all are in the same field of endeavor dealing with neural networks to perform medical image processing.
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Liao, Kearney and Dewey to incorporate the teachings of Sofka to be able to implement effective multi-shot magnetic resonance (MR) imaging as taught by Sofka at paragraph 2.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Athanasiou, US 2025/0134387
Guerreiro et al., US 2024/0212102
Kudo, US 2023/0214664
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAWANDEEP DHINGRA whose telephone number is (571) 270-1231. The examiner can normally be reached 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abderrahim Merouan can be reached at (571) 270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAWAN DHINGRA/Examiner, Art Unit 2683
/ABDERRAHIM MEROUAN/Supervisory Patent Examiner, Art Unit 2683
Prosecution Insights