DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of the application’s status as a continuation of EP 23195170.8
Information Disclosure Statement
The information disclosure statement (IDS) was submitted on 08/30/2024. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Status
Claims 1, 3, 11, and 13 are interpreted under 112(f).
Claim(s) 1, 5, 12, 14, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Vazquez (US 20230129194 A1) in view of Ghodrati (US 20240037815 A1).
Claim(s) 3, 4, 6, 7, 9, 10, 15, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Vazquez (US 20230129194 A1) in view of Ghodrati (US 20240037815 A1) and in further view of Park (US 20200034948 A1).
Claims 2, 8, 11, 13, 16, 18-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“selection unit configured to…” in claims 1 and 13.
“reconstruction module configured to…” in claims 3 and 11
“loss module configured to…” in claim 3
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5, 12, 14, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Vazquez (US 20230129194 A1) in view of Ghodrati (US 20240037815 A1).
Regarding claim 1, discloses A reconstruction network for reconstructing cine MRI images, the reconstruction network being a variational network configured to reconstruct images from cine MRI data, the reconstruction network comprising: (¶41 “the reconstruction of a 3D MRI volume from partial 2D Cine-MRI images using a conditional variational auto-encoder architecture along with a spatial transformation network layer.”)
Vazquez fails to specifically disclose an architecture with a cascade of cascade modules, wherein
an input of a first cascade module is an input-stack of a plurality of N frames, and
an input of each cascade module, following the first cascade module, is the input-stack and an output-stack of a preceding cascade module; and
a selection unit configured to select, as a basis for an output dataset, a single frame being processed by the cascade modules and that corresponds to an i-th frame of the input-stack.
In related art, Ghodrati discloses an architecture with a cascade of cascade modules, wherein (Ghodrati: ¶35 :Deep learning system 40 includes a series of cascading image enhancing stages/convolutional stages 44”)
an input of a first cascade module is an input-stack of a plurality of N frames, and (Ghodrati: ¶1 discloses capturing cinematic frames from cine-mode MRI. ¶35 discloses passing image frames to the cascading image enhancing stages )
an input of each cascade module, following the first cascade module, is the input-stack and an output-stack of a preceding cascade module; and (Ghodrati: ¶35 “a series of cascading image enhancing stages/convolutional stages 44 that each produce enhanced output image data from input image data, a first stage receiving the undersampled MRI image data while each remaining stage receives the output image data from a previous stage.”)
a selection unit configured to select, as a basis for an output dataset, a single frame being processed by the cascade modules and that corresponds to an i-th frame of the input-stack. (Ghodrati: ¶35 “each produce enhanced output image data from input image data, a first stage receiving the undersampled MRI image data while each remaining stage receives the output image data from a previous stage.” ¶42 “Undersampled images 42 include a series of undersampled dynamic/cine MRI images that progress temporally.” Ghodrati discloses while a group of images are considered, the system processes images frame-by-frame )
Therefore, it would have been obvious to for one of ordinary skill in the art before the effective filing date to incorporate recreating under sampled MRI image data using cascading stages disclosed by Ghodrati into the method reconstructing 3D MRI anatomy from 2D cine-MRI images using a variational autoencoder disclosed by Vazquez to improve quality of reconstructed MRI images from video image frames using series of image enhancing steps.
Regarding claim 5, Vazquez, as modified by Ghodrati, disclose wherein the reconstruction method comprising: (Ghodrati: ¶2 “According to one embodiment, a system for recreating images from undersampled MRI image data”)
recording a plurality of MRI dataframes from a region of interest at different points in time, wherein the region of interest includes a heart; (Ghodrati: ¶42 “To train and evaluate the network, the experiment used retrospectively acquired clinical breath-held 2D multi-slice, ECG-triggered, GRAPPA 2X, bSSFP cardiac cine MR images”)
forming the input-stack from a temporally last recorded plurality of N MRI dataframes; (Ghodrati: ¶36 “Undersampled images 42 include a series of undersampled dynamic/cine MRI images that progress temporally.”)
inputting the input-stack into the reconstruction network and reconstructing an image (Ghodrati: ¶2 “According to one embodiment, a system for recreating images from undersampled MRI image data”) from a single frame chosen by the selection unit of the reconstruction network; (Ghodrati: ¶3 “Each CNN in each stage can consider the group of sequentially captured images to create the regularizer term for each individual image.” Ghodrati discloses an input stack but processing each image in the stack frame-by-frame)
displaying the image; and (Ghodrati: ¶54 discloses a display. Fig. 7 and 8 disclose displaying the image )
in case there is another recorded MRI dataframe, repeating the forming, the inputting, the reconstructing and the displaying (Ghodrati: Fi.7 and 8 disclose the reference and reconstructed image) until a termination condition is reached. (Ghodrati: ¶2 “According to one embodiment, a system for recreating images from undersampled MRI image data…undersampled MRI image data that include one or more images”)
Regarding claim 12, Vazquez, as modified by Ghodrati, disclose wherein An MRI-System comprising a reconstruction network (Vazquez: ¶41 “the methods and systems described herein allow the reconstruction of a 3D MRI volume from partial 2D Cine-MRI images”)
Regarding claim 14, Vazquez, as modified by Ghodrati, disclose wherein A non-transitory computer program product comprising instructions that, when executed by a computer, cause the computer to carry out the reconstruction method of claim 5 (Vazquez: ¶38 “The memory 504 may comprise non-transitory computer readable storage medium… storing machine-readable instructions 506 executable by processing unit 502.”)
Regarding claim 21, Vazquez, as modified by Ghodrati, disclose wherein A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the reconstruction method according to claim 5. (Vazquez: ¶38 “The memory 504 may comprise non-transitory computer readable storage medium… storing machine-readable instructions 506 executable by processing unit 502.”)
Claim(s) 3, 4, 6, 7, 9, 10, 15, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Vazquez (US 20230129194 A1) in view of Ghodrati (US 20240037815 A1) and in further view of Park (US 20200034948 A1).
Regarding claim 3, Vazquez, as modified by Ghodrati, disclose the claimed invention except for wherein comprising at least one of:
an image reconstruction module configured to
receive the single frame selected by the selection unit, and
reconstruct an image based on the single frame, or a loss module configured to
receive a reference frame and the single frame selected by the selection unit, and
calculate a loss based on a comparison between data based on the single frame and the reference frame.
In related art, Park discloses comprising at least one of: (Park: ¶9 “The optimization of the conventional method employs the minimization of a pixel-wise difference (e.g. mean squared error between SR images and ground truth HR images)”)
an image reconstruction module configured to (Park: ¶5 “The aim of SR reconstruction is to reconstruct high resolution (HR) images from a single or a set of low resolution (LR) images to improve the visibility of, or recover, image details.” Park discloses a system for image reconstruction)
receive the single frame selected by the selection unit, and (Park: ¶5 “The aim of SR reconstruction is to reconstruct high resolution (HR) images from a single or a set of low resolution (LR) images to improve the visibility of, or recover, image details.” Park discloses selecting a single image of low resolution)
reconstruct an image based on the single frame, (Park: ¶87 “In various aspects, a robust MRI SR reconstruction method is provided that includes a cascaded deep learning (DL) framework that overcomes the limitations of previous methods in an efficient and practical manner.”) or a loss module configured to (Park: ¶105 “the reconstruction error (content loss)”)
receive a reference frame and the single frame selected by the selection unit, and (Park: ¶105 “The first loss represents the reconstruction error (content loss) between the generated image and the ground truth image.” Park receives a generated frame from the reconstruction model and compares it against the ground truth image)
calculate a loss based on a comparison between data based on the single frame and the reference frame. (Park: ¶105 “The first loss represents the reconstruction error (content loss) between the generated image and the ground truth image.”)
Therefore, it would have been obvious to for one of ordinary skill in the art before the effective filing date to incorporate calculating loss between a generated image and reference image disclosed by Park into the method of cine MRI image reconstruction disclosed by Vazquez, as modified Ghodrati by to aid the learning process by minimizing differences between the generated image and ground truth image.
Regarding claim 4, Vazquez, as modified by Ghodrati, disclose wherein the reconstruction network is trained according to a method comprising: (Vazquez: ¶23 “FIG. 1A illustrates schematically a first embodiment for the training phase of the 3D reconstruction system.”)
recording a plurality of MRI dataframes in a time period, wherein the plurality of MRI dataframes show a region of interest at different points of time, (Vazquez: ¶5 “acquiring 2D images of the anatomical structure at m prior times T.sub.in={t−m, . . . , t−2, t−1};”)
selecting a subset of N training-frames consecutive in time from the plurality of MRI dataframes as an input-stack, (Vazquez: ¶23 “A training volume V.sub.t corresponding to any other respiratory phase is also input into the alignment network 102. Each volume V.sub.ref, V.sub.t is composed of several stacked 2D images of a given imaging technology, such as MRI, CT, ultrasound, etc.”)
inputting the subset of N training-frames into the reconstruction network and generating an output dataset based on a single frame chosen by the selection unit of the reconstruction network, (Vazquez: ¶23 “In this embodiment, a reference volume V.sub.ref taken at a reference respiratory phase is input into an alignment network 102.”)
computing a loss based on the output dataset and updating parameters of the reconstruction network based on the loss, (Vazquez: ¶31 “The embodiment of FIG. 1B is optimized by minimizing the following composite loss function:” Vazquez discloses training the reconstruction system by minimizing a composite loss function)
Vazquez, as modified by Ghodrati, fails to specifically disclose selecting a further subset of N training-frames consecutive in time from the plurality of MRI dataframes, and
repeating the inputting, the generating, the computing, the updating and the selecting until a termination condition is reached.
In related art, Park discloses selecting a further subset of N training-frames consecutive in time from the plurality of MRI dataframes, and (Park: ¶96 “q=1 . . . Q is the subset of training data from total number of training sets k=1 . . . K” Park discloses the training process progressing through sets of training MRI data)
repeating the inputting, the generating, the computing, the updating and the selecting until a termination condition is reached. (Park: ¶147 “a generative model G and a discriminator D, that are trained iteratively to improve the capabilities of each network G in the GAN framework learns the mapping between MR and CT images, producing sCT outputs given an MR image input.)
Therefore, it would have been obvious to for one of ordinary skill in the art before the effective filing date to incorporate the iteratively training through a training set disclosed by Park into the method of minimizing loss to improve a model disclosed by Vazquez, as modified by Ghodrati, to train a model that can effectively reconstruct images from MRI data with minimal differences from the ground truth data.
Regarding claim 6, Vazquez, as modified by Ghodrati, disclose wherein the training method comprising: (Vazquez: ¶23 “FIG. 1A illustrates schematically a first embodiment for the training phase of the 3D reconstruction system.”)
recording a plurality of MRI dataframes in a time period, wherein the plurality of MRI dataframes show a region of interest at different points in time; (Vazquez: ¶5 “acquiring 2D images of the anatomical structure at m prior times T.sub.in={t−m, . . . , t−2, t−1};”)
selecting a subset of N training-frames consecutive in time from the plurality of MRI dataframes as an input-stack; (Vazquez: ¶23 “A training volume V.sub.t corresponding to any other respiratory phase is also input into the alignment network 102. Each volume V.sub.ref, V.sub.t is composed of several stacked 2D images of a given imaging technology, such as MRI, CT, ultrasound, etc.”)
inputting the subset of training-frames into the reconstruction network and generating an output dataset based on a single frame chosen by the selection unit of the reconstruction network; (Vazquez: ¶23 “In this embodiment, a reference volume V.sub.ref taken at a reference respiratory phase is input into an alignment network 102.”)
computing a loss based on the output dataset and updating parameters of the reconstruction network based on the loss; (Vazquez: ¶31 “The embodiment of FIG. 1B is optimized by minimizing the following composite loss function:” Vazquez discloses training the reconstruction system by minimizing a composite loss function)
Vazquez, as modified by Ghodrati, fails to specifically disclose
selecting a further subset of N training-frames consecutive in time from the plurality of MRI dataframes; and
repeating the inputting, the generating, the computing, the updating and the selecting until a termination condition is reached.
In related art, Park discloses selecting a further subset of N training-frames consecutive in time from the plurality of MRI dataframes; and (Park: ¶96 “q=1 . . . Q is the subset of training data from total number of training sets k=1 . . . K” Park discloses the training process progressing through sets of training MRI data)
repeating the inputting, the generating, the computing, the updating and the selecting until a termination condition is reached. (Park: ¶147 “a generative model G and a discriminator D, that are trained iteratively to improve the capabilities of each network G in the GAN framework learns the mapping between MR and CT images, producing sCT outputs given an MR image input.)
Therefore, it would have been obvious to for one of ordinary skill in the art before the effective filing date to incorporate the iteratively training through a training set disclosed by Park into the method of minimizing loss to improve a model disclosed by Vazquez, as modified by Ghodrati, to train a model that can effectively reconstruct images from MRI data with minimal differences from the ground truth data.
Regarding claim 7, Vazquez, as modified by Ghodrati and Park, disclose wherein the plurality of MRI dataframes are processed to be under-sampled by a factor F (Ghodrati: ¶18 “Experiments using these embodiments reveal that artifact-corrected dynamic/cine MRI images can be acquired and reconstructed with 8x-10x undersampling,”) and the subset of N training-frames is selected from the under-sampled MRI dataframes, (Ghodrati: ¶36 “While the images change with time, the features largely relate frame-by-frame. This allows heavy under sampling, such as 8x or 10x as shown by experiments using some embodiments. Learning system 40 includes a sequentially cascading series of convolutional stages 44 (44a, 44b, 44n shown, where n is the total number of convolutional stages) forming an image reconstruction pipeline.”) and wherein a temporally varying under-sampling pattern is used for each frame. (Ghodrati: ¶23 “Although FIG. 1 shows an example undersampling k-space mask with a single k-space center line and randomly sampled individual points… as well as sampled individual points following specific algorithms or patterns.”)
Regarding claim 9, Vazquez, as modified by Ghodrati and Park, disclose wherein selected single frame has a same relative position in the respective subset of N training-frames, (Ghodrati: ¶36 “While the images change with time, the features largely relate frame-by-frame.”) and wherein the selected single frame corresponds to a newest training-frame in the subset. (Ghodrati: ¶35 “a first stage receiving the undersampled MRI image data” ¶36 “Undersampled images 42 include a series of undersampled dynamic/cine MRI images that progress temporally.”)
Regarding claim 10, Vazquez, as modified by Ghodrati and Park, disclose the claimed invention except for wherein a first selected subset of N training-frames starts with a frame of the plurality of MRI dataframes, and subsequent subsets of N training frames start with a second frame of a respective preceding subset of N training-frames. (Park: ¶96 “p=1 . . . P is the subset of training data from the total number of training sets k=1 . . . K” Park discloses progressing through multiple subsets of training data)
Regarding claim 15, Vazquez, as modified by Ghodrati and Park, disclose wherein A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the training method of claim 6. (Vazquez: ¶38 “The memory 504 may comprise non-transitory computer readable storage medium… storing machine-readable instructions 506 executable by processing unit 502.”)
Regarding claim 17, Vazquez, as modified by Ghodrati and Park, disclose wherein each selected single frame has a same relative position in the respective subset of N training-frames. (Ghodrati: ¶36 “In some embodiments, system 40 considers groups of images 42 together, rather than wholly individually. While the images change with time, the features largely relate frame-by-frame.”)
Regarding claim 20, Vazquez, as modified by Ghodrati and Park, disclose wherein An MRI system comprising a reconstruction network trained according to the training method of claim 6. (Vazquez: ¶23 “FIG. 1A illustrates schematically a first embodiment for the training phase of the 3D reconstruction system.”)
Allowable Subject Matter
Claims 2, 8, 11, 13, 16, 18-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Aggarwal (US 20060224062 A1) discloses a method for acquiring MR data from a beating heart during subject respiration includes a prescan phase in which a respiratory compensation table and a k-space sampling schedule are produced. The k-space sampling table is produced using a spatio-temporal model of the beating heart and time sequential sampling theory. During the subsequent scan an imaging pulse sequence which is prospectively compensated for respiratory motion is used to acquire k-space data from the subject. The imaging pulse sequence is repeated to play out the phase encodings in the order listed in the k-space sampling schedule.
Kim (US 12005271 B2) discloses a computer implemented method of treatment targeting includes receiving magnetic resonance (MR) images of a subject including a target region, generating at least one contour of at least one surrogate element apart from the target region in the MR images, and determining a location of the target region in each of the MR images based on a location of the at least one contour in the MR images.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL KIM MAIDEN whose telephone number is (703)756-1264. The examiner can normally be reached Monday - Friday 7:30 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Koziol can be reached at 4089187630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL KIM MAIDEN/Examiner, Art Unit 2665
/Stephen R Koziol/Supervisory Patent Examiner, Art Unit 2665
Prosecution Insights