Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed on 18 December, 2023.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 27 September, 2023 and 22 January, 2024 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1, 8, and 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "by photographing the same part of a body" in line 5. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation “the second medical image” in lines 2, 4, 6, and 8, and “a second medical image” in lines 8 – 9. The examiner believes there is an issue with antecedent basis that renders the claim indefinite.
As written, claim 1 refers to transforming a first medical image into a second medical image in line 1 – 2. In line 4 the claim states “receive the first medical image and second medical image obtained by photographing the same part of a body”. Finally in lines 7 – 9 the limitation states “an artificial neural network module configured to receive the preprocessed first medical image and second medical image, respectively, and transform the first medical image into a second medical image”. It is unclear to the examiner if “a second medical image” in this limitation is intended to be the same second medical image as “the second medical image” or if this is a new “second medical image”. Referring to figure 3 of the applicant’s specification, in view of page 9 of the applicants specification filed 29 November, 2023, the first medical image (CT image) is input into a neural network along with a second medical image (MR Image) to create an intermediate transformed image, and finally a Generated MR Image. It is the examiner’s understanding the Generated MR image is not the MR Image, but instead a conversion of the CT image using a neural network that is modelling the Generated MR Image after the second image (MR Image). As such “a second medical image” from line 8 – 9 of claim 1 should be a new term such as “a third medical image” or a “generated medical image” to distinguish it from “the second medical image” as described in lines 2, 4, 6, and 8.
Claims 8 and 13 recite the limitation “wherein the first generator is configured to generate the intermediate transformed image so that a global feature of boundaries of respective regions for internal parts of the body is revealed in the CT image”. As claimed, the examiner is interpreting this limitation as the first generator generates the intermediate transformed image so that a global feature of boundaries… for internal body parts is revealed in the CT image. This is unclear as the creation of an intermediate transformed image, which is separate from the CT image, would not reveal new information in the input CT image. Instead, this information would be preserved during the creation of, or revealed/made clear in the intermediate generated image. Page 10 of applicant’s specification filed 29 November, 2023, specifically lines 10 – 16 appears to describe just that: “In this case, the first generator 111a may receive the CT image and generate the intermediate transformed image that reveals the global feature of internal parts of the body (i.e., feature of overall shape of respective regions of the internal parts of the body) in CT images.”. It is clear that the specification is describing the intermediate generated image shows this global feature of the internal parts of the body, whereas the claim being written differently appears to describe the creation of the image somehow makes the input CT image more detailed/defined.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) are: “a preprocessing module configured to”, and “an artificial neural network configured to” in claim 1; “first generator configured to” and “first discriminator configured to” in claim 5; “a down block configured to” and “an up block configured to” in claim 7; “A first concatenation layer configured to”, “a second concatenation layer configured to”, and “a discrimination later configured to” in claim 9; “a second generator configured to” and “a second discriminator configured to” in claim 10; “a concatenation layer configured to”, and “a plurality of concatenation layers that are configured to”, and “a discrimination later configured to” in claim 12.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 3 are rejected under 35 U.S.C. 103 as being unpatentable over Amirkolaee et al (Hamed Amini Amirkolaee, Development of a GAN architecture based on integrating global and local information for paired and unpaired medical image translation, Expert Systems with Applications, Volume 203, 2022, 117421,ISSN 0957-4174,https://doi.org/10.1016/j.eswa.2022.117421., hereinafter “Amirkolaee”) in view of Jin et al (U.S. Patent Publication No. 2021/0225491 A1, hereinafter “Jin”).
Regarding claim 1, Amirkolaee teaches an apparatus for image transformation for transforming a first medical image into a second medical image based on an artificial neural network technology (Abstract: The purpose of this study is to develop and evaluate a novel end-to-end generative adversarial network for image-to-image translation of medical paired and unpaired data.), the apparatus comprising:
an artificial neural network module configured to receive the preprocessed first medical image and second medical image, respectively, and transform the first medical image into a second medical image (Figure 1, G2; Figure 5, CT to MR: The results of using the proposed deep GAN in different medical tasks, (a) input image, (b) translated image, and (c) the reference image.).
Amirkolaee does not explicitly teach a preprocessing module configured to receive the first medical image and second medical image obtained by photographing the same part of a body using different photographing techniques and perform preprocessing on the first and second medical images.
However, Jin does teach a preprocessing module configured to receive the first medical image and second medical image obtained by photographing the same part of a body using different photographing techniques and perform preprocessing on the first and second medical images (Figure 9; ¶ 0129: Registration between these paired data can provide the desired paired data of the CT image and the MRI image as shown at the bottom of FIG. 9.; ¶ 0130: In the example shown in FIG. 9, CT and MRI images of the same patient are aligned using affine transformation based on mutual information. As shown in FIG. 9, it can be seen that the CT and MRI images after registration are well aligned spatially and temporally.);
A preprocessing module (¶ 0039: According to some embodiments of the present invention, a non-transitory computer readable recording medium stores a computer program including computer executable instructions for causing, when executed by a processor, the processor to perform the method of converting a diagnostic image according to some embodiments of the present invention.); and
an artificial neural network module (¶ 0039: According to some embodiments of the present invention, a non-transitory computer readable recording medium stores a computer program including computer executable instructions for causing, when executed by a processor, the processor to perform the method of converting a diagnostic image according to some embodiments of the present invention.).
Amirkolaee and Jin are considered to be analogous art as both pertain to CT to MRI image conversion. Therefore, it would have been obvious to one of ordinary skill in the art to combine the GAN architecture based on integrating global and local information (as taught by Amirkolaee) and the diagnostic image converting apparatus (as taught by Jin) before the effective filing date of the claimed invention. The motivation for this combination of references would be the method of Jin trains the network using paired and unpaired data which provides robustness in details as well as structure during the network operation. (See ¶ 0142).
This motivation for the combination of Amirkolaee and Jin is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim 3, the Amirkolaee and Jin combination teaches the apparatus of claim 1.
Additionally, Amirkolaee teaches wherein the first medical image is a computer tomography (CT) image, and the second medical image is a magnetic resonance imaging (MRI) image (Figure 1, G2; Figure 5, CT to MR: The results of using the proposed deep GAN in different medical tasks, (a) input image, (b) translated image, and (c) the reference image.; Page 4, Section 3.2; For example, in CT to MR translation, the reference image is MR image and the input data is CT image.).
Allowable Subject Matter
Claims 14 - 18 are allowed.
The following is a statement of reasons for the indication of allowable subject matter:
The closest prior art Amirkolaee et al (Hamed Amini Amirkolaee, Development of a GAN architecture based on integrating global and local information for paired and unpaired medical image translation, Expert Systems with Applications, Volume 203, 2022, 117421,ISSN 0957-4174,https://doi.org/10.1016/j.eswa.2022.117421., hereinafter “Amirkolaee”) in combination with a second art Jin et al (U.S. Patent Publication No. 2021/0225491 A1, hereinafter “Jin”) teaches that it was known at the time the application was filed to perform a method for image transformation performed on a computing device including a first artificial neural network model and a second artificial neural network model.
However, the prior art, alone or in combination, does not appear to teach or suggest training the first artificial neural network model to generate an intermediate transformed image by receiving a CT image; and
training the second artificial neural network model to generate an MRI image by receiving the intermediate transformed image generated by the first artificial neural network model.
Claims 15 – 17 depend from independent claim 14.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Claims 2, and 4 – 7, and 9 – 12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kudo et al (U.S. Patent Publication No 2023/0368442 A1) teaches a method of using a neural network comprising a generator and discriminator to take a three-dimensional image input of a first domain and output a three-dimensional image of a second domain different than the first domain. Figure 5 shows conversion from CT to MRI.
Park et al (U.S. Patent Publication No. 2020/0034948 A1) teaches a computer implemented method of converting low-resolution MR images to high-resolution MR images as well as transforming MR images into synthetic CT images.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW JONES whose telephone number is (703)756-4573. The examiner can normally be reached Monday - Friday 8:00-5:00 EST, off Every Other Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW B. JONES/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667