DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
This action is in response to the amendments and remarks filed on 05/04/2026.
Claims 1-20 are pending.
Corrective Actions by Applicant
Claims 1, 17-18, and 20 have been amended.
Response to Arguments
The examiner has fully considered Applicant’s presented arguments.
On page 6 of the remarks, Applicant argues that The amendment to claim 17 overcomes the objection to claim 17. This is persuasive. The objection to claim 17 has been withdrawn.
On page 6 of the remarks, Applicant requests abeyance of the provisional nonstatutory double patenting rejections in view of copending U.S. Application 18/918,677 in view of Chatterjee, until one or more of the cited claims of either Application are allowed.
The examiner acknowledges Applicant’s request. The rejections will respectively be maintained until either the filing of a terminal disclaimer, or until withdrawal of the rejections in view of claim amendments at a later date.
The examiner notes that the present amendments do not overcome these double patenting rejections, and the updated rejections are presented below for a clear record (the presently-amended limitations of claim 1 correspond to rows 5 and 6 of the claim comparison table in the updating double patenting rejections below).
On page 6 of the remarks, Applicant argues that the amendments to claims 1, 18, and 20 overcome the nonstatutory double patenting rejections in view of U.S. Patent No. 12,154,285 and Chatterjee. The examiner respectfully disagrees.
The present amendments to claims 1, 18, and 20 specify that the features output by the first neural network include a PET feature map and a co-modality feature map; these are regarded as natural results of inputting PET and co-modality images into the first neural network, which the reference patent also does.
The present amendments to claims 1, 18, and 20 also re-word the use of the second neural network; however, the claim scope in the context of double patenting remains the same, and as Applicant points out in the 35 U.S.C. 101 arguments, these limitations were primarily amended to address 101 issues.
The above presently-amended limitations of claim 1 correspond to rows 4 and 5 of the claim comparison table in the updating double patenting rejections below.
On pages 7-9 of the remarks, Applicant argues that all claims are directed to eligible subject matter under 35 U.S.C. 101 based on the amendments to claims 1, 18, and 20. This is persuasive. More specifically, upon reconsideration of the amended claims and of Applicant’s arguments, the examiner argues that the claims still recite a mental process, but agrees that the additional elements of the claims integrate into a practical application at Step 2A Prong Two, as detailed below in the “Updated 35 U.S.C. 101 Analysis” section below.
On pages 9-10 of the remarks, Applicant argues that Chatterjee fails to disclose or reasonably suggest “inputting the PET image and the co-modality image to a first trained neural network and, based on inputting the PET image and the co-modality image to the first trained neural network, generating first features of the PET image and second features of the co-modality image, wherein the first features comprise a PET feature map and the second features comprise a co-modality feature map” of amended claims 1, 18, and 20.
This is persuasive, particularly regarding “wherein the first features comprise a PET feature map and the second features comprise a co-modality feature map”. More specifically, Chatterjee’s modality-neutral images 404 and 406 are not feature maps, but rather modified, modality-neutral images. Thus, all 35 U.S.C. 102 rejections have been withdrawn, except for that of claim 20, which does not include “wherein the first features comprise a PET feature map and the second features comprise a co-modality feature map”.
Applicant further points to Chatterjee’s paragraph 0091, arguing that in this paragraph Chatterjee argues away from using feature maps. However, the examiner interprets this paragraph to only teach away from using feature maps of the original modality images, i.e. the original movable image 104 and the original fixed image 106 of figure 5. This is because the different imaging modalities can represent the same edges with vastly different intensity values that are not simply comparable, for example.
However, Chatterjee’s modality-neutral images do not appear to pose this same problem of being incomparable using feature maps. Chatterjee’s paragraphs 0084-0087 describe how the original CT image 202 and original MRI image 204 of figure 2 suffer from this incomparability, and also how the respective modality-neutral images 602 and 604 of figure 6 are comparable due to now having “very similar pixel/voxel intensity distributions as each other” (see paragraph 0087).
Thus, Chatterjee would be combinable with other references that compare feature maps of images for purposes such as alignment or registration. Respectively, new 35 U.S.C. 103 rejections are presented below.
On page 10 of the remarks, Applicant argues that Chatterjee fails to disclose or reasonably suggest “generating display data based on the displacement data” of amended claims 1, 18, and 20. The examiner respectfully disagrees.
The broadest reasonable interpretation of “display data” that is generated “based on the displacement data” is any kind of visual data that illustrates features or characteristics of the displacement data. Chatterjee’s final registered image is interpreted as an example of display data, as it visually represents the results of the displacement data modifying the two images, which would be especially evident by comparing the images before and after registration. Applicant’s argument gives a heat map as an example of display data, which is notably different from Chatterjee’s registered image, but as Applicant points out, this is only an example; the display data as presently claimed is broad enough to encompass both of these examples.
On pages 11-12 of the remarks, Applicant argues that each of Paragios, Madabhushi, and Laaksonen fail to overcome the above deficiencies of Chatterjee. The examiner posits that these arguments are rendered moot by the above response to the 35 U.S.C. 102 arguments.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-6, 10-11, and 18-20 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 7, 9, and 19 of copending Application No. 18/918,677 in view of Chatterjee et al. (U.S. Publ. US-2023/0260142-A1).
Regarding claim 1, the claim language of present claim 1 and reference claim 7 is substantially similar, as described below (rows 5 and 6 contain the presently-amended limitations, which are still substantially similar):
Row #
Present Application
18/618,372
Claim 1
Reference Application
18/918,677
Claim 7
Notes
1
A computer-implemented method comprising:
A method for image registration, comprising:
Present claim 1’s preamble is broader.
2
receiving positron emission tomography (PET) measurement data and co-modality measurement data from an image scanning system;
receiving an anatomical image and a functional image of a structure of interest, and first and second trained convolutional neural networks,
The reference Application only claims “receiving” the two images, not additionally generating them from respective measurement data as in present claim 1.
However, paragraph 0003 of the reference Application’s publication states that PET images can be an instance of “functional images”, and that CT images can be an instance of “anatomical images.” The present application makes clear that CT images are also an example of “co-modality images.” Thus, the examiner considers PET images analogous to functional images, and co-modality images analogous to anatomical images.
3
generating a PET image based on the PET measurement data and a co-modality image based on the co-modality measurement data;
4
wherein the anatomical image and the functional image are acquired at different scan times or there is movement of the structure of interest between the different scan times;
Present claim 1 is broader in that it does not recite a similar limitation.
5
inputting the PET image and the co-modality image to a first trained neural network and, based on inputting the PET image and the co-modality image to the first trained neural network, generating first features of the PET image and second features of the co-modality image, wherein the first features comprise a PET feature map, and the second features comprise a co-modality feature map;
extracting features by applying the anatomical image and the functional image as input to the first trained convolutional neural network;
Essentially the same. The reference claim 7 extracting features from the anatomical and functional images would naturally result in feature maps being generated from each image.
6
generating, by a second trained neural network, displacement data based on inputting the first features and the second features to the second trained neural network wherein the displacement data characterizes a displacement between the first features and the second features;
estimating a deformation field by applying the features as input to the second trained convolutional neural network;
The examiner still interprets reference claim 7’s “deformation field” as a narrower instance of present claim 1’s “displacement data.”
7
generating display data based on the displacement data;
and applying the deformation field to the anatomical image to generate a registered anatomical image.
The examiner interprets reference claim 7’s “registered anatomical image” to be a narrower instance of present claim 1’s “display data.”
8
and transmitting the display data for display.
The reference application does not claim any form of transmitting output data.
As seen above, the only limitations of present claim 1 not present in reference claim 7 are those of rows 2-3 and 8. In other words, present claim 1 is only narrower than reference claim 7 in that present claim 1 additionally recites first generating the images from their respective measurement data before the image processing, and at last transmitting the final display/output data.
Pertaining to the same field of endeavor, Chatterjee discloses receiving positron emission tomography (PET) measurement data and co-modality measurement data from an image scanning system; generating a PET image based on the PET measurement data and a co-modality image based on the co-modality measurement data (see figure 5, movable image 104, fixed image 106 and paragraphs 0061-0062, where the movable image can be a CT scan/co-modality image generated from an image scanning system, and the fixed image can be a PET image generated from an image scanning system); and transmitting the display data for display (see paragraph 0107).
The reference Application and Chatterjee are considered analogous art, as they are both directed to neural networks for generating displacement fields between medical images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Chatterjee into the reference Application because doing so allows for obtaining medical images of different modalities (see Chatterjee paragraphs 0061-0062), and because doing so allows for rendering the output data on any suitable computer screen and/or display as desired (see Chatterjee paragraph 0107).
Regarding claim 2, present claim 2 is rejected in view of reference claim 9, with similar analysis to the rejection of present claim 1 in view of reference claim 7 above. The only difference here is that present claim 2 narrows the co-modality images to be CT images, which reference claim 9 also does by narrowing the anatomical images to be CT images.
Regarding claim 3, present claim 3 is rejected in view of reference claim 7, with similar analysis to the rejection of present claim 1 in view of reference claim 7 above. The only difference here is that present claim 3 narrows the first trained network to be a CNN, which reference claim 7 also does in row 5 above.
Regarding claim 4, present claim 4 is rejected in view of reference claim 7, with similar analysis to the rejection of present claim 1 in view of reference claim 7 above. The only difference here is that present claim 4 narrows the second trained network to be a CNN, which reference claim 7 also does in row 6 above.
Regarding claim 5, present claim 5 is rejected in view of reference claim 7, with similar analysis to the rejection of present claim 1 in view of reference claim 7 above. The only difference here is that present claim 5 states that the images share common features, which is implied by reference claim 7 in rows 4 and 6-7 above.
Regarding claim 6, present claim 6 is rejected in view of reference claim 7, with similar analysis to the rejection of present claim 1 in view of reference claim 7 above. The only difference here is that present claim 6 narrows the displacement data to comprise at least one displacement value for each of a plurality of pixels of the PET image and the co-modality image, which is within the broadest reasonable interpretation of the “deformation field” recited by reference claim 7.
Regarding claim 10, present claim 10 is rejected in view of reference claim 7, with similar analysis to the rejection of present claim 1 in view of reference claim 7 above. The only difference here is that present claim 10 narrows the displacement data to comprise displacement values identifying pixel offsets between the PET image and the co-modality image, which is within the broadest reasonable interpretation of the “deformation field” recited by reference claim 7.
Regarding claim 11, present claim 11 is rejected in view of reference claim 7, with similar analysis to the rejection of present claim 1 in view of reference claim 7 above. The only difference here is that present claim 11 narrows the measurement data to be based on corresponding scans of a same subject, which is implied by reference claim 1 in rows 4 and 6-7 above.
Regarding claim 18, present claim 18 is rejected in view of reference claim 19, with similar analysis to the rejection of present claim 1 in view of reference claim 7 above. The only difference here is that present claim 18 and reference claim 19 are instead directed to non-transitory computer readable mediums that perform the same method.
Regarding claim 19, present claim 19 is rejected in view of reference claim 9, with similar analysis to the rejection of present claim 2 in view of reference claim 9 above.
Regarding claim 20, present claim 20 is rejected in view of reference claim 1, with similar analysis to the rejection of present claim 1 in view of reference claim 7 above. The only difference here is that present claim 20 and reference claim 1 are instead directed to systems that perform the same method.
This is a provisional nonstatutory double patenting rejection.
Claims 1-6, 10-11, and 18-20 are rejected on the grounds of nonstatutory obviousness-type double patenting as being unpatentable over claims 1, 6-7, and 18 of U.S. Patent No. US-12154285-B2 in view of Chatterjee et al. (U.S. Publ. US-2023/0260142-A1).
Regarding claim 1, the claim language of present claim 1 and reference claim 6 is substantially similar, as described below (rows 4 and 5 contain the presently-amended limitations, which are still substantially similar):
Row #
Present Application
18/618,372
Claim 1
Reference Patent
US-12154285-B2
Claim 6
Notes
1
A computer-implemented method comprising:
A method for image registration, comprising:
Present Application’s preamble is broader.
2
receiving positron emission tomography (PET) measurement data and co-modality measurement data from an image scanning system;
receiving an anatomical image and a functional image of a structure of interest, and first and second trained convolutional neural networks;
The reference patent only claims “receiving” the two images, not additionally generating them from respective measurement data as in present claim 1.
However, column 1, lines 12-26 of the reference patent states that PET images can be an instance of “functional images”, and that CT images can be an instance of “anatomical images.” The present application makes clear that CT images are also an example of “co-modality images.” Thus, the examiner considers PET images analogous to functional images, and co-modality images analogous to anatomical images.
3
generating a PET image based on the PET measurement data and a co-modality image based on the co-modality measurement data;
4
inputting the PET image and the co-modality image to a first trained neural network and, based on inputting the PET image and the co-modality image to the first trained neural network, generating first features of the PET image and second features of the co-modality image, wherein the first features comprise a PET feature map, and the second features comprise a co-modality feature map;
extracting features by applying the anatomical image and the functional image as input to the first trained convolutional neural network;
Essentially the same. The reference claim 6 extracting features from the anatomical and functional images would naturally result in feature maps being generated from each image.
5
generating, by a second trained neural network, displacement data based on inputting the first features and the second features to the second trained neural network wherein the displacement data characterizes a displacement between the first features and the second features;
estimating a deformation field by applying the features as input to the second trained convolutional neural network, wherein the second trained convolutional neural network comprises a deformation vector field regressor that regresses a relative motion displacement matrix between the anatomical image and the functional image;
The examiner still interprets reference claim 6’s “deformation field” as a narrower instance of present claim 1’s “displacement data.”
6
and generating display data based on the displacement data,
and applying the deformation field to the anatomical image to generate a registered anatomical image.
The examiner interprets reference claim 6’s “registered anatomical image” as a narrower instance of the present claim 1’s “display data.”
7
and transmitting the display data for display.
The reference patent does not claim any form of transmitting output data.
As seen above, the only limitations of present claim 1 not present in reference claim 6 are those of rows 2-3 and 7. In other words, present claim 1 is only narrower than reference claim 6 in that present claim 1 additionally recites first generating the images from their respective measurement data before the image processing, and at last transmitting the final display/output data.
Pertaining to the same field of endeavor, Chatterjee discloses receiving positron emission tomography (PET) measurement data and co-modality measurement data from an image scanning system; generating a PET image based on the PET measurement data and a co-modality image based on the co-modality measurement data (see figure 5, movable image 104, fixed image 106 and paragraphs 0061-0062, where the movable image can be a CT scan/co-modality image generated from an image scanning system, and the fixed image can be a PET image generated from an image scanning system); and transmitting the display data for display (see paragraph 0107).
The reference Application and Chatterjee are considered analogous art, as they are both directed to neural networks for generating displacement fields between medical images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Chatterjee into the reference Application because doing so allows for obtaining medical images of different modalities (see Chatterjee paragraphs 0061-0062), and because doing so allows for rendering the output data on any suitable computer screen and/or display as desired (see Chatterjee paragraph 0107).
Regarding claim 2, present claim 2 is rejected in view of reference claim 7, with similar analysis to the rejection of present claim 1 in view of reference claim 6 above. The only difference here is that present claim 2 narrows the co-modality images to be CT images, which reference claim 7 also does by narrowing the anatomical images to be CT images.
Regarding claim 3, present claim 3 is rejected in view of reference claim 6, with similar analysis to the rejection of present claim 1 in view of reference claim 6 above. The only difference here is that present claim 3 narrows the first trained network to be a CNN, which reference claim 6 does in row 4 above.
Regarding claim 4, present claim 4 is rejected in view of reference claim 6, with similar analysis to the rejection of present claim 1 in view of reference claim 6 above. The only difference here is that present claim 4 narrows the second trained network to be a CNN, which reference claim 6 does in row 5 above.
Regarding claim 5, present claim 5 is rejected in view of reference claim 6, with similar analysis to the rejection of present claim 1 in view of reference claim 6 above. The only difference here is that present claim 5 states that the images share common features, which is implied by reference claim 5 in rows 5-6 above.
Regarding claim 6, present claim 6 is rejected in view of reference claim 6, with similar analysis to the rejection of present claim 1 in view of reference claim 6 above. The only difference here is that present claim 6 narrows the displacement data to comprise at least one displacement value for each of a plurality of pixels of the PET image and the co-modality image, which is within the broadest reasonable interpretation of the “deformation field” recited by reference claim 6.
Regarding claim 10, present claim 10 is rejected in view of reference claim 6, with similar analysis to the rejection of present claim 1 in view of reference claim 6 above. The only difference here is that present claim 10 narrows the displacement data to comprise displacement values identifying pixel offsets between the PET image and the co-modality image, which is within the broadest reasonable interpretation of the “deformation field” recited by reference claim 6.
Regarding claim 11, present claim 11 is rejected in view of reference claim 6, with similar analysis to the rejection of present claim 1 in view of reference claim 6 above. The only difference here is that present claim 11 narrows the measurement data to be based on corresponding scans of a same subject, which is implied by reference claim 6 in rows 5-6 above.
Regarding claim 18, present claim 18 is rejected in view of reference claim 18, with similar analysis to the rejection of present claim 1 in view of reference claim 6 above. The only difference here is that present claim 18 and reference claim 18 are instead directed to non-transitory computer readable mediums that perform the same method.
Regarding claim 19, present claim 19 is rejected in view of reference claim 7, with similar analysis to the rejection of present claim 2 in view of reference claim 7 above.
Regarding claim 20, present claim 20 is rejected in view of reference claim 1, with similar analysis to the rejection of present claim 1 in view of reference claim 6 above. The only difference here is that present claim 20 and reference claim 1 are instead directed to systems that perform the same method.
35 USC § 101 Analysis
The examiner has determined that the amended claims are now eligible under 101. The analysis is provided below for the purpose of a clear record.
The updated analysis for claim 1 is provided in the following. Claim 1 is reproduced in the following (annotation added):
A computer-implemented method comprising:
receiving positron emission tomography (PET) measurement data and co-modality measurement data from an image scanning system;
generating a PET image based on the PET measurement data and a co-modality image based on the co-modality measurement data;
inputting the PET image and the co-modality image to a first trained neural network and, based on inputting the PET image and the co-modality image to the first trained neural network, generating first features of the PET image and second features of the co-modality image, wherein the first features comprise a PET feature map and the second features comprise a co-modality feature map;
generating, by a second trained neural network, displacement data based on inputting the first features and the second features to the second trained neural network wherein the displacement data characterizes a displacement between the first features and the second features;
generating display data based on the displacement data;
and transmitting the display data for display.
Step 1: Does the claim belong to one of the statutory categories? Claim 1 is directed to a process, which is a statutory category of invention (YES).
Step 2A Prong One: Does the claim recite a judicial exception? Step e is still regarded as practically performable in the human mind.
First, the term “displacement data” is broad and reads on any data representing any kind of displacement of similar features in the two feature maps; this type of data can certainly be mentally generated by a human, or a human using pen and paper by recognizing similar features and noting their displacements across the feature maps.
Second, the courts do not distinguish between claims that recite mental processes performed by humans and claims that recite mental processes performed on a computer (see MPEP 2106.04(a)(2).III). Even though the claim now specifies that the displacement data is positively generated by the “second neural network”, this action is also performable by a human as detailed above. The claim does not offer any details of how the second neural network determines the displacement data, in such a way that could not also be performed mentally or using pen and paper (YES).
Step 2A Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application? The examiner still argues that step b amounts to mere data gathering, and that step g amounts to mere data output, but agrees with Applicant’s arguments that steps b-d and f amount to more than these processes. More specifically, step c requires the generation of the PET image and the co-modality image from their respective measurement data, which as argued by Applicant is a complex process that transforms the measurement data into displayable images. Step f recites generating display data based on the displacement data; this is not mere data output as originally interpreted by the examiner, but rather reflects the improvements discussed in at least figure 4C and paragraphs 0056-0059 of the originally-filed specification, where the display data can take the form of warnings, heat maps, etc. representative of the degree of the displacements. Paragraphs 0003, 0029, and 0035 specify that this display data informs clinicians of misalignments between the PET and co-modality images, preventing diagnoses based on inaccurate image data. Thus, the displacement data obtained from mental processes is integrated into a practical application (YES). Claim 1 is eligible.
Similar analysis is applicable to independent claims 18 and 20. Claims 18 and 20 are eligible.
Dependent claims 2-11 and 19 do not introduce any new judicial exceptions. Claims 2-11 and 19 are eligible.
Dependent claims 12-17 recite processes of training the first and second neural networks, involving determinations of training parameters that read on mental processes. However, the limitations of claim 1 above also integrate these into a practical application. Claims 12-17 are eligible.
Claim Rejections – 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim 20 is rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Chatterjee et al. (U.S. Publ. US-2023/0260142-A1).
Regarding claim 20, Chatterjee discloses a system (see figure 18) comprising: a memory device storing instructions (see figure 18, system memory 1806, RAM 1812, ROM 1810); and at least one processor communicatively coupled the memory device, the at least one processor configured to execute the instructions to (see figure 18, processing unit 1804):
receive positron emission tomography (PET) measurement data and co-modality measurement data from an image scanning system; generate a PET image based on the PET measurement data and a co-modality image based on the co-modality measurement data (see figure 5, movable image 104, fixed image 106 and paragraphs 0061-0062, where the movable image can be a CT scan/co-modality image generated from an image scanning system, and the fixed image can be a PET image generated from an image scanning system; both images can be two- or three-dimensional; paragraph 0063 specifies that the two images depict the same subject);
input the PET image and the co-modality image to a first trained neural network (see figure 5, where the movable and fixed images are input to a machine learning model 402);
and, based on inputting the PET image and the co-modality image to the first trained neural network, generate first features of the PET image and second features of the co-modality image (see figure 5, modality-neutral movable image 404, modality-neutral image 406, and paragraphs 0078-0083, where the machine learning model outputs the modality-neutral images; paragraph 0040 specifies that this can involve applying a convolutional neural network that generates features of the images);
generate, by a second trained neural network, displacement data based on inputting the first features and the second features to the second trained neural network wherein the displacement data characterizes a displacement between the first features and the second features (see figure 9, scenario 904, where the modality-neutral images 404 and 406 are input to a deep learning registration model 906; see figure 9, where the deep learning model 906 generates the registration field 802; paragraphs 0093 and 0096 specify that the registration field can include a vector field/displacement data that maps the movement of pixels/voxels/features from one image to the other);
generate display data based on the displacement data (see paragraphs 0103-0106, where the movable and fixed images can be registered/aligned with each other via the registration field to generate a registered image/display data),
transmit the display data for display (see paragraph 0107).
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 10-11, and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Chatterjee et al. (U.S. Publ. US-2023/0260142-A1) in view of Paragios et al. (U.S. Publ. US-2024/0087270-A1).
Regarding claim 1, Chatterjee discloses a computer-implemented method (see figures 5 and 9) comprising:
receiving positron emission tomography (PET) measurement data and co-modality measurement data from an image scanning system; generating a PET image based on the PET measurement data and a co-modality image based on the co-modality measurement data (see figure 5, movable image 104, fixed image 106 and paragraphs 0061-0062, where the movable image can be a CT scan/co-modality image generated from an image scanning system, and the fixed image can be a PET image generated from an image scanning system; both images can be two- or three-dimensional; paragraph 0063 specifies that the two images depict the same subject);
inputting the PET image and the co-modality image to a first trained neural network (see figure 5, where the movable and fixed images are input to a machine learning model 402);
and, based on inputting the PET image and the co-modality image to the first trained neural network, generating first features of the PET image and second features of the co-modality image (see figure 5, modality-neutral movable image 404, modality-neutral image 406, and paragraphs 0078-0083, where the machine learning model outputs the modality-neutral images; paragraph 0040 specifies that this can involve applying a convolutional neural network that generates features of the images),
generating, by a second trained neural network, displacement data based on inputting the first features and the second features to the second trained neural network wherein the displacement data characterizes a displacement between the first features and the second features (see figure 9, scenario 904, where the modality-neutral images 404 and 406 are input to a deep learning registration model 906; see figure 9, where the deep learning model 906 generates the registration field 802; paragraphs 0093 and 0096 specify that the registration field can include a vector field/displacement data that maps the movement of pixels/voxels/features from one image to the other);
generating display data based on the displacement data (see paragraphs 0103-0106, where the movable and fixed images can be registered/aligned with each other via the registration field to generate a registered image/display data),
and transmitting the display data for display (see paragraph 0107).
Chatterjee fails to disclose wherein the first features comprise a PET feature map and the second features comprise a co-modality feature map. More specifically, Chatterjee only now fails to disclose every element of claim 1 because the modality-neutral images themselves are input into the second network, instead of feature maps of said modality-neutral images.
Pertaining to the same field of endeavor, Paragios discloses wherein the first features comprise a PET feature map and the second features comprise a co-modality feature map (first see paragraphs 0080-0083, where two images of different modalities are first transformed to emulate the same modality; in this example, with a CT image and MRI image, the MRI is transformed into a "pseudo-CT image"; paragraph 0023 specifies that any imaging modalities can be used, including PET and CT; then, see figure 5 and paragraphs 0109-0113, where a machine learning architecture uses a CNN, which is known in the art to produce feature maps, to extract features from each of the above images for subsequent displacement field generation).
Chatterjee and Paragios are considered analogous art, as they are both directed to neural networks for generating displacement fields between medical images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Paragios into Chatterjee by additionally extracting feature maps from Chatterjee’s modality-neutral images for the disparity map calculations, because CNNs are well-known tools in the art for performing image registration (see Paragios paragraph 0112).
Regarding claim 2, Chatterjee in view of Paragios discloses wherein the co-modality measurement data is computed tomography (CT) measurement data and the co-modality images are CT images (see Chatterjee paragraphs 0061-0062, where the movable image can be a CT scan/co-modality image generated from an image scanning system).
Regarding claim 3, Chatterjee in view of Paragios discloses wherein the first trained neural network is a convolutional neural network (CNN) (see Chatterjee paragraphs 0040-0041).
Regarding claim 4, Chatterjee fails to disclose the limitations of claim 4.
Pertaining to the same field of endeavor, Paragios discloses wherein the second trained neural network is a convolutional neural network (CNN) (see figure 5 and paragraphs 0109-0113, where a second machine learning architecture uses a CNN to generate a displacement field between sets of medical images).
Chatterjee and Paragios are considered analogous art, as they are both directed to neural networks for generating displacement fields between medical images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Paragios into Chatterjee because CNNs are well-known tools in the art for performing image registration (see Paragios paragraph 0112).
Regarding claim 5, Chatterjee in view of Paragios discloses wherein the first features of the PET images and the second features of the co-modality images include common features (see Chatterjee paragraph 0063, where the two images depict common features of the same subject).
Regarding claim 6, Chatterjee in view of Paragios discloses wherein the displacement data comprises at least one displacement value for each of a plurality of pixels of the PET image and the co-modality image (Chatterjee paragraphs 0093 and 0096 specify that the registration field can include a vector field/displacement data that maps the movement of pixels/voxels/features from one image to the other).
Regarding claim 7, Chatterjee in view of Paragios discloses wherein the at least one displacement value for each of the plurality of pixels comprises a first displacement value for a first direction, a second displacement value for a second direction, and a third displacement value for a third direction (see Chatterjee paragraph 0093, where the vector field can also apply to voxels if the image is three-dimensional, in which case the vectors would have three direction components).
Regarding claim 10, Chatterjee in view of Paragios discloses wherein the displacement data comprises displacement values identifying pixel offsets between the PET image and the co-modality image (Chatterjee paragraphs 0093 and 0096 specify that the registration field can include a vector field/displacement data that maps the movement of pixels/voxels/features from one image to the other).
Regarding claim 11, Chatterjee in view of Paragios discloses wherein the PET measurement data and the co-modality measurement data are based on corresponding scans of a same subject (see Chatterjee paragraph 0063, where the two images depict common features of the same subject).
Regarding claim 15, Chatterjee in view of Paragios discloses training the second trained neural network (Chatterjee paragraphs 0097-0099 provide an overview of training the deep learning registration model 906), the training comprising:
inputting labelled PET features and labelled CT features to a neural network (see Chatterjee paragraph 0098, where training movable and fixed images can be labeled and input to the model)
and, based on inputting the labelled PET features and the labelled CT features to the neural network, generating output data characterizing displacement values between the labelled PET features and labelled CT features (see Chatterjee paragraph 0099, where the model generates output registration fields representing displacement values between the training images);
and determining the neural network is trained based on the output data (see Chatterjee paragraph 0099, where the output can be used for training termination criteria, such as error, loss, or objective functions, that determine when the model is appropriately trained).
Regarding claim 16, Chatterjee in view of Paragios discloses determining at least one metric value based on the output data; and determining the neural network is trained based on the at least one metric value (see Chatterjee paragraph 0099, where the output can be used for training termination criteria, such as error, loss, or objective functions, that determine when the model is appropriately trained).
Regarding claim 17, Chatterjee in view of Paragios discloses storing parameters characterizing the second trained neural network in a data repository (see Chatterjee paragraph 0096, where the field component can electronically store the deep learning registration model).
Regarding claim 18, Chatterjee in view of Paragios discloses a non-transitory computer readable medium (see Chatterjee figure 18, system memory 1806, RAM 1812, ROM 1810) storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising (see Chatterjee figure 18, processing unit 1804).
The remainder of claim 18 recites steps identical to those of claim 1. Therefore, Chatterjee in view of Paragios discloses claim 18 as applied to claim 1 above.
Regarding claim 19, Chatterjee in view of Paragios discloses claim 19 as applied to claim 2 above.
Claims 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Chatterjee et al. (U.S. Publ. US-2023/0260142-A1) in view of Paragios et al. (U.S. Publ. US-2024/0087270-A1), and further in view of Madabhushi et al. (U.S. Publ. US-2022/0012902-A1).
Regarding claim 8, Chatterjee in view of Paragios discloses determining, for each of the plurality of pixels, a magnitude value (see Chatterjee paragraph 0093, where the registration field is a matrix with each element having a vector with a calculated direction and magnitude)
and generating the display data based on the magnitude values (see Chatterjee paragraphs 0103-0106, where the movable and fixed images can be registered/aligned with each other via the vectors in the registration field to generate a registered image/display data).
Chatterjee fails to disclose determining, for each of the plurality of pixels, a magnitude value based on the first displacement value, the second displacement value, and the third displacement value (emphasis added via underline).
Pertaining to the same field of endeavor, Madabhushi discloses determining, for each of the plurality of pixels, a magnitude value based on the first displacement value, the second displacement value, and the third displacement value (see paragraph 0066, where the formula for the magnitude of a 3D vector,
x
2
+
y
2
+
z
2
, is used to find the magnitudes "D(c)" of displacement vectors in medical imaging data).
Chatterjee and Madabhushi are considered analogous art, as they are both directed to neural networks for generating displacement fields in medical images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Madabhushi into Chatterjee and Paragios because doing so enables extraction of structural deformation information in medical images (see Madabhushi paragraphs 0065-0066).
Regarding claim 9, Chatterjee in view of Paragios fails to disclose the limitations of claim 9.
Pertaining to the same field of endeavor, Madabhushi discloses wherein the display data characterizes a heat map (see figures 5-6 and paragraphs 0076-0077, where the magnitude values "D(c)" are visualized using heatmaps).
Chatterjee and Madabhushi are considered analogous art, as they are both directed to neural networks for generating displacement fields in medical images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Madabhushi into Chatterjee and Paragios because doing so allows for visualization of the deformation of biological structures in imaging data (see Madabhushi figures 5-6 and paragraphs 0076-0077).
Claims 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Chatterjee et al. (U.S. Publ. US-2023/0260142-A1) in view of Paragios et al. (U.S. Publ. US-2024/0087270-A1), and further in view of Laaksonen et al. (U.S. Publ. US-2021/0192719-A1).
Regarding claim 12, Chatterjee in view of Paragios discloses training the first trained neural network (see Chatterjee paragraphs 0111-0118, where the machine learning model is trained only with unsupervised learning), the training comprising:
inputting (see Chatterjee paragraphs 0111 and 0118, where the model outputs predicted modality-neutral images based on the unlabeled input images);
and determining the neural network is trained based on the output data (see Chatterjee paragraphs 0123-0124, where any training termination criteria can be used, such as loss functions, for determining that the model is appropriately trained).
Chatterjee in view of Paragios fails to disclose inputting labelled PET images and labelled CT images to a neural network and, based on inputting the labelled PET images and the labelled CT images to the neural network, generating output data characterizing PET features and CT features (emphasis added via underline). In other words, Chatterjee only discloses training the first network via unsupervised learning, not via supervised learning as required by the claim.
Pertaining to the same field of endeavor, Laaksonen discloses inputting labelled PET images and labelled CT images to a neural network and, based on inputting the labelled PET images and the labelled CT images to the neural network, generating output data characterizing PET features and CT features (see figure 4 and paragraphs 0038-0044, where both labeled and unlabeled training data for multiple modalities, including PET and CT, can be input to a model that generates features/output data from said images).
Chatterjee and Laaksonen are considered analogous art, as they are both directed to neural networks for processing co-modality image sets. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have integrated the teachings of Laaksonen into Chatterjee and Paragios because using semi-supervised approaches is useful when training data is scarce and/or expensive (see Laaksonen paragraph 0044).
Regarding claim 13, Chatterjee in view of Paragios and Laaksonen discloses determining at least one metric value based on the output data; and determining the neural network is trained based on the at least one metric value (see Chatterjee paragraphs 0123-0124, where any training termination criteria can be used, such as loss functions, for determining that the model is appropriately trained).
Regarding claim 14, Chatterjee in view of Paragios and Laaksonen discloses storing parameters characterizing the first trained neural network in a data repository (see Chatterjee paragraph 0040, where the modality-neutral component can electronically store the machine learning model).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS JOHN HELCO whose telephone number is (703)756-5539. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella, can be reached at telephone number 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/NICHOLAS JOHN HELCO/Examiner, Art Unit 2667 /MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights