DETAILED ACTION
Response to Arguments
Applicant’s argument: Saha discloses the use of DCE-MRI images to obtain BPE scores. However, Saha only discloses BPE scores obtained from patients without breast cancer. The purpose of Saha is to determine how good BPE predicts future development of cancer (see Saha, Background). This determination is based on DCE-MRI images obtained from patients before they developed cancer. Thus, none of Saha's DCE-MRI images have a lesion.
The Office Action states: It would have been obvious to one of ordinary skill in the art...to include the segmentation of sequence of MRI slices, as well as projecting the sequence to form a maximum intensity projection image as suggested by Saha to Sakashita's deletion of a lesion from a 3D image. (Office Action, p. 5).
We respectfully disagree. The person of ordinary skill in the art would immediately recognize that using Sakashita's device to delete lesions from Saha's lesion-free DCE-MRI images would render Sakashita's device inoperable for its intended purpose, as there is, in fact, no lesion in Saha's images for Sakashita's device to delete.
Examiner’s response: The motivation to combine the references has been changed to only include motivations as to why one would create MIPS images from a sequence of 3D medical images. This motivation does not require the obtaining of the BPE scores, and claim 1 also does not require BPE score calculation.
Sakashita is being modified by Saha, not the other way around. Therefore, Sakashita is used to create the lesion deleted images, and then Saha is used to create a MIPS image. This combination would be operable since a lesion free image can then be projected to create a lesion free MIPS image for reasons that have nothing to do with BPE scores.
Applicant’s argument: The Office Action provides no reasoning or evidence to support its conclusion that the motivation to combine Sakashita and Saha is "to more accurately calculate the BPE features or scores without the tumor interfering with the calculations." (Office Action, p. 5). Accordingly, the Office Action has failed to provide any motivation to combine these references.
Examiner’s response: The motivation to combine has been changed as disclosed in the rejection of the independent claims. These reasons have nothing to do with BPE features. While the motivation does not come from either Sakashita or Saha it is common knowledge in the art.
Applicant’s argument: The Office Action merely reinstates the applicant's own inventive concept as if it were a known motivation to combine. The proposed motivation to combine (i.e., improved accuracy of BPE scores) is not based on any fact taught, disclosed, or suggested by Saha or Sakashita. Therefore, we respectfully submit that this proposed motivation to combine is impermissible hindsight.
Examiner’s response: It is agreed that the previous motivation to combine was likely impermissible hindsight. Therefore, the motivation to combine has been changed as disclosed in the independent claims.
However, claims 3 and 13 which do claim obtaining a BPE score have now been objected to since it would not be obvious to combine the portions of Saha that teach these limitations.
Since the prior art rejections have not changed, this office action has been made Final.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 4-5, 9-12, 14-15, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Sakashita (US Pub. No. 2022/0358640 A1) in view of Saha et al. (“Machine learning based prediction of future breast cancer using algorithmically measured background parenchymal enhancement on high-risk screening MRI”).
Regarding claim 1, Sakashita discloses, a method for electronically removing a lesion from a three-dimensional (3D) medical image, comprising: segmenting each two-dimensional (2D) slice of a sequence of 2D slices of the 3D medical image to identify the lesion within any one or more of the 2D slices; (See Sakashita ¶60, “For example, information of the position and range of the lesion 31 may be acquired by executing known image processing on the medical image 30 and segmenting the region of the lesion 31.” Further see Sakashita ¶42, “However, at least part of the technology exemplified by the present disclosure can be applied to even a case in which the medical image 30 and the predicted disease image 60 are images other than fundus front images. … Furthermore, the medical image 30 and the predicted disease image 60 may be medical images of a living tissue other than the eye to be examined (for example, images of internal organs).” Further see Sakashita ¶42, “For example, the medical image 30 and the predicted disease image 60 may be two-dimensional tomographic images or three-dimensional tomographic images including information in the depth direction.” Further see Sakashita ¶54, “For example, a scanning laser ophthalmoscope (SLO), an OCT device, a corneal endothelial cell photographing device (CEM), a computed tomography (CT) device, or the like may be used as the medical image photographing device 11.”)
deleting the lesion from each 2D slice in which the lesion was identified to create a sequence of lesion-deleted slices; (See Sakashita ¶97, In addition, the user can input an instruction to designate deletion of the selected lesion 31 by operating the deletion button in a state in which the lesion 31 on the pre-modification image 50 is selected. If an instruction to change the range of the lesion 31 is input (S17: YES), the CPU 23 changes (enlarge, reduce, or delete) the information of the range of the selected lesion 31 in the lesion information 73 (see FIG. 2) to the range designated in S17 (S18).” Further see Sakashita ¶98, “More specifically, in S20 in the present embodiment, the CPU 23 inputs the lesion removed image 40 acquired in S2 on the basis of the medical image 30 and the added or changed lesion information 73 to the predicted disease image generation model 74 (see FIG. 2). In S21, the CPU 23 acquires the predicted disease image 60 output by the predicted disease image generation model 74.”)
Sakashita discloses the above limitations, but he fails to disclose the following limitations.
However, Saha discloses, segmenting each two-dimensional (2D) slice of a sequence of 2D slices of the 3D medical image (See Saha p. 4, “(a) Breast mask generation: The breast mask segmentation T1 fat saturated sequence was carried out as described in (12). This mask was registered to the first post contrast sequence.”)
and constructing, based on the sequence of lesion-deleted slices, a lesion-deleted intensity- based projection image. (See Saha p. 4, “(d) Heart Mask extraction from the MIP: The maximum intensity values of the subtracted sequence were projected in z-direction to form the maximum intensity projection (MIP) image.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the segmentation of sequence of image slices, as well as projecting the sequence to form a maximum intensity projection image as suggested by Saha to Sakashita’s deletion of a lesion from a 3D image. This can be done using known engineering techniques, with a reasonable expectation of success. The motivation for doing so is because MIPs offer offers significant advantages in visualization, diagnostic efficiency, and analysis. MIP works by projecting the highest intensity voxel along a viewing ray onto a 2D image, which is especially beneficial for highlighting high-contrast structures. By summarizing a thick slab of 3D data into a single 2D image, MIPs reduce the need for radiologists to scroll through hundreds of individual slices, speeding up interpretation. It provides enhanced visualization of vascular structures by displaying the entire course of a vessel in a single image, which is crucial for assessing vascular anatomy.
Regarding claim 2, Sakashita and Saha disclose, the method of claim 1, wherein said constructing comprises constructing a lesion-deleted maximum-intensity projection image. (See Saha p. 4, left col, 1st para, “(d) Heart Mask extraction from the MIP: The maximum intensity values of the subtracted sequence were projected in z-direction to form the maximum intensity projection (MIP) image.” Further see Saha p. 4 left col, 2nd para, “Voxel enhancement is defined as a change in the intensity of the voxel in the first postcontrast image compared with that in the precontrast image. In our work, enhancement in MR volumes is quantified in terms of the percentage change in the intensity of the voxel in the first postcontrast image compared with the precontrast image.”)
Regarding claim 4, Sakashita and Saha disclose, the method of claim 1, wherein each 2D slice comprises a dynamic contrast enhanced magnetic resonance image. (See Saha p. 4, “(d) Heart Mask extraction from the MIP: The maximum intensity values of the subtracted sequence were projected in z-direction to form the maximum intensity projection (MIP) image.” Further see Saha p. 4, left
Regarding claim 5, Sakashita and Saha disclose, the method of claim 1, wherein said constructing comprises blocking, with a mask, one or more voxels of any one or more of the 2D slices. (See Saha p. 4, “(a) Breast mask generation: The breast mask segmentation T1 fat saturated sequence was carried out as described in (12). This mask was registered to the first post contrast sequence.”)
Regarding claim 9, Sakashita and Saha disclose, the method of claim 1, wherein said deleting comprises replacing, for each voxel of a plurality of voxels forming the lesion, a value of said each voxel with a replacement value. (See Sakashita Fig. 2 which shows the lesion 30 in image 30, has been replaced in the output image 60 with other non-lesion pixels.)
Regarding claim 10, Sakashita discloses, a method for electronically removing a lesion from a three-dimensional (3D) medical image, comprising: segmenting the intensity-based projection image to identify the projection of the lesion; (See Sakashita ¶60, “For example, information of the position and range of the lesion 31 may be acquired by executing known image processing on the medical image 30 and segmenting the region of the lesion 31.” Further see Sakashita ¶42, “However, at least part of the technology exemplified by the present disclosure can be applied to even a case in which the medical image 30 and the predicted disease image 60 are images other than fundus front images. … Furthermore, the medical image 30 and the predicted disease image 60 may be medical images of a living tissue other than the eye to be examined (for example, images of internal organs).” Further see Sakashita ¶42, “For example, the medical image 30 and the predicted disease image 60 may be two-dimensional tomographic images or three-dimensional tomographic images including information in the depth direction.”)
and deleting the projection of the lesion from the intensity-based projection image. (See Sakashita ¶97, In addition, the user can input an instruction to designate deletion of the selected lesion 31 by operating the deletion button in a state in which the lesion 31 on the pre-modification image 50 is selected. If an instruction to change the range of the lesion 31 is input (S17: YES), the CPU 23 changes (enlarge, reduce, or delete) the information of the range of the selected lesion 31 in the lesion information 73 (see FIG. 2) to the range designated in S17 (S18).” Further see Sakashita ¶98, “More specifically, in S20 in the present embodiment, the CPU 23 inputs the lesion removed image 40 acquired in S2 on the basis of the medical image 30 and the added or changed lesion information 73 to the predicted disease image generation model 74 (see FIG. 2). In S21, the CPU 23 acquires the predicted disease image 60 output by the predicted disease image generation model 74.”)
Sakashita discloses the above limitations but he fails to disclose the following limitations.
However, Saha discloses, constructing, based on a sequence of two-dimensional (2D) slices of the 3D medical image, an intensity-based projection image containing a projection of the lesion; (See Saha p. 4, “(d) Heart Mask extraction from the MIP: The maximum intensity values of the subtracted sequence were projected in z-direction to form the maximum intensity projection (MIP) image.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include projecting to form a maximum intensity projection image as suggested by Saha to Sakashita’s deletion of a lesion from a 3D image. This can be done using known engineering techniques, with a reasonable expectation of success. The motivation for doing so is because MIPs offer offers significant advantages in visualization, diagnostic efficiency, and analysis. MIP works by projecting the highest intensity voxel along a viewing ray onto a 2D image, which is especially beneficial for highlighting high-contrast structures. By summarizing a thick slab of 3D data into a single 2D image, MIPs reduce the need for radiologists to scroll through hundreds of individual slices, speeding up interpretation. It provides enhanced visualization of vascular structures by displaying the entire course of a vessel in a single image, which is crucial for assessing vascular anatomy.
Regarding claim 11, Sakashita and Saha disclose, a system for electronically removing a lesion from a three-dimensional (3D) medical image, comprising: a processor; a memory communicably coupled with the processor; (See Sakashita ¶46, “The control unit 2 includes a CPU 3 that is a controller configured to perform control and a storage device 4 that can store programs, data, and the like.”)
and a lesion deleted implemented as machine-readable instructions that are stored in the memory and, when executed by the processor, control the system to: segment each two-dimensional (2D) slice of a sequence of 2D slices of the 3D medical image to identify the lesion within any one or more of the 2D slices, delete the lesion from each 2D slice in which the lesion was identified to create a sequence of lesion-deleted slices, and construct, based on the sequence of lesion-deleted slices, a lesion-deleted intensity-based projection image. (See the rejection of claim 1 as it is equally appliable for claim 11 as well.)
Regarding claim 12, Sakashita and Saha disclose, the system of claim 11, wherein the machine-readable instructions that, when executed by the processor, control the system to construct include machine-readable instructions that, when executed by the processor, control the system to construct a lesion-deleted maximum-intensity projection image. (See the rejection of claim 2 as it is equally appliable for claim 12 as well.)
Regarding claim 14, Sakashita and Saha disclose, the system of claim 11, wherein each 2D slice is a dynamic contrast enhanced magnetic resonance image. (See the rejection of claim 4 as it is equally appliable for claim 14 as well.)
Regarding claim 15, Sakashita and Saha disclose, the system of claim 11, further comprising a masker implemented as machine-readable instructions that are stored in the memory and, when executed by the processor, control the system to block, with a mask, one or more voxels of any one or more of the 2D slices. (See the rejection of claim 5 as it is equally appliable for claim 15 as well.)
Regarding claim 19, Sakashita and Saha disclose, the system of claim 11, wherein the machine-readable instructions that, when executed by the processor, control the system to segment include machine-readable instructions that, when executed by the processor, control the system to cluster. (See the rejection of claim 9 as it is equally appliable for claim 19 as well.)
Regarding claim 20, Sakashita and Saha disclose, the system of claim 11, further comprising a medical imaging device for capturing the 3D medical image. (See Sakashita ¶54, “For example, a scanning laser ophthalmoscope (SLO), an OCT device, a corneal endothelial cell photographing device (CEM), a computed tomography (CT) device, or the like may be used as the medical image photographing device 11.”)
Claims 6-8 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Sakashita (US Pub. No. 2022/0358640 A1) in view of Saha et al. (“Machine learning0based prediction of future breast cancer using algorithmically measured background parenchymal enhancement on high-risk screening MRI”) and in further view of Bernard et al. (US Pub. No. 2023/0023042 A1).
Regarding claim 6, Sakashita and Saha disclose, the method of claim 5, but they fail to disclose the following limitations.
However, Bernard discloses, further comprising generating a mask for each 2D slice. (See Bernard ¶80, “At 354, method 350 includes generating a mask for the new input image. For embodiments where the new input image is an FFDM image, the mask may be a 2D breast mask, such as the 2D breast mask 218 of FIG. 2A described above.” Further see Bernard ¶18, “The CNN may be deployed to classify and/or detect lesions in the 2D FFDM and/or biopsy images, or the 3D DBT and/or DBT biopsy images, in accordance with method 350 of FIG. 3B.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the creation of a segmentation mask for a 3D volume of MRI images as suggested by Bernard to Sakashita and Saha’s segmentation of breast images using known engineering techniques, with a reasonable expectation of success. The motivation for doing so is in order to accurately segment the breast tissue from other body tissue.
Regarding claim 7, Sakashita, Saha, and Bernard disclose, the method of claim 6, wherein said generating the mask comprises: inputting said each 2D slice to a trained convolutional neural network (CNN) to obtain a corresponding region-of-interest; (See Bernard ¶80, “For example, the mask generator may use a previously trained neural network and/or a machine learning algorithm to detect and/or segment a breast of the new input image (e.g., a breast segmentation model), or the mask generator may generate a 2D biopsy window mask in accordance with a local biopsy procedure or equipment.”)
and binarizing the region-of-interest to obtain the one mask. (See Bernard ¶80, “The mask generator may subsequently generate an array of 1s and 0s of a size of the new input image, where either a 1 or a 0 is assigned to each pixel of the new input image. If a pixel of the new input image is included within the boundary of the breast or biopsy window, a 1 may be assigned to the array of 1s and 0s at a location corresponding to the pixel. Alternatively, if the pixel is not included within the boundary of the breast or biopsy window, a 0 may be assigned to the array of 1s and 0s at the location corresponding to the pixel.”)
Regarding claim 8, Sakashita, Saha, and Bernard disclose, the method of claim 7, wherein the trained CNN identifies a class label for each voxel of said each 2D slice. ¶80, “If a pixel of the new input image is included within the boundary of the breast or biopsy window, a 1 may be assigned to the array of 1s and 0s at a location corresponding to the pixel. Alternatively, if the pixel is not included within the boundary of the breast or biopsy window, a 0 may be assigned to the array of 1s and 0s at the location corresponding to the pixel.”)
Regarding claim 16, Sakashita, Saha, and Bernard disclose, the system of claim 15, further comprising a mask generator implemented as machine- readable instructions that are stored in the memory and, when executed by the processor, control the system to generate a mask for each 2D slice. (See the rejection of claim 6 as it is equally appliable for claim 16 as well.)
Regarding claim 17, Sakashita, Saha, and Bernard disclose, the system of claim 16, the mask generator including additional machine-readable instructions that, when executed by the processor, control the system to: input said each 2D slice to a trained convolutional neural network (CNN) to obtain a corresponding region-of-interest, and binarize the region-of-interest to obtain the mask. (See the rejection of claim 7 as it is equally appliable for claim 17 as well.)
Regarding claim 18, Sakashita, Saha, and Bernard disclose, the system of claim 17, wherein the trained CNN identifies a class label for each voxel of said each 2D slice. (See the rejection of claim 8 as it is equally appliable for claim 18 as well.)
Allowable Subject Matter
Claims 3 and 13 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 3, Sakashita and Saha disclose, the method of claim 1, further comprising processing the lesion-deleted intensity-based projection image to obtain a background parenchymal enhancement (BPE) score. (The disclosed prior art of record fails to disclose the limitations of this claim, or it would not be obvious to combine references to teach these limitations.)
Regarding claim 13, Sakashita and Saha disclose, the system of claim 11, further comprising a background parenchymal enhancement (BPE) scorer implemented as machine-readable instructions that are stored in the memory and, when executed by the processor, control the system to process the lesion- deleted intensity-based projection image to obtain a BPE score. (The disclosed prior art of record fails to disclose the limitations of this claim, or it would not be obvious to combine references to teach these limitations.)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID PERLMAN whose telephone number is (571) 270-1417.
The examiner can normally be reached on Monday - Friday; 10:00am -6:30pm.
Examiner interviews are available via telephone and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is
(571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at (866) 217-9197 (toll-free). If You would like assistance from a USPTO Customer Service Representative, call (800) 786-9199 (IN USA OR CANADA) or (571) 272-1000.
/DAVID PERLMAN/Primary Examiner, Art Unit 2673