DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see Remarks pages 7-8, filed 07/29/2026, with respect to the rejection of claim(s) 1 and 6-7 under 35 U.S.C. 102(a)(1) have been fully considered and are moot in view of the new grounds of rejection (detailed in the rejections below) necessitated by Applicant’s amendment to the claim(s).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2 and 6-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Periaswamy et al. (US-20140147025-A1) hereinafter referenced as Periaswamy, in view of Uppaluri et al. (US-20030215120-A1) hereinafter referenced as Uppaluri.
Regarding claim 1, Periaswamy discloses: An image processing apparatus comprising: at least one processor, wherein the at least one processor is configured (Periaswamy: Abstract) to execute: calcification image detection processing of detecting a calcification image based on a plurality of tomographic images obtained from a series of a plurality of projection images obtained by tomosynthesis imaging of a breast (Periaswamy: 0017: “According to an embodiment, the three-dimensional medical image source 110 is a digital tomosynthesis imaging system such as offered by the General Electric Company of Fairfield, Conn. (GE); Hologic, Inc, of Bedford, Mass. (Hologic); or Siemens AG of Munich, Germany (Siemens). Digital tomosynthesis imaging systems image an anatomical region by moving a source, and acquiring a plurality of projection images (e.g., 10-25 direct projections) at different angles (e.g., at 4-degree increments).”;
0022: “the detector 124 can include a calcification detector, blob detector, a spiculation detector, or combinations thereof. As illustrated in FIG. 1, the three-dimensional ROI detector 124 produces an ROI response image 126 that contains this characterization information for every image slice in the three-dimensional image 112.”);
region-of-interest image generation processing of generating a region-of-interest image based on a detection result by the calcification image detection processing by cutting out a region including the calcification image, from a projection image obtained at an irradiation position among the plurality of tomographic images, the irradiation position being closest to a position facing a detection surface of a radiation detector (Periaswamy: 0022-0023: “As illustrated in FIG. 1, the three-dimensional ROI detector 124 produces an ROI response image 126 that contains this characterization information for every image slice in the three-dimensional image 112. The two-dimensional ROI extractor 128 extracts two-dimensional information from portions of the three-dimensional image 112 that include the points or regions of interest exhibiting the characteristics of interest.”; Wherein the extraction of 2D ROI information from the 3D image, constructed from a plurality of projection images, includes a “closest” irradiation position.);
shape restoration processing of restoring a shape of the calcification image based on the region-of-interest image generated by the region-of-interest image generation processing (Periaswamy: 0023: “According to an embodiment, the extractor 128 extracts a 2D binary mask 130, also referred to herein as a chip 130 , for each ROI.”); and
synthesized two-dimensional image generation processing of generating a synthesized two-dimensional image based on a shape restoration result by the shape restoration processing and the plurality of tomographic images (Periaswamy: 0024: “the image blending unit 132 includes a blending function or process that combines the two-dimensional information extracted by the extractor 128 with the two-dimensional image 118 provided by source 116. The blending function/process forms the ROI-enhanced two-dimensional image 140.”).
Periaswamy does not disclose expressly: region-of-interest image generation processing of generating a region-of-interest image based on a detection result by the calcification image detection processing by cutting out a region including the calcification image, from a projection image obtained at an irradiation position among the plurality of projection images, the irradiation position being closest to a position facing a detection surface of a radiation detector.
Uppaluri discloses: region-of-interest image generation processing of generating a region-of-interest image based on a detection result by a calcification image detection processing by cutting out a region including a calcification image (Uppaluri: 0057-0058: “Referring now to either FIGS. 6 or 7, for the data source 210, data may be obtained from a combination of one or more sources…Dual energy image sets 215…are an additional source of data for the data source 210…
On the image-based data 215, a region of interest 220 can be defined from which to calculate features. The region of interest 220 can be defined several ways. For example, the entire image 215 could be used as the region of interest 220. Alternatively, a part of the image, such as a candidate nodule region in the apical lung field could be selected as the region of interest 220. The segmentation of the region of interest 220 can be performed either manually or automatically.”;
0060: “Once the features, such as shape, size, density, gradient, edges, texture, etc., are computed as described above in the feature selection algorithm 230 and an optimal set of features 280 is produced, a pre-trained classification algorithm 240 can be used to classify the regions of interest 220 into benign or malignant nodules, calcifications, fractures or metastases, or whatever classifications are employed for the particular medical condition involved.”), from a projection image obtained at an irradiation position among a plurality of projection images, the irradiation position being closest to a position facing a detection surface of a radiation detector (Uppaluri: Figure 18; 0084: “an imaging mode where limited angle x-ray tomosynthesis acquisition is performed and reconstructed may be combined with the computer aided detection and diagnosis methods described above and as shown in FIG. 18…The tomosynthesis CAD system 800 shown in FIG. 18 is similar to the CT CAD system shown in FIG. 16 except that the image data 815 includes tomosynthesis images 1...N, such that data source 810 includes tomosynthesis images 815, image acquisition data 212, and patient demographic data 214. Other than data source 810, the system 800 is similar to system 200 described with respect to FIG. 6. Alternatively, the operations 220, 230, 240 may be performed in parallel on each tomosynthesis image as described with respect to FIG. 7.”; Wherein the processing of a set of tomosynthesis images includes the processing of a projection image with a “closest” irradiation position.).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to incorporate the algorithms for extracting and classifying ROI from a set of tomosynthesis images taught by Uppaluri by extracting calcification ROIs from the projection images disclosed by Periaswamy and incorporating them into the image mask extraction and image blending algorithms disclosed by Periaswamy. The suggestion/motivation for doing so would have been “The feature selection algorithm 230 is then employed to sort through the candidate features and select only the useful ones and remove those that provide no information or redundant information…the multiple feature measures 270 from the high energy image, low energy image, soft image, and bone images or a combination of those images are extracted…Step 274 ranks all the features based on the distance criteria. That is, the features are ranked based on their ability to differentiate between different classes, their discrimination capability. The feature selection algorithm 230 is also used to reduce the dimensionality from a practical standpoint…the output 280 provides an optimal set of features.
Once the features, such as shape, size, density, gradient, edges, texture, etc., are computed as described above in the feature selection algorithm 230 and an optimal set of features 280 is produced, a pre-trained classification algorithm 240 can be used to classify the regions of interest 220 into benign or malignant nodules, calcifications, fractures or metastases, or whatever classifications are employed for the particular medical condition involved.” (Uppaluri: 0059-0060; Wherein the processing of features extracted from the image set, which includes distinctly acquired images, allows for the optimization of features, thus improving ROI classification accuracy.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Periaswamy with Uppaluri to obtain the invention as specified in claim 1.
Regarding claim 2, Periaswamy in view of Uppaluri discloses: The image processing apparatus according to claim 1, wherein the at least one processor is configured to: in a case where a plurality of the calcification images are detected in the calcification image detection processing, individually generate the region-of-interest image for each of the plurality of calcification images in the region-of-interest image generation processing (Periaswamy: 0022-0023: “the three-dimensional ROI detector 124 produces an ROI response image 126 that contains this characterization information for every image slice in the three-dimensional image 112. The two-dimensional ROI extractor 128 extracts two-dimensional information from portions of the three-dimensional image 112 that include the points or regions of interest exhibiting the characteristics of interest. According to an embodiment, the extractor 128 extracts a 2D binary mask 130 , also referred to herein as a chip 130, for each ROI.”)
(Uppaluri: 0056: “ It should be noted here that dual energy CAD 200 may be performed once by incorporating features from all images 215 or may be performed in parallel…the ROI selection 220 can be performed on each image 216, 217,218, and 219 to provide a low energy image ROI 221, a high energy image ROI 222, a soft tissue image ROI 223, and a bone image ROI 224.”;
0084: “The tomosynthesis CAD system 800 shown in FIG. 18 is similar to the CT CAD system shown in FIG. 16 except that the image data 815 includes tomosynthesis images 1 . . . N, such that data source 810 includes tomosynthesis images 815, image acquisition data 212, and patient demographic data 214. Other than data source 810, the system 800 is similar to system 200 described with respect to FIG. 6. Alternatively, the operations 220, 230, 240 may be performed in parallel on each tomosynthesis image as described with respect to FIG. 7.”).
As per claim(s) 6, arguments made in rejecting claim(s) 1 are analogous.
As per claim(s) 7, arguments made in rejecting claim(s) 1 are analogous. In addition, 0020 of Periaswamy discloses the limitation “A non-transitory computer-readable storage medium storing a program executable by a computer to perform”.
Claim(s) 3-5 /are rejected under 35 U.S.C. 103 as being unpatentable over Periaswamy in view of Uppaluri, and further in view of Koike (WO-2021182229-A1).
Regarding claim 3, Periaswamy in view of Uppaluri discloses: The image processing apparatus according to claim 1.
Periaswamy in view of Uppaluri does not disclose expressly: wherein the at least one processor is configured to: execute the shape restoration processing by inputting the region-of-interest image into a machine-learned model.
Thus, Periaswamy in view of Uppaluri does not disclose expressly: the extraction of the 2D binary mask for an ROI image by inputting the ROI image into a machine-learned model.
Koike discloses: a detection unit comprising a learning unit trained to determine and extract whether an input image comprises a lesion (Koike: 0062-0063: “a convolutional neural network is used as the machine learning model for constructing the learning model 36A. The learning model 36A is constructed by training a machine learning model so that when an abnormal image Dj included in the training data is input, it outputs the probability (likelihood) that each pixel of the abnormal image Dj is in the region of a lesion.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the convolutional neural network taught by Koike in order to perform the 2D binary mask extraction disclosed by Periaswamy in view of Uppaluri. The suggestion/motivation for doing so would have been “The image processing program provided in this disclosure causes a computer to perform the following steps: acquiring a target image and detecting lesions contained in the target image using a model constructed by the learning device provided in this disclosure. According to this disclosure, lesions can be detected with high accuracy from images” (Koike: 0022-0023; Wherein the disclosed training of the neural network model provides accurate results.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Periaswamy in view of Uppaluri with Koike to obtain the invention as specified in claim 3.
Regarding claim 4, Periaswamy in view of Uppaluri and Koike discloses: The image processing apparatus according to claim 3, wherein the machine-learned model is a neural network obtained by performing machine learning by using, as an input image, the region-of-interest image (Periaswamy: 0022-0023: “The three-dimensional ROI detector 124 characterizes the degree to which various points or regions in an image exhibit characteristics of particular interest…the detector 124 can include a calcification detector, blob detector, a spiculation detector, or combinations thereof. As illustrated in FIG. 1, the three-dimensional ROI detector 124 produces an ROI response image 126 that contains this characterization information for every image slice in the three-dimensional image 112.”)
(Uppaluri: 0058: “On the image-based data 215, a region of interest 220 can be defined from which to calculate features. The region of interest 220 can be defined several ways. For example, the entire image 215 could be used as the region of interest 220. Alternatively, a part of the image, such as a candidate nodule region in the apical lung field could be selected as the region of interest 220. The segmentation of the region of interest 220 can be performed either manually or automatically.”)
(Koike: 0076: “In the second embodiment, a radiographic image H0 of the breast M is obtained by performing a simple mammography in which radiation is irradiated to the breast M only from the reference source position Sc in the mammography apparatus 1”;
0079: “The training data generation unit 34 generates training data HT which includes the abnormal image H1 generated as described above and data representing the location of lesion images included in the abnormal image H1 (which will be called the correct answer data). Figure 19 schematically shows the training data generated in the second embodiment. As shown in Figure 19, the training data HT consists of an abnormal image H1 containing a lesion image 44 of a simulated lesion 40, and ground truth data HC representing the position of the lesion image 44 of the simulated lesion 40 in the abnormal image H1. The training data HT generated by the training data generation unit 34 is stored in the storage unit 23”; Wherein the ROIs as disclosed by Periaswamy and Uppaluri are further processed for the generation of the training data.)
and using, as a correct answer image, an image generated by cutting out a region including the calcification image from a projection image obtained by plain radiography in which radiation is emitted from a position facing a detection surface of a radiation detector (Koike: 0079: “The training data generation unit 34 generates training data HT which includes the abnormal image H1 generated as described above and data representing the location of lesion images included in the abnormal image H1 (which will be called the correct answer data). Figure 19 schematically shows the training data generated in the second embodiment. As shown in Figure 19, the training data HT consists of an abnormal image H1 containing a lesion image 44 of a simulated lesion 40, and ground truth data HC representing the position of the lesion image 44 of the simulated lesion 40 in the abnormal image H1. The training data HT generated by the training data generation unit 34 is stored in the storage unit 23”;
0081: “the learning unit 35 constructs a learning model 36A for the detection unit 36 by training a machine learning model to determine the area of a lesion in the target image acquired by the input simple imaging, using the training data HT for the abnormal image H1 as the first training data”).
Regarding claim 5, Periaswamy in view of Uppaluri and Koike discloses: The image processing apparatus according to claim 3, wherein the machine-learned model is a neural network obtained by performing machine learning by using an input image and a correct answer image that are generated by a simulation or by imaging using a phantom (Koike: 0084: “First, the image acquisition unit 31 acquires a radiation image H0 for generating an abnormal image (step ST31). Next, the synthesis unit 32 generates an abnormal image H1 by synthesizing the lesion image with the radiographic image H0 based on the geometric relationship between the radiation source position when the mammography device 1 performed the imaging and the position of a simulated lesion virtually placed on the breast M (step ST32)”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J RODRIGUEZ whose telephone number is (703)756-5821. The examiner can normally be reached Monday-Friday 10am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY J RODRIGUEZ/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights