DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Response to Amendment
The amendment filed on April 20, 2026 has been entered.
The amendment of claims 1, 3, 4, 6, 7, 9, 10, 11, 12, 14, 15, and 16 and addition of claims 17-20 have been acknowledged.
In view of the amendment, the 35 U.S.C. 112(b) and 101 rejections have been withdrawn. The claims are no longer interpreted under 35 U.S.C. 112(f).
Response to Arguments
Applicant's arguments filed on April 20, 2026, with respect to the pending claims, have been fully considered but they are not persuasive.
Applicant’s Representative submits that the prior art (Bai) does not teach the claims because Bai’s method for mammogram classification does not correspond to object detection because Bai analyzes an entire pair of images to produce a single classification for the pair as a whole.
The examiner respectfully disagrees. The images are classified as either negative (normal) or positive (cancer). The neural networks used to classify the images are trained with the features obtained from the objects being detected, e.g., mass, microcalcifications, architectural distortion, etc.
In addition, Applicant’s arguments are moot because the arguments rely on newly added and/or amended claim limitations (e.g., bounding box). The examiner has revised the rejections to match the new claim limitations.
Claim Rejections - 35 USC § 103
Claim(s) 1, 2, 6-9, 11-16, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bai et al. (“Feature fusion Siamese network for breast cancer detection comparing current and prior mammograms,” Med Phys. 2022;49:3654–3669, DOI: 10.1002/mp.15598), in view of Nguyen et al. (US 10,499,857 B1), hereinafter referred to as Bai and Nguyen, respectively.
Regarding claim 1, Bai teaches a method for processing a medical image, the method comprising:
receiving a medical image (Bai pg. 3656 left column: “developing a novel end-to-end model based on the Siamese CNN model that uses previous year and current year images as paired inputs to predict the probability of malignancy”);
performing an image content analysis for object detection and classification on said medical image (Bai pg. 3656 left column: “extract intraimage and interimage features from pairs of patients’ previous and current year FFDMs for more accurate breast cancer classification”; Bai pg. 3656 right column: “Vector d1 is concatenated with scalar d2 to build the distance feature for classification”), comprising:
propagating the medical image in one iteration through at least one convolutional neural network (Bai pg. 3666 left column: “The one-shot learning characteristic of Siamese-based models can contribute to the superior performance of the proposed model”; Bai Figs. 1-3: the CNNs show the images are propagated in one iteration, indicated by an arrow), and
determining after said one iteration one or more detected objects together with a respective classification label identifying one of two or more different available classes and with respective positional parameters relative to the medical image (Bai Table 3 & pg. 3661 left column: “493 mammogram pairs are labeled cancer … and 581 mammogram pares are labelled normal”; Bai pg. 3656: “designing a new distance learning function … The distance learning network measures the distance between the feature maps from the twin networks and employs a fully connected (FC) network to learn the differences between the feature maps (interimage features) … d1 measures the pixel-wise distance of fc and fp, d2 measures the Euclidean distance between fc and fp … Vector d1 is concatenated with scalar d2 to build the distance feature for classification”; Bai Fig. 7: “from the same location”); and
storing the determined classification label and positional parameters of one or more detected objects in association with the medical image (Bai Fig. 3 & pg. 3659-3660: “We used four datasets (three for pretraining and one for training and testing): (1) Digital Database for Screening Mammography (DDSM) … ”; Bai pg. 3662 right column: “We used Tesla V100 GPUs with 32 GB memory to train and test all models”).
Bai further teaches that the detected objects are indicated with boundaries or bounding boxes (Bai Figs. 4 & 7). However, Bai does not appear to explicitly teach that the positional parameters define a bounding box for each detected object.
Pertaining to the same field of endeavor, Nguyen teaches that the positional parameters define a bounding box for each detected object (Nguyen col. 3 lines 46-61: “Analysis consists of, at least, the binary classification of each imaging study into two classes, normal or abnormal. This task can be referred to as classification (110). In addition, in the case of a positive (abnormal) study, suitable analysis typically provides indication of where in the imaging study (e.g. which slide, which locations within the slide) the abnormality is present. This task is sometimes referred to as detection, or as localization (115), which could be limited to a location or a coarse region of an image, like a quadrant or a bounding box, or the fine delineation of the region where the pathology is manifested (segmentation). Finally, a complete diagnosis requires the categorization (120) of the pathology, to determine the nature and severity of the abnormality. This step is carried out by classifying the abnormality as one of a number of abnormalities recognized by medical professional protocols”; Nguyen col. 7 line 55-col. 8 line 2: “For detection/localization, the function f is defined at each voxel or pixel in each image x, where it returns a class label y. Detectors may also return bounding box coordinates, or perhaps a parametric curve”; Nguyen col. 13 lines 17-38: “These labels may be image/volume wide or include a detailed location of the pathology i.e. in which image, and which pixels/voxels in that image, including possibly a bounding box or even a curve outlining where the pathology is located physically in the body. Labels may also include a wide variety of fine-grained categories indicating the region/anatomy that is normal/abnormal, as well as different medical conditions associated with that abnormality”).
Bai and Nguyen are considered to be analogous art because they are directed to medical image processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the feature fusion Siamese network for breast cancer detection (as taught by Bai) to define bounding boxes for each detected object (as taught by Nguyen) because the combination provides a detailed location of the pathology, including which regions of the body contains abnormality (Nguyen col. 13 lines 17-38).
Regarding claim 2, Bai, in view of Nguyen, teaches the method of claim 1, wherein the image content analysis is configured to detect and classify also partially cropped and/or at least partially overlapped objects and the determined classification label and positional parameters of said partially cropped and/or at least partially overlapped objects detected in the medical image (Bai pg. 3655 left column: “In a mammography there is a likelihood of missing small tumors surrounded by dense fibroglandular breast tissue, resulting in delays in the diagnosis and missing early detection”; Bai pg. 3661: “In order to increase the generalizability of the data set, we included a variety of tumor and breast density types. The mass type in the data set contains round, oval, architectural distortion, irregular, and lobulated … The data set contains all types of breast density including fatty breast, fibroglandular dense breast, heterogeneously dense breast, and extremely dense breast … All mammograms … where N is height, and M is width, are cropped”; Bai Fig. 4: shows that the cancer is detected and classified even in dense breast tissue, in which the cancer is cropped/overlaid by the dense tissue).
Regarding claim 6, Bai, in view of Nguyen, teaches the method of claim 1, wherein the object classification is configured to discriminate laterality of the detected objects (Bai pg. 3661 discussed above teaches distinguishing between different mass types; also see Bai pg. 3656 left column: “learn both interimage (between images) and intraimage features form both Craniocaudal (CC) and (mediolateral oblique) MLO views of patient’s particular breast”).
Regarding claim 7, Bai, in view of Nguyen, teaches the method of claim 6, wherein two or more specialized processing modules are selected based on at least one matching parameter, wherein the image is processed with all of the selected processing modules, wherein labels corresponding to medical conditions detected by different processing modules are collectively stored in association with the same image or stored in association with separate copies of the image (Bai pg. 3656 right column: “The goal of the proposed model is to predict the similarity between a current year image, denoted by C, and its corresponding previous year image, denoted by P, where ‘similar’ means normal and ‘dissimilar’ means cancer”; Bai pg. 3657 left column: “at the output layer a sigmoid function, as given in Equation (3) is applied to the distance feature to predict the probability of dissimilarity (cancer) or similarity (normal) … the similarity probability represents the likelihood of abnormal changes between current year and previous year images”; Bai Figs. 1-3).
Regarding claim 8, Bai, in view of Nguyen, teaches the method of claim 1, wherein the object classification is configured to discriminate view position of the detected objects (Bai pg. 3661 left column: “The FFDMS of two breasts and two view for each breast (LCC, RCC, LMLO, and RMLO) from a majority of patients are included in the data set … The cancer cases were defined as labeled breast CC views and MLO views with biopsy confirmed cancerous breast lesions”).
Regarding claim 9, Bai, in view of Nguyen, teaches the method of claim 1, further comprising:
providing two or more specialized processing modules, wherein each specialized processing module is associated with one or more compatible mandatory object classes (Bai pg. 3656-3657 & Figs. 1-3 discussed above);
comparing the one or more detected object classes associated with the image with each of the one or more compatible mandatory object classes to determine at least one matching parameter for each specialized processing module (Bai pg. 3656-3657 & Figs. 1-3 discussed above),
selecting at least one of the two more specialized processing modules based on the at least one matching parameter (Bai pg. 3656-3657 discussed above; see Bai Eq. (3)-(7) for FFS-CNN & Eq. (8) for Siamese CNN), and
processing the image with the selected at least one processing module, wherein the selected processing module detects one or more medical conditions and stores one or more corresponding labels in association with the image for displaying to a viewer of the image (Bai pg. 3656-3657 & Figs. 1-3 discussed above; also see Bai Fig. 4 & 7).
Regarding claim 11, Bai, in view of Nguyen, teaches the method of claim 9, wherein each specialized processing module is associated with zero or more compatible optional object classes, wherein the comparing comprises comparing the one or more detected object classes associated with the image with each of the one or more compatible mandatory object classes and each of the zero or more compatible optional object classes to determine at least one matching parameter for each specialized processing module (Bai pg. 3656-3657, Figs. 1-3, & Eq. (3)-(8) discussed above).
Regarding claim 12, Bai, in view of Nguyen, teaches the method of claim 1, wherein the medical image is a radiographic image (Bai Figs. 1-4 & pg. 3661 left column: “493 mammogram pairs are labeled cancer … 581 mammogram pairs are labeled normal”).
Regarding claim 13, Bai, in view of Nguyen, teaches the method of claim 1, wherein the medical image is received in the Digital Imaging and Communications in Medicine (DICOM) format (Bai pg. 3661 left column: “the DICOMs were exported from Picture Archiving and Communication Systems (PACS)”).
Regarding claim 14, Bai, in view of Nguyen, teaches a medical image processing system adapted to execute the method described in claim 1 (Bai Figs. 1-3 & pg. 3662 right column: “We used Tesla V100 GPUs with 32 GB memory”). Therefore, claim 14 is rejected using the same rationale as applied to claim 1 discussed above.
Regarding claim 15, Bai, in view of Nguyen, teaches a non-transitory computer readable medium containing a computer program product comprising instructions to cause a medical image processing system to execute the method described in claim 1 (Bai Figs. 1-3 & pg. 3662 right column: “We used Tesla V100 GPUs with 32 GB memory”). Therefore, claim 15 is rejected using the same rationale as applied to claim 1 discussed above.
Regarding claim 16, Bai, in view of Nguyen, teaches the computer-readable medium of claim 15 (Bai Fig. 3 & pg. 3662 right column discussed above), wherein the image content analysis is configured to detect and classify partially cropped and/or at least partially overlapped objects and the determined classification label and positional parameters of said partially cropped and/or at least partially overlapped objects detected in the medical image (Bai pg. 3655 left column & pg. 3661 discussed above).
Regarding claim 19, Bai, in view of Nguyen, teaches the method of claim 12, wherein the medical image is a two-dimensional x-ray image, an ultrasound image, a computer tomography image, a magnetic resonance image, or a positron emission image (Bai Figs. 1-4 & pg. 3661 left column discussed above).
Regarding claim 20, Bai, in view of Nguyen, teaches the medical image processing system of claim 14, wherein the image content analysis is configured to detect and classify partially cropped and/or at least partially overlapped objects and the determined classification label and positional parameters of said partially cropped and/or at least partially overlapped objects detected in the medical image (Bai pg. 3655 left column & pg. 3661 discussed above).
Claim(s) 3 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bai et al. (Med Phys. 2022;49:3654–3669, DOI: 10.1002/mp.15598), in view of Nguyen et al. (US 10,499,857 B1), and further in view of Poltaretskyi et al. (US 2019/0380792 A1), hereinafter referred to as Bai, Nguyen, Poltaretskyi, respectively.
Regarding claim 3, Bai, in view of Nguyen, teaches the method of claim 1, wherein the object detection and classification is configured for objects detecting and classifying one or more instances of body parts (Bai Fig. 4).
However, Bai, in view of Nguyen, does not appear to explicitly teach detecting body implants and outside structures e.g., measurement/calibration objects.
Pertaining to the same field of endeavor, Poltaretskyi teaches detecting body implants and outside structures (Poltaretskyi Fig. 28: 2200 bone structure and 1506 implant components & Poltaretskyi ¶¶0277: “The optimization process can employ any suitable optimization algorithm (e.g., a minimization algorithm such as an Iterative Closest Point or genetic algorithm) to perfect alignment of virtual bone model 1008 with observed bone structure 2200. At block 2016 of FIG. 20A, upon completion of execution of the optimization algorithm, the registration procedure is complete”; Poltaretskyi Fig. 30 & ¶¶0286-¶¶0290: “fixed optical marker 3010 may include a planar fiducial marker … stickers 3016A-3016C that include planar fiducial markers”).
Bai, in view of Nguyen, and Poltaretskyi are considered to be analogous art because they are directed to medical image processing. It would have been obvious to one of ordinary skill in the art at the time the invention was made to have modified the feature fusion Siamese network for breast cancer detection (as taught by Bai, in view of Nguyen,) to detect implants and other structures (as taught by Poltaretskyi) because the combination aids the surgeons by guiding tools to correct locations during surgical procedures (Poltaretskyi Abstract & ¶¶0162).
Regarding claim 17, Bai, in view of Nguyen and Poltaretskyi, teaches the method of claim 3, wherein the outside structures are one or more annotation, measurement and calibration objects (Poltaretskyi Figs. 28, 30, ¶¶0277, & ¶¶0286-¶¶0290 discussed above).
Allowable Subject Matter
Claims 4-5, 10, and 18 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 4, the prior art of record teaches that it was known at the time the application was filed to use the method of claim 1, wherein the at least one convolutional neural network is trained with training data comprising medical images with annotated and classified objects, wherein the annotated and classified objects are one or more of body parts, body implants and outside structures (Bai pg. 3656: “We used pretrained ResNEt as the backbone … Pairs of current and previous mammogram images are inputs of the proposed model, FFS-CNN … Define S … to present the training data set, where yi represent the class label”; Bai Fig. 2; Poltaretskyi Figs. 28, 30 & ¶¶0277, ¶¶0286-¶¶0290).
However, the prior art, alone or in combination, does not appear to teach or suggest that the annotated and classified objects are one or more from a group consisting of body parts, body implants and outside structures (i.e., the group consists of no other additional objects and no less than the listed objects due to the use of the phrase “consisting of”).
Claim 5 is objected to for the same reason as claim 4 discussed above due to dependency.
Regarding claim 10, the prior art of record teaches that it was known at the time the application was filed to use the method of claim 9, wherein the selecting uses a distance measure applied to the at least one matching parameter and selects processing module(s) corresponding to the smallest distance measure of the two or more specialized processing modules (Bai pg. 3656: “The proposed model consists of two identical parallel CNNS (twin CNNs) with shared weights as twin networks followed by a distance learning network … These feature vectors are input to the distance learning functions … measures the Euclidean distance between fc and fp”; Bai pg. 3656-3657 & Eq. (3)-(8) discussed above).
However, the prior art, alone or in combination, does not appear to teach or suggest selecting exactly one processing module corresponding to the smallest distance measure of the two or more specialized processing modules.
Claim 18 depends from claim 4 and therefore is objected to for the same reason as claim 3 discussed above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOO J SHIN whose telephone number is (571)272-9753. The examiner can normally be reached M-F; 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Soo Shin/Primary Examiner, Art Unit 2667
Prosecution Insights