Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed August 3 2026 has been entered and considered. Claims 1, 11, and 14 have been amended. Claims 2 and 12 have been canceled. The amendment does not overcome the rejection under 35 U.S.C. 102 previously set forth. The rejection is maintained and is restated below with additional discussion of the amended limitations; accordingly, this action is made final.
Specification Objections –
In view of the amendment to the specification, the objection is withdrawn as moot.
Claim Objections –
In view of the amendment to claim 14, the objection is withdrawn as moot.
Response to Arguments
Applicant's arguments filed 8/3/2026, see Remarks Pgs. 9-10, have been fully considered but they are not persuasive.
Applicant argues that the prior art does not disclose the amended limitations.
Examiner respectfully disagrees.
Applicant argues (Remarks Pgs. 9-10):
For example, Applicant submits that Zaman fails to teach or suggest converting the output segmentation image having the second data format to have the same data format as the first data format of the input medical image.
Examiner responds:
Applicant’s specification expressly provides that different “data formats” may include data differing in “type” (Paras. 45 and 58). Zaman teaches an input medical image as real-valued data and the segmentation image as integer-valued data being jointly provided as a two-channel input to the CNN. Zaman further discloses that the proposed method was implemented using TensorFlow (Pg. 5, Col. 1). TensorFlow does not automatically perform type conversions and operations involving incompatible tensor data types result in an error, with conversion between types being required where appropriate (See TensorFlow in Pertinent Prior Art). Specifically, because Zaman expressly identifies the input medical image as real-valued and the segmentation image as integer-valued, yet processes the two together as channels of the disclosed network input, the differing data types necessarily must be reconciled to a common data type in forming the disclosed input.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 4-7, 9, 11, 14-17, and 19-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zaman et al. (NPL, “Segmentation quality assessment by automated detection of erroneous surface regions in medical images”, published 2022, pdf attached).
Regarding claim 1, Zaman teaches a method of auditing of artificial intelligence-based medical image segmentation, the method comprising: performing preprocessing to generate a preprocessed segmentation image by receiving an input medical image and an output segmentation image provided from a medical image segmentation device and preprocessing the output segmentation image based on the input medical image (Fig. 1, Pg. 3, Col. 1, “Iin and Iseg are used as two-channel input to the network. The output of the final layer has the same resolution as Iin”, the segmentation image is stacked with the input image as separate channels to form the preprocessed segmentation image); generating a heatmap image to generate a segmentation error heatmap image, which comprises a segmentation error region in the preprocessed segmentation image (Fig. 6, Pg. 3, Col. 1, “The predicted error mask Epred of the network is a prediction map of the multi-class erroneous voxels, conditioned on Iin and Iseg.”, a prediction map of erroneous voxels is a heatmap of error pixels), by inputting the preprocessed segmentation image to a deep learning model trained in advance (Fig. 1); calculating an error risk to calculate a segmentation error risk for the segmentation error region based on pixel values of the segmentation error heatmap image (Pg. 3, “Each channel represents the probability distribution of the erroneous voxels corresponding to a particular class or background. We assign each of the error voxels to a particular class depending on the corresponding maximum probability score.”; Pg. 4, Col. 1, “We remove all erroneous surface patches with less number of connected voxels than the optimized threshold. This reduces the over-sensitivity of the CNN in II-A in tracing the actual surface topology and prioritizes only the significant errors on the segmented surface.”, thresholds are used to focus on the higher risk errors); and providing auditing information to provide the auditing information for auditing accuracy of the output segmentation image based on the calculated segmentation error risk to a user (Pg. 7, Col. 2, “the method enables quick navigation to the error locations of segmentation, which would significantly improve the efficiency of user interaction.”), wherein the performance of the preprocessing comprises: receiving the input medical image having a first data format and the output segmentation image having a second data format provided from the medical image segmentation device; and converting the output segmentation image having the second data format to have the same data format as the first data format of the input medical image (Pg. 3, Col. 2, “Iin ∈ Rn× n× n, conditioned on the corresponding segmentation mask, Iseg ∈ Zn× n× n”; Pg. 4, Col. 1, “Iin and Iseg are used as two-channel input to the network.”, See analysis under Response to Arguments above).
Regarding claim 4, Zaman teaches all of the elements of claim 1, as stated above, as well as wherein the generation of the heatmap image comprises: receiving the preprocessed segmentation image; and applying the preprocessed segmentation image to the deep learning model, which distinguishes between a normal image and an abnormal image, to output the segmentation error region in a heatmap format (Fig. 1, Fig. 6, shows the usage of the preprocessed segmentation image applied to the deep learning model and the resulting classification.).
Regarding claim 5, Zaman teaches all of the elements of claim 4, as stated above, as well as in training the deep learning model, generating a plurality of output segmentation images by inputting the plurality of input medical images to the medical image segmentation device, labeling the plurality of output segmentation images divisionally with normal images and abnormal images to make up a training data set, and inputting the training data set to the deep learning model so that the deep learning model can be trained to output a segmentation error heatmap image for the segmentation error region (Fig. 1, “The ground truth of the error image mask Egt is used to calculate the loss for the training of the CNN.”, Zaman discloses a typical training pipeline for a neural network, including ground truth labeled training images).
Regarding claim 6, Zaman teaches all of the elements of claim 5, as stated above, as well as wherein, in the training, the deep learning model is trained to compare the normal image and the abnormal image, and identify a segmentation error region based on a changed region in the abnormal image compared to the normal image (Fig. 6).
Regarding claim 7, Zaman teaches all of the elements of claim 6, as stated above, as well as wherein the deep learning model is provided as a single model to identify the segmentation error region from the preprocessed segmentation image upon the preprocessed segmentation image being provided, and apply a heatmap format to the segmentation error region (Fig. 1).
Regarding claim 9, Zaman teaches all of the elements of claim 1, as stated above, as well as wherein the calculation of the error risk comprises calculating the error risk based on at least one of a maximum pixel value or a pixel value sum of the segmentation error heatmap image (Pg. 3, “Each channel represents the probability distribution of the erroneous voxels corresponding to a particular class or background. We assign each of the error voxels to a particular class depending on the corresponding maximum probability score.”).
Claim 11 corresponds to claim 1 and is rejected under the same analysis.
Claim 14 corresponds to claim 4 and is rejected under the same analysis.
Claim 15 corresponds to claim 5 and is rejected under the same analysis.
Claim 16 corresponds to claim 6 and is rejected under the same analysis.
Claim 17 corresponds to claim 7 and is rejected under the same analysis.
Claim 19 corresponds to claim 9 and is rejected under the same analysis.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 8, 10, 18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zaman.
Regarding claim 8, Zaman teaches all of the elements of claim 7, as stated above, as well as wherein the deep learning model comprises: a first deep learning model configured to identify the segmentation error region from the preprocessed segmentation image upon the preprocessed segmentation image being provided; and a second deep learning model linked to the first deep learning model and configured to apply a heatmap format to the segmentation error region (Fig. 1, although Zaman describes these operations within a single model, they also describe the possibility of using a GAN architecture for improved accuracy(Pg. 8, Col. 1). It would have been obvious to one of ordinary skill in the art to implement the same functionality using two linked models. Modularizing these functions across two models is a predictable design choice offering ease of training and flexibility in deployment.).
Regarding claim 10, Zaman teaches all of the elements of claim 1, as stated above, as well as wherein the provision of the auditing information comprises: assessing the plurality of segmentation error heatmap images, in which the segmentation error risks have been calculated, for significant errors (Pg. 4, “This reduces the over-sensitivity of the CNN in II-A in tracing the actual surface topology and prioritizes only the significant errors on the segmented surface.”;), and providing the input medical image, the output segmentation image, and the segmentation error heatmap image, in which the segmentation error risk is high, to the user (Fig. 6, Pg. 7, “First of all, the method enables quick navigation to the error locations of segmentation, which would significantly improve the efficiency of user interaction.”).
Zaman as modified above does not explicitly disclose sorting the plurality of segmentation error heatmap images in order of high segmentation error risk.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zaman to include sorting the plurality of segmentation error heatmap images in order of high segmentation error risk. Zaman teaches generating segmentation error maps and computing error-likelihood for each region in order for a user to review, as well as a focus on significant errors. Given multiple heatmap images with associated error-risk values, it would have been obvious to one of ordinary skill in the art to sort them in order of high segmentation risk so that the most problematic cases can be reviewed first. Sorting quality assessment results by severity is a routine and predictable design choice.
Claim 18 corresponds to claim 8 and is rejected under the same analysis.
Claim 20 corresponds to claim 10 and is rejected under the same analysis.
Claim(s) 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Zaman in view of Dong et al. (Korean Patent Number 102313652 B1, published 2021, machine translation attached).
Regarding claim 3, Zaman teaches all of the elements of claim 1, as stated above, as well as converting an image corresponding to the segmentation components to have the same format as a format of the input medical image (Fig. 1).
Zaman does not explicitly disclose receiving the input medical image and the output segmentation image, in which a segment region of the input medical image is subjected to a color overlay or separating the output segmentation image into color components and gray components.
Dong teaches receiving the input medical image and the output segmentation image, in which a segment region of the input medical image is subjected to a color overlay, provided from the medical image segmentation device (Pg. 40, “In this case, the result output through the output layer of the neural network model may be in a form in which a first color is overlaid on a first area of the target image and a second color is overlaid on a second area of the target image based on the first label. Through this, the first area and the second area may be more easily distinguished.”).
Dong does not explicitly disclose separating the output segmentation image into color components and gray components.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Zaman to incorporate the teachings of Dong to include receiving the input medical image and the output segmentation image, in which a segment region of the input medical image is subjected to a color overlay and separating the output segmentation image into color components and gray components. Dong discloses a medical image segmentation device that outputs a segmentation result in the form of a target medical image with colored overlays applied to segmented regions to distinguish them. One of ordinary skill in the art would have understood that using the color overlays of Dong with the segmentation results of Zaman to distinguish it is a routine technique in the art. Although Zaman and Dong do not explicitly disclose separating the color-overlay segmentation image into color components and gray components, the color overlaid output of Dong inherently comprises color components and gray components. One of ordinary skill in the art would have understood that separating these components is common practice in the art and possible using well-known pixel classification techniques. Extending the image normalization taught by Zaman to convert the image corresponding to the color components to have the same format of the input medical image would have been a routine design choice.
Regarding claim 13, the recited elements perform variably the same function as that of claim 3. It is rejected under the same analysis.
Conclusion
Pertinent Prior Art: TensorFlow. (2021). Introduction_to_TensorFlow_part_1_-_basics.Ipynb at master · Google/TF-quant-finance. GitHub. https://github.com/google/tf-quant-finance/blob/master/tf_quant_finance/examples/jupyter_notebooks/Introduction_to_TensorFlow_Part_1_-_Basics.ipynb Introduction and overview of TensorFlow and how it is used. Relevant pages attached, where it goes over data types and the processing needed when using different data types.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID A WAMBST whose telephone number is (703)756-1750. The examiner can normally be reached M-F 9-6:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID ALEXANDER WAMBST/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
