DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 07/21/2026 has been entered.
Response to Arguments
Applicant’s arguments with respect to claims 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-12, 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (US 2022/0351347 A1) in view of Tong et al. (US 2021/0241122 A1) and further in view of Kim et al. (US 2022/0207721 A1).
With respect to Claim 1, Yang’347 shows one or more computer-readable non-transitory storage media (figure 1, 170) comprising instructions executable by one or more processors (160) of a digital pathology image processing system (10 paragraph [0040]) for:
receiving a plurality of digital pathology images (paragraph [0040] interpreting image data from the plurality of raw images acquired during image acquisition process and step 7 generating a representative image from a pathology image (the reconstructed image from steps 2-4)) of histologic samples (paragraphs [0033] and [0104]-[0106] archived clinical histologic/cytologic slides from available databases may be used to take raw images);
assessing physical characteristics of a first histologic sample (paragraphs [0033] and [0104]-[0106]) associated with a first digital pathology image of the plurality of digital pathology images (paragraph [0040] step 8 enumerate/assess abnormalities (tumor cells paragraph [0083]) in the sample from raw and pathology images, paragraph [0190] describes identifying spatial/physical relationships between normal and abnormal tissue);
[ ];
performing, [ ], a [ ] segmentation of the first digital pathology image (paragraph [0040] step 10) based on one or more regions of the first digital pathology image corresponding to one or more predetermined histologic features (paragraph [0108] a demarcated or segmented image may appear to have clearly defined or delineated boundaries (regions) drawn over the image to identify the portions of interest such as viable tumor cells (predetermined histologic features));
generating an assessment regarding a specified condition in the first histologic sample based on the one or more regions of the first digital pathology image corresponding to tumor bed and the one or more regions of the first digital pathology image corresponding to the one or more predetermined histologic features (paragraph [0040] steps 11-12 regard using the segmented image for training a neural network, figure 15 and paragraph [0138] training of the neural network (DCNN 1504) to output the segmented image 1506 with assessment 1508 for, paragraph [0142] image analysis including diagnostic indicator 1516); and
generating a user interface comprising a display of the assessment (paragraph [0142] the diagnostic indicator 1516 may be provided to a user on an interface).
Yang’347 does not specifically show performing a first segmentation of the first digital pathology image based on one or more regions of the first digital pathology image corresponding to tumor bed.
Tong’122 shows performing a first segmentation of the first digital pathology image based on one or more regions of the first digital pathology image corresponding to tumor bed (Figure 1 and paragraph [0027] annotated tissue sample images 111 used for training a prediction model 110. Paragraph [0033] automatically segmenting and classifying areas including tumor bed detection. Paragraph [0034] the segmentation and classification is performed by leverage masks of image bands wherein each image band is a set of image bands representing a corresponding cell type which requires identifying the area via different image bands. Paragraph [0050] multiple masks for multiple image bands each for segmentation including tumor bed detection.); and performing a second segmentation of the first digital pathology image based on one or more regions of the first digital pathology image corresponding to one or more predetermined histologic features (Paragraph [0033] automatically segmenting and classifying areas including predetermined histologic features such as necrosis, stomal tissue, or any other cell type. Paragraph [0034] the segmentation and classification is performed by leverage masks of image bands wherein each image band is a set of image bands representing a corresponding cell type which requires identifying the area via different image bands. Paragraph [0050] multiple masks for multiple image bands each for segmentation including predetermined histologic features such as necrosis, stomal tissue, or any other cell type.).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Yang’347 to include performing a first segmentation of the first digital pathology image based on one or more regions of the first digital pathology image corresponding to tumor bed method taught by Tong’122. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to have a singular step of plurality of segmentations and classifying areas (paragraphs [0034] and [0050]).
Yang’347 and Tong’122 do not show wherein the first segmentation using a first machine-learning model, and a second segmentation using a second machine-learning model different from the first machine-learning model.
Kim’721 shows first segmentation using a first machine-learning model, and a second segmentation using a second machine-learning model different from the first machine-learning model (Figure 17 shows segmenting a target medical image by the medical image analysis device 2000, medical image analysis device 2000 may input the target medical image into an input layer of the artificial neural network model and obtain first segmentation information output through an output layer on the basis of the trained artificial neural network model including the first parameter set included in at least one node of the feature layer and medical image analysis device 2000 may input the target medical image into the input layer of the artificial neural network model and obtain second segmentation information output through the output layer on the basis of the trained artificial neural network model including the second parameter set included in at least one node of the feature layer using at least two trained neural network models in parallel to segment the target medical image on the basis of a combination of various target feature information in paragraphs [0453]-[0468]).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Yang’347 and Tong’122 to include first segmentation using a first machine-learning model, and a second segmentation using a second machine-learning model different from the first machine-learning model method taught by Kim’721. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to improve accuracy of the data and provide a user with accurate treatment assistance information while reflecting the feature information in relation to an elaborate tumor treatment (paragraph [0507]).
With respect to Claim 2, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 1, wherein the first machine-learning model (in Kim’721: figure 17) is trained to segment tumor bed from non-tumor bed (in Tong’122: paragraph [0020] segmentation task of areas of interest cell types or biomarkers may include, e.g., necrotic tissue cells, tumor bed, stromal tissue, immune cells, immune inflamed tumors, among others or any combination thereof in paragraph [0033], paragraph [0050] ).
With respect to Claim 3, Yang’347, Tong’122 and Kim’721 shows the
one or more computer-readable non-transitory storage media of claim 1, wherein the second machine-learning model (in Kim’721: figure 17) is trained to segment the one or more predetermined histologic features from tumor bed (in Yang’347: paragraph [0040] steps 11-12 regard using the segmented image for training a neural network, figure 15 and paragraph [0138] training of the neural network (DCNN 1504) to output the segmented image 1506 with assessment 1508 for, paragraph [0142] image analysis including diagnostic indicator 1516 and in Tong’122: paragraph [0020] segmentation task of areas of interest and the classification task in a single step using a generative adversarial network (or GAN) using masks covering each region of interest. The invention similarly employs a GAN to perform both segmentation and classification by using a separate image band for each class).
With respect to Claim 4, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 3, further comprising instructions executable by one or more processors of the digital pathology image processing system for: receiving, by the digital pathology image processing system, feedback from a user operator regarding the assessment; and training the second machine-learning model (in Kim’721: figure 17) to segment the one or more predetermined histologic features from tumor bed based on the feedback (in Yang’347: paragraph [0108] a human annotator can annotate training images in a training dataset to identify and/or enumerate one or more portions of interest. For example, an annotator may identify and/or enumerate viable tumor cells or collections of viable tumor cells. The annotation may serve as the gold standard, or ground truth, for training a model).
With respect to Claim 5, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 1, wherein the one or more predetermined histologic features comprise necrotic tumor cells, regions of necrosis, viable tumor cells (in Yang’347: tumor cells paragraph [0083]), regions of viable tumor (in Yang’347: paragraph [0040] step 8 enumerate/assess abnormalities (tumor cells paragraph [0083]) in the sample from raw and pathology images, paragraph [0190] describes identifying spatial/physical relationships between normal and abnormal tissue), tumor stroma cells, or regions of tumor stroma.
With respect to Claim 6, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 1, wherein generating the assessment regarding the specified condition comprises determining whether the specified condition is present (in Yang’347: figure 15 1512 paragraph [0140]).
With respect to Claim 7, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 1, further comprising instructions executable by one or more processors of the digital pathology image processing system for: computing a first area value of the first histologic sample corresponding to tumor bed based on the one or more regions of the first digital pathology image corresponding to tumor bed and the physical characteristics of the first histologic sample; and computing a second area value of the first histologic sample corresponding to each of the one or more predetermined histologic features based on the one or more regions of the first digital pathology image corresponding to the one or more predetermined histologic features and the physical characteristics of the first histologic sample; wherein the assessment regarding the specified condition in the first histologic sample is generated based on the first area value and the second area value (in Yang’347: figure 15 paragraph [0138] segmented image 1506 comprises demarcated area 1508 wherein boundary lines are determined based on a probability of pixel associated with the boundary line exceeding a threshold compared to other pixels, this describes pixels of the first area/demarcated area 1508 exceed the threshold compared with pixels outside of the demarcated area 1508 falling below the threshold to identify the abnormality and in Kim’721: figure 17 each segmentation of the data can specify plurality of areas to be specific to the first target feature information).
With respect to Claim 8, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 7, wherein determining whether the specified condition is detected in the first histologic sample based on the first area value and the second area value comprises computing a percentage of the second area relative to the first area corresponding to each of the one or more predetermined histologic features (in Yang’347: paragraphs [0088] and [0141] describes generating a percentage coverage metric of the viable tumor).
With respect to Claim 9, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 8, wherein determining whether the specified condition is detected in the first histologic sample based on the first area value and the second area value comprises determining whether the percentage of the second area value relative to the first area value satisfies one or more predetermined thresholds, wherein the one or more predetermined thresholds are based on the specified condition (in Yang’347: paragraph [0138] segmented image 1506 comprises demarcated area 1508 wherein boundary lines are determined based on a probability of pixel associated with the boundary line exceeding a threshold compared to other pixels, paragraph [0141] describes the diagnostic indicator is achieved based on the percentage area coverage describing a relationship between the two).
With respect to Claim 10, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 1, wherein: performing, using the first machine-learning model (in Kim’721: figure 17), the first segmentation of the first digital pathology image based on the one or more regions of the first digital pathology image corresponding to tumor bed comprises producing a first instance of the first digital pathology image including annotations corresponding to the regions of the first digital pathology image corresponding to tumor bed (in Tong’122: Figure 1 and paragraph [0027] annotated tissue sample images 111 used for training a prediction model 110. Paragraph [0033] automatically segmenting and classifying areas including tumor bed detection. Paragraph [0034] the segmentation and classification is performed by leverage masks of image bands wherein each image band is a set of image bands representing a corresponding cell type which requires identifying the area via different image bands. Paragraph [0050] multiple masks for multiple image bands each for segmentation including tumor bed detection.); and performing, using the second machine-learning model (in Kim’721: figure 17), the second segmentation of the first digital pathology image based on the one or more regions of the first digital pathology image corresponding to the one or more predetermined histologic features comprises producing a second instance of the first digital pathology image including annotations corresponding to the regions of the first digital pathology image corresponding to the one or more predetermined histologic features (in Yang’347: paragraphs [0033] and [0104]-[0106] archived clinical histologic/cytologic slides from available databases may be used to take raw images, paragraph [0040] steps 11-12 regard using the segmented image for training a neural network, figure 15 and paragraph [0138] training of the neural network (DCNN 1504) to output the segmented image 1506 with assessment 1508).
With respect to Claim 11, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 1, wherein the first histologic sample is further associated with a set of one or more second digital pathology images (in Yang’347: paragraphs [0033] and [0104]-[0106] archived clinical histologic/cytologic slides from available databases may be used to take raw (digital) images; the interpretation here for the second digital pathology image regards the trained model analysis wherein the first digital pathology images regards training the model); wherein the one or more computer-readable non-transitory storage media further comprising comprise instructions executable by one or more processors of the digital pathology image processing system for, for each of the second digital pathology images: assessing physical characteristics of the first histologic sample associated with the second digital pathology image (in Yang’347: figure 15 image analysis staring with 1512 paragraph [0140] assess abnormalities (tumor cells paragraph [0083]) in the sample from raw and pathology images in application outside of training the model as disclosed in paragraph [0040], paragraph [0190] describes identifying spatial/physical relationships between normal and abnormal tissue); performing, using the first machine-learning model (in Kim’721: figure 17), a first segmentation of the second digital pathology image based on one or more regions of the second digital pathology image corresponding to tumor bed (in Tong’122: Figure 1 and paragraph [0027] annotated tissue sample images 111 used for training a prediction model 110. Paragraph [0033] automatically segmenting and classifying areas including tumor bed detection. Paragraph [0034] the segmentation and classification is performed by leverage masks of image bands wherein each image band is a set of image bands representing a corresponding cell type which requires identifying the area via different image bands. Paragraph [0050] multiple masks for multiple image bands each for segmentation including tumor bed detection.); and performing, using the second machine-learning model (in Kim’721: figure 17), a second segmentation of the second digital pathology image based on one or more regions of the second digital pathology image corresponding to the one or more predetermined histologic features (in Yang’347: paragraph [0040] interpreting image data from the plurality of raw images acquired during image acquisition process and step 7 generating a representative image from a pathology image (the reconstructed image from steps 2-4), paragraphs [0033] and [0104]-[0106] archived clinical histologic/cytologic slides from available databases may be used to take raw images); wherein generating the assessment regarding the specified condition in the first histologic sample is further based on the one or more regions of the set of second digital pathology images corresponding to tumor bed and the one or more regions of the set of second digital pathology images corresponding to the one or more predetermined histologic features (in Yang’347: paragraph [0040] steps 11-12 regard using the segmented image for training a neural network, figure 15 and paragraph [0138] training of the neural network (DCNN 1504) to output the segmented image 1506 with assessment 1508 for, paragraph [0142] image analysis including diagnostic indicator 1516).
With respect to Claim 12, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 1, further comprising instructions executable by one or more processors of the digital pathology image processing system for: receiving a human-generated assessment of the first histologic sample (in Yang’347: Figure 15 1510 paragraphs [0101], [0108], and [0139] wherein manual and/or automated annotations for ground truth image); comparing (in Yan’347: 1518) the assessment generated by the first digital pathology image processing system (In Yang’347: 1506) to the human-generated assessment; and generating a user interface comprising a display of the comparison (In Yang’347: paragraph [0142] the diagnostic indicator 1516 may be provided to a user on an interface).
With respect to Claim 14, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 1, further comprising instructions executable by one or more processors of the digital pathology image processing system for: generating a level of confidence in the assessment (In Yang’347: figure 15 1518 paragraph [0145]).
With respect to Claim 15, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 1, wherein the user interface comprising the display of the assessment further comprises a display of annotations for the first digital pathology image associated with the segmentations based on the one or more regions of the first digital pathology image corresponding to tumor bed and the one or more regions of the first digital pathology image corresponding to the one or more predetermined histologic features (In Yang’347: figure 1 displays 180 described in paragraph [0042] to display the data of the CRADL system 10 to a pathologist, paragraph [0040] step 6 to display the images on the displays).
With respect to Claims 16 and 19, rejection analogous to those presented for claim 1, are applicable.
With respect to Claims 17 and 20, rejection analogous to those presented for claim 2, are applicable.
With respect to Claim 18, rejection analogous to those presented for claim 3, are applicable.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (US 2022/0351347 A1) in view of Tong et al. (US 2021/0241122 A1), further in view of Kim et al. (US 2022/0207721 A1) and further in view of Peng et al. (US 2021/0019342 A1).
With respect to Claim 13, Yang’347, Tong’122 and Kim’721 shows the one or more computer-readable non-transitory storage media of claim 1, wherein the assessment regarding the specified condition is further generated based on [ ] and additional data associated with the first histologic sample (in Yang’347: paragraph [0144] describes successful identification of tumor cells is based on additional data of the sample including accurate delineation with boundaries and percentage area coverage metrics).
Peng’342 discloses the use of metadata (paragraph [0041]).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Yang’347, Tong’122 and Kim’721 to include the use of metadata method taught by Peng’342. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to enabling the model to learn the name of the feature (paragraph [0041]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Saltz et al. (US 2024/0177301 A1): shows first classification model and the second classification model to iteratively classify a first segmentation quality and a second segmentation quality of each patch of the plurality of patches based on the intensity and texture features of the plurality of patches as computed during classification and the patch-level intensity and texture features as computed during the training phase, the first segmentation quality and a second segmentation quality of each patch as classified predicting a segmentation quality of the segmented objects in each patch of the plurality of patches in the test image.
Chen et al. (US 2021/0304896 A1): figure 500 and paragraphs [0007] and [0106] generate at least one first segmentation image and at least one second segmentation image based on the target image, the system may generate the at least one first segmentation image by processing the target image using a first segmentation model for segmenting the at least one target region. The system may further generate the at least one second segmentation image by processing the at least one first segmentation image and the target image using a second segmentation model for segmenting the at least one lesion region.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IRIANA CRUZ whose telephone number is (571)270-3246. The examiner can normally be reached 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi M. Sarpong can be reached at (571) 270-3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IRIANA CRUZ/Primary Examiner, Art Unit 2681
