DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application is being examined under the pre-AIA first to invent provisions.
Information Disclosure Statement
The IDS(s) submitted on June 22, 2026, has/have been considered and placed in the application file.
Status of Claims
This action is in response to the amendment filed August 11, 2026, which amends claims 21, 22, 25, 26, 31, 33-36 and 38-40. Claims 21-40 are pending and under examination in this office action.
Response to Arguments
This final office action is in response to the amendment filed August 11, 2026. Claims 21-40 are pending in this application and have been considered below.
The rejection under 35 U.S.C. 101 is withdrawn. Amended claims 21, 34 and 38 recite the tile-resolution FCN formed by converting a CNN, classification without the later-layer upsampling of a pixel-resolution FCN, and single-pass classification of a large square tile. The specification ties that architecture to a computational improvement: “the computations for each pixel are only performed once” (SPEC ¶ 309), and upsampling “can be avoided with the present architecture” (SPEC ¶ 312). The claims integrate the recited operations into a practical application under MPEP 2106.04(d)(1) and 2106.05(a).
Applicant argues that Tunstall has no neural network, no FCN and no single-pass processing, and that the stated motivation does not reach the new limitations. Those limitations were added by amendment. The new ground relies on Sermanet for the convolutional network, its conversion, and its single-pass evaluation.
Applicant argues that Tunstall’s boundary comes from an edge detection algorithm, so it is not based on the tumor tissue class. The edge detection runs on the three-state map, and that map is built from the tumor classification “combine 1012 the tiles to provide a two state map (e.g. binary) identifying tumour, and non-tumour regions of the tissue” (Tunstall ¶ 40). The claim requires only that the boundary be “based at least in part on the tumor tissue class.”
Applicant’s arguments with respect to claims 21-40 have been considered but are moot in view of new ground(s) of rejection because of the amendments.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 21, 34, and 38 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Each of independent claims 21, 34, and 38 recites “without the upsampling performed in later layers of a pixel-resolution FCN.” There is insufficient antecedent basis for “the upsampling” in the claims, and no pixel-resolution FCN is otherwise recited, so it cannot be determined which upsampling operation the limitation excludes (MPEP 2173.05(e)). Claims 22-33, 35-37 and 39-40 are rejected by virtue of their dependence.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 21-40 rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1-20 of U.S. Patent No. 12,165,236 B2 (“the ‘236 patent,” issued 2024-12-10 from Application No. 17/829,228, titled “Predicting Total Nucleic Acid Yield and Dissection Boundaries for Histology Slides,” same inventive entity) in view of claims 1 and 10 of U.S. Patent No. 11,935,152 B2 (“the ‘152 patent”). Although the claims at issue are not identical, they are not patentably distinct from each other because Instant claim 27 (“when a predicted expected yield of nucleic acid fails to satisfy a target total nucleic acid yield: identifying a number of associated unstained slides that satisfies the target total nucleic acid yield; and accepting the number of associated unstained slides for next generation sequencing”) is verbatim claim 8 of the ‘236 patent.
Instant claims 23-26 (tumor shape, cell shape, and cell texture imaging features including RGB, grayscale, and gradient patterns) are substantially identical claims 2-5 of the ‘236 patent.
Instant claim 28 (target yield range 50- 2000 ng) is encompassed by '236 claim 7 (minimum threshold 50 ng).
Instant claims 31-32 (applying tile images to a trained cell segmentation model; identifying pixels as cell interior, cell border, or cell exterior) are encompassed by '236 claim 1 (pixel-resolution three-dimensional cell-segmentation model classifying cell interior, cell border, and cell exterior).
Instant independent claims 21 , 34, and 38 (“receive digital image; classify tissue with a tile-resolution FCN; generate a tumor area mask defining the dissection boundary; detect excess tissue outside the dissection boundary; label slide as having excess tissue”) differ from '236 claim 1 (predict NA yield from tumor cells within dissection boundary) only in cosmetic relabeling and in the tile-resolution FCN: “excess tissue” is defined as tissue outside the dissection boundary defined by the '236 tumor area mask, and tile-resolution FCN tissue classification is claimed in the '152 patent, claim 1 (“predicting a respective tissue classification for each tile image using one or more trained deep learning classifier models”) and claim 10 (“at least one of the trained deep learning classifier models is a tile-resolution Fully Convolutional Network (FCN) classification model”).
The ‘152 claim term is read in light of the ‘152 specification, which describes the same architecture as the instant disclosure (added and replacement layers that “convert a CNN to a tile-resolution FCN without requiring the upsampling performed in the later layers of traditional pixel-resolution FCNs”): the reference specification is used only to construe the reference claim term. The single-pass evaluation of a large square tile is shown by Sermanet as applied below and is an obvious implementation of a tile-resolution FCN (MPEP 804, subsection II.B.2). Substituting the ‘152 tile-resolution FCN tissue classifier for the tumor-identification step of the ‘236 claims, both operating on the same H&E slide image, would have been an obvious substitution of one known classifier for another.
This is a routine and obvious application of the predictive yield analysis and the tile classification already patented by the same inventive entity.
Claims 21-40 rejected on the ground of nonstatutory double patenting as being unpatentable over the claims of the following commonly-owned issued patents, patents, the ‘661, ‘236, ‘239 and ‘240 claims each taken in view of claims 1 and 10 of the ‘152 patent for the tile-resolution FCN limitation as set out above.
Over US 11,348,661 B2 (Yip et al. – issued from 17/139,765, parent application).
Instant 18/936,982 – Claim 21
'661 – Claim 1 (verbatim, in part)
Why not patentably distinct
receiving, via one or more
processors, a digital image of an histology slide
"generating a digital image of each of the one or more H&E slides prepared from the specimen at an image-based nucleic acid yield prediction system having one or more processors"
Identical step. An H&E slide is an histology slide.
detecting, via one or more
processors, an excess tissue on the slide
"identifying ... tumor cells of the H&E slide from the digital image; generating a tumor area mask based at least in part on the identified tumor cells, wherein the tumor area
mask defines a dissection boundary"
Under BRI of "excess tissue"(§ 112(b) above), the '661 claim's identification of non-tumor regions outside the dissection boundary
reads on detecting "excess tissue."
labeling, via one or more
processors, the slide as having excess tissue
(implicit: the tumor area mask classifies tumor vs. non-tumor regions on the slide)
Labeling a slide based on the same image classification is an obvious naming convention.
generating, via one or more
processors, a notification report indicating the excess tissue
"providing one or more of the associated unstained slides for nucleic acid analysis when the summation of expected yield of nucleic acid predicted for each digital image exceeds a predetermined threshold"
A report/notification indicating slide tissue status is the obvious automation of the disclosed unstained-slide-provisioning trigger.
The '661 claim discloses the same automated decision pipeline; the present claim merely recites the report output.
Dependent instant claims 22- 33 are likewise not patentably distinct from corresponding limitations of '661 claims 2-29, including: imaging features (cf. '661 cl. 11-12), 50-2000 ng
range ( cf. '661 cl. 10), surface area / cell count specifications (cf '661 cl. 6-8), microdissection/macrodissection/whole-slide boundaries (cf. ' 661 cl. 13-15), tile-based pixel classification (cf. '661 cl. 17), cell interior border exterior pixel classification (cf. '661 cl. 18), pixel-resolution UNet (cf '661 cl. 19), and H&E annotation training (cf. '661 cl. 21). Independent instant claims 34 (system) and 38 (CRM) and their dependents are likewise not patentably distinct from '661 cl. 24-25 (system form) and ‘152 cl. 18 (computer-readable medium form).
Over US 12,165,236 B2 (Yip et al. – issued from 17/829,228, immediate parent).
Instant - Claim 21
'236 - Claim 1
Why not patentably distinct
receiving ... a digital image of
an histology slide
"receiving a digital image of the histology slide at an image-based nucleic acid yield prediction system
having one or more processors"
Identical.
detecting ... an excess tissue on the slide
"identifying ... tumor cells of the histology slide from the digital image using a trained cell segmentation model ... defining a dissection boundary of the histology
slide"
Same as above.
labeling ... the slide as having
excess tissue
(implicit in dissection boundary classification)
Obvious automation.
generating ... a notification
report indicating the excess
tissue
'236 cl. 13-14: generating tumor area mask + viewer overlay indicating tumor cells to scrape
Notification report is an obvious form of the disclosed
overlay/indication.
Instant cl. 23-26 (imaging features: tumor shape, cell shape, cell texture) are anticipated by '236
cl. 2-5 (substantially identical). Instant cl. 27-30 (yield-fail trigger, 50-2000 ng range, FFPE, scrap flag) map
to '236 cl. 8-11. Instant cl. 31-32 (tile-based pixel classification; cell interior/border/exterior)
map to '236 cl. 17-18. Instant cl. 34 (system) and 38 (CRM) forms map to '236 cl. 19- 20
(computing device / computer system).
Over US 11,348,239 B2 (Yip et al. – issued from 17/139,784).
Instant – Claim 21
'239 – Claim 1
Why not patentably distinct
receiving ... a digital image of an histology slide
"receiving a first hematoxylin and eosin (H&E) slide prepared from a tumor block at an image-based nucleic acid yield prediction system
having one or more processors"
Identical (H&E slide is histology slide).
detecting ... an excess tissue on the slide
"generating a tumor area mask … wherein the tumor area mask defines a dissection boundary"
Same as above.
labeling / notification report
"based on the predicted expected yield of nucleic acid, identifying a number of H&E slides to prepare from the tumor block to satisfy a
total nucleic yield"
Identifying the number of additional slides is a notification regarding
slide tissue sufficiency.
Instant cl. 27 (yield-fail trigger I unstained slides for NGS) maps directly to '239 cl. 1’s
“identifying a number of H&E slides to prepare ... to satisfy a total nucleic yield.” Instant cl. 28 (50-2000 ng) mpas to ‘239 cl. 7.
Over US 11,348,240 B2 (Yip et al. – issued from 17/139,798).
Instant – Claim 21
'240 – Claim 1
Why not patentably distinct
receiving .. . a digital image of
an histology slide
"receiving a digital image of the H&E slide at an image-based nucleic acid yield prediction system having one or more processors"
Identical.
detecting ... an excess tissue on the slide
" identifying ... tumor cells of the H&E slide from the digital image using a trained cell segmentation model; generating a tumor area
mask ... wherein the tumor area mask defines the dissection boundary"
Same as above.
labeling / notification report
"predicting the expected yield of slide nucleic acid ... by providing the digital image of the H&E slide to a model trained on a plurality of H&E
slides having dissection labels and total nucleic yield labels"
The trained model output is a slide-level label/notification regarding tissue sufficiency.
Instant cl. 27 (yield-fail trigger) maps to '240 cl. 4. Instant cl. 28 (50- 2000 ng) maps to '240 cl. 5.
Over US 11,935,152 B2 (Yip et al. – sibling biomarker patent).
Instant- Claim 34 (system)
'152 - Claim 1 (computing system)
Why not patentably distinct
one or more processors; one or more memories
"one or more processors; an
electronic network; and one or more memories having stored thereon computer-executable instructions"
Identical structural recitation.
receive … a digital image of an histology slide
"process a plurality of segmented tile images each corresponding to a different respective portion of the
digital image"
Same H&E slide processing
pipeline.
detect / label / notification report
"predicting a respective biomarker classification for each tile image … determining a predicted presence of one or more biomarkers; transmit …
the predicted presence of the one or more biomarkers"
Detection-label-notification reads on biomarker-classification-prediction transmit.
Instant cl. 31- 32 (tile-based pixel classification / cell interior-border-exterior) map directly to '152 cl. 2-4 (tile separation, pixel-based cell segmentation, trained cell segmentation model). The tile-resolution FCN tissue classification of instant claims 21, 34 and 38 maps to ‘152 cl. 1 and 10; the ‘152 patent claims no dissection boundary, so it is applied in combination with the ‘661, ‘236, ‘239 and ‘240 claims above.
A terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to obviate the rejection. The USPTO Internet website contains terminal disclaimer forms which may be used.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 21, 22, 27-30, 34, 35, 38, and 39 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tunstall et al., US 2016/0063724 A1 (“Tunstall”) in view of Sermanet et al., “OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks,” (“Sermanet”).
Claim 21.
Tunstall discloses a computer-implemented method, comprising: receiving, via one or more processors, a digital image of an histology slide (Tunstall ¶ 28-29: the processor 22 obtains 1000 from memory 24 an image of a section of a tissue sample imaged from a microscope slide stained using Hematoxylin and Eosin; Fig. 3, 1000);
classifying, via the one or more processors, tissue depicted in the digital image into a plurality
of tissue classes including a tumor tissue class (Tunstall ¶ 39: “the processor compares 1008 the concatenated feature vector from the tile with the selected subset of the classification data” and classifies the tile as tumour or non-tumour; ¶ 40: “a spatial map of the image data which classifies regions of the image into one of three states e.g. background, tumour tissue, and non-tumour tissue" – each tile is assigned one of a plurality of tissue classes, one of which is tumour tissue) using a tile-resolution fully convolutional network (FCN), the tile-resolution FCN being formed by converting a convolutional neural network with added and replacement layers that classify each tile image of a plurality of tile images formed from the digital image without the upsampling performed in later layers of a pixel-resolution FCN, wherein the classifying is performed on a large square tile that comprises multiple of the plurality of tile images, the large square tile being applied to the tile-resolution FCN in a single pass to assign a respective tissue class to each of the tile images in the large square tile such that computations for each pixel of the large square tile are performed once instead of once per tile image (Tunstall ¶ 64: “the classification data may be derived from a supervised learning model … comprises a feature vector … and an indication of whether that feature vector relates to tumour or non-tumour image data.” Tunstall classifies each tile by comparing its feature vector with stored classification data and does not use a convolutional network; see Sermanet below);
generating, via the one or more processors, a tumor area mask that defines a dissection
boundary of the histology slide based at least in part on the tumor tissue class (Tunstall ¶ 40:
“combine 1012 the tiles to provide a two state map (e.g. binary) identifying tumour, and nontumour regions of the tissue”; ¶ 41: “identify 1014 a boundary between regions in the three state map”; ¶ 48: a template guides cutting “along a path corresponding to a boundary shown
on the template or to scrape tissue with a spatula from inside the boundary;” where, the binary tumour map is a tumor area mask, and the boundary identified from it and used to guide cutting is a dissection boundary derived from the tumour class);
detecting, via the one or more processors, an excess tissue on the histology slide, wherein the
excess tissue comprises tissue of the histology slide located outside the dissection boundary (Tunstall ¶40: combining 1012 the tiles to provide a two state map (e.g. binary) identifying tumour, and non-tumour regions of the tissue with the tissue/non-tissue mask generated by the segmentation 1002 to provide a spatial map of the image data which classifies regions of the image into one of three states e.g. background, tumour tissue, and non-tumour tissue; ¶43: the map is “overlaid with the smoothed boundary data, to provide a user with an estimate of the location of tumour and non-tumour regions in the image”; ¶ 49: “The non-tumour regions 8 may also contain tumour tissue, but in a lower concentration than present in their tumour regions 6”; where, the non-tumour regions located outside the tumour boundary are tissue located outside the dissection boundary); and
labeling, via the one or more processors, the histology slide as having excess tissue (Tunstall ¶41: identify 1014 a boundary between regions in the three state map, encode the resulting boundary; ¶43: a color map overlaid with the smoothed boundary data, to provide a user with an estimate of the location of tumour and non-tumour regions in the image – the boundary classification labels the slide regions accordingly; ¶ 167: “a display for
displaying the boundary as a template to guide dissection of the sample”).
Tunstall discloses all of the subject matter as described above except for specifically teaching the tile-resolution FCN: converting a convolutional neural network with added and replacement layers that classify each tile image without the upsampling performed in later layers of a pixel-resolution FCN, applied to a large square tile in a single pass so that computations for each pixel are performed once instead of once per tile image.
However, Sermanet, which is reasonably pertinent to the problem of classifying every tile of a large image with a convolutional network without redundant computation (MPEP 2141.01(a); the problem the specification identifies at ¶¶ 11-12), teaches this (Sermanet p. 8: “the last layers of our architecture are fully connected linear layers. At test time, these layers are effectively replaced by convolution operations with kernels of 1x1 spatial extent”; p. 5: “apply the last subsampling operation at every offset." The fully connected layers are replaced by 1x1 convolutions and an offset pooling stage is added; p. 7: “eventually producing a map of output class predictions, with one spatial location for each "window" (field of view) of input”; p. 5: “this architecture can only produce a classification vector every 36 pixels in the input dimension along each axis”; p. 8: “The entire ConvNet is then simply a sequence of convolutions, max-pooling and thresholding operations exclusively (emphasis added)” one class prediction per window of the input and no layer that upsamples the predictions to pixel resolution; p. 7: “ConvNets are inherently efficient when applied in a sliding fashion because they naturally share computations common to overlapping regions”; p. 7: “When applying our network to larger images at test time, we simply apply each convolution over the extent of the full image”; p. 8: “Convolutions are applied bottom-up, so that the computations common to neighboring windows need only be done once”; Fig. 5, p. 8: “Since all layers are applied convolutionally, the extra computation required for the larger image is limited to the yellow regions”; Table 5, p. 16: a square 245x245 input produces a 3x3xC map of class predictions, nine windows classified in one pass, larger square input containing many windows is processed in one pass and each pixel’s computation is shared by every window that contains it).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Tunstall’s tile classifier as Sermanet’s convolutional network applied over a square region of the sliding window image containing many tiles in one pass, because the classifier of Tunstall may be a learned model (Tunstall ¶ 64), and Sermanet's convolutional network would run over the input region in one pass because “ConvNets are inherently efficient when applied in a sliding fashion because they naturally share computations common to overlapping regions” (Sermanet p. 7); this yields one class per window. KSR lnt'I Co. v. Teleflex Inc., 550 U.S. 398 (2007).
Claim 30.
Tunstall in view of Sermanet teaches the computer-implemented method of claim 21, wherein associated unstained slides include tissue from a formalin-fixed paraffin embedded specimen (Tunstall ¶3 FFPE, ¶49).
Claim 34.
Tunstall in view of Sermanet teaches a computing system, comprising: one or more processors (Tunstall ¶29: a processor 22; Fig. 3); and one or more memories (Tunstall ¶29: memory 24; 1120), having stored thereon computer-executable instructions that, when executed, cause the computing system to:
receive, via the one or more processors, a digital image of an histology slide (see claim 21 mapping; Tunstall ¶28-29);
classify tissue depicted in the digital image into a plurality of tissue classes including a tumor
tissue class using a tile-resolution FCN formed as recited in claim 21 (see claim 21 mapping;
Tunstall ¶¶ 39-40; Sermanet pp. 5-8);
generate a tumor area mask that defines a dissection boundary of the histology slide based at
least in part on the tumor tissue class (see claim 21 mapping; Tunstall ¶¶ 40-41, 48);
detect an excess tissue on the histology slide, wherein the excess tissue comprises tissue of the
histology slide located outside the dissection boundary (see claim 21 mapping; Tunstall ¶¶ 40,
43, 49); and
label the histology slide as having excess tissue (see claim 21 mapping; Tunstall ¶¶ 41, 43, 167).
The rationale provided for the rejection of claim(s) 21 is applicable to claim 34, mutatis mutandis.
Claim 38.
Tunstall in view of Sermanet teaches a computer-readable medium having stored thereon computer-executable instructions (Tunstall ¶20: a computer with a processor; ¶52: a communications link such as a network) that, when executed, cause a computer to: [same mapping as claim 21].
Claims 22, 27-29, 35, and 39 are rejected under 35 U.S.C. 103 as being unpatentable
over Tunstall in view of Sermanet as applied above, and further in view of Bellevicine et al.,
“How to Prepare Cytological Samples for Molecular Testing,” (“Bellevicine”).
Claim 22.
Tunstall in view of Sermanet and Bellevicine teaches the computer-implemented method of claim 21, further comprising: collecting respective nucleic acid yield of a plurality of cells corresponding to the histology slide (Bellevicine p. 823, right column: "NUCLEIC ACID SAMPLE QUANTITY AND QUALITY ASSESSMENT ... Spectrophotometer analysis ... is largely used to quantify DNA or RNA. Between nucleic acid correlation and absorbance (A), there is a linear correlation able to predict the DNA or RNA quantity in the solution"), and wherein detecting the excess tissue on the histology slide includes detecting excess tissue of the plurality of cells (Tunstall, ¶40: spatial map classifies regions into background, tumour tissue, and non-tumour tissue; ¶42: obtaining a probability estimate based on comparing the feature vectors of tiles in tissue regions of the image with the selected subset of model image data).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Tunstall and Sermanet with Bellevicine. The motivation for this combination would have been to conserve tumor tissue and improve efficiency by avoiding studies done in a piecemeal fashion (Bellevicine p. 821, right column: “Cutting extra, unstained CB sections up front to avoid refacing block would be ideal to save as much tumour tissue as possible for molecular testing, avoiding that ancillary studies are performed in a piecemeal fashion”), and to measure the nucleic acid yield of the dissected cells because "The accurate analysis of input nucleic acid sample quantity, purity and integrity is crucial, especially on scant routine cytological samples" (Bellevicine p. 823). KSR lnt'I Co. v. Teleflex Inc., 550 U.S. 398 (2007).
Claim 27.
Tunstall in view of Sermanet and Bellevicine teaches the computer-implemented method of claim 21, wherein when a predicted expected yield of nucleic acid fails to satisfy a target total nucleic acid yield: identifying a number of associated unstained slides that satisfies the target total nucleic acid yield (Bellevicine p. 821, right column: "Cutting extra, unstained CB sections up front to avoid refacing block would be ideal to save as much tumour tissue as possible for molecular testing, avoiding that ancillary studies are performed in a piecemeal fashion"); and accepting the number of associated unstained slides for next-generation sequencing (Bellevicine p. 825, left column: "NGS and fully automated platforms may necessitate specific sample requirements").
Claim 28.
Tunstall in view of Sermanet and Bellevicine teaches the computer-implemented method of claim 27, wherein the target total nucleic acid yield is selected from a range between and including 50 ng - 2000 ng (Bellevicine p. 825, left column: DNA concentration below the manufacturer's recommended threshold of 10 ng; p. 824: “the yield of DNA is a critical
preanalytical factor that determines the success of molecular analysis”; p. 824: “As the input quantity is decreased, the total mutation burden may drop below the lower threshold of detection for the assay”; p. 825: “Illumina NGS usually requires more cells and/or higher DNA input than Ion Torrent NGS” – the nucleic acid input required by the assay is a recognized
result-effective variable that differs by platform, and selecting the target from the range the
platform in use requires is routine optimization, MPEP 2144.05(ll)(B)).
Claim 29.
Tunstall in view of Sermanet teaches the computer-implemented method of claim 21, wherein associated unstained slides are flagged for scraping (Tunstall ¶48: a boundary shown on the template or to scrape tissue with a spatula from inside the boundary; Bellevicine p. 821, right column: unstained sections lined up with corresponding tumour-mapped H&E-stained slide as a guide for macrodissection).
Claim 35. (New)
The computing system of claim 34 – same mapping as claim 22.
Claim 39. (New)
The computer-readable medium of claim 38 – same mapping as claim 22.
Claims 23-26, 33, 36, 37, and 40 are rejected under 35 U.S.C. 103 as being unpatentable over Tunstall in view of Sermanet and Bellevicine, and further in view of Coudray et al.. “Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning” (“Coudray”).
Claim 23.
Tunstall in view of Sermanet and Bellevicine teaches the computer-implemented method of claim 21, further comprising: processing imaging features including tumor shape features, cell shape features, and/or cell texture features (Tunstall ¶ 35: the properties determined for each
tile include “texture, statistical moments … morphological features” such as “average
nuclear size and/or shape, nuclear concentration in a spatial region” – nuclear size and shape
are cell shape features and texture is a cell texture feature; ¶40: “the tiles” are processed for “spatial map of the image data which classifies regions”; Coudray p. 1560: “the network was instead trained, validated and tested using 512x512 pixel tiles” processed via the inception v3 deep learning architecture i.e. image features extracted include tumor and tissue morphology).
It would have been obvious to combine Tunstall, Sermanet, and Bellevicine with Coudray. The motivation would have been to apply the routine deep-learning feature-extraction paradigm of Coudray to the tile-based tumor classification pipeline of Tunstall to extract richer morphological features for downstream analysis (Coudray Abstract: "we trained a deep convolutional neural network (inception v3) on whole-slide images obtained from The Cancer Genome Atlas to accurately and automatically classify them into LUAD, LUSC or normal lung tissue"). KSR exemplary rationale (A), combining prior art elements according to known methods to yield predictable results, MPEP 2141 (III).
Claim 24.
Tunstall, in view of Sermanet and Bellevicine, further in view of Coudray teaches the computer-implemented method of claim 23, wherein the imaging features include at least one of tumor shape features of tumor area, tumor perimeter, tumor circularity, tumor density, or number of tumors (Tunstall ¶4: a boundary of a tumour region of a tissue section on a histology slide; ¶40: spatial map identifying tumour regions; ¶43: estimate of the location of tumour and non-tumour regions – area, perimeter and density of tumour regions are known properties of the identified region (IPR2026-00185, EX1002,¶6 “automatically quantifying a wide array of morphometric features” (p. 5), ¶26-27 “classify tissues, detect cells, and identify biomarkers” + “multiscale approaches to feature extraction”); Coudray, p. 1560: tile-level prediction includes tumour-area metrics).
Claim 25.
Tunstall, in view of Sermanet and Bellevicine, further in view of Coudray teaches the computer-implemented method of claim 23, wherein the imaging features includes cell shape features of cell area, cell perimeter, cell circularity, and/or cell density (Tunstall ¶35: each cell nucleus as a vertex; ¶65: boundary pixels – cell-area and density are known properties (IPR2026-00185, EX1002,¶6 “automatically quantifying a wide array of morphometric features” (p. 5), ¶26-27 “classify tissues, detect cells, and identify biomarkers” + “multiscale approaches to feature extraction”); Coudray, p. 1560: per-tile probability features include cellular composition).
Claim 26.
Tunstall, in view of Sermanet and Bellevicine, further in view of Coudray teaches the computer-implemented method of claim 23, wherein the imaging features includes cell texture features of RGB texture patterns, grayscale texture patterns, gradient and/or features (Tunstall ¶64: classification data may be derived from a supervised learning model in which the classification data comprises a feature vector derived from an image of a tissue sample; Coudray p. 1560: inception v3 convolutional architecture extracts color and gradient features at multiple scales over 512 x 512 RGB tiles).
Claim 33.
Tunstall in view of Sermanet and Bellevicine teaches the computer-implemented method of claim 21, further comprising:
receiving, at an image-based nucleic acid yield prediction system having one or more processors, digital images of a plurality of hematoxylin and eosin (H&E) slides prepared from a tumor block (Tunstall ¶28-29: processor obtains image from memory; Fig. 3; Bellevicine p. 821: H&E sections from cell block tumour samples); identifying, via the one or more processors, tumor cells within each digital image of the digital images (Tunstall ¶ 40, ¶42: tumour vs. non-tumour classification of tile regions;);
predicting, for each digital image of the digital images, an expected yield of nucleic acid for the tumor cells (Bellevicine p. 823, right column: predicting DNA/RNA quantity from spectrophotometric analysis correlated with cellularity; Coudray, Abstract: we trained a deep convolutional neural network (inception v3) on whole-slide images to predict molecular outputs (mutations) from the image);
determining, based on the predicted expected yield of nucleic acid and a predetermined threshold, a quality control (QC) status for each H&E slide of the plurality of H&E slides (Tunstall ¶¶ 31, 56: predetermined threshold; Bellevicine p. 825, left column: NGS and fully automated platforms may necessitate specific sample requirements – sample requirements define QC status);
flagging a respective QC status in one or more of the H&E slides of the plurality of H&E slides when the predicted expected yield of nucleic acid or an amount of tumor tissue exceeds he predetermined threshold, indicating potential for more efficient use of the slide for diagnostic or research purposes (Tunstall ¶56: threshold used to provide the mask can also be predefined; ¶43: color map overlaid with the smoothed boundary data, to provide a user with an estimate of the location of tumour and non-tumour regions); and
generating a report indicating the QC status for each H&E slide of the plurality of H&E
slides, the report including recommendations for further processing based on the QC status (Tunstall ¶166-167: displaying the tumor boundary as a template to guide dissection; Bellevicine p. 821: tumour-mapped H&E-stained slide as a guide for macrodissection), wherein slides flagged for manual review are presented with overlays indicating the tumor area mask and a rationale for the manual review status (Tunstall ¶43: a color map overlaid with the smoothed boundary data, to provide a user with an estimate of the location of tumour and non-tumour regions in the image – the overlay constitutes the rationale).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Tunstall, Sermanet, Bellevicine, and Coudray before the effective filing date of the claimed invention. The motivation for this combination of references would have been to apply Coudray 's deep-learning architecture (a single trained CNN producing multiple correlated slide-level outputs from H&E images) (Coudray Methods: “we replaced the final softmax layer of the network with a sigmoid layer, to allow each sample to be associated with several binary labels") to Tunstall's tumour-region pipeline and Bellevicine 's nucleic-acid-yield workflow, in order to provide a single image-driven pipeline that simultaneously outputs both dissection-boundary classification and predicted nucleic acid yield. KSR rationale (A) (combining prior art elements according to known methods to yield predictable results) and (G) (some teaching, suggestion, or motivation in the prior art that would have led PHOSITA to combine references), MPEP 2141 (III). The combination is further motivated by the routine application of deep learning to histopathology images at the time of the invention.
Claim 36.
The computing system of claim 34 – same mapping as claim 23.
Claim 37.
The computing system of claim 36 – same mapping as claim 24.
Claim 40.
The computer-readable medium of claim 38 – same mapping as claim 23.
Claims 31-32 are rejected under 35 U.S.C. 103 as being unpatentable over Tunstall in view of Sermanet as applied above, and further in view Pan et al. ("Cell detection in pathology and microscopy images with multi-scale fully convolutional neural networks," Exhibit 1016, IPR2026-00185) (“Pan”).
Claim 31. (New)
Tunstall in view of Sermanet teaches the computer-implemented method of claim 21, further comprising: applying, via the one or more processors, a second plurality of tile images formed from the digital image to a (Tunstall ¶40: combining 1012 the tiles; ¶47: storing results of the image processing method for later user reference and/or for updating the classification data using a learning method; and, for each tile image of the second plurality of tile images, assigning a cell classification to one or more pixels within the tile image.
Tunstall in view of Sermanet does do not teach applying the tile images to a trained cell segmentation model and assigning a cell classification to pixels. However, Pan teaches a trained fully convolutional cell model applied to image patches (Pan p. 3: “Since the networks are fully convolutional, they can predict a density map for an input image with arbitrary size, and we exploit this to improve efficiency by end-to-end training on image patches.”) and, for each tile, assigning a cell classification to one or more pixels within the tile image (Pan p. 1: “regression on the proposed structured proximity space for patches is performed so that centers of image patches are explicitly forced to produce larger values than their adjacent areas”).
It would have been obvious to combine Tunstall and Sermanet with Pan. The motivation would have been to use a multi-scale FCN to provide pixel-level cell classification, as taught by Pan, in place of Tunstall's coarser tile-level region classification, to improve cell-level resolution of the segmentation (Pan p. 1: motivation is improved cell-level accuracy).
Claim 32.
Tunstall in view of Sermanet and Pan teaches the computer-implemented method of claim 31, further comprising: identifying, using the one or more processors, the one or more pixels as a cell interior, a cell border, or a cell exterior and classifying the one or more pixels as the cell interior, the cell border, or the cell exterior (Tunstall ¶35: image properties are selected from "morphological features, such as average nuclear size and/or shape, nuclear
concentration in a spatial region, and high level spatial relationships between image
objects, which may be derived from Delaunay Triangulation, Voronoi diagram and/or a
minimal expanding tree algorithm which treats each cell nucleus as a vertex" – Tunstall identifies cell nuclei (cell-interior centroids) as discrete spatial vertices; ¶65: "The boundary may be encoded in the form of a sequence of transition indicators indicating the direction of a transition between two consecutive boundary pixels" – Tunstall encodes boundary pixels distinguishing the interior of an identified region from its exterior; Pan § 3.1, p. 5: per-pixel proximity-map values M_ij characterize each pixel by its distance to the nearest cell center, such that a high value (D
≤
r) indicates a pixel near a cell center ( cell-interior region), an intermediate value indicates a pixel near the cell boundary, and a value of 0 indicates a pixel exterior to the cell).
The combination of Tunstall and Pan as applied to claim 31 above teaches treating each cell nucleus as a vertex (Tunstall ¶35), encoding boundary pixels (Tunstall ¶65), and assigning a
per-pixel value characterizing each pixel's relationship to the nearest cell center (Pan § 3.1).
The rationale provided for the rejection of claim(s) 31 is applicable to claim 32, mutatis mutandis.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication, earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on (313) 446-4912. The fax phone number for the organization where this application, proceeding is assigned is 571-273-8300.
Information regarding the status of published, unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA, CANADA), 571-272-1000.
/Ross Varndell/Primary Examiner, Art Unit 2674
Prosecution Insights