DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claim 21-10 are rejected under the judicially created doctrine of obviousness-type double patenting as being unpatentable over claim 1 of U.S. Patent No. 11,176,676, US 11,640,719, and US Patent 11,423,547. The conflicting claims are not identical because patent claim 1 requires the additional elements "the
machine learning prediction model having been generated by processing a plurality of training images
by receiving a plurality of synoptic annotations comprising one or more labels for each of the plurality of
training images", not required by claim 1 of the instant application. However, the conflicting claims are not patentably distinct from each other because:
• Claims 21 of application '080 and claim 1 of patent '676; of patent '547; of patent '719 recite common subject matter;
• Whereby claim 21, which recites the open ended transitional phrase “comprising”, does not preclude the additional elements recited by claim 1 of the patent, and
• Whereby the elements of claim 1 are fully anticipated by patent claim 1, and anticipation is “the ultimate or epitome of obviousness” (In re Kalm, 154 USPQ 10 (CCPA 1967), also In re Dailey, 178 USPQ 293 (CCPA 1973) and In re Pearson, 181 USPQ 641 (CCPA 1974)).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 21-40 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al (US 2016/0232425) and Jaber et al (WO 2019/018695)
As to claim 21, Huang et al teaches the computer-implemented method for processing electronic slide images, the method comprising:
receiving one or more electronic slide images associated with a tissue specimen ( Figure 2 Huang step 205 paragraph [0086] " process 200 begins in step 205 by capturing an original image from a tissue sample using imaging device 110);
generating a machine learning prediction model (figure 2 steps 210 and 215 and paragraph [0086, 0088-0099] step 210
the processor 122 performs texture analysis and pattern recognition analysis of the received tissue
image by executing the control logic steps of the software into the computer system 120. In step 215,
the processor 122 classifies the types of tissues found in the original tissue image wherein the tissue
classification is performed by a neural network trained on sample tissue images) by:
partitioning one of a plurality of training images into a plurality of training tiles for the plurality of training images ( the original digital tissue image 401 is divided into a plurality of smaller tissue blocks or simply tissue blocks 402 (see, e.g. FIG. 4) having a size less than the complete original tissue image. Any
suitable block size may be used, paragraph [0092]);
and training the machine learning prediction model to infer at least one tile- level prediction using at least one label of a plurality of annotations of the plurality of training images (figure 2 steps 210 and 215 and paragraph [0086, 0088-0099] step 210
the processor 122 performs texture analysis and pattern recognition analysis of the received tissue
image by executing the control logic steps of the software into the computer system 120. In step 215,
the processor 122 classifies the types of tissues found in the original tissue image wherein the tissue
classification is performed by a neural network trained on sample tissue images);
and outputting a prediction generated by the machine learning prediction model of at least one label of at least one electronic slide image of the one or more electronic slide images ((step
220 the processor 122 generates a heat map, which visually indicates or displays the tissue types of
interest or medically suspect high-risk tissues. In step 225, the processor 122 overlays the heat map onto
the original tissue image to form a composite image that is digitally enhanced in a manner, which
visually highlights the tissue types of interest or medical high-risk tissues"
While Huang teaches the limitation above. Huang fails to teach " removing at least one of the plurality of training tiles detected to be non- tissue”.
However, Jaber et al teaches system 200, a fixed-size scaled patch-based approach allows
analysis of regions as well as capturing micro- and macroscopic characteristics of a SI simultaneously. In
an embodiment, Sis (e.g., breast invasive carcinoma (BRCA) diagnostic whole-slide images of formalin -
fixed paraffin-embedded (FFPE) blocks with associated PAM50 labels obtained from TCGA data sources)
may be segmented or tiled into 1600 X 1600-pixel patches 202 at the 20x zoom level. The 1600 X 1600-
pixel patches 202 may be filtered for a minimum color variance to eliminate empty (i.e., background)
patches from further processing. Further, each 1600 X 1600-pixel patch 202 may be converted into 400x
400-pixel patch representations 204 at, for example, one or more of 5x, IOx, 20x, and 40x magnification
scales centered on a same location or point by down-sampling and cropping to the center 400 X 400-
pixels (paragraph [0047-0048]). Additionally, training engine 310 may segment each of the training Sis 1
to N 302, 304, 306 into a plurality of scaled patches 204, where each scaled patch of the plurality of
scaled patches 204 comprises one or more patch representations at one or more zoom levels that are
centered at a location within a corresponding training SI. Training engine 310 may then convert each
scaled patch of the plurality of scaled patches 204 into a multiscale descriptor using a deep-learning
neural network 206 (e.g., one of an Inception-v3, resnet34, resnetl52, densenetl69, densenet201 or
other deep-learning convolutional neural network) by, for each scaled patch, mapping each of the one
or more patch representations to a patch-level descriptor 208 and combining the patch- level
descriptors to generate a multiscale descriptor 210. For example, the patch- level descriptors may be
one or more of concatenated, averaged, stacked, or mathematically or empirically mixed or manipulated
to generate a multiscale descriptor 210. Training engine 310 may configure and train classifier model
214 to process the multiscale descriptors 210 such that, for each training SI 1 to N 302,304, 306 classifier model 214 is operable to assign a patch-level molecular subtype classification 216 to each of the
plurality of scaled patches corresponding to a training SI, and determine a Si-level molecular subtype
classification 218 or heterogeneous classification 220 based on the patch-level molecular subtype
classifications 216 ( paragraph[0051]). Therefore, it would have been obvious to one of ordinary skill in
the art before the effective filing date of the claimed invention to use the segmentation technique of the
pathology slide image using deep learning in order to enable the pathologist to select appropriate
visualization of annotated areas of interest on the slides.
As to claim 22, Jabber et al teaches the computer-implemented method of claim 21, wherein the plurality of training tiles that are determined to be non-tissue are further determined to be a background of the tissue specimen (The subset of the plurality of scaled patches may be selected to summarize tumor content within a training SI. plurality of scaled patches may be filtered for a minimum color variance, and each scaled patch determined to be empty space or background may be eliminated from further processing based on the filtering, paragraph [0013-0014]).
As to claim 23, Jabber et al teaches the computer-implemented method of claim 21, further comprising: creating a training tissue mask by detecting at least one tissue region from a background of the one or more electronic slide images; and detecting a plurality of tissue regions of the one or more electronic slide images and/or plurality of tiles by segmenting the at least one tissue region from the background((The 1600 X 1600- pixel patches 202 may be filtered for a minimum color variance to eliminate empty (i.e., background) patches from further processing. Further, each 1600 X 1600-pixel patch 202 may be converted into 400 X 400- pixel patch representations 204 at, for example, one or more of 5x, IOx, 20x, and 40x magnification scales centered on a same location or point by down-sampling and cropping to the center 400 X 400- pixels; at least one of an Inception-v3, resnet34, resnetl52, densenetl69, densenet201 or other deep-learning convolutional neural network may be used to map each 400 X 400-pixel color patch 204 to patch-level descriptors (i.e., descriptive vectors) 208 at each zoom level, paragraph [0047-0048]).
As to claim 24, Jabber et al teaches the computer-implemented method of claim 23, wherein the segmenting comprises using thresholding based on color, color intensity, and/or texture features(The 1600 X 1600- pixel patches 202 may be filtered for a minimum color variance to eliminate empty (i.e., background) patches from further processing. Further, each 1600 X 1600-pixel patch 202 may be converted into 400 X 400- pixel patch representations 204 at, for example, one or more of 5x, IOx, 20x, and 40x magnification scales centered on a same location or point by down-sampling and cropping to the center 400 X 400- pixels; at least one of an Inception-v3, resnet34, resnetl52, densenetl69, densenet201 or other deep-learning convolutional neural network may be used to map each 400 X 400-pixel color patch 204 to patch-level descriptors (i.e., descriptive vectors) 208 at each zoom level, paragraph [0047-0048])..
As to claim 25, Jabber et al teaches the computer-implemented method of claim 21, wherein the plurality of training images comprise a plurality of electronic slide images and a plurality of target labels (In an embodiment, Sis (e.g., breast invasive carcinoma (BRCA) diagnostic whole-slide images of formalin -fixed paraffin-embedded (FFPE) blocks with associated PAM50 labels obtained from TCGA data sources) may be segmented or tiled into 1600 X 1600-pixel patches 202 at the 20x zoom level. The 1600 1600- pixel patches 202 may be filtered for a minimum color variance to eliminate empty (i.e., background) patches from further processing, paragraph [0047])
As to claim 26, Huang et al teaches the computer-implemented method of claim 21, wherein using the machine learning prediction model under weak supervision comprises using at least one of multiple-instance learning (MIL), Multiple Instance Multiple Label Learning (MIMLL), self-supervised learning, and unsupervised clustering (Fig. 2, steps 210 & 215, paragraph [0086, 0088-0099]; "In step 210, the processor 122 performs texture analysis and pattern recognition analysis of the received tissue image by executing the control logic steps of the program software or instructions programmed into the computer system 120. Instep 215, the processor 122 classifies the types of tissues found in the original tissue image).
As to claim 27, Huang et al teaches the computer-implemented method of claim 21, wherein using the machine learning prediction model under weak supervision comprises using at least one of Multiple Instance Multiple Label Learning (MIMLL), self-supervised learning, and unsupervised clustering ( Fig. 2, steps 210 & 215, paragraph [0086, 0088-0099]; In step 210, the processor 122 performs texture analysis and pattern recognition analysis of the received tissue image by executing the control logic steps of the program software or instructions programmed into the computer system 120. In step 215, the processor 122 classifies the types of tissues found in the original tissue image." The tissue classification is performed by a neural network trained on sample tissue images.).
As to claim 28, Huang et al teaches the computer-implemented method of claim 21, further comprising: receiving a plurality of predictions of at least one feature from a weakly- supervised tile-level learning module for the plurality of training tiles; applying the machine learning prediction model to take, as an input, the plurality of predictions of the at least one feature from the weakly-supervised tile-level learning module for the plurality of training tiles; and predicting a plurality of labels for a slide or a patient specimen, using the plurality of training tiles( Fig. 2, steps 210 & 215, paragraph [0086, 0088-0099]; In step 210, the processor 122 performs texture analysis and pattern recognition analysis of the received tissue image by executing the control logic steps of the program software or instructions programmed into the computer system 120. In step 215, the processor 122 classifies the types of tissues found in the original tissue image." The tissue classification is performed by a neural network trained on sample tissue images.).
As to claim 29, Jabber et al teaches the computer-implemented method of claim 28, wherein at least one of the plurality of labels is binary, categorical, ordinal or real-valued (table 4).
As to claim 30, Huang et al teaches the computer-implemented method of claim 28, wherein applying the machine learning prediction model to take, as the input, the plurality of predictions of the at least one feature from the weakly-supervised tile-level learning module for the plurality of training tiles comprises a plurality of image features(paragraph [0143]; [0144]).
As to claim 31, Jabber et al teaches the computer-implemented method of claim 21, wherein the machine learning prediction model predicts at least one label using at least one unseen slide(paragraph p0047][0049]).
The limitation of claims 32-40 has been addressed above.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NANCY BITAR whose telephone number is (571)270-1041. The examiner can normally be reached Mon-Friday from 8:00 am to 5:00 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mrs. Jennifer Mehmood can be reached at 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
NANCY . BITAR
Examiner
Art Unit 2664
/NANCY BITAR/Primary Examiner, Art Unit 2664
Prosecution Insights