DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 8, 11, and 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tian (CN 110136809 A) in view of Schleifer (PGPUB: 20190368982 A1), and further in view of Ozcan (PGPUB: 20230030424 A1).
Regarding claims 1, 11, and 17. A system for using a machine learning model to select a formalin fixed paraffin embedded (FFPE) tissue block in a genomic assay, the system comprising:
at least one memory storing instructions; and at least one processor configured to execute the instructions to perform operations (see Fig. 10, page 46, lines 22-23, the processor 1004 can be used in call memory 1008 storing a computer program) comprising:
receiving a collection of digital images, at a digital storage device (see page 19, lines 19-23, to the user terminal 10a as an example, when the user terminal 10a receives the medical image comprises a lesion, the said medical image 10d to the transmitting server 10f through the switch 10e and a communication bus. server 10f can identify the category and focus belongs to the quadrant position information in the medical image, and according to the identification result to generate medical service data);
applying a machine learning model to the collection, the machine learning model determining whether the collection contains a presence of an attribute of a tissue beyond a predetermined threshold (see page 23, lines 26-32, based on the classifier in the focus detection model, to identify each image block belonging to the focus detection model corresponding to focus probability of the focus attribute, if the focus probability image blocks identified probability greater than the focus threshold, indicates that the image block contained in the focus object, and determining the focus property of the focus object is focus attribute corresponding to the focus detecting model, candidate region so that the image block of the position region in the biological tissue image as a focus object; see page 21, line 3-6, the user terminal 10a obtains the image semantic segmentation model, image semantic segmentation model can be tissue classes belonging to the each pixel point in the recognition image, tissue classes comprises: nipple type, muscle class and a background class);
determining a location of the tissue (see Fig. 2, page 23, lines 9-17, if there are 2 focus detection model, focus attribute corresponding to focus detection model A is lump, focus attribute corresponding to focus detection model B of calcification. the biological tissue image input focus detection model A, focus detection model A output location area if, illustrate focus object is located in the location area, and the focus property of the focus object area after the position is lump, if focus detection model A does not output position area, the lump is not the focus property of the focus object. the focus detection model B can adopt the same method, determines whether the focus object focus attribute is the calcification, if focus attribute is calcification, may also determine the focus position of the object region in a biological tissue image).
However, Tian does not expressly teach an adequate presence of the tissue exists within the FFPE tissue block.
Schleifer teaches that a FFPE human tissue block was obtained. The FFPE block was analyzed to determine if fluorescence of paraffin can be used to differentiate tissue in the FFPE block from paraffin. In particular, the experiment was conducted to determine if an image could be obtained of paraffin fluorescence in the presence of the endogenous fluorophores in tissue. Flourescence intensity of tissue in a FFPE sample was measured using a 365 nm LED excitation source with an emission filter centered at 560 nm (55 nm bandpass). Flourescence intensity of the paraffin surrounding the tissue in an FFPE sample was measured with a 280 nm LED excitation source and an emission filter centered at 405 nm (20 nm wide bandpass) (see paragraph 89).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tian by Schleifer to obtain a FFPE human tissue block was obtained. The FFPE block was analyzed to determine if fluorescence of paraffin can be used to differentiate tissue in the FFPE block from paraffin. In particular, the experiment was conducted to determine if an image could be obtained of paraffin fluorescence in the presence of the endogenous fluorophores in tissue, in order to provide an adequate presence of the tissue exists within the FFPE tissue block. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
However, the combination does not expressly teach outputting a confirmation indicating tissue.
Ozcan teaches that deep neural network 10 in response to the input image 20 outputs or generates a digitally stained or labelled output image 40. The digitally stained output image 40 has “staining” that has been digitally integrated into the stained output image 40 using the trained, deep neural network 10. In some embodiments, such as those involved tissue sections, the trained, deep neural network 10 appears to a skilled observer (e.g., a trained histopathologist) to be substantially equivalent to a corresponding brightfield image of the same tissue section sample 22 that has been chemically stained (see Fig. 1, paragraph 48); it was also confirmed by a pathologist that the neural network output images FIGS. 4C and 5G correctly reveal the histological features corresponding to hepatocytes, sinusoidal spaces, collagen and fat droplets (FIG. 5G), consistent with the way that they appear in the brightfield images 48 of the same tissue samples 22, captured after the chemical staining (FIGS. 5D and 5H). Similarly, it was also confirmed by the same expert that the deep neural network output images 40 reported in FIGS. 5K and 5O (lung) reveal consistently stained histological features corresponding to vessels, collagen and alveolar spaces as they appear in the brightfield images 48 of the same tissue sample 22 imaged after the chemical staining (FIGS. 6L and 6P) (see Fig. 3-5, paragraph 78).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination by Ozcan to obtain deep neural network 10 in response to the input image 20 outputs or generates a digitally stained or labelled output image 40. The digitally stained output image 40 has “staining” that has been digitally integrated into the stained output image 40 using the trained, deep neural network 10. In some embodiments, such as those involved tissue sections, the trained, deep neural network 10 appears to a skilled observer (e.g., a trained histopathologist) to be substantially equivalent to a corresponding brightfield image of the same tissue section sample 22, in order to provide outputting a confirmation indicating tissue. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
Regarding claims 8 and 16. The computer-implemented method of claim 1, further comprising:
outputting a binary image indicating where the tissue is located (see Schleifer, paragraph 97, determining the location of at least a portion of the tissue in the embedded sample based on the fluorescence emission).
Claim(s) 2-4, 6-7, 12-15, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tian (CN 110136809 A) in view of Schleifer (PGPUB: 20190368982 A1), and further in view of Ozcan (PGPUB: 20230030424 A1), and further in view of Mitra (PGPUB: 20210263055).
Regarding claims 2, 12, and 18. The combination teaches the computer-implemented method of claim 1, further comprising:
partitioning each of the collection into a collection of tiles associated with one or more of the collection of digital images (see Tian, Fig. 8, page , lines , a dividing module 12, for the image area of the biological tissue in the biological tissue image, which is divided into a plurality of quadrants position area);
detecting and/or segmenting a tissue region from a background of each digital image to create a tissue mask (see Tian, page 27, lines 24-31, finally obtaining can extract the full convolution of the biological tissue in the image first tissue object area and the area where the second tissue object dividing network. the following image-based semantic segmentation model to determine the first biological tissue image in the identified region with the first tissue particularly shown image semantic segmentation model comprising forward convolution and transpose convolution layer, forward convolutional layer is for forward convolution operation, forward convolution operation can reduce the size of the characteristic pattern; see Tian, page 27-28, lines 1-7, wherein H and W respectively represent the height and width of the first biological tissue image, the H * W * 3 mask comprising: a background mask belonging to the background class, first tissue mask belonging to the first tissue properties, and second tissue mask belonging to the second tissue property. tissue mask, the background mask, the first size of the second tissue mask H x W, here a first tissue attribute is used to identify the object of the first tissue and the second tissue property is used for identifying the second tissue object, when the biological tissue is breast, first tissue object may be the nipple, the second tissue object can be muscle).
However, the combination does not expressly teach removing all tiles in the collection of tiles that comprise the background.
Mitra teaches that when the system detects a certain color that is significant to the lab it could cross check the sample type with the lab data to ensure the colors match. This provides an additional backup of the quality control system. Also, this data can be used during image processing so that variations in image background is effectively filtered out (see Fig. 1, paragraph 79).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination by Mitra to obtain when the system detects a certain color that is significant to the lab it could cross check the sample type with the lab data to ensure the colors match. This provides an additional backup of the quality control system. Also, this data can be used during image processing so that variations in image background is effectively filtered out, in order to provide removing all tiles in the collection of tiles that comprise the background. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
Regarding claims 3, 13, and 19. The combination teaches the computer-implemented method of claim 2, wherein detecting and/or segmenting comprises using thresholding-based methods and running a connected components algorithm (see Tian, page 24, lines 9-13, the terminal device can be greater than the pixel point of focus probability threshold value to form the image area as the focus object of the first region in the image of biological tissue, and may identify a focus attribute of the focus object in the first area is equal to focus attribute corresponding to focus detection model).
Regarding claims 4, 14, and 20. The combination teaches the computer-implemented method of claim 2, wherein detecting and/or segmenting comprises using one or more segmentation algorithms (see Tian, page 25, lines 30-34, wherein determining the second area may be image-based semantic segmentation model, image semantic segmentation model is used for identifying object attribute of each pixel point in the biological tissue image, the working process of image semantic segmentation model in the above focus detection model to determine a second manner attribute similar to the first area and the focus).
Regarding claim 6. The combination teaches the computer-implemented method of claim 1, further comprising:
determining that the presence of the tissue is sufficiently low (see Tian, page 23, lines 32-36, if there is no focus probability of any one image block is greater than the probability threshold, the focus attribute of the focus object in the image of biological tissue is not the focus attribute corresponding to the focus detecting model, i.e. focus detecting module does not detect the position area).
The combination does not expressly teach indicating to a user to prepare a new block for testing.
Mitra teaches that communicate with users of the automated tape transfer apparatus 1 and/or communicate with the microtome 4 to which the automated tape transfer apparatus 1 is connected. There are many motions that can be controlled within the automated tape transfer apparatus 1. Examples of these motions include the movement of the feed mechanism 3 and the take-up mechanism 6, movement of the lower portion 30 and the translation portion of the slide station 5, movement of the linear actuator member etc. The controller may also provide information to users of the functions or conditions of the automated tape transfer apparatus 1 such as the number of slides that have been prepared, the number of sections that have been transferred, the amount of tape remaining on the roll, etc. The controller is capable of receiving any types of input (e.g., mechanical, visual, electrical, etc.) to perform its control functions (see paragraph 251); utilizing the same camera or imaging device, or alternatively, utilizing another camera or imaging device, as the tape with the adhered sections cut from the sample block advances to the slide station and the section is transferred to a slide, a photo (or other imaging technique) is taken of each slide containing the sample to enable real time analysis to make sure the section has been properly, i.e., completely, transferred to the slide (see paragraph 260).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination by Mitra to obtain that the controller may also provide information to users of the functions or conditions of the automated tape transfer apparatus 1 such as the number of slides that have been prepared, the number of sections that have been transferred, the amount of tape remaining on the roll and utilizing the same camera or imaging device, or alternatively, utilizing another camera or imaging device, as the tape with the adhered sections cut from the sample block advances to the slide station and the section is transferred to a slide, a photo (or other imaging technique) is taken of each slide containing the sample to enable real time analysis to make sure the section has been properly, i.e., completely, transferred to the slide, in order to provide indicating to a user to prepare a new block for testing. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
Regarding claims 7 and 15. The combination teaches the computer-implemented method of claim 1, further comprising:
outputting a binary confirmation indicating the FFPE tissue block (see Schleifer, paragraph 35, the present methods can be used to locate the surface of tissue in a formalin-fixed paraffin-embedded (FFPE) tissue block. After the tissue is located in the embedded sample, the tissue may be further processed by trimming or slicing to obtain one or more tissue sections).
However, the combination does not expressly teach to contain enough tissue to test.
Mitra teaches that this process for removing this paraffin layer and exposing the large cross section of the tissue is referred to as block facing. After removal of this superficial paraffin layer, the tissue sample is exposed and ready to be sectioned and put on the tape for transfer to a glass slide for analysis, e.g., pathology or histology. That is, when enough paraffin has been removed (the block is referred to as “faced”), subsequent block sectioning provides tissue sections for placement on glass slides for analysis (processed further for evaluation) (see paragraph 80).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination by Mitra to obtain when enough paraffin has been removed (the block is referred to as “faced”), subsequent block sectioning provides tissue sections for placement on glass slides for analysis (processed further for evaluation), in order to provide to contain enough tissue to test. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tian (CN 110136809 A) in view of Schleifer (PGPUB: 20190368982 A1), in view of Ozcan (PGPUB: 20230030424 A1), and further in view of GODRICH (WO 2020243550 A1).
Regarding claim 9. The combination does not expressly teach computer-implemented method of claim 1, further comprising:
receiving a synoptic annotation comprising one or more label for each digital image.
GODRICH teaches that a CNN may learn feature representations for classification tasks directly from pixels, which may lead to better diagnostic performance. When detailed annotations for regions or pixel-wise labels are available, a CNN may be trained directly if there is a large amount of labeled data (see paragraph 66).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination by xxx to obtain that when detailed annotations for regions or pixel-wise labels are available, a CNN may be trained directly if there is a large amount of labeled data, in order to provide receiving a synoptic annotation comprising one or more label for each digital image. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tian (CN 110136809 A) in view of Schleifer (PGPUB: 20190368982 A1), and further in view of Ozcan (PGPUB: 20230030424 A1), in view of GODRICH (WO 2020243550 A1), and further in view of Mitra (PGPUB: 20210263055)
Regarding claim 10. The combination does not expressly teach the computer-implemented method of claim 9,
wherein the one or more label is at one or more of a pixel-level label, a tile level label, a slide-level label, and/or a part specimen-level label.
Mitra teaches that a just-in-time glass slide label printing protocol can be implemented where the labels for the slides are printed after the tissue samples are cut from the tissue block. In this manner, the glass slide is barcoded or labeled by a just-in-time printer with the barcode derived from the block that was just sectioned at the microtome, such that the immediately cut tissue section is then placed on the newly printed barcoded slide. In some embodiments, the next tissue section is cut and label printed only after the preceding tissue section has been placed on the slide and labeled, and optionally confirmed to be associated with the tissue block (see paragraph 209).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination by Mitra to obtain a just-in-time glass slide label printing protocol can be implemented where the labels for the slides are printed after the tissue samples are cut from the tissue block, in order to provide wherein the one or more label is at one or more of a pixel-level label, a tile level label, a slide-level label, and/or a part specimen-level label. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tian (CN 110136809 A) in view of Schleifer (PGPUB: 20190368982 A1), in view of Ozcan (PGPUB: 20230030424 A1), and further in view of KRIZMAN (PGPUB: 2018/0112277 A1).
Regarding claim 5. The combination does not expressly teach computer-implemented method of claim 1, further comprising:
identifying a tissue block with an amount of tissue for subsequent testing see page 23, lines 26-32, based on the classifier in the focus detection model, to identify each image block belonging to the focus detection model corresponding to focus probability of the focus attribute, if the focus probability image blocks identified probability greater than the focus threshold, indicates that the image block contained in the focus object, and determining the focus property of the focus object is focus attribute corresponding to the focus detecting model, candidate region so that the image block of the position region in the biological tissue image as a focus object); and
However, the combination teach indicating to a user that the tissue block has at least one additional slide to prepare for testing.
KRIZMAN teaches that Such an instrument comprises at least the following parts and operational functions: 1) a moveable microscope slide stage to hold and immobilize a DIRECTOR slide, or slides, that will precisely move, as designated by the inclusion and exclusion signals, in relation to a fixed laser source, 2) a computer monitor to isplay a live image of the tissue section to be microdissected as visualized through either a digital camera or microscope objective that magnifies an image of the tissue section through a digital camera, 3) a computer and computer software that digitally compares and precisely aligns the virtual imprinted digital image of the tissue section containing the inclusion/exclusion signals with the live image of the exact same tissue section on the DIRECTOR slide as it resides within the moveable microscope slide stage (see paragraph 33); the presently described signal directed tissue microdissection method is the cornerstone for a multistep panomics process comprising: 1) obtaining formalin fixed paraffin embedded tumor tissue via a physician and/or healthcare team accompanied by a test requisition form describing the requested tests, 2) isolating and collecting a purified population of patient tumor cells directly from said patient tumor tissue using the presently described signal directed tissue microdissection method, 3) reducing said population of patient tumor cells to a soluble and liquefied state using standard tissue sample preparation protocols and reagents and/or the Liquid Tissue protocol and reagents, 4) detecting and quantifying targeted proteins in said Liquid Tissue lysate using mass spectrometry to develop protein expression profiles and hence the proteomic status of the patient's tumor cells (see paragraph 33).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination by KRIZMAN to obtain a computer and computer software that digitally compares and precisely aligns the virtual imprinted digital image of the tissue section containing the inclusion/exclusion signals with the live image of the exact same tissue section on the DIRECTOR slide as it resides within the moveable microscope slide stage and obtaining formalin fixed paraffin embedded tumor tissue via a physician and/or healthcare team accompanied by a test requisition form describing the requested tests, 2) isolating and collecting a purified population of patient tumor cells directly from said patient tumor tissue using the presently described signal directed tissue microdissection method, in order to provide indicating to a user that the tissue block has at least one additional slide to prepare for testing. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
The combination does not expressly teach the adequate amount of tissue.
The examiner is taking "Official Notice" that the limitation about the adequate amount of tissue is well known in the art.
Therefore, it would have been obvious to a person having ordinary skill in the art at the time the invention was made to have modified the combination so that the adequate amount of tissue would be available.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIN JIA whose telephone number is (571)270-5536. The examiner can normally be reached 9:00 am-7:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XIN JIA/Primary Examiner, Art Unit 2663
Prosecution Insights