DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority/Continuity
Number
Filing Date
Type
Support
Notes
63546144
10/27/2023
Provisional
All claims appear to be supported
PCT/US24/52940
10/25/2024
Continuation of a PCT
All claims appear to be supported
18991159
12/20/2024
Continuation
All claims appear to be supported
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/14/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-12 and 14-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-12 and 14-20 of U.S. Patent No. 12450926. Although the claims at issue are not identical, they are not patentably distinct from each other because the instant pending claims are fully encompassed and broader than the claims of the patent No. 12450926.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, and 8-10 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Comiter (US 20240371184) in view of Ho (US 20240161519) and Choi (US 20240194292).
Regarding claim 1:
Comiter discloses: a system for generating synthetic spatial omics images, comprising:
one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors (¶ [0013, ¶ [0037], FIG. 3);
the one or more programs including instructions for:
receiving a histopathology image depicting a diseased region of interest of an input tissue sample (¶¶ [0008], [0023], [0037], and [0124] teach receiving microscopy imaging data of biological samples, including histopathology images for omics profile determination ); and
generating a synthetic spatial omics image depicting one or more stained structures of interest within the diseased region of interest (¶¶ [0009], [0011], [0039], [0140], and FIGS. 4 and 5a-5c describe generating spatial expression profiles from microscopy images which involved synthesizing spatial omics data using AI models);
receiving a training histopathology image of a training tissue sample, wherein the training histopathology image comprises a plurality of identified landmarks (¶¶ [0011], [0016], [0026], [0040] – [0042] , [0045], and [0084]; describe training models using histopathology images and detecting landmarks such as nuclei staining and endogenous Oct4-GFP (iPSC marker));
receiving a training spatial omics image of the training tissue sample, wherein the training spatial omics image comprises a plurality of identified landmarks (¶ [0030], [0040] – [0042], [0048, [0052], [0060], [0084], and [0162] describe training models using spatial omics data and detecting landmarks such as nuclei stationing and endogenous Oct4-GFP (iPSC marker));
registering the training histopathology image and the training spatial omics image (FIGS. 10 and 31 ¶ [0040], [0048], [0058], [0442], and [0558] disclose aligning histology images and spatial omics data);
Comiter does not explicitly teach inputting the histopathology image into a trained model for the claimed generation step of synthetic image, the registration is based on the plurality of landmarks identified in the training histopathology image and the plurality of landmarks identified in the training spatial omics image; generating, based on the registration of the training histopathology image and the training spatial omics image, a training dataset comprising a histopathology image set and a corresponding spatial omics image set; and training the model based on the generated training dataset.
However, in a related field, Ho teaches: Ho teaches: inputting the histopathology image into a trained model (¶ [0010] “…(ii) process the histology image and the cell segmentation map using a trained machine learning model to generate a cell expression map including a plurality of cell contours, each including a respective predicted aggregated gene expression, wherein the machine learning model is trained using training data”))
receiving training images including histopathology images and spatial omics images comprising landmarks (¶ [0070] – [0075], Nuclei-based registration in special omics, the multiplex immunofluorescence (IF) imaging includes nuclei markers, which are reference points for spatial omics data alignment, and the spatial transcriptomics images are aligned with the (IF) images).
the registration is based on the plurality of landmarks identified in the training histopathology image and the plurality of landmarks identified in the training spatial omics image (¶ [0074]; Nuclei-based registration in special omics, the multiplex immunofluorescence (IF) imaging includes nuclei markers, which are reference points for spatial omics data alignment, and the spatial transcriptomics images are aligned with the (IF) images);
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Comiter to incorporate the teachings of Ho in order to improve accuracy and reliability of spatial omics image generation and analysis. By 1) enhancing registration accuracy using nuclei based landmarks, Comiter describes receiving histopathology images and spatial omics images as well as processing them for spatial feature extraction and omics profile generation, but lacks to explicitly teach that a landmark-based registration approach for aligning histology and spatial omics images, which is taught by Ho and enables precise alignment of histopathology images with spatial transcriptomics data. 2) Ho also describes SSIM-based computational registration techniques for aligning multiplexed IF images and spatial transcriptomics maps.
Comiter in view of Ho does not specifically teach: generating, based on the registration of the training histopathology image and the training spatial omics image, a training dataset comprising a histopathology image set and a corresponding spatial omics image set; and training the model based on the generated training dataset.
However, in a related field, Choi teaches: generating, based on the registration of the training histopathology image and the training spatial omics image, a training dataset comprising a histopathology image set and a corresponding spatial omics image set (¶ [0009] “…a cell type enrichment prediction model which is trained using training data that consists of spatially resolved transcriptome information and tissue images spatially aligned with the spatially resolved transcriptome information”; ¶ [0044] “…matching the transcriptome data with the tissue image data based on the coordinates of the spots 212, a process of placing a rectangular box of a preset size in the tissue image data including the multiple spots 212, and a process of extracting the tissue image data into at least one patch tissue image 211 such that the coordinates of a central spot 212 among the multiple spots 212 become central coordinates 213 of the rectangular box.”; ¶ [0040]’ “…the transcriptome data and the tissue image data are spatially matched to each other based on the coordinates of the spots 212”); and
training the model based on the generated training dataset (¶ [0041] “The cell type enrichment prediction model 200 is constructed based on training data in which the spatially resolved transcriptome information 20 for each of previously collected human or animal tissues is matched to cell type enrichment information 240 with respect to the transcriptome data classified according to the coordinates of each of the spots 212.”; ¶ [0052] “…For example, the patch tissue image 211 may be augmented through rotation, operation for bilateral/vertical symmetry, zooming-in and zooming-out (in 20% steps), and variation for each RGB channel as random functions with respect to data input to a training process of a convolutional neural network.”
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Comiter in view of Ho to incorporate the teachings of Choi by including: generating, based on the registration of the training histopathology image and the training spatial omics image, a training dataset comprising a histopathology image set and a corresponding spatial omics image set; and training the model based on the generated training dataset in order to provide an apparatus and method for predicting complex cell type enrichment information in tissues by inputting general tissue images free from spatially resolved transcriptome information to a cell type enrichment prediction model trained based on spatially resolved transcriptome information which includes a spatial data-sharing transcriptome and tissue images.
Regarding claim 2: Comiter in view Ho and Choi teaches the limitations of claim 1 as applied above.
Comiter further teaches: wherein the histopathology image depicting the diseased region of interest of the input tissue sample comprises a hematoxylin and eosin (H&E) stain image, a trichrome stain image, an immuno-histochemistry (IHC) stain image, or a Fluorescence In Situ Hybridization (FISH) stain image (¶ [0030]).
Regarding claim 3: Comiter in view Ho and Choi teaches the limitations of claim 1 as applied above.
Comiter further teaches: wherein the generated synthetic spatial omics image comprises a synthetic spatial proteomics image, a synthetic spatial transcriptomics image, or a synthetic spatial epigenetic image (¶ [0233], and see Rahman Abstract and Ho ¶ [0002]).
Regarding claim 4: Comiter in view Ho and Choi teaches the limitations of claim 1 as applied above.
Comiter further teaches: wherein the one or more stained structures of interest comprise one or more proteins of interest, DNA, autofluorescence of tissues, or any combination thereof (¶ [0142] and ¶ [0153]).
Regarding claim 5: Comiter in view Ho and Choi teaches the limitations of claim 4 as applied above.
Comiter further teaches: DAPI, Cy5, TRITC, FOXP3 (D2W8E), Perforin_C, p-STAT3_C, HLA-I (A, B, or C), TGF-beta, GRZMB_A, GFAP_D, LAG-3_B, TMEM119_C, CD45RO_A, PD-1 EPR4877(2), HLA-DR_B, PD-L1, CD68 PG-M1, CD19_A, NKG2D_B, TIM-3_A, CD20 L26, LCK_B, CD8 4B11, P2RY12_A, CD4 EPR6854, CD11c, CD205_D, CD163, CD31, or any combination thereof (¶ [0040]).
Regarding claim 8: Comiter in view Ho and Choi teaches the limitations of claim 1 as applied above.
Comiter further teaches: wherein the plurality of landmarks identified in the training histopathology image or the plurality of landmarks identified in the training spatial omics image comprise: one or more cell nuclei (¶ [0040], and ¶ [0570]. Also see FIG. 2 of Rahman as applied in claim 7 below).
Regarding claim 9: Comiter in view Ho and Choi teaches the limitations of claim 1 as applied above.
Comiter further teaches: wherein registering the training histopathology image and the training spatial omics image comprising obtaining a transformation function (¶¶ [0442], and [0558] and FIGs 10 and 31, feature based registration involves computing transformation functions).
Regarding claim 10: Comiter in view Ho and Choi teaches the limitations of claim 1 as applied above.
Comiter further teaches: the one or more programs further including instructions for: performing noise removal on the spatial omics image set of the training dataset (¶ [0564]).
Regarding claims 19 and 20: the claims limitations are similar to those of claim 1; therefore, rejected in the same manner as applied above.
Claim(s) 7, 11-12, and 14-18 rejected under 35 U.S.C. 103 as being unpatentable over Comiter (US 20240371184) in view of Ho (US 20240161519), Choi (US 20240194292), and Rahman ("A SSIM guided cGAN architecture for clinically driven generative image synthesis of multiplexed spatial proteomics channels." In 2023 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), pp. 1-8. IEEE, August 29, 2023.”).
Regarding claim 7: Comiter in view Ho and Choi teach the limitations of claim 1 as applied above.
Comiter in view Ho and Choi teach does not specifically teach: identifying the plurality of landmarks in the training spatial omics image by isolating, from the training spatial omics image, image data corresponding to the plurality of landmarks by selecting a channel from a plurality of channels.
However, in a related field, Rahman further teaches: identifying the plurality of landmarks in the training spatial omics image by isolating, from the training spatial omics image, image data corresponding to the plurality of landmarks by selecting a channel from a plurality of channels (II. “OVERVIEW OF MODEL ARCHITECTURE” and FIG. 3, channel selection algorithm for training a generative cGAN to synthesize missing channels from multiplexed spatial proteomics images).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Comiter in view of Ho and Choi to incorporate the teachings of Rahman by including: identifying the plurality of landmarks in the training spatial omics image by isolating, from the training spatial omics image, image data corresponding to the plurality of landmarks by selecting a channel from a plurality of channels in order to enhance spatial omics image generation using synthetic data augmentation, and extend the functionality of the system from spatial omics to synthetic spatial omics generation to allow for a trained model synthesis of missing or predicted molecular feature, and therefore, provided a more complete spatial omics dataset.
Regarding claim 11: Comiter in view Ho and Choi teaches the limitations of claim 1 as applied above.
Rahman further teaches: wherein the generator of the GAN model comprises a convolutional neural network (Section II, “Figure 1A shows an overview of the U-NET based cGAN network architecture consisting of a generator and discriminator network”).
Regarding claim 12: Comiter in view Ho and Choi teaches the limitations of claim 11 as applied above.
Rahman further teaches: wherein the generator of the GAN model is based on a U-Net model (Section II, “Figure 1A shows an overview of the U-NET based cGAN network architecture consisting of a generator and discriminator network”).
Regarding claim 14: Comiter in view Ho and Choi teaches the limitations of claim 11 as applied above.
Comiter further teaches: predicting, based on the synthetic spatial omics image, an outcome of a patient associated with the input tissue sample (¶ [0228]).
Regarding claim 15: Comiter in view Ho and Choi teaches the limitations of claim 11 as applied above.
Comiter further teaches: identifying, based on the synthetic spatial comics image, a treatment for a patient associated with the input tissue sample (¶ [0228]).
Regarding claim 16: Comiter in view Ho and Choi teaches the limitations of claim 11 as applied above.
Comiter further teaches: identifying, based on the synthetic spatial comics image, a biomarker (¶¶ [0102] and [0228]).
Regarding claim 17: Comiter in view Ho and Choi teaches the limitations of claim 11 as applied above.
Comiter further teaches: identifying, based on the synthetic spatial comics image, one or more spatial motifs (¶ [0301]).
Regarding claim 18: Comiter in view Ho and Choi teaches the limitations of claim 11 as applied above.
Ho further teaches: the one or more programs further including instructions for: displaying the synthetic spatial omics image as an overlay over the histopathology image of the input tissue sample (¶ [0074] “These regions may be registered by eye for visualization purposes and may not need computational registration”).
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Comiter (US 20240371184) in view of Ho (US 20240161519) and Choi (US 20240194292), and further in view of Madabhushi (US 9286672).
Regarding claim 6: Comiter in view Ho and Choi teaches the limitations of claim 1 as applied above.
Comiter in view Ho and Rahman does not specifically teach: identifying the plurality of landmarks in the training histopathology image by isolating, from the training histopathology image, image data corresponding to the plurality of landmarks by performing color deconvolution.
However, in a related field, Madabhushi: identifying the plurality of landmarks in the training histopathology image by isolating, from the training histopathology image, image data corresponding to the plurality of landmarks by performing color deconvolution (column 11, lines 11-14 “The nuclei are first identified automatically by isolating the blue hematoxylin stain, which preferentially stains nuclear material, via color deconvolution”).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Comiter in view of Ho and Rahman to incorporate the teachings of Madabhushi by including: identifying the plurality of landmarks in the training histopathology image by isolating, from the training histopathology image, image data corresponding to the plurality of landmarks by performing color deconvolution in order to produce an improved predictor of patient outcome.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Verma (US 20230117405) discloses GANs including DCGANs and Wasserstein GANs (WGANs) for histology image processing (¶ [0172], and also discusses creating training datasets from registered histology-genomic images pairs (¶ [0045], and ¶ [0050]),
Mahmood (US 20220367053) teaches: using GAN for histology image processing and segmentation (¶ [0077]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WASSIM MAHROUKA whose telephone number is (571)272-2945. The examiner can normally be reached Monday-Thursday 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Koziol can be reached on (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WASSIM MAHROUKA/Primary Examiner, Art Unit 2665