DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
1. The information disclosure statements (IDS) submitted on 11/19/2024, 01/03/2024 have been considered by the examiner.
Claim Rejections - 35 USC § 102
2. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
4. Claims 164-174 and 179-183 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Yao et al.; 2019, “Cell Type classification and Unsupervised Morphological Phenotyping from Low-resolution Images Using Deep Learning” (pp. 1-13).
Regarding claim 164, Yao discloses A method, comprising:
generating a cell morphology map using image data comprising tag-free images of a plurality of single cells, wherein the cell morphology map comprises a plurality of morphologically-distinct clusters corresponding to different types or states of the cells (page 2 – 1st para – “we propose a label-free classification scheme using typical thick-bottomed, plastic cell culture flasks, in which live cells are most commonly maintained in cell biology laboratories, allowing for a larger amount of data to be collected with less expense; page 2 -2nd para – “Here we establish a method for cell type classification utilizing only brightfield images taken on a benchtop microscope directly from cell-culture flask”; page 2- 3rd para – “To uncover potential biological significance of these morphological phenotypes, we developed Self-Label Clustering (SLC), a novel unsupervised clustering method designed to cluster and identify distinct morphological phenotypes within a single cell type. Single cells undergo significant morphological changes throughout the cell cycle, and even clonal populations span a wide range of single cell morphologies which may benefit from a larger vocabulary of identifiers than currently available. Using SLC, we are able to perform clustering of single cell morphologies from low-res images taken in cell culture flasks and describe cell density-dependent phenotypes. This result highlights the use of unsupervised methods in machine learning (ML) to establish novel morphological identifiers”; page 7 – 5th para – “We sought to construct a novel, unsupervised clustering technique to uncover morphological identifiers of potential biological significance. For this section of the work we used images of only cell type A (HEK-293A). We began by constructing “self-classes” for each cell, each class representing a series of (50) images generated out of the same cell through the augmentation outlined in (Fig. S1a). We assigned each class a unique label and replaced the final two-class classification layer in the network architecture described in (Fig. 1d) with an N -class classification layer where N is the number of classes determined by the number of cells in the database (Fig. 5a). In this way, we constructed what we call a Self-Label ConvNet where the groups of augmentations of each cell are considered unique classes”; page 7 – 6th and 7th paras - All cells analyzed from every density-controlled flask were included in the training data of Self-Label ConvNet in order to span the entire morphological feature space of HEK cells…This feature space vector of size 288 × 1 is composed of the [l]ast [c]onvolutional layer’s [a]ctivations, which we call the “LCA Feature Space”. A matrix with rows containing LCA vectors for each cell was displayed in (Fig. 5d) – LCA Feature Space here is considered as Cell Morphology Map as also shown figure 5a”); and
using a classifier to automatically classify a cellular image sample using a proximity, a correlation, a commonality, or a combination thereof, with one or more of the morphologically- distinct clusters, the classifier having been trained using the cell morphology map (figure 5c – the input cell(s) are classified through the trained Self-Label ConvNet; page 7 – para 5 – “When given each original image used to generate these classes, the trained Self-Label ConvNet model is able to return a representation of the similarities and differences among any group of the original images based on learned features present in the hidden layers of the network. These similarities and differences are in the vocabulary of novel features learned by the network training without relying on any predetermined” – where similarities and differences provide proximity, a correlation, a commonality, or a combination thereof); further see page 7 – 6th and 7th paras).
Regarding claim 165, Yao discloses “The method of claim 164, further comprising: annotating each morphologically-distinct cluster of the morphologically-distinct clusters using at least a predefined annotation schema” (page 11 – 3rd para - The number of cells in the ensemble was indicated by N (in this Self-Label ConvNet study N = 2208). N classes were constructed in Self-Label ConvNet in the final layer (Softmax classification) instead of two classes for the cell type classification, while other layers before the final layer remained unchanged from (Fig. 1d), the cell type classification ConvNet. Each class in Self-Label ConvNet represents the combination of a series of m images (in this study m = 50) generated out of the same cell image through the augmentation outlined in (Fig. S1), and each class was then assigned a unique label of (“1”,“2”, through “N”) indicating N categories of distinguished Self-Label ConvNet morphological phenotypes throughout the ensemble. The training data of Self-Label ConvNet was then composed of N × m single cell images, leading to a much heavier computational cost for neural network training with around 3 million iterations to Self-Label ConvNet achieve stable accuracy and loss (Fig. 2b)).
Regarding claim 166, Yao discloses “The method of claim 164, further comprising: automatically classifying, using the classifier, the cellular image sample, without prior knowledge or information about a type, state, or characteristic of one or more cells of the cellular image sample” (as cited in the rejection of claim 164 – Yao teaches and unsupervised phenotyping using deep learning, and unsupervised do not need prior knowledge or information of one or more cells).
Regarding claim 167, Yao disclose “The method of claim 164, wherein the cell morphology map is generated using at least one or more morphological features from the image data, wherein the cell morphology map comprises an ontology of the one or more morphological features” (page 8 – figure 5 description – LCA feature space is the structure of interest for the following morphological phenotype clustering…where assigning a class is a fundamental and essential part of developing and using an ontology of morphological feature(s), LCA Feature Maps for many cells across all densities (2208 cells total) were displayed as rows in a matrix (size 2208 × 288) with each column representing one feature in the LCA).
Regarding claim 168, Yao disclose “The method of claim 167, wherein the one or more morphological features are attributable to one or more unique groups of pixels in the image data” (page 2- 3rd para – “To uncover potential biological significance of these morphological phenotypes, we developed Self-Label Clustering (SLC), a novel unsupervised clustering method designed to cluster and identify distinct morphological phenotypes within a single cell type – where distinct morphological phenotypes are separate and observable physical forms in the image and thus representing a unique group of pixels in the image data).
Regarding claim 169, Yao discloses “The method of claim 164, wherein the generating the cell morphology map using image data comprises processing image data using a machine learning algorithm to group the images of the plurality of single cells into the plurality of morphologically-distinct clusters” (page 2- 3rd para – “To uncover potential biological significance of these morphological phenotypes, we developed Self-Label Clustering (SLC), a novel unsupervised clustering method designed to cluster and identify distinct morphological phenotypes within a single cell type; page 7 – 5th para – “We sought to construct a novel, unsupervised clustering technique to uncover morphological identifiers of potential biological significance. For this section of the work we used images of only cell type A (HEK-293A). We began by constructing “self-classes” for each cell, each class representing a series of (50) images generated out of the same cell through the augmentation outlined in (Fig. S1a). We assigned each class a unique label and replaced the final two-class classification layer in the network architecture described in (Fig. 1d) with an N -class classification layer where N is the number of classes determined by the number of cells in the database (Fig. 5a). In this way, we constructed what we call a Self-Label ConvNet where the groups of augmentations of each cell are considered unique classes”; page 7 – 6th and 7th paras - All cells analyzed from every density-controlled flask were included in the training data of Self-Label ConvNet in order to span the entire morphological feature space of HEK cells…This feature space vector of size 288 × 1 is composed of the [l]ast [c]onvolutional layer’s [a]ctivations, which we call the “LCA Feature Space”. A matrix with rows containing LCA vectors for each cell was displayed in (Fig. 5d) – LCA Feature Space here is considered as Cell Morphology Map as also shown figure 5a”); page 11 – 3rd para - The number of cells in the ensemble was indicated by N (in this Self-Label ConvNet study N = 2208). N classes were constructed in Self-Label ConvNet in the final layer (Softmax classification) instead of two classes for the cell type classification, while other layers before the final layer remained unchanged from (Fig. 1d), the cell type classification ConvNet. Each class in Self-Label ConvNet represents the combination of a series of m images (in this study m = 50) generated out of the same cell image through the augmentation outlined in (Fig. S1), and each class was then assigned a unique label of (“1”,“2”, through “N”) indicating N categories of distinguished Self-Label ConvNet morphological phenotypes throughout the ensemble. The training data of Self-Label ConvNet was then composed of N × m single cell images, leading to a much heavier computational cost for neural network training with around 3 million iterations to Self-Label ConvNet achieve stable accuracy and loss (Fig. 2b)).
Regarding claim 170, Yao discloses “The method of claim 169, further comprising: extracting, using the machine learning algorithm, one or more morphological features of the image data from each cell of the plurality of single cells” (page 2- 3rd para – “To uncover potential biological significance of these morphological phenotypes, we developed Self-Label Clustering (SLC), a novel unsupervised clustering method designed to cluster and identify distinct morphological phenotypes within a single cell type).
Regarding claim 171, Yao discloses “The method of claim 164, further comprising: annotating each morphologically-distinct cluster of the morphologically-distinct clusters to generate a plurality of annotated cell images belonging to each cluster of the morphologically-distinct clusters” (as cited in the rejections of claims 164 and 169 - each class was then assigned a unique label of (“1”,“2”, through “N”)).
Regarding claim 172, Yao discloses “The method of claim 164, wherein the using the classifier to automatically classify the cellular image sample is performed on (i) a known population of cells in the cellular image sample, (ii)an unknown population of cells in the cellular image sample, or both (i) and (ii)” (page 7 – 2nd para – 2nd line – “distinguish cell types from a mixed population of cells of unknown origin”).
Regarding claim 173, Yao discloses “The method of claim 164, wherein the one or more of the clusters comprise sub-clusters” (page 8 – figure 5 description - LCA Feature Maps for many cells across all densities (2208 cells total) were displayed as rows in a matrix (size 2208 × 288) with each column representing one feature in the LCA).
Regarding claim 174, Yao discloses “The method of claim 164, wherein two or more of the clusters overlap.” (page 7 – 2nd para - The classification quality of individual frames (Fig. 4c) indicates that even though in the mixed-cell population flask there exist regions with uneven cell type distributions – where mixed cell population provides at least 2 overlapping clusters of cells).
Regarding claim 179, claim 179 has been similarly analyzed and rejected as per claim 164 rejection citations.
Regarding claim 180, claim 180 has been similarly analyzed and rejected as per claim 169 and 170 rejection citations.
Regarding claim 181, claim 181 has been similarly analyzed and rejected as per claim 171 rejection citations.
Regarding claim 182, claim 182 has been similarly analyzed and rejected as per claim 164 rejection citations.
Regarding claim 183, claim 183 has been similarly analyzed and rejected as per claim 169 and 170 rejection citations.
Claim Rejections - 35 USC § 103
5. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
6. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 175-178 are rejected under 35 U.S.C. 103 as being unpatentable over Yao et al.; 2019, “Cell Type classification and Unsupervised Morphological Phenotyping from Low-resolution Images Using Deep Learning” (pp. 1-13), and further in view of Schroder et al., 2020, “MorphoCluster: Efficient Annotation of Plankkton Images by clustering” (pp. 1-26).
Regarding claim 175, claim 175 recites “The method of claim 164, further comprising: curating, verifying, editing, annotating, or a combination thereof, using an interactive annotation tool, the morphologically- distinct clusters”. Yao in page 7 – 5th para teaches – We assigned each class a unique label – which implies annotating was done to the clusters, but does not explicitly teaches using an interactive annotation tool for curating, verifying, editing, annotating, or a combination thereof, the morphologically- distinct clusters. However, Schroder teaches MorphoCluster, a tool for data-driven, fast, and accurate annotation of large data sets of single object images…we combine unsupervised clustering with an interactive tool to revise the initial clusters, arrange them hierarchically, manually correct the hierarchy, and annotate the clusters (see Schroder – page 3 – topic – MorphoCluster). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to use the teachings of using an interactive annotation tool for curating, verifying, editing, annotating, or a combination thereof, the morphologically- distinct clusters as taught by Schroder in the invention of Yao. A person having ordinary skill in the art would have been motivated before the effective filing date of the claimed invention to use the teachings of using an interactive annotation tool for curating, verifying, editing, annotating, or a combination thereof, the morphologically- distinct clusters as taught by Schroder in the invention of Yao, in order to bring reliability and accuracy to the classification including annotation by combining unsupervised clustering with an interactive tool to revise the initial clusters, arrange them hierarchically, manually correct the hierarchy, and annotate the clusters.
Regarding claim 176, the combined invention of Yao and Schroder teaches “The method of claim 175, further comprising: excluding, using the interactive annotation tool, one or more cells of the plurality of single cells that are incorrectly clustered” (see Schroder - page 3 – last para – visually pure clusters are validated and mixed cluster are manually rejected – which is nothing but excluding cells in the mixed clusters where mixed is seen as incorrectly clustered; further see page 6 – topic 2.5 – Cluster Validation – “Impure cluster seeds are deleted).
Regarding claim 177, the combined invention of Yao and Schroder teaches “The method of claim 175, further comprising: excluding, using the interactive annotation tool, debris or one or more cell clumps from the plurality of morphologically-distinct clusters” (see Schroder - page 3 – last para – visually pure clusters are validated and mixed cluster are manually rejected – which is nothing but excluding cells in the mixed clusters where mixed is seen as incorrectly clustered; further see page 6 – topic 2.5 – Cluster Validation – “Impure cluster seeds are deleted).
Regarding claim 178, the combined invention of Yao and Schroder teaches “The method of claim 175, further comprising: assigning weights, using the interactive annotation tool, to one or more clusters of the plurality of morphologically-distinct clusters” (see Schroder – page 3 – topic – MorphoCluster – “we combine unsupervised clustering with an interactive tool to revise the initial clusters, arrange them hierarchically, manually correct the hierarchy, and annotate the clusters – assigning hierarchy here is considered as assigning weights).
8. Claim 184 is rejected under 35 U.S.C. 103 as being unpatentable over Yao et al.; 2019, “Cell Type classification and Unsupervised Morphological Phenotyping from Low-resolution Images Using Deep Learning” (pp. 1-13), and further in view of Reddy et al., 2019, “Review on cloud-based Image Classification” (pp. 679-686).
Regarding claim 184, claim 184 recites “The system of Claim 182, wherein the automatically classifying is performed at least partially with a computer-implemented cloud-based system”. Yao as discussed in the rejection of claims 164 and 166 teaches automatically classifying the cells, but does not explicitly teach classifying is performed using a cloud-based system. However, Reddy discloses automatically classifying is performed at least partially with a computer-implemented cloud-based system (page 680 – left column – “IMACEL is a cloud-based image analysis framework developed for automatic classification and morphological analysis. As all image processing and machine learning techniques are performed by virtual machines in the cloud, it’s not needed to set up powerful computers in laboratory. IMACEL’s target data includes different types of microscopic bioimages”). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to use the teachings of using a cloud-based system as taught by Reddy in the invention of Yao. A person having ordinary skill in the art would have been motivated before the effective filing date of the claimed invention to use the teachings of using a cloud-based system as taught by Reddy in the invention of Yao, as all image processing and machine learning techniques are performed by virtual machines in the cloud, it’s not needed to set up powerful computers in laboratory (see Reddy - page 680 – left column).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Manav Seth whose telephone number is (571) 272-7456. The examiner can normally be reached on Monday to Friday from 8:30 am to 5:00 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Sumati Lefkowitz, can be reached on (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https:/Awww.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000
/Manav Seth/
Primary Examiner, Art Unit 2672
November 9, 2025