DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to the Applicant
Limitations appearing inside {} are intended to indicate the limitations not taught by said prior art(s)/combinations.
Response to Amendments
The Amendment filled 06/23/2026 in response to Non-Final Office Action mailed 03/04/2026 has been entered. Claims 19-25, and 27-28 have been amended. Claims 1-18, 30-67, 26, and 30-69 are withdrawn. Rejections under 35 USC 112(b) have been withdrawn in light of amended claims. Rejections under 35 USC §103 have been withdrawn in light of the amended claims.
Claims 19-25, and 27-29 are currently pending.
Response to Arguments/Remarks
Applicant’s arguments, see Remarks pages 1-2, filed 06/23/2026, with respect to claims 19-25, and 27-29 have been fully considered and are persuasive. The rejection under 35 USC 112(b) of 03/04/2026 has been withdrawn.
Applicant’s arguments with respect to claims 19-25, and 27-29 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Information Disclosure Statement
No Information Disclosure Statement (IDS) was filed; therefore, no applicant-submitted references were considered.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 19, 20, 24, and 25 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by “Dutta” (Dutta et al., US 20220147760 A1).
19. Dutta teaches a method for training a machine learning system with a candidate extraction model for extracting candidate signal series from an image series,
wherein the image series is generated by marking analytes with markers in a plurality of coloring rounds and detecting the markers with a camera,
the camera captures an image (Dutta, ¶[0548]; over-head cameras (e.g., Illumina's GAIIx's CCD camera taking images of the clusters on the biosensor 7812 from the top; ¶[0205]; The imaging device 106 (e.g., a solid-state imager such as a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) sensor) takes snapshots at multiple locations along the lanes 204 in a series of non-overlapping regions called tiles 206) of the image series in each coloring round, the markers are selected in such a way that image signals from a respective one of the analytes in an image area across the image series comprise colored and uncolored signals
(¶[0684]; each image in the set of images includes color signals, wherein a different color corresponds to a different nucleotide base….producing a plurality of cycles of color images. ¶[0698]; The color at which individual analytes appear may be a function of the dye employed as well as of the wavelength of the light used by the imaging system for imaging purposes. Analytes to which targets are not bound or that are otherwise devoid of a particular label can be identified according to other characteristics,),
comprising:
- providing an annotated data set (Dutta; Fig 15, and ¶[0277]; Training Set Generator 1502), and
- optimizing an objective function by adjusting the model parameters of the candidate extraction model wherein the objective function detects a difference between a result output from the candidate extraction model and a target output (Dutta, ¶[0380]; The training 2800 includes iteratively optimizing a loss function that minimizes error 2806 …and updating parameters of the regression model 2600 based on the error 2806),
wherein the annotated data set comprises signal series from respective image areas of the image series (Dutta, ¶[0277]; the image patch covers a portion of the particular one of the tiles), the signal series including (Dutta, ¶[0359]; Each of the two hundred sequencing images 108) at least one analyte signal series from a respective image area in which image signals from a respective one of the analytes are captured (Dutta, ¶[0359]; depicts intensity emissions of clusters on tile A), and
at least one background signal series from a respective background image area in which image signals from a background are captured (Dutta, ¶[0359]; and their surrounding background captured in a particular image channel at a particular sequencing cycle), and
for each of the signal series, a target output indicating whether or not the signal series comprises image signals from one of the analytes (Dutta, ¶[0359]; the ground truth data is generated for tile A that is on lane A of flow cell A. The ground truth data is generated from the sequencing images 108 of tile A captured during sequencing run A).
20. Dutta teaches the method according to claim 19. Dutta further teaches wherein the candidate extraction model is trained to identify the candidate signal series on the basis of a number of colored signals among the colored and uncolored signals (Dutta, ¶[0591]; the analytes may appear different with regard to the color of a given analyte detected in different images, a change in the intensity of signal detected for a given analyte in different images, or even the appearance of a signal for a given analyte in one image and disappearance of the signal for the analyte in another image.),
wherein the colored and uncolored signals are identified based on at least one particular ratio of one of the colored and/or uncolored signals of the respective signal series to at least one other of the colored and/or uncolored signals of the respective signal series and/or to identify the candidate signal series, respectively, based on a characteristic signature with the at least one particular ratio (Dutta, ¶[0505]; ratio of the brightest base intensity divided by the sum of the brightest and the second brightest base intensities.).
24. Dutta teaches the method according to claim 19. Dutta further teaches wherein the candidate extraction model is implemented as a detection model and outputs a list of image areas that detect a candidate signal series (Dutta, ¶[0341]; a patch extractor 2202 extracts patches from the sequencing images 108 in the series of image sets 2100 and produces a series of down-sized image sets 2206, 2208, 2210, and 2212. Each image in the series of down-sized image sets is a patch of size M×M (e.g., 20×20) that is extracted from a corresponding sequencing image in the series of image sets 2100. ¶[1005] A neural network-implemented method of determining analyte data from image data, … , ¶[1009]; wherein the input image data comprises image patches extracted from each image in the sequence of images).
25. Dutta teaches the method according to claim 19. Dutta further teaches wherein the annotated data set is generated with at least one of the following steps:
- simulating signals of the different markers by using a representative background image and a known point spread function of a microscope,
- generating the annotated data set by means of a generative model trained on the basis of comparable data,
- acquiring reference images comprising at least one background image and, for each of the background images, at least one image in which each of the analytes to be identified is marked,
- performing a classical method for the spatial identification of analytes
(Dutta, ¶[0744]; The method includes generating at least one ground truth data representation for each of the training examples, the ground truth data representation identifying at least one of spatial distribution of analytes and their surrounding background on the particular one of the tiles whose intensity emissions are depicted by the image data, including at least one of analyte shapes, analyte sizes, and/or analyte boundaries, and/or centers of the analytes.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Dutta in view of “Murzova” (Murzova A. (2020 July 13). CNN Fully Convolutional Image Classification (FCN CNN) with TensorFlow. LearnOpenCV. https://learnopencv.com/cnn-fully-convolutional-image-classification-with-tensorflow/).
21. Dutta teaches the method according to claim 19. Dutta further teaches wherein the candidate extraction model is a fully convolutional network, which has been trained as a classification model with fully connected layers (Dutta, ¶[0838]; training a classifier based upon the determined plurality of disjointed regions of contiguous subpixels, the classifier being a neural network-based template generator for processing input image data to generate a decay map, a ternary map, or a binary map, representing one or more properties of each of a plurality of analytes represented in the input image data for base calling by a neural network-based base caller) using individual ones of the signal series of the annotated data set, the individual ones of the signal series being from respective individual image areas (Dutta, ¶[0786]; in the training data, multiple training examples respectively include as image data different image patches from each image in a sequence of image sets), and {after the training, the classification model is transformed, through replacement of the fully connected layers with convolutional layers}, into the fully convolutional network that can process respective signal series from all image areas of the image series simultaneously (Dutta, ¶[0327]; classification model 5400 is a fully convolutional network).
Dutta does not explicitly disclose after the training, the classification model is transformed, through replacement of the fully connected layers with convolutional layers.
However, Murzova, a similar field of endeavor of transforming fully connected layers to a fully convolutional network, teaches after the training, the classification model is transformed, through replacement of the fully connected layers with convolutional layers, into the fully convolutional network that can process the signal series of all image areas of the image series simultaneously (Murzova, [p 2, ¶3]; To feed an arbitrary-sized image into the network, we need to replace all FC layers with convolutional layers).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include transforming fully connected layers to fully convolutional network as taught by Murzova to the invention of Dutta. The motivation to do so is because the fully convolutional layer does not require a fixed input size.
Claims 22, 23, 28, and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Dutta, and further in view of “Wang” (Wang A, et al. A novel deep learning-based 3D cell segmentation framework for future image-based disease detection. Sci Rep. 2022 Jan 10;12(1):342. doi: 10.1038/s41598-021-04048-3. PMID: 35013443; PMCID: PMC8748745.).
22. Dutta teaches the method according to claim 19. Dutta further teaches wherein the candidate extraction model is a semantic segmentation model (Dutta, ¶[0290]; the neural network-based template generator 1512 uses semantic segmentation techniques to produce an output value for each unit in the input array) and
the annotated data set comprises, for each image of the image series, a segmentation mask that assigns to each image area a value indicating whether or not the image area is a candidate image area having a corresponding analyte signal series, wherein the value is a bit indicating whether or not the image area is a candidate area (Dutta, ¶[0500]; binary mask of ground truth ternary map 6008 that assigns each unit the class label that has the highest output value; ¶[0575]; processes which calculate a tile cluster mask for each tile in the flow cell, which identifies pixels in the array of sensor data that correspond to clusters of genetic material on the corresponding tile of the flow cell. ¶[0275]; the ground truth data representation is a ground truth mask; ¶[0291]; the objective function is set to reconstruct a segmentation mask; but Dutta does not explicitly disclose that the segmentation mask indicates whether or not the image area is a candidate image. ¶[0322]; The template image can, in some implementations, serve as a mask for intensity extraction. The template image is not the annotated data. It may be argued that Dutta does not explicitly disclose the claim limitations.)
However, Wang, a similar field of endeavor of cell instance segmentation, teaches the annotated data set comprises, for each image of the image series, a segmentation mask that assigns to each image area a value indicating whether or not the image area is a candidate image area having a corresponding analyte signal series, wherein the value is, for example, a bit indicating whether or not the image area is a candidate area (See Wang, Fig 1, shown below, exhibits a semantic segmentation model in Stage I, with three masks, one of which indicates whether a voxel is the cell foreground, and replaces the value
p
k
with a value
p
k
p
k
+
α
indicating confidence that the voxel is the cell foreground [§Results. Loss Function., p3-4]).
PNG
media_image1.png
460
828
media_image1.png
Greyscale
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include segmentation masks as taught by Wang to the invention of Dutta. The motivation to do so would be to improve the accuracy of segmentation.
23. Dutta teaches the method according to claim 19. Dutta does not explicitly disclose wherein the candidate extraction model is an image-to-image model and a processing map is an image-to-image map, and the target output in the annotated data set is either a distance value indicating how far the image area is from a closest image area having a corresponding analyte signal series or a probability value indicating the probability that a signal series from the image area is an analyte signal series.
However, Wang teaches wherein the candidate extraction model is an image-to-image model and a processing map is an image-to-image map (See Fig 3 exhibits an image-to-image map produced by 3DSegCell), and the target output in the annotated data set is either
a distance value indicating how far the image area is from a closest image area having a corresponding analyte signal series or
a probability value indicating the probability that a signal series from the image area is an analyte signal series (Wang, [§METHODS. 3DCellSeg Loss to tackle the clumped cell problem., p 11, ¶1];
p
k
∈
[
0,1
]
is the model confidence of a voxel being the cell foreground).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include an image-to-image map as taught by Wang to the invention of Dutta. The motivation to do so would be to for voxel-by-voxel labeling.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include confidence value as taught by Wang to the invention of Dutta. The motivation to do so would be to improve the accuracy of the loss function.
28. Dutta teaches method according to claim 19. Su further teaches wherein the optimization of the objective function comprises a plurality of training rounds, a training round comprising:
- selecting training data from the annotated data set (Su, See Fig 1b Training Slides selected from the labeled Dataset),
- determining the objective function on the basis of the training data (Su, See Fig 1d, Training Neural Network),
{- identifying misclassified signal series of a background area within a first predetermined radius around the respective image area of an individual analyte signal series of the at least one analyte signal series, and outside a second predetermined radius around the respective image area, wherein the first predetermined radius is larger than the second predetermined radius},
- using the identified misclassified signal series as training data in a next training round, in addition to the training data selected in the next training round (Dutta, ¶[0434]; The custom-weighted loss function gives more weight to the COM subpixels, such that the cross-entropy loss is multiplied by a corresponding reward (or penalty) weight specified in a reward (or penalty) matrix whenever a COM subpixel is misclassified.).
Dutta does not explicitly disclose
- identifying misclassified signal series of a background area within a first predetermined radius around the respective image area of an individual analyte signal series of the at least one analyte signal series, and outside a second predetermined radius around the respective image area, wherein the first predetermined radius is larger than the second predetermined radius,
However, Wang teaches identifying misclassified signal series of a background area within a first predetermined radius around the respective image area of an individual analyte signal series of the at least one analyte signal series, and outside a second predetermined radius around the respective image area, wherein the first predetermined radius is larger than the second predetermined radius(Wang, [§Results, p 3, ¶1-2]; super voxels across two neighboring cells are caused by membrane voxels being misclassified as the cell foreground, and this misclassified area is usually small in practice),
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include identifying misclassified regions as taught by Wang to the invention of Dutta. The motivation to do so would be to improve the model by enabling more accurate cell instance clustering.
29. The combination of Dutta and Wang teaches the method according to claim 28. Wang further teaches/suggests wherein a misclassified signal series that is immediately adjacent to a pixel to which an analyte has been correctly assigned is not used as training data in the next training round (Wang, [§METHODS. 3DCellSeg Loss to tackle the clumped cell problem., p12, ¶1]; the mis-classified voxels around the edge are penalized more heavily than those in the center. (This implies that misclassification near the center near true positives will not be penalized heavily) Since mis-classification of the cell foreground often occurs near cell membranes, 3DCellSeg Loss helps address the adhesions of the cell foreground masks).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include exclude misclassified cells near true positives as taught by Wang to the invention of Dutta. The motivation to do so would be to prioritize occurrences near the cell membrane.
Claim 27 are rejected under 35 U.S.C. 103 as being unpatentable over Dutta in view of “Way” (Way, Gregory P., et al. "Morphology and gene expression profiling provide complementary information for mapping cell state." Cell systems 13.11 (2022): 911-923.).
27. Dutta teaches the method according to claim 19. Dutta does not explicitly disclose further comprising, before inputting into the candidate extraction model an individual analyte signal series of the at least one analyte signal series, swapping positions of image signals corresponding to different coloring rounds within the individual analyte signal series.
However, Way, a similar field of endeavor of applying supervised methods to predict cell painted targets, teaches further comprising, before inputting into the candidate extraction model an individual analyte signal series of the at least one analyte signal series, swapping positions of image signals corresponding to different coloring rounds within the individual analyte signal series (Way, [§METHOD DETAILS, Supervised mechanism of action prediction: Training and test splits, p. e7. ¶2] we created a shuffle data set using data in the training set.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include shuffling the training data as taught by Way to the invention of Dutta . The motivation to do so would be to ascertain and verify that the classification models are learning from the training set and that they could generalize well on test set data.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sarkar (US 10235559 B2) teaches counting dots in an image of a tissue specimen comprising detecting dots in an image of the tissue sample that meet criteria for absorbance strength, classifying the detected dots, and would have been relied upon for teaching segmentation masks and calculating a ratio of dots.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHANDHANA PEDAPATI whose telephone number is 571-272-5325. The examiner can normally be reached M-F 8:30am-6pm (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHANDHANA PEDAPATI/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights