DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed.
Response to Amendment
Claims 1-17 were previously pending and subject to non-final action filed on 03/24/2026. In the response filed 06/16/2026, claims 1 and 13 were amended. Therefore, claims 1-17 are currently pending and subject to the final action below.
Response to Arguments
Applicant’s arguments, see page 7, filed 06/16/2026, with respect to the Drawing have been fully considered and are persuasive. The objection of the Drawing has been withdrawn.
Applicant’s arguments, see page 7-9, filed 06/16/2026, with respect to claim 1-17 under 35 U.S.C. 101 have been fully considered and are persuasive. The 101 rejection of the Drawing has been withdrawn.
Applicant’s arguments, see pages 9-13, filed on 6/16/2026 with respect to claims 1-17 under the prior rejections have been considered but are moot with respect to the prior rejection because 1-17 are currently rejected under 35 U.S.C. 103 set forth below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-17 are rejected under 35 U.S.C. 103 as being unpatentable over KAPPEL (WO 2020244779 A1, Pub Date: Dec. 10, 2020) in view of Ando (US PAT: US 10769501 B1, Pub Date: Sep. 8, 2020).
Regarding independent claim 1, KAPPEL teaches: A method for processing images of an optical imaging device, the method comprising: ([pdf page 18 ll. 1-5] The system 100 may be implemented in a microscope, may be connected to a microscope or may comprise a microscope. The microscope may be configured to obtain the biology related image-based search data 103.)
obtaining embeddings of a plurality of candidate molecules (KAPPEL − [pdf page 8 ll. 14-17] A high-dimensional representation (e.g. first and second high-dimensional representation) may be a,… an embedding, a sematic embedding and/or a token embedding and/or…, an embedding, a semantic embedding and/or a token embedding. [pdf page 18 ll. 30-35)
PNG
media_image1.png
209
893
media_image1.png
Greyscale
PNG
media_image2.png
331
823
media_image2.png
Greyscale
wherein embedding of each candidate molecule is derived from a chemical or sequence representation of the respective candidate molecule; (KAPPEL – [pdf page 12 ll. 18-28] Each biology related language based input data set… may be nucleotide sequence, a protein sequence, a description of a biological molecule or biological structure…; [pdf page 18 ll. 15-25] visual model 220… was trained on the semantic embedding of a textual model, which was trained on a large body of protein sequences… nucleotide sequence)
PNG
media_image3.png
436
960
media_image3.png
Greyscale
PNG
media_image4.png
434
896
media_image4.png
Greyscale
obtaining, for each candidate molecule, one or more images of the optical imaging device, (KAPPEL − [pdf page 18-19, ll. 31-32, 1-4] The image data in the database 240 (e.g. stored by the one or more storage devices) or as part of a running experiment in a microscope may be transformed into their respective embeddings 250 (plurality of second high-dimensional representations) via a forward pass through a pre-trained visual model 220.) the microscope is an optical imaging device.
PNG
media_image5.png
505
826
media_image5.png
Greyscale
the one or more images showing a visual representation of a target property exhibited by the candidate molecule in a biological sample; (KAPPEL − [pdf page 7, ll. 25-30] The biology-related image-based search data 103 may be image data ( e.g. pixel data of an image) of an image of a biological structure comprising…, a protein or a protein sequence, a biological molecule; [pdf page 23 ll. 24-30] The system 400 may be configured to determine a microscope target position based on the selected second high-dimensional representation.) positioning the microscope to show image data of protein (target property) by using the selected high-dimensional representation (candidate molecule).
PNG
media_image6.png
392
1368
media_image6.png
Greyscale
PNG
media_image7.png
400
1379
media_image7.png
Greyscale
processing, using a machine-learning model, for each candidate molecule, the one or more images and/or information derived from the one or more images to generate a predicted embedding of the candidate molecule, (KAPPEL − [pdf page 20, ll. 10-25] The visual model 220 may have been pre-trained to predict these semantic embedding during training. [pdf page 36 ll. 18-19] A visual model trained to predict semantic token embeddings from images may convert a query image to its related semantic embedding. [pdf page 42 ll. 19-20] Embodiments may be based on using a machine-learning model or machine-learning algorithm.)
PNG
media_image8.png
426
765
media_image8.png
Greyscale
the machine-learning model being trained to output the predicted embedding for an input comprising the one or more images and/or the information derived from the one or more images (KAPPEL − [pdf page 20, ll. 10-25] The visual model 220 may have been pre-trained to predict these semantic embedding during training. [pdf page 36 ll. 18-19] A visual model trained to predict semantic token embeddings from images may convert a query image to its related semantic embedding.) generate predict semantic embedding from trained visual model.
comparing the embeddings of the candidate molecules with the predicted embeddings of the candidate molecules; (KAPPEL − [pdf page 34 ll. 5-10] the method 1100 comprises selecting 1130 a second high-dimensional representation from the plurality of second high-dimensional representation based on a comparison of the first high-dimensional representation with each or a subset of the second high-dimensional representation of the plurality of second high-dimensional representations.) comparing first high-dimensional representation (embeddings) with the each or subset of second high-dimensional representation (predict semantic embedding).
and selecting one or more candidate molecules based on the comparison; (KAPPEL − [pdf page 34 ll. 5-10] the method 1100 comprises selecting 1130 a second high-dimensional representation from the plurality of second high-dimensional representation based on a comparison of the first high-dimensional representation with each or a subset of the second high-dimensional representation of the plurality of second high-dimensional representations.)
KAPPEL does not explicitly teach: that can be used as labels or tags in life-science microscopy, wherein the predicted embedding is generated based on an effect of a respective candidate molecule on the biological sample;
However, Ando teaches: a plurality of candidate molecules that can be used as labels or tags in life-science microscopy, (Ando – [Col. 14. ll. 35-40] Only one of the candidate phenotypes 322 is labelled in FIG. 3C, in order to avoid cluttering the figure [Col. 23, ll. 40-45] For example, the cells 512, or part of the cells 512, such as the nuclei 514, may be dyed with a fluorescent compound that fluoresces at a specific wavelength range.)
wherein the predicted embedding is generated based on an effect of a respective candidate molecule on the biological sample; (Ando – Fig. 11, [Col. 42 ll. 32-40] At block 1108, the method 1100 includes recording candidate images of each of the candidate biological cells, each having a respective candidate phenotype arising in response to the candidate treatment regimen being applied. [Col. 43 ll. 16-24] At block 1112, the method 1100 includes obtaining, by the computing device, a semantic embedding associated with the target image. Likewise, at block 1114, the method 1100 includes obtaining, by the computing device for each candidate image, a semantic embedding associated with the respective candidate image. deep metric network model to identify similarities and differences between images.)
and outputting the selected one or more candidate molecules as validated labels or tags for use in imaging a biological sample with the optical imaging device. (Ando – [Col. 42 ll. 3-10] applying candidate treatment regimens may include various candidate treatment compounds to candidate biological cells; [Col. 43 ll. 65-67] [Col. 44 ll. 15] selecting a preferred treatment regimen among a variety of candidate treatment regimen among variety of candidate treatment regimens based on the similarity scores.)
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teaching of KAPPEL, and Ando as each invention in the same field of image processing of biological molecules. One of ordinary skill in the art would have been motivated to make these modification to improve the level of effective treatment of disease in patients.
Regarding dependent claim 2, depends on claim 1, KAPPEL teaches: wherein the target property is one of a spatial distribution, a spatio-temporal distribution, an intensity distribution, and a cell fate. (KAPPEL − [pdf page 35, ll. 18-19] Images may encompass multidimensional image tensors with three spatial dimensional, a time dimensional)
Regarding dependent claim 3, depends on claim 1, KAPPEL does not explicitly teach: transporting or sequestering one or more payloads
However, Ando teaches: wherein the candidate molecules are molecules for transporting or sequestering one or more payloads to a target region. (Ando − [col. 44, ll. 56-58] another example embodiment, certain cellular organelles may be targeted by one or more fluorophores.)
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teaching of KAPPEL, and Ando as each invention in the same field of image processing of biological molecules. One of ordinary skill in the art would have been motivated to make these modification to improve the level of effective treatment of disease in patients.
Regarding dependent claim 4, depends on claim 3, KAPPEL does not explicitly teach: transporting or sequestering one or more payloads
However, Ando teaches: wherein the one or more payloads comprise one or more of a fluorophore, a drug for influencing gene expression, a drug for binding as a ligand to a receptor or an enzyme, a drug acting as an allosteric regulator of an enzyme, and a drug competing for a binding site as an antagonist. (Ando − [col. 44, ll. 56-58] another example embodiment, certain cellular organelles may be targeted by one or more fluorophores.)
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teaching of KAPPEL, and Ando as each invention in the same field of image processing of biological molecules. One of ordinary skill in the art would have been motivated to make these modification to improve the level of effective treatment of disease in patients.
Regarding dependent claim 5, depends on claim 1, KAPPEL teaches: wherein the method comprises determining one or more imaging parameters based on the target property of the candidate molecule, and obtaining the one or more images based on the determined one or more imaging parameters, and/or wherein the method comprises determining, for each candidate molecule, one or more parameters related to sample preparation for preparing the sample with the respective candidate molecule and outputting the one or more parameters related to sample preparation. (KAPPEL − [pdf page 41 ll. 21-24] The visual recognition machine-learning algorithm may be trained by adjusting parameters of the visual recognition machine-learning algorithm.)
Regarding dependent claim 6, depends on claim 1, KAPPEL teaches: wherein the machine-learning model is trained to process a set of images showing the biological sample at two or more points in time to output the predicted embedding of the candidate molecule. (KAPPEL − [pdf page 8 ll. 4-5] The biology-related image-based input data set may be an XY pixel map, volumetric data (XYZ), time series data (XY+T) or combinations) time series data used for predicted embedding.
Regarding dependent claim 7, depends on claim 1, KAPPEL teaches: wherein the method comprises training the machine-learning model, using supervised learning and using a set of training data, to output the predicted embedding of the candidate molecule based on the one or more images or the information derived from the one or more images. (KAPPEL − [pdf page 20, ll. 10-25] The visual model 220 may have been pre-trained to predict these semantic embedding during training. [pdf page 36 ll. 18-19] A visual model trained to predict semantic token embeddings from images may convert a query image to its related semantic embedding. [pdf page 42 ll. 19-20] Embodiments may be based on using a machine-learning model or machine-learning algorithm. [pdf page 43 ll. 5-15] Machine-learning models may be trained using training input data. training method called "supervised learning".)
Regarding dependent claim 8, depends on claim 1, KAPPEL teaches: generating, using a second machine-learning model, a plurality of embeddings of molecules, and selecting the plurality of candidate molecules and corresponding embeddings from the plurality of embeddings of molecules according to a selection criterion. (KAPPEL – [pdf page 20-21] a plurality of trained visual recognition machine-learning algorithms [pdf page 34 ll. 5-10] the method 1100 comprises selecting 1130 a second high-dimensional representation from the plurality of second high-dimensional representation based on a comparison of the first high-dimensional representation with each or a subset of the second high-dimensional representation of the plurality of second high-dimensional representations.)
Regarding dependent claim 9, depends on claim 8, KAPPEL teaches: wherein the method comprises comparing the embeddings of the molecules with one or more embeddings of one or more molecules with respect to the target property, and selecting the plurality of candidate molecules and corresponding embeddings based on the comparison, or wherein the second machine-learning model has an output indicating the molecule with respect to the target property, with the selection of the plurality of candidate molecules and corresponding embeddings being based on the output indicating the molecule with respect to the target property. (KAPPEL − [pdf page 20-21] a plurality of trained visual recognition machine-learning algorithms [pdf page 7, ll. 25-30] The biology-related image-based search data 103 may be image data ( e.g. pixel data of an image) of an image of a biological structure comprising…, a protein or a protein sequence, a biological molecule; [pdf page 23 ll. 24-30] The system 400 may be configured to determine a microscope target position based on the selected second high-dimensional representation.)
KAPPEL does not explicitly teach: quality of the molecule
However, Ando teaches: wherein the method comprises comparing the embeddings of the molecules with one or more embeddings of one or more molecules having a desired quality with respect to the target property, and selecting the plurality of candidate molecules and corresponding embeddings based on the comparison, or wherein the second machine-learning model has an output indicating a quality of the molecule with respect to the target property, with the selection of the plurality of candidate molecules and corresponding embeddings being based on the output indicating the quality of the molecule with respect to the target property. (Ando − [Col. 7 ll. 20-30] For example, a semantic embedding may have multiple dimensions (e.g., 64 dimensions) that correspond to various qualities of an image (e.g., shapes, textures, image content, relative sizes of objects, and perspective). The dimensions of the semantic embeddings could be either human-interpretable or non-human interpretable. In some embodiments, for example, one or more of the dimensions may be superpositions of human-interpretable features.)
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teaching of KAPPEL, and Ando as each invention in the same field of image processing of biological molecules. One of ordinary skill in the art would have been motivated to make these modification to improve the level of effective treatment of disease in patients.
Regarding dependent claim 10, depends on claim 8, KAPPEL teaches: wherein the plurality of embeddings are generated autoregressively, by using the second machine-learning model to select, based on a starter token representing a portion of a molecule, one or more additional tokens representing one or more additional portions of the molecule, and generating the respective embeddings by combining the respective starter tokens with the corresponding one or more additional tokens. (KAPPEL − [pdf page 20-21] a plurality of trained visual recognition machine-learning algorithms [pdf page 20, ll. 10-25] The visual model 220 may have been pre-trained to predict these semantic embedding during training. [pdf page 36 ll. 18-19] A visual model trained to predict semantic token embeddings from images may convert a query image to its related semantic embedding. [pdf page 42 ll. 19-20] Embodiments may be based on using a machine-learning model or machine-learning algorithm.)
Regarding dependent claim 11, depends on claim 8, KAPPEL teaches: wherein the method comprises training the second machine-learning model using the corpus of tokenized representations of different molecules, with the training being performed using the denoising target and/or with the second machine-learning model being trained to predict the one or more additional tokens given the one or more starter tokens. (KAPPEL − [pdf page 20-21] a plurality of trained visual recognition machine-learning algorithms [pdf page 20, ll. 10-25] The visual model 220 may have been pre-trained to predict these semantic embedding during training. [pdf page 36 ll. 18-19] A visual model trained to predict semantic token embeddings from images may convert a query image to its related semantic embedding. [pdf page 42 ll. 19-20] Embodiments may be based on using a machine-learning model or machine-learning algorithm.)
Regarding dependent claim 12, depends on claim 8, KAPPEL teaches: wherein the second machine-learning model is trained to output an embedding of a molecule based on an input comprising a representation of at least a portion of the molecule. ([pdf page 20-21] a plurality of trained visual recognition machine-learning algorithms [pdf page 20, ll. 10-25] The visual model 220 may have been pre-trained to predict these semantic embedding during training. [pdf page 36 ll. 18-19] A visual model trained to predict semantic token embeddings from images may convert a query image to its related semantic embedding. [pdf page 42 ll. 19-20] Embodiments may be based on using a machine-learning model or machine-learning algorithm.)
Regarding independent claim 13, KAPPEL teaches: A method for training a machine-learning model, the method comprising:
obtaining a set of training data, the set of training data comprising a plurality of sets of training samples, (KAPPEL − [pdf page 38, ll. 25-27] The system may be configured to receive biology-related language-based input training data.)
each training sample comprising, as training input data, a) one or more images showing a visual representation of a target property exhibited by a candidate molecule in a biological sample or b) information derived from the one or more images, and, as desired training output, an embedding of the molecule; (KAPPEL − [pdf page 7, ll. 25-30] The biology-related image-based search data 103 may be image data ( e.g. pixel data of an image) of an image of a biological structure comprising…, a protein or a protein sequence, a biological molecule; [pdf page 23 ll. 24-30] The system 400 may be configured to determine a microscope target position based on the selected second high-dimensional representation.)
derived from a chemical or sequence representation of the molecule; (KAPPEL – [pdf page 12 ll. 18-28] Each biology related language based input data set… may be nucleotide sequence, a protein sequence, a description of a biological molecule or biological structure…; [pdf page 18 ll. 15-25] visual model 220… was trained on the semantic embedding of a textual model, which was trained on a large body of protein sequences… nucleotide sequence)
and training the machine-learning model, using supervised learning and using the set of training data, to output a predicted embedding of the candidate molecule based on the one or more images or the information derived from the one or more images. (KAPPEL − [pdf page 20, ll. 10-25] The visual model 220 may have been pre-trained to predict these semantic embedding during training. [pdf page 36 ll. 18-19] A visual model trained to predict semantic token embeddings from images may convert a query image to its related semantic embedding. [pdf page 42 ll. 19-20] Embodiments may be based on using a machine-learning model or machine-learning algorithm. [pdf page 43 ll. 5-15] Machine-learning models may be trained using training input data. training method called "supervised learning".)
KAPPEL does not explicitly teach: wherein the candidate molecule is a molecule that can be used as label or tag in life-science microscopy, wherein the predicted embedding is generated based on an effect of a respective candidate molecule on the biological sample;
However, Ando teaches: wherein the candidate molecule is a molecule that can be used as label or tag in life-science microscopy(Ando – [Col. 14. ll. 35-40] Only one of the candidate phenotypes 322 is labelled in FIG. 3C, in order to avoid cluttering the figure [Col. 23, ll. 40-45] For example, the cells 512, or part of the cells 512, such as the nuclei 514, may be dyed with a fluorescent compound that fluoresces at a specific wavelength range.)
wherein the predicted embedding is generated based on an effect of the candidate molecule on the biological sample. (Ando – Fig. 11, [Col. 42 ll. 32-40] At block 1108, the method 1100 includes recording candidate images of each of the candidate biological cells, each having a respective candidate phenotype arising in response to the candidate treatment regimen being applied. [Col. 43 ll. 16-24] At block 1112, the method 1100 includes obtaining, by the computing device, a semantic embedding associated with the target image. Likewise, at block 1114, the method 1100 includes obtaining, by the computing device for each candidate image, a semantic embedding associated with the respective candidate image. deep metric network model to identify similarities and differences between images.)
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the teaching of KAPPEL, and Ando as each invention in the same field of image processing of biological molecules. One of ordinary skill in the art would have been motivated to make these modification to improve the level of effective treatment of disease in patients.
Regarding dependent claim 14, depends on claim 1, KAPPEL teaches: A system comprising one or more processors and one or more storage devices, wherein the system is configured to perform the method of claim 1. (KAPPEL − [pdf page 38 ll. 24-25] A system for training machine-learning algorithms for processing biology-related data may comprise one or more processors and one or more storage devices.)
Regarding dependent claim 15, depends on claim 1, KAPPEL teaches: A non-transitory machine-readable storage medium including a program code configured to perform the method according to claim 1 when the program code is executed on a processor. (KAPPEL − [pdf page 45 ll. 18-20] The implementation can be performed using a non-transitory storage medium such as a digital storage medium)
Regarding dependent claim 16, depends on claim 13, KAPPEL teaches: A system comprising one or more processors and one or more storage devices, wherein the system is configured to perform the method of claim 13. (KAPPEL − [pdf page 38 ll. 24-25] A system for training machine-learning algorithms for processing biology-related data may comprise one or more processors and one or more storage devices.)
Regarding dependent claim 17, depends on claim 13, KAPPEL teaches: A non-transitory machine-readable storage medium including a program code configured to perform the method according to claim 13 when the program code is executed on a processor. (KAPPEL − [pdf page 45 ll. 18-20] The implementation can be performed using a non-transitory storage medium such as a digital storage medium)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARL E BARNES JR whose telephone number is (571)270-3395. The examiner can normally be reached Monday-Friday 9am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Hong can be reached at (571) 272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CARL E BARNES JR/Examiner, Art Unit 2178
/STEPHEN S HONG/Supervisory Patent Examiner, Art Unit 2178
Prosecution Insights