DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The reply filed on 31 March 2026 has been entered. Applicant’s arguments with respect to claims 1-2 and 4-16 have been considered but are moot in view of new ground(s) of rejection caused by the amendments.
Claims 1-2 and 4-16are pending in this application and have been considered below. Claim 3 is canceled by the applicant.
Priority
Receipt is acknowledged that application is a National Stage application of PCT EP2022/063463. Priority to DE10 2021 114 349.7 with a priority date of 2 June 2021 is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The IDS dated 30 November 2023 that has been previously considered remains placed in the application file.
Specification - Drawings
Corrected drawings were submitted. The objection to the drawings is withdrawn.
Claim Interpretation
Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009).
Claims 2, 7, 8, 9, 12 recite “or” or “one or more of the following” or “from the following group.” Since all of the above are disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Claim Interpretation
Claim 14 has been amended. The interpretation of claim 14 under 35 USC 112(f) is withdrawn.
1st Claim Rejections - 35 USC § 101
Claim 1 has been amended. The rejection of claims 1-2 and 4-16 under 35 USC 101 as not being a statutory category is withdrawn.
2nd Claim Rejections - 35 USC § 101
Claim 1 has been amended. The rejection of claims 1-2 and 4-16 under 35 USC 101 as being an abstract idea is withdrawn, as using the output images to train a neural network is a concrete application.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2 and 4-16 (all claims) are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2025 0259462 A1, (Ozcan et al.) in view of US Patent Publication 2023 0058876 A1, (Zhang et al.). The references are listed in a PTO-892 from the Office Action in which they are first used.
[AltContent: textbox (Ozcan et al. Fig. 8, showing using a neural network to stain images in a realistic manner so the images can be used for network training.)]
PNG
media_image1.png
666
968
media_image1.png
Greyscale
Claim 1
Regarding Claim 1, Ozcan et al. teach a computer-implemented method for acquiring images for training data to train a statistical model (130) by machine learning for image processing in microscopy, wherein the training data comprise pairs of input images (110) and output images (120) from image processing ("the technical field relates to microscopy methods and systems that utilize deep neural network learning for digitally or virtually staining of images of unstained or unlabeled tissue," paragraph [0002]), the method comprising
acquiring at least one image ("An out-of-focus or in-focus image of the sample is obtained using the incoherent microscope ," paragraph [0007]),
analyzing the at least one image according to predetermined criteria ("The standard auto-focusing methods require the acquisition of multiple images, to be used with the autofocusing algorithm, which selects the most in focus image, according to a pre-defined criterion," paragraph [0030]),
determining acquisition parameters for acquiring output images (120) on the basis of analysis results ("The standard auto-focusing methods require the acquisition of multiple images, to be used with the autofocusing algorithm, which selects the most in focus image, according to a pre-defined criterion," paragraph [0030]),
acquiring output images (120) on the basis of the determined acquisition parameters ("standard auto-focusing methods require the acquisition of multiple images, to be used with the autofocusing algorithm, which selects the most in focus image, according to a pre-defined criterion," paragraph [0030] and Fig. 13).
Ozcan et al. is not relied upon to explicitly teach all of using the output images acquired on the basis of the determined acquisition parameters to train the model by machine learning.
However, Zhang et al. teach using the output images (120) acquired on the basis of the determined acquisition parameters to train the model (130) by machine learning [AltContent: textbox (Zhang et al. Fig. 6, showing using modified images as training images.)]
PNG
media_image2.png
796
818
media_image2.png
Greyscale
for image processing in microscopy ("perform simulation processing on a standard image based on the light field brightness variation range and the light field variation parameters to obtain a simulated image and a light field image matched with the simulated image; and take a set of the simulated image and the light field image matched with the simulated image as an image training sample set matched with the use environment of the image processing model" paragraph [0097]).
Therefore, taking the teachings of Ozcan et al. and Zhang et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify “Method and System for Digital Staining of Microscopy images using Deep Learning” as taught by Ozcan et al. to use “Image processing method and Apparatus Based on Image Processing Model” as taught by Zhang et al. The suggestion/motivation for doing so would have been that, “in related art, the acquired images are usually subject to light field correction due to uneven ambient light source and jitter in an imaging process of a camera. In order to acquire a clear image, a plurality of images of the same target often need to be collected repeatedly. The light field correction may be realized only after determining corresponding equations of the images, to not only increase photographing costs and image storage costs, but also increase time of light field correction, thereby wasting a lot of storage space and computing resources.” as noted by the Zhang et al. disclosure in paragraph [0004], which also motivates combination because the combination would predictably have a higher productivity as there is a reasonable expectation that additional training images will be needed to accelerate recognition; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
The rejection of method claim 1 above applies mutatis mutandis to the corresponding limitations of apparatus claim 14 while noting that the rejection above cites to both device and method disclosures. Claim 14 is mapped below for clarity of the record and to specify any new limitations not included in claim 1.
Claim 2
Regarding claim 2, Ozcan et al. teach the method according to claim 1,wherein the image processing is virtual staining, noise reduction, super resolution, deconvolution, compressed sensing, or another type of image optimization ("The trained, deep neural network 10 in response to the input image 20 outputs or generates a digitally stained or labelled output image 40," paragraph [0050] where digitally stained is virtually stained).
Claim 4
Regarding claim 4, Ozcan et al. teach the method according to claim 1,wherein the training of the model (130) is an adjustment of the model (130) for image processing ("Typically, an artificial neural network model 10 converges after -30 hours on two Nvidia 1080Ti GPUs," paragraph [0055] where converges is an adjustment of the model).
Claim 5
Regarding claim 5, Ozcan et al. teach the method according to claim 1,wherein the analysis and determination steps of the method are performed by a further statistical model of the machine learning ("The last layer is a convolutional layer (CL) mapping 32 channels into 3 channels, represented by the YcbCr color map. Both the generator and the discriminator networks were trained with a patch size of 256x256 pixels," paragraph [0105] where a discriminator network is an analysis model and the generator network is an determination network).
Claim 6
Regarding claim 6, Ozcan et al. teach the method according to claim 1,wherein the statistical model (13) and/or the further statistical model is a neural network ("The GAN-based neural network 10 is composed of two deep neural networks, a generator network (G) and a discriminator network (D)," paragraph [0116]).
Claim 7
Regarding claim 7, Ozcan et al. teach the method according to claim 1,wherein the at least one image to be analyzed is one or more of the following:
an overview image, comprising one or more input (110) or output (120) images at lower magnification than the input (110) and/or output (120) images ("the bright-field microscope images 48 were down-sampled to 75.85% of their original size so that they match with the lower magnification images," paragraph [0098]),
one or more of the input images (110) ("An out-of-focus or in-focus image of the sample is obtained using the incoherent microscope ," paragraph [0007]).
Claim 8
Regarding claim 8, Ozcan et al. teach the method according to claim 1,wherein the acquisition parameters are one or more of the following:
depth plane of samples ("the deep neural network 10a can be used to refocus aberrated images from a single defocused image, as demonstrated in FIG. 13, in contrast to standard autofocusing techniques, which require the acquisition of multiple images through multiple depth planes," paragraph [0059]).
Claim 9
Regarding claim 9, Ozcan et al. teach the method according to claim 1,wherein the predetermined criteria are one or more of the following:
results of a detection of anomalies and/or novelties ("standard auto-focusing methods require the acquisition of multiple images, to be used with the autofocusing algorithm, which selects the most in focus image, according to a pre-defined criterion," paragraph [0030] where focus is the anomaly or novelty).
Claim 10
Regarding claim 10, Ozcan et al. teach the method according to claim 1,wherein the predetermined criteria are selected by the model (130) ("standard auto-focusing methods require the acquisition of multiple images, to be used with the autofocusing algorithm, which selects the most in focus image, according to a pre-defined criterion," paragraph [0030]).
Claim 11
Regarding claim 11, Ozcan et al. teach the method according to claim 1,wherein the method further comprises a determination of acquisition parameters for the acquisition of additional input images (110) on the basis of the results of the analysis, and an acquisition of the additional input images (110) on the basis of the determined acquisition parameters ("After this initial training phase, the output images 40 of each sample in the available image set can be screened against their corresponding brightfield images 48 to set a more refined threshold to reject some additional images and further clean the training/ validation image set. With a few iterations of this process, one can, not only further refine the image set, but also improve the performance of the final trained deep neural network 10," paragraph [0087]).
Claim 12
Regarding claim 12, Ozcan et al. teach the method according to claim 1,wherein one of the input images (110) and one of the output images (120) are assigned to one another and show the same,
wherein the input (110) and output (120) images differ in the acquisition method and/or acquisition contrast,
wherein the different acquisition contrasts are from the following group: non-fluorescent contrast, fluorescent contrast, color contrast, phase contrast, differential interference contrast, electron microscopy and x-ray microscopy ("As explained herein, the input image 20 is a fluorescence image 20 of a sample 22 (such as tissue in one embodiment) that is not stained or labeled with a fluorescent stain or label. Namely, the input image 20 is an autofluorescence image 20 of the sample 22 in which the fluorescent light that is emitted by the sample 22 is the result of one or more endogenous fluorophores or other endogenous emitters of frequency shifted light contained therein," paragraph [0045]) and/or
wherein the acquisition methods are from the following group: bright field method, wide field method, dark field method, phase contrast method, polarization method, differential interference contrast method, incident light microscopy method, digital contrast method, electron microscopy method and x-ray microscopy method ("In some embodiments, the input image 20 ( e.g., the raw fluorescent image) is subject to one or more linear or non-linear pre-processing operations selected from contrast enhancement, contrast reversal, image filtering," paragraph [0045]).
Claim 13
Regarding claim 13, Ozcan et al. teach the method according to claim 1,wherein the method comprises the acquisition of further images, input images (110) and/or output images (120) ("standard auto-focusing methods require the acquisition of multiple images, to be used with the autofocusing algorithm, which selects the most in focus image, according to a pre-defined criterion," paragraph [0030] and Fig. 13).
Claim 14
Regarding claim 14, Ozcan et al. teach a device for acquiring images for training data to train a statistical model (130) by machine learning for image processing in microscopy, wherein the training data comprise pairs of input images (110) and output images (120) of image processing, wherein the device is configured to perform the method according to claim 1 and wherein the device comprises:
a microscope which is configured to acquire images ("An out-of-focus or in-focus image of the sample is obtained using the incoherent microscope ," paragraph [0007]),
a computer processor which is configured to analyze images according to predetermined criteria, and to determine acquisition parameters for the acquisition of output images (120) on the basis of analysis results ("a desktop computer with an Intel Xeon W-2195 CPU at 2.30 GHz and 256 GB RAM, running a Microsoft Windows 10 operating system. Network training and testing were performed using four NVIDIA Geforce RTX 2080 Ti GPUs," paragraph[ 0150]), and
an image generator comprising a machine learning model which is configured to acquire the output images (120) on the basis of the determined acquisition parameters ("For example, while various embodiments have been described as generating digitally/virtually stained microscopic images of label-free or unstained samples," paragraph [0168]).
Claim 15
Regarding claim 15, Ozcan et al. teach a computer program product with a program stored on a non-transitory medium for a data processing device, comprising software code sections for performing the steps according to claim 1 if the program is run on the data processing device ("The software may be implemented on any computing device 100," paragraph [0150]).
Claim 16
Regarding claim 16, Ozcan et al. teach the computer program product according to claim 15, wherein the computer program product comprises a non-transitory computer-readable medium upon which the software code sections are saved, wherein the program can be loaded directly into an internal memory of the data processing device ("a trained, deep neural network that is executed by image processing software using one or more processors of a computing device, wherein the trained, deep neural network is trained with a plurality of pairs of out-of-focus and/or in-focus microscopy images or image patches that are used as input images to the deep neural network," paragraph [0008]).
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
US Patent Publication 2022 0026699 A1 to Jackson et al. discloses a method for acquiring a single, in-focus two-dimensional projection image of a live, three-dimensional cell culture sample, with a fluorescence microscope. One or more long-exposure "Z-sweep" images are obtained, i.e. via a single or series of continuous acquisitions, while moving the Z-focal plane of a camera through the sample, to produce one or more two-dimensional images of fluorescence intensity integrated over the Z-dimension.
US Patent Publication 2025 0217443 A1 to Marie-Nelly et al. discloses systems and methods for training a machine learning model to generate image of biological samples, and systems and methods for generating enhanced images of biological samples. The method for training a machine learning model to generate images of biological samples may include obtaining a plurality of training images comprising a training image of a first type, and a training image of a second type.
Non Patent Publication “Deep learning-based transformation of H&E stained tissue into special stains” to Haan et al. discloses the utility of supervised learning-based computational stain transformation from H&E to different special stains (Masson’s Trichrome, periodic acid-Schiff and Jones silver stain) using tissue sections from kidney needle core biopsies.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HEATH E WELLS whose telephone number is (703)756-4696. The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Jennifer Mehmood can be reached on 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.E.W/Examiner, Art Unit 2664
Date: 28 May 2026
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights