DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/06/2024, 12/17/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Election/Restrictions
Applicant's election with traverse of group I ( Claims 1-9) in the reply filed on 04/08/2026 is acknowledged. The traversal is on the ground(s) that Non-elected Group II (claims 10-16) and Group III (Claims 17-20) is withdrawn from consideration. This is not found persuasive because
Applicant's arguments: in the Remark, filed on 04/08/2026, the applicants argued:” Claims 1-9 are hereby provisionally elected for further examination on the merits with traverse. The Examiner has indicated that there would be a serious search burden to examine the claim groups together. The only reasoning provided included citations of the preambles of each independent claim associated with each claim group and a conclusory statement that they are different embodiments. No specific claim language was discussed or cited for causing the serious search burden. MPEP 808.02 explains what is required by an Examiner to establish burden. The Examiner must show one of the following: (A) Separate classification thereof, (B) A separate status in the art when they are classifiable together, or (C) A different field of search. Without admitting or acquiescing to the statements that they are directed to different "embodiments", the term "embodiment" is not mentioned in MPEP 808 "Reasons for Insisting Upon Restriction".
Thus, the Applicant believes that the Office has not set forth the reasoning required to establish that there would be a serious search burden regarding Claims 1-20.
The examiner’s response: Applicant's election with traverse of Species 1 (Claims 1-9) in the reply filed on 04/08/2026 is acknowledged. The traversal is on the ground(s) that the invention of Species 1 (Claims 1-9), is basically the same as that of Species 2 (Claims 10-16), and Species 3 (Claims 17-20), and any prior art searched for one group is applicable to the other group. This is not found persuasive because:
The applicant is referred to MPEP 809.02 (a), which states the requirements for an election of species requirement. Note especially section (B) which states that “the species are preferably identified as the species of figures 4 (Claims 1-9), 5 (Claims 10-16), and 6 (Claims 17-20)” and that the distinguishing characteristics of the species should be states only “in the absence of distinct figures of examples”.
Should applicant traverse on the ground that the species are not patentably distinct, applicant should submit evidence or identify such evidence now of record showing the species to be obvious variants or clearly admit on the record that this is the case. In either instance, if the examiner finds one of the inventions unpatentable over the prior art, the evidence or admission may be used in a rejection under 35 U.S.C. 103(a) of the other invention.
The requirement is still deemed proper and is therefore made FINAL.
Claim Status
Claim(s) 1 and 5-9 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ye, Cassandra Tong, et al. (“Learned, uncertainty-driven adaptive acquisition for photon-efficient multiphoton microscopy.”; Ye).
Claim(s) 2-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ye, Cassandra Tong, et al. (“Learned, uncertainty-driven adaptive acquisition for photon-efficient multiphoton microscopy.”; Ye), in view of Kooijman et al (U.S. 20130015351 A1; Kooijman).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1 and 5-9 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ye, Cassandra Tong, et al. (“Learned, uncertainty-driven adaptive acquisition for photon-efficient multiphoton microscopy.”; Ye).
Regarding claim 1, Ye discloses a method for classifying microscopic components of a physical sample, (Abstract: a method to simultaneously denoise and predict pixel-wise uncertainty for multiphoton imaging measurements, improving algorithm trustworthiness and providing statistical guarantees for the deep learning predictions.”) comprising:
generating a set of regions-of-interest (ROIs) in an image representative of the physical sample, wherein the image is generated by a microscopy system using a first analysis mode; (Fig.1a “A noisy measurement is acquired with a scanning multiphoton microscope (MPM) and passed into a deep learning model that predicts a denoised image and its associated pixel-wise uncertainty. Subsequently, the top N uncertain pixels are selected for a rescan, obtaining more measurements at only the uncertain regions. As more adaptive measurements are taken, the deep learning model predicts a denoised image with lower uncertainty. Scan duration and power are minimized, limiting sample damage while maintaining high confidence in the model prediction.”; 2.4. Adaptive acquisition in microscopy: “Many of these methods rely on detecting the sample or regions of interest, either by a coarse prescan [60] or real-time specimen tracking …)
generating an initial classification for an ROI by applying a trained machine-learning model to at least a portion of the image associated with the ROI; (Fig.1 ; 3. Method : “Our denoiser, 𝐹𝜃, predicts both the denoised image and the uncertainty at each pixel ˆ x𝑇, ˆ u𝑇 = 𝐹𝜃(y0,...,y𝑇).(2) For single-image denoising, the denoiser uses a single measurement, y0. For multi-image denoising, the denoiser uses 𝑇 measurements y0, ...,y𝑇. Using multiple measurements improves the prediction and reduces uncertainty at each pixel. We can use our denoiser as is to obtain a denoised image and uncertainty estimate, or we can use the uncertainty to drive our scanning process for adaptive microscopy”; 4.3. Training Details: “Additional finetuning on our model was conducted with our SHG dataset for 77 epochs. After training, we calibrated the uncertainty interval using conformal risk control and our calibration SHG dataset, as explained in Sec. 3.2. We set 𝛼 = 0.1 to select a 90% confidence interval, finding ˆ𝜆=1.64, ˆ𝜆=1.80, ˆ𝜆=1.72 for 1, 3, and 5 images respectively. To highlight darker regions and features within our images, we display results with a gamma correction of value 2.2 for all images (measurements, denoised, ground truth), excluding uncertainty images (images with a red and blue colormap)”)
generating a confidence score associated with the initial classification; (Fig. 2. Learned Uncertainty Quantification: n. (a) During training, the modified network returns three channels: the lower uncertainty quantile, the denoised prediction, and the upper uncertainty quantile. A quantile loss function defines the loss for the upper and lower quantiles, encouraging underestimates and overestimates. (b) After training, a calibration step is needed to adjust the upper and lower bounds and provide statistical guarantees for this predicted interval.”; 1. Introduction: “To incorporate uncertainty quantification into any supervised deep denoising method, we need to make two simple modifications: we modify the network to predict a lower bound and upper bound for each image (i.e. quantile regression) … With these modifications, a denoiser will predict not only a denoised image but also a confidence interval for each pixel with a statistical guarantee that a percentage of the true values will fall within this interval.” 3.1. Learning uncertainty through quantile regression: “we use pixel-wise quantile regression as a metric for uncertainty [35]. Given a deep network, we add a modified last layer to the network that returns two additional channels: the lower bound (ulow) and the upper bound (uhigh) for each predicted pixel, Fig. 2. These bounds form a confidence interval for our image estimate and provide a notion of uncertainty at each pixel. If we want 90% coverage, i.e., a 90% probability that our confidence interval will include the true value, we can set the upper and lower bounds to be the 95% quantile and 5% quantile, respectively.”; 3.2. Calibrating Uncertainty through Conformal Risk Control: “During calibration, the lower and upper bounds, (ˆ ulow, ˆ uhigh), are scaled by ˆ𝜆 until they contain the correct fraction of ground truth pixels based on the calibration dataset. … This gauges the predictive risk associated with the chosen 𝜆 by calculating the number of pixels that are not within the confidence interval given 𝜆. The choice of ˆ𝜆 statistically guarantees that the confidence interval will cover 1 − 𝛼 fraction of future pixels,”) and
when the confidence score for an initial classification of an ROI does not satisfy a set of confidence criteria, causing the microscopy system to re-analyze at least a portion of the sample associated with the ROI using a second analysis mode different than the first analysis mode. (Fig.1(b) ; 1. Introduction: “From this confidence interval, we can identify regions of our denoised image with the highest uncertainty, which can be caused by uncertainty in the model or signal … Taking it one step further, we propose to leverage the learned uncertainty to drive adaptive acquisition: we capture more measurements of our sample only at the most uncertain regions rather than rescanning the whole sample.”; 3.3. Uncertainty-driven Adaptive acquisition: “Pixel-wise uncertainty is used to drive our adaptive microscopy scans. Given an initial measurement at time 𝑡, our model predicts the denoised image and its associated uncertainty, u𝑡. To choose which coordinate store scan at time 𝑡+1, we select coordinates within the image that have an uncertainty above the threshold 𝑢 thresh. … we only rescan the most uncertain pixels within the sample (Fig. 1b), with the uncertainty at time 𝑡 driving the adaptive scan at time 𝑡 + 1. In each subsequent pass, the model takes in the original noisy measurement and superimposed rescans to perform multi-image denoising. With each pass, our model has more data at the uncertain pixels and can improve its image prediction and confidence. This iterative process increases the number of observations at the most uncertain positions within the sample until the model’s prediction is within an acceptable uncertainty level (Fig. 1a).”; 5.3. Adaptive Sample Acquisition and Uncertainty Informed Denoising)
Regarding claim 5, Ye discloses some but not all of the ROIs are re-analyzed using the second analysis mode. (3.3. Uncertainty-driven Adaptive acquisition: “To choose which coordinates to rescan at time 𝑡+1, we select coordinates within the image that have an uncertainty above the threshold 𝑢thresh. … Thus, we only rescan the most uncertain pixels within the sample (Fig. 1b), with the uncertainty at time 𝑡 driving the adaptive scan at time 𝑡 + 1.”)
Regarding claim 6, Ye discloses an individual ROI in the image corresponds to an individual particle in the physical sample. (Fig. 1. (a) Uncertainty-based Adaptive Imaging: A noisy measurement is acquired with a scanning multiphoton microscope (MPM) and passed into a deep learning model that predicts a denoised image and its associated pixel-wise uncertainty. Subsequently, the top N uncertain pixels are selected for a rescan, obtaining more measurements at only the uncertain regions.”)
Regarding claim 7, Ye discloses further comprising: when the confidence score for an initial classification of an ROI does not satisfy the set of confidence criteria, using data generated by the re-analyzing to generate a final classification for the ROI. (Fig.1a and 3.3. Uncertainty-driven Adaptive acquisition: “To choose which coordinates to rescan at time 𝑡+1, we select coordinates within the image that have an uncertainty above the threshold 𝑢thresh. … Thus, we only rescan the most uncertain pixels within the sample (Fig. 1b), with the uncertainty at time 𝑡 driving the adaptive scan at time 𝑡 + 1. In each subsequent pass, the model takes in the original noisy measurement and superimposed rescans to perform multi-image denoising. With each pass, our model has more data at the uncertain pixels and can improve its image prediction and confidence. This iterative process increases the number of observations at the most uncertain positions within the sample until the model’s prediction is within an acceptable uncertainty level (Fig. 1a).”)
Regarding claim 8, Ye discloses further comprising: when the confidence score for an initial classification of an ROI does satisfy a set of confidence criteria, using the initial classification of the ROI as a final classification of the ROI.( (3.2. Calibrating Uncertainty through Conformal Risk Control: “During calibration, the lower and upper bounds, (ˆ ulow, ˆ uhigh), are scaled by ˆ𝜆 until they contain the correct fraction of ground truth pixels based on the calibration dataset. … This gauges the predictive risk associated with the chosen 𝜆 by calculating the number of pixels that are not within the confidence interval given 𝜆. The choice of ˆ𝜆 statistically guarantees that the confidence interval will cover 1 − 𝛼 fraction of future pixels, given that the calibration set and new test data points are exchangeable”; 3.3. Uncertainty-driven Adaptive acquisition: “To choose which coordinates to rescan at time 𝑡+1, we select coordinates within the image that have an uncertainty above the threshold 𝑢thresh. … Thus, we only rescan the most uncertain pixels within the sample (Fig. 1b), with the uncertainty at time 𝑡 driving the adaptive scan at time 𝑡 + 1. In each subsequent pass, the model takes in the original noisy measurement and superimposed rescans to perform multi-image denoising. With each pass, our model has more data at the uncertain pixels and can improve its image prediction and confidence. This iterative process increases the number of observations at the most uncertain positions within the sample until the model’s prediction is within an acceptable uncertainty level (Fig. 1a).”)
Regarding claim 9, Ye discloses further comprising: outputting a classification report that includes the final classification of individual ROIs in the set of ROIs. (Fig.1 Adaptive Scanning Pipeline and 3. Uncertainty-driven Adaptive acquisition: “To choose which coordinates to rescan at time 𝑡+1, we select coordinates within the image that have an uncertainty above the threshold 𝑢thresh. … Thus, we only rescan the most uncertain pixels within the sample (Fig. 1b), with the uncertainty at time 𝑡 driving the adaptive scan at time 𝑡 + 1. In each subsequent pass, the model takes in the original noisy measurement and superimposed rescans to perform multi-image denoising. With each pass, our model has more data at the uncertain pixels and can improve its image prediction and confidence. This iterative process increases the number of observations at the most uncertain positions within the sample until the model’s prediction is within an acceptable uncertainty level (Fig. 1a).”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ye, Cassandra Tong, et al. (“Learned, uncertainty-driven adaptive acquisition for photon-efficient multiphoton microscopy.”; Ye), in view of Kooijman et al (U.S. 20130015351 A1; Kooijman).
Regarding claim 2, Ye discloses all the claims invention except the microscopy system takes less time to image the portion of the sample associated with the ROI using the first analysis mode than using the second analysis mode.
Kooijman discloses the microscopy system takes less time to image the portion of the sample associated with the ROI using the first analysis mode than using the second analysis mode. (Fig.4 Paragraph 32: “An electron beam is directed toward a sample and scanned across regions having different characteristics, such as different mineral compositions. A first detector may provide information about contour or topography, for example, by detecting backscattered electrons, while a second detector provides information about composition, for example, by detecting characteristic x-rays. … For an image of 1k.times.1k pixels, backscattered electron data to determine a contour can therefore be acquired in about one second, whereas obtaining the compositional information from the x-ray detector can take from about fifteen minutes to a few hours. Thus, the x-ray data is about 10.sup.4 times more sparse than the data from the backscattered electron detector.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Ye by including different analysis modalities to determine properties of a sample that is taught by Kooijman, to make the invention that a method and apparatus for more rapidly acquiring information about a sample.; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the resolution of a detector that provides compositional information as well as reducing the need for exact image registration methods. (Kooijman: Paragraph 47)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 3, Ye discloses all the claims invention except the first analysis mode includes backscattered electron detection (BSED).
Kooijman discloses the first analysis mode includes backscattered electron detection (BSED). (Fig.4 and Paragraph 45: “FIGS. 3 and 4 show how the lower resolution of one technique can be improved by the higher resolution of a second technique. In the method of FIGS. 3 and 4, the higher resolution of one type of detector, the backscattered electron detector, is thus "transferred" to the other type of detector, the x-ray detector, to provide the compositional information from the first, low resolution detector, the EDS detector, at the higher resolution of the second detector, the backscattered electron detector.” ; Paragraphs 70-71)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Ye by including different analysis modalities to determine properties of a sample that is taught by Kooijman, to make the invention that a method and apparatus for more rapidly acquiring information about a sample.; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the resolution of a detector that provides compositional information as well as reducing the need for exact image registration methods. (Kooijman: Paragraph 47)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 4, Ye, as modified by Kooijman discloses all the claims invention. Kooijman further discloses the second analysis mode includes energy dispersive spectroscopy (EDS). (Fig.4 and Paragraph 45: “FIGS. 3 and 4 show how the lower resolution of one technique can be improved by the higher resolution of a second technique. In the method of FIGS. 3 and 4, the higher resolution of one type of detector, the backscattered electron detector, is thus "transferred" to the other type of detector, the x-ray detector, to provide the compositional information from the first, low resolution detector, the EDS detector, at the higher resolution of the second detector, the backscattered electron detector.”; Paragraphs 70-71).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ozcan et al (U.S. 20190333199 A1), “System and Method for Deep Learning Microscopy”, teaches about the trained deep neural network trained with a training set of images comprising co-registered pairs of high-resolution microscopy images or image patches of a sample and their corresponding low-resolution microscopy images or image patches of the same sample. A microscopy input image of a sample to be imaged is input to the trained deep neural network which rapidly outputs an output image of the sample, the output image having improved one or more of spatial resolution, depth-of-field, signal-to-noise ratio, and/or image contrast.
Kaplemko et al (U.S. 20220065804 A1), “Method of Examining a Sample Using a Charged Particle Beam Apparatus”, teaches about the method comprises the step of detecting, using a first detector, emissions of a first type from the sample in response to the charged particle beam illuminating the sample; acquiring spectral information on emissions of a second type from the sample in response to the charged particle beam illuminating the sample; providing a spectral information prediction algorithm and using said algorithm for predicting said spectral information based on detected emissions of the first type as an input parameter of said algorithm. With this it is possible to gather EDS data using only a BSE detector.
Zeineh et al (U.S. 20060159367 A1), “System And Method For Creating Variable Quality Images Of A Slide”, teaches about systems, devices and methods for creating variable quality images of a slide. It also teaches about an image verification device includes a processor having instructions which, when executed, cause the processor to: resolve whether a first image of a specimen is accepted or rejected for use in diagnosis; forward, if the first image is accepted, the first image to a diagnoser; forward, if the first image is rejected, the first image to an image refiner, the image refiner altering at least one parameter related to image capture; capture, if the first image is rejected, a second image of the specimen, with the at least one parameter altered with respect to the capture of the second image; and forward, if the second image is captured, the second image to the diagnoser.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Duy A Tran whose telephone number is (571)272-4887. The examiner can normally be reached Monday-Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ONEAL R MISTRY can be reached at (313)-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUY TRAN/ Examiner, Art Unit 2674
/ONEAL R MISTRY/ Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights