Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
Applicant may wish to submit an Information Disclosure Statement for certain references cited in Kim D, Paik S, Park J, Hwang SJ, Onoda S, Ohshima T, Kim DH, Lee SY. Classification of Single‐Photon Emitters in Confocal Fluorescence Microscope Images by Deep Convolutional Neural Networks. Advanced Quantum Technologies. 2024 Nov;7(11):2400173.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. Specifically, the title must distinguish from the inventors’ other applications and patents.
The abstract of the disclosure is objected to because it does not “enable the Office and the public generally to determine quickly from a cursory inspection the nature and gist of the technical disclosure.” 37 CFR 1.72(b). Specifically, the abstract suffers from the issues in the 112 rejections for claim 1.
A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Drawings
The drawings are objected to because they are not clear enough. Submitting larger drawings with better resolution is expected to overcome this objection.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 (all claims) are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1, 15, and 16 recite “trained artificial neural network model,” but this is unlimited functional claiming given the wide array of different neural network models and different ways that they could be trained. MPEP 2173.05(g). Limiting the claim to a known architecture and training, such as “is a convolutional neural network trained on single- photon point light source images” overcomes this rejection. Claim 9 does not overcome this rejection because it recites “comprises” a CNN.
Dependent claims are likewise rejected.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 (all claims) are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 15, and 16 recite “determination information expected values,” but this is new terminology. MPEP 2173.05(a). Specification [0146] states “The determination information expected value is an expected probability including information that may determine whether an emitter is a single-photon emitter or a non single- photon emitter.” Note that US practice does not import limitations from the specification. Additionally, given this meaning, the last claim step (i.e., “determine whether …”) appears trivial at best, which is a surprising situation.
Claims 2, 3, 10, and 19 recite “and/or,” but this is exemplary claim language because “and” is an example of an open-ended “or.” MPEP 2173.05(d).
Claims 4 and 7 recite “one single-photon point light source cluster,” but it is unclear what the cluster is of. Is the intent that this is a cluster of pixels that have detected light?
Claims 4, 7, 8, and 16 recite “small” or “large,” but these are relative terms without sufficient guidance. MPEP 2173.05(b).
Claims 4, 7, and 16 recite “a photon count rate of a preset standard,” but it is not clear what the plain meaning of this is.
Claims 4, 7, and 16 recite “within a preset area,” but it is unclear how to determine where the preset area is. MPEP 2173.05(b)(IV). In particular, wouldn’t the image itself be a preset area?
Claim 7 recites “performing normalization to a normally distributed value,” but does not specify what is being normalized, or what it is being normalized to. Further, it is unclear how to tell if a single value (or a small number of values) is normally distributed or not.
The image of claim 8 is best understood as a product-by-process (because the claim steps are not recited as an active part of a method). However, it is unclear what the implied structure is, and thus this claim is indefinite. MPEP 2113(I).
Claim 8 recites “a pixel corresponding to a local maximum of a photon count rate,” but this lacks a plain meaning. Is the intent to select the pixel where the most photos arrived?
Claims 11 and 20 recite “distinguished according to a correlation,” but this lacks a plain meaning.
Dependent claims are likewise rejected.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-11 and 15 (all claims except those rejected under 103, below) are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kudyshev ZA, Bogdanov SI, Isacsson T, Kildishev AV, Boltasseva A, Shalaev VM. Rapid classification of quantum sources enabled by machine learning. Advanced Quantum Technologies. 2020 Oct;3(10):2000067 (“Kudyshev”). Kudyshev was retrieved from https://arxiv.org/pdf/1908.08577
1. A method for determining a single-photon emitter based on deep learning, performed by at least one electronic device, the method comprising: (Kudyshev, title, “Rapid classification of quantum sources enabled by machine learning”)
acquiring input data based on a single-photon point light source image; (Kudyshev, abstract, “solid-state single-photon emitters by the means of optical characterization”)
generating determination information expected values by inputting the input data to a trained artificial neural network model; and (Kudyshev, abstract, “We have implemented supervised machine learning-based classification of quantum emitters as “single” or “not-single” based on their sparse autocorrelation data.”)
determining whether an emitter providing the single-photon point light source image is a single-photon emitter or a non- single-photon emitter, based on the determination information expected values. (Kudyshev, abstract, “We have implemented supervised machine learning-based classification of quantum emitters as “single” or “not-single” based on their sparse autocorrelation data.”)
2. The method of claim 1, wherein the single-photon point light source image includes an image acquired using confocal fluorescence microscopy, scanning tunneling microscopy (STM), and/or nanoscale-magnetic resonance imaging (nano-MRI). (Kudyshev, p. 23, top, “All the optical characterization was performed using a custom-made scanning confocal microscope”)
3. The method of claim 1, wherein the single-photon point light source include at least one of isolated single atoms, single molecules, single dye molecules, and/or point defects in solids. (Kudyshev, p. 15, bottom, “The proposed approach could also have a strong impact on single-molecule spectroscopy.”)
4. The method of claim 1, wherein the input data is generated based on an image of a small area having a photon count rate of a preset standard within a preset area in a large-scan image including one single-photon point light source cluster. (Kudyshev, Fig. 3. See 3a showing that he laser only goes to a small part of the nanodiamonds and 3b, describing the circle radius as “sparse datasets.” Compare also Fig. 3a to Specification, Fig. 2A. The present standard is arbitrarily chosen such that the photon count rate is met.)
5. The method of claim 1, further comprising training the artificial neural network model, wherein the training of the artificial neural network model comprises constructing image training data based on the single- photon point light source image. (Kudyshev, Fig. 1)
6. The method of claim 5, wherein the training of the artificial neural network model further comprises constructing laser power training data based on laser power irradiated on a target sample. (Kudyshev, Fig. 1a. See also Fig. 3a that explicitly identifies the beam as a laser.)
7. The method of claim 5, wherein the constructing of the image training data comprises:
generating an image of a small area having a photon count rate of a preset standard within a preset area in a large-scan image including one single-photon point light source cluster; (Kudyshev, Fig. 3. See 3a showing that he laser only goes to a small part of the nanodiamonds and 3b, describing the circle radius as “sparse datasets.” Compare also Fig. 3a to Specification, Fig. 2A. The present standard is arbitrarily chosen such that the photon count rate is met.)
removing background noise from the generated image; (Kudyshev, p. 23, bottom “The results
of individual measurements were summed up to obtain low-noise autocorrelation data.”)
determining and labeling whether the emitter is the single- photon emitter or the non-single-photon emitter; and (Kudyshev, Fig. 1 caption, “The trained model classifies the previously unencountered emission sources as “single” and “not-single” emitters based on sparse autocorrelation data.”)
performing normalization to a normally distributed value. (Kudyshev, Fig. 1 caption, “The trained model classifies the previously unencountered emission sources as “single” and “not-single” emitters based on sparse autocorrelation data.” Kudyshev’s “single” and “not-single” teach the claimed normally distributed values.)
8. The method of claim 7, wherein the generated image includes an image focused on an individual emitter, and the image focused on the individual emitter is an image of a small area raster-scanned again based on a pixel corresponding to a local maximum of a photon count rate of the individual emitter in a raster-scanned image for the individual emitter within the one single-photon point light source cluster. (Kudyshev, p. 23, top, “All the optical characterization was performed using a custom-made scanning confocal microscope with a 50 μm pinhole” Kudyshev’s pinhole teaches the claimed raster scanning, see specification [0085].)
9. The method of claim 1, wherein the trained artificial neural network model comprises a convolutional neural network (CNN)-based deep learning model. (Kudyshev, Fig. 2 caption, “(a) Accuracy distribution of the CNN Classifier”)
10. The method of claim 1, wherein the trained artificial neural network model comprises a convolutional layer, a pooling layer, and/or a fully-connected layer. (Kudyshev, p. 8, bottom, “The CNN binary classifier consists of one input layer, three hidden convolutional layers, one max pooling layer followed by two fully connected layers.”)
Examiner Note: While claim 6 has been mapped according to the literal meaning (as best understood), Kudyshev Fig. S1b (p. 26) appears to be more conceptually similar than what is mapped.
11. The method of claim 6, wherein the trained artificial neural network model comprises:
a first learning model that is distinguished according to a correlation between the input data and the laser power irradiated on the target sample; and (Kudyshev, p. 8, bottom, “The CNN binary classifier consists of one input layer, three hidden convolutional layers, one max pooling layer followed by two fully connected layers.” The layers are within the BRI of model, and the input layer teaches the claimed correlation.)
a second learning model that is not distinguished according to the correlation between the input data and the laser power irradiated on the target sample. (Kudyshev, p. 8, bottom, “The CNN binary classifier consists of one input layer, three hidden convolutional layers, one max pooling layer followed by two fully connected layers.” The layers are within the BRI of model, and the pooling layer lacks the claimed correlation.)
Claim 15 is rejected as per claim 1.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 12, 13, 14, and 16-20 (all claims not rejected under 102, above) are rejected under 35 U.S.C. 103 as being unpatentable over Kudyshev ZA, Bogdanov SI, Isacsson T, Kildishev AV, Boltasseva A, Shalaev VM. Rapid classification of quantum sources enabled by machine learning. Advanced Quantum Technologies. 2020 Oct;3(10):2000067 (“Kudyshev”) in view of Peyton T, Carpenter JL, Camp S, Fadul M, Dean B, Reising DR, Loveless TD. Supervised deep learning and classification of single-event transients. IEEE Transactions on Nuclear Science. 2023 Apr 24;70(8):1740-6. (“Peyton”). Peyton was retrieved from https://par.nsf.gov/servlets/purl/10426772.
12. Kudyshev teaches the method of claim 6, (See the mapping of claim 6)
Kudyshev is not relied on for the below claim language.
However, Peyton teaches wherein the training of the artificial neural network model comprises constructing first laser power training data learned by setting the laser power irradiated on the target sample to first laser power and second laser power training data learned by setting the laser power irradiated on the target sample to second laser power, and the first laser power and the second laser power have different power values. (Peyton, Fig. 3. Peyton, Fig. 3 shows varying power, and the claim does not require the data sets to be kept separate.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Peyton to the teachings of Kudyshev such that Peyton’s measurements and validation techniques are used with Kudyshev for the purpose of implementation details (note that Peyton is a student publication, and thus demonstrates what would have been understood by one of ordinary skill)
Based on the above, this is an example of “combining prior art elements according to known methods to yield predictable results.” MPEP 2143.
13. The method of claim 1, wherein the trained artificial neural network model is trained using a binary cross-entropy loss function. (Peyton, Fig. 7 caption, “Cross entropy loss of model over 75 epochs.”)
Peyton and Kudyshev are combined as per claim 12.
14. The method of claim 1, wherein the trained artificial neural network model determines whether the determination information expected values are appropriate by using K-fold cross validation, where k is a natural number greater than or equal to 3, and the K-fold cross validation is performed by randomly classifying training data into k-folds and using k-1 folds as a training set and the remaining one fold as a testing set. (Peyton, Fig. 4 caption, “An example of k-fold validation with 5 folds.”)
Peyton and Kudyshev are combined as per claim 12.
Claims 16-20 are rejected as per their counterpart claims. Note that there may be more than one counterpart claim, e.g., claim 17 combines limitations from claims 12 and 13.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 12561951 B2, titled “System For Low-photon-count Visual Object Detection And Classification,” see [0054] “For instance, after a single photon signature is received”
US 12159369 B2, titled “Machine Learning Assisted Super Resolution Microscopy”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID ORANGE whose telephone number is (571)270-1799. The examiner can normally be reached Mon-Fri, 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID ORANGE/Primary Examiner, Art Unit 2663
Prosecution Insights