DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 12-14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite in that it fails to point out what is included or excluded by the claim language. These claims are omnibus type claims.
Claim 12 is directed to “A data processing device for computing an estimate of a first signal of a plurality of signals, the plurality of signals being contained in a digital color input image, each signal of the plurality of signals having a different ground truth spectrum, wherein the data processing device is configured to: …”, Claim 13 is directed to “An optical observation device, being configured to record …,” and Claim 14 is directed to “The optical observation device according to claim 13, wherein the data processing device comprises or consists of an embedded system of the optical observation device.” However, the bodies of these claims do not recite a limitation on the structure of the claimed devices or system. Thus it is not clear what structures are included or excluded by the claims.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 9, 11 are rejected under 35 U.S.C. 102(a) as being anticipated by US 20190271592 to Gray (“Gray”).
Regarding Claim 1: “A computer-implemented method for computing an estimate of a first signal of a plurality of signals, the plurality of signals being contained in a digital color input image, each signal of the plurality of signals having a different ground truth spectrum, the digital color input image comprising a plurality of pixels; the computer-implemented method comprising: (Note that prior art applies its methodology in a similar digital imaging context and in imaging similar fluorescence subject matter having an expected “ground truth” spectrum. See Gray, Paragraphs 3-5, 81)
extracting a subset from the plurality of pixels; (Under the broadest reasonable interpretation consistent with the specification and ordinary skill in the art, a subset can be a region of one or more pixels and/or a color or spectrum range. Prior art teaches an example separated spectral subsets: “separated in emission wavelength by 50-100 nm are chosen so that their signals can be cleanly separated with emission filters.” Gray, Paragraph 4. Also note that separation can be both by pixel and color because “each color arrives at a slightly different pixel on the detector.” Gray, Paragraph 45.)
extracting from the subset by spectral unmixing: … a preliminary estimate of the first signal as a first unmixed signal, and … a preliminary estimate of at least one further signal of the plurality of signals as at least one further unmixed signal; (Under the broadest reasonable interpretation consistent with the specification and ordinary skill in the art, the preliminary estimate of an unmixed signal can be a signal having a particular color such as a recorded color, an expected color, or an expected spectrum range. According to Specification, Page 28, an estimate may be a simple spatial or spectral partition of an image signal into pixels and/or colors.
Prior art indicates that during recording pixels can be separated by pixel color as in Gray, Paragraphs 45, 47, 38, which is similar to the examples in Specification, Paragraphs 19 and 28. Also note that colors can be spatially or spectrally separated based on known properties of the target, such as known properties of fluorophores. See Gray, Paragraph 47. See similarly in Zimmermann Figs. 1 and 2.)
computing from the subset an estimate of a dependency of the first unmixed signal on the at least one further signal as a crosstalk quantity for the subset; and (“ssd values that fall in the border region between two adjacent peaks” corresponding to a crosstalk between two adjacent colors. Gray, Paragraphs 34, 62. Also see embodiments of more complex spatial and spectral overlap in Gray, Paragraphs 70-71 and Figs. 3A, 4A, 4C, 8.)
removing the crosstalk quantity from the first unmixed signal to obtain the estimate of the first signal for the subset.” (“To reduce spectral crosstalk, localization events associated with ssd values that fall in the border region between two adjacent peaks were discarded.” Gray, Paragraphs 34, 62 and Figs. 8-9.)
Regarding Claim 2: “The computer-implemented method according to claim 1, wherein the extracting by spectral unmixing from the subset the preliminary estimate of the at least one further signal of the plurality of signals as the at least one further unmixed signal comprises:
extracting by spectral unmixing from the subset a preliminary estimate of a second signal of the plurality of signals as a second unmixed signal; (Prior art indicates that multiple signals can be separated by pixel and by color, for example where “each color arrives at a slightly different pixel on the detector” with examples of using 4 separable colors, as in Gray, Paragraphs 45, 47, 38, which is similar to the example in Specification, Paragraph 19. Also note that colors can be spatially or spectrally separated based on known properties of the target, such as known properties of fluorophores. See Gray, Paragraph 47.)
wherein the computer-implemented method further comprises: … computing an estimate of a dependency of the second unmixed signal on the first unmixed signal as a second crosstalk quantity in the subset; and (“ssd values that fall in the border region between two adjacent peaks” corresponding to a crosstalk between two adjacent colors. Gray, Paragraphs 34, 62 and Figs. 3A, 4A, 4C, 8. Also see embodiments of more complex spatial and spectral overlap in Gray, Paragraphs 70-71.)
removing the second crosstalk quantity from the second unmixed signal to obtain the estimate of the second signal in the subset.” (“To reduce spectral crosstalk, localization events associated with ssd values that fall in the border region between two adjacent peaks were discarded.” Gray, Paragraphs 34, 62 and Figs. 8-9.)
Regarding Claim 3: “The computer-implemented method according to claim 1, wherein the extracting by spectral unmixing comprises one of:
extracting by linear spectral unmixing; or (For example see: “a corresponding 'center' position for each molecule in the spectral channel was computed based on a [linear] transformation matrix, which was determined by registering positional and spectral images of gold fiduciaries” Gray, Paragraph 84.)
extracting by non-negative tensor factorization.” (Under the broadest reasonable interpretation consistent with the specification and ordinary skill in the art, a non-negative tensor can be embodied as a matrix which is a “two-dimensional case of a tensor.” See Specification, Paragraph 44. Prior art teaches this, for example: “a corresponding 'center' position for each molecule in the spectral channel was computed based on a [linear] transformation matrix, which was determined by registering positional and spectral images of gold fiduciaries” Gray, Paragraph 84.)
Regarding Claim 4: “The computer-implemented according to claim 1, wherein the computing the crosstalk quantity comprises: computing a probability that the first unmixed signal corresponds to the first signal.” (For example, “Diffusion constants of state i may be labeled as D, probability of molecules the state is Pl” See Gray, Paragraph 30.)
Regarding Claim 9: “The computer-implemented method according to claim 1, wherein the subset corresponds to a single pixel of the plurality of pixels of the digital color input image.” (For example, “each color arrives at a slightly different pixel on the detector.” Gray, Paragraphs 45, 57, 68.)
Regarding Claim 11: “A method of using a medical optical observation device, the method comprising:
recording at least one color image and generating a color input image from the at least one color image; and (“a series of single-molecule positional and spectral images of CF680R recorded at 20 ms per frame and shown [generated] in 40 ms ( every 2 frames) intervals.” Gray, Paragraph 26.
executing the computer-implemented method according to claim 1.” (See reasons for rejection in Claim 1.)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 5-6, 8 are rejected under 35 U.S.C. 103 as being unpatentable over US 20190271592 to Gray (“Gray”) in view of T. Zimmermann, “Spectral Imaging and Linear Unmixing in Light Microscopy,” Adv Biochem Engin/Biotechol, Springer-Verlag (2005) 95: 245-265 (“Zimmerman”) provided in an IDS.
Regarding Claim 5: “The computer-implemented according to claim 1, wherein the computing the crosstalk quantity comprises computing a correlation.” (“In addition to the linear unmixing method described above, alternative approaches for the spectral unmixing of microscopic data have also been implemented, especially for datasets consisting of only few spectral channels [14]. These approaches are based on the correlation of the intensity values of a pixel in different image channels” Zimmarman, Page 253, Paragraphs 3-4.)
Regarding Claim 6: “The computer-implemented according to claim 1, wherein the computing the crosstalk quantity comprises computing a crosstalk tensor of which the crosstalk quantity is an element.” (Gray generally teaches unmixing “based on a [linear] transformation matrix, which was determined by registering positional and spectral images of gold fiduciaries” Gray, Paragraph 84. Zimmermann teaches a specific example: “After the calculation of the weighing matrix A; clear representations of the separated fluorophores can be created (Figs, 2 and 3),” on Page 252, third paragraph.)
Regarding Claim 8: “The computer-implemented according to claim 1, wherein the removing the crosstalk quantity from the first unmixed signal comprises one of: … subtracting the crosstalk quantity from the first unmixed signal; … and multiplying the first unmixed signal by the crosstalk quantity.” (For example, “To reduce spectral crosstalk, localization events associated with ssd values that fall in the border region between two adjacent peaks were discarded,” that is subtracted from the signal. See Gray, Paragraph 34 and Fig. 8. More explicitly, “subtraction is an essential prerequisite for the linear unmixing analysis” in Zimmermann, Page 257, Second Paragraph.)
Claims 7, 10, 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over US 20190271592 to Gray (“Gray”) in view of T. Zimmermann, “Spectral Imaging and Linear Unmixing in Light Microscopy,” Adv Biochem Engin/Biotechol, Springer-Verlag (2005) 95: 245-265 (“Zimmermann”) provided in an IDS, and in view of US 20040086174 to Sun (“Sun”).
Regarding Claim 7: “The computer-implemented according to claim 6, wherein the computing the crosstalk tensor comprises computing at least one of:”
Gray and Zimmermann do not teach ”computing a covariance tensor; or … computing a Fisher information tensor.”
Sun teaches examples of the above computations in the context of processing pixelwise correspondences between images. For example, “The output covariance 350 of any unbiased estimator can not be smaller than the inverse of the Fisher information matrix.” Sun, Paragraphs 61-63 and Fig. 4. Cumulatively, see an example of a covariance tensor in Sun, Paragraph 46.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to supplement the teachings of Gray and Zimmermann to compute a covariance tensor or a Fisher information tenson in the manner taught in Sun, in order to represent metrics of correspondence in tensor format. See Sun, Paragraphs 3, 6.
Finally, in reviewing the present application, there does not seem to be objective evidence that the claim limitations are particularly directed to: addressing a particular problem which was recognized but unsolved in the art, producing unexpected results at the level of the ordinary skill in the art, or any other objective indicators of non-obviousness.
Claim 10: “A non-transitory computer-readable medium having a computer program stored thereon, the computer program, when executed by a computer processor, causing performance of the method of claim 1.” This claim is rejected for reasons stated for Claim 1, and because prior art teaches: “the computer program may be stored in a computer readable storage medium” Sun, Paragraph 30. See similar uses of software and scripting in Gray, Paragraph 83. See statement of motivation in Claim 7.)
Regarding Claim 12: “A data processing device for computing an estimate of a first signal of a plurality of signals, the plurality of signals being contained in a digital color input image, each signal of the plurality of signals having a different ground truth spectrum, wherein the data processing device is configured to: …” is rejected for reasons stated for Claim 1 and because prior art teaches: “a computer system 110 for implementing the present invention.” Sun, Paragraph 31 and Fig. 1. See statement of motivation in Claim 7.
Regarding Claim 13: “An optical observation device, being configured to record a digital color input image, the medical optical observation device comprising: the data processing device according to claim 12.” This claim is rejected for reasons stated for Claims 11 and 12.
Regarding Claim 14: “The optical observation device according to claim 13, wherein the data processing device comprises or consists of an embedded system of the optical observation device.” This claim is rejected for reasons stated for Claim 13, because prior art teaches: “the present invention is not limited to the computer system 110 shown, but may be used on any electronic processing system such as found in home computers, kiosks, retail or wholesale photofinishing, or any other system for the processing of digital images.” Sun, Paragraph 31.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20120038797 to Jang (“Jang”) relevant for teaching various tensor techniques used in image processing.
L. Sun, F. Wu, T. Zhan, W. Liu, J. Wang and B. Jeon, "Weighted Nonlocal Low-Rank Tensor Decomposition Method for Sparse Unmixing of Hyperspectral Images," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 13, pp. 1174-1188, 2020. (“Sun2”) This reference explains use of tensor math in spectral unmixing similarly to Sun.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MIKHAIL ITSKOVICH whose telephone number is (571)270-7940. The examiner can normally be reached Mon. - Thu. 9am - 8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Joseph Ustaris can be reached at (571)272-7383. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MIKHAIL ITSKOVICH/Primary Examiner, Art Unit 2483
Prosecution Insights