DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 1 is objected to because of the following informalities: “the image the processor” and “im-aged” appear to contain typographical errors.
Claim 13 is objected to because of the following informalities: There is missing a period at the end of the claim.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim(s) 5 is/are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Regarding claim 5, it is unclear as to how a color white-light image is to be converted to a location not on the white light (visible) spectrum.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1-5, 9, 10, and 12-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Themelis (US 2017/0237958, of record) in view of Steiner (US 2017/0280029, of record).
Regarding claim 1, 10, and 13-15, Themelis discloses an image processor for a medical fluorescence observation device comprising fluorescence-light and white-light color cameras, for a computer-implemented image processing method, the image processor being configured to: retrieve a digital white-light color image of an object recorded in a first imaged spectrum ([0048]: “full spectrum of visible light”); retrieve a digital fluorescence-light color image of the object recorded in a second imaged spectrum, the second imaged spectrum overlapping with a fluorescence emissions spectrum of at least one fluorophore ([0042], [0043]: “fluorescence of the at least one fluorophore”), the second imaged spectrum being different from the first imaged spectrum, both the first imaged spectrum and the second imaged spectrum overlapping with a visible spectrum ([0048]: “near infrared” – the visible and near-infrared regions of the EM spectrum are considered to be overlapping since they are adjacent regions). Themelis discloses an operational mode wherein the image processor is configured to perform an image-combining that combines the digital white-light color image and the digital fluorescence-light color image ([0063]: “visible-light image is merged with the fluorescent-light image). Themelis does not explicitly disclose a second operation mode, wherein the image processor is configured to perform a color-conversion in addition to an image-combining; and wherein the color-conversion comprises applying at least one of a first color conversion function to be applied to the digital white-light color image or a second color conversion function to be applied to the digital fluorescence-light color image. However, Steiner teaches fluorescence image processing wherein image data may be converted from an initial color space to an alternate color space ([0040]…[0045]). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to apply the color space conversion of Steiner to the image data of Themelis, as to provide enhanced and robust visualization of image data.
Regarding claim 2, Themelis discloses that the first imaged spectrum and the second imaged spectrum are complementary to one another ([0014]: “different types of tissues marked by the different types of fluorophores can be discerned in the pseudocolor image” – the first and second spectrum complement each other with regard to what can be visualized in the tissue being imaged).
Regarding claim 3, Themelis discloses that colors of the digital white-light color image, colors of the digital fluorescence-light color image, and colors of the digital output image are represented in a same color space ([0076]: the color space of the image data is presented in RGB).
Regarding claim 4, while Themelis does not explicitly disclose additive combining color space coordinates of white and fluorescent color image data, Themelis does teach adding color space values ([0077]). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to apply an additive combination of pixel values, as to provide a conventional and intuitive manner of combining image data values.
Regarding claim 5, Themelis does not explicitly disclose that the first color conversion function is configured to convert a recorded color in the digital white-light color image to a color that is not located in the first imaged spectrum, and/or the first conversion function is configured to convert a recorded color in the digital fluorescence-light color image to a color that is not located in the second imaged spectrum. However, Steiner teaches fluorescence image processing wherein image data may be converted from an initial color space to an alternate color space ([0040]…[0045]). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to apply the color space conversion of Steiner to the image data of Themelis, as to provide enhanced and robust visualization of image data.
Regarding claim 9, Themelis does not explicitly disclose that the first color conversion function and/or the second color conversion function is configured to convert a region of recorded colors into a larger region in a predetermined color space. However, Steiner teaches converting image data to a larger color space ([0040]; [0043]). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to apply the up-conversion of Steiner to the image data of Themelis, as to provide color space accommodation for fluoresced signals.
Regarding claim 12, Themelis discloses that the medical fluorescence observation device is a surgical fluorescence microscope ([0003], [0022], [0041], [0042]).
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Themelis (US 2017/0237958, of record) in view of Steiner (US 2017/0280029, of record), as applied to claim 1 above, in view of Sugano (US 2015/0042774).
Regarding claim 5, neither Themelis nor Steiner explicitly disclose that the first color conversion function is configured to convert a recorded color in the digital white-light color image to a color that is not located in the first imaged spectrum, and/or the first conversion function is configured to convert a recorded color in the digital fluorescence-light color image to a color that is not located in the second imaged spectrum. However, Sugano teaches converting fluorescent image signals into color signals ([0044], [0071], [0072]). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to apply the conversion of Sugano to the image data of Themelis and Steiner, as to provide conventional and robust color conversion for fluorescent image data.
Claim(s) 6-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Themelis (US 2017/0237958, of record) in view of Steiner (US 2017/0280029, of record), as applied to claim 1 above, in view of Iguchi (US 2003/0174216).
Regarding claim 6, neither Themelis nor Steiner explicitly disclose that the first color conversion function that is configured to color-balance the digital white-light color image. However, Iguchi teaches conventional color balancing of image data ([0097]: “color balance” is corrected). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to apply the correction of Iguchi to the image data of Themelis and Steiner, as to provide corrected images.
Regarding claim 7, neither Themelis nor Steiner explicitly disclose that the first color conversion function is configured to shift a recorded color of the digital white-light color image to a first predetermined color in color space, and/or the second color conversion function is configured to shift a recorded color of the digital fluorescence-light color image to a second predetermined color in color space. However, Iguchi teaches conventional color conversion within a color space ([0098]…[0103]). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to apply the color conversion of Iguchi to the image data of Themelis and Steiner, as to provide corrected images.
Regarding claim 8, neither Themelis nor Steiner explicitly disclose that the image processor is configured to shift different recorded colors by a different amount and/or in a different direction in a predetermined color space. However, Iguchi teaches conventional color shifting within a color space ([0098]…[0103]). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to apply the color conversion of Iguchi to the image data of Themelis and Steiner, as to provide corrected images.
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Themelis (US 2017/0237958, of record) in view of Steiner (US 2017/0280029, of record), as applied to claim 10 above, in view of Baumann (US 2021/0267443).
Regarding claim 11, neither Themelis nor Steiner explicitly disclose that a fluorescence-light camera has a higher integration time than the white-light color camera. However, Baumann teaches that the integration time for a fluorescence light sensor may be higher than that of a visible light sensor ([0067]). Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to apply the integration time of Baumann to the sensors of Themelis, as to provide exposure control for different types of sensed data.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jason Ip whose telephone number is (571) 270-5387. The examiner can normally be reached Monday - Friday 9a-5p PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Koharski can be reached on (571) 272-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JASON M IP/Primary Examiner, Art Unit 3793
Prosecution Insights