Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on July 7, 2026 has been entered.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, 7-8, 10-11, 13, and 22-23 are rejected under 35 U.S.C. 103 as being unpatentable over Themelis (US PGPUB 2020/0204776 – “Themelis”) in view of Blumzvig et al. (US PGPUB 2005/0267328 – “Blumzvig”) and Hasegawa (US PGPUB 2011/0018971 – “Hasegawa”).
Regarding Claim 1, Themelis discloses:
An imaging system (Themelis FIG. 1, endoscope 1), comprising:
an endoscope tube;
an illumination system (Themelis FIG. 1, illumination system 10) coupled to the endoscope tube and configured to emit first illumination light (Themelis FIG. 1, white light source 34) having a first wavelength profile (Themelis FIG. 1, illumination white light 36) and excitation light (Themelis FIG. 1, fluorescence light source 35) having an excitation wavelength profile (Themelis FIG. 1, fluorescence excitation light 40) outside of the first wavelength profile (Themelis paragraph [0073], “fluorescence excitation light 40 preferably only contains wavelengths λ. which are in the absorption spectrum 16 of the at least fluorophore 8”);
a first image sensor (Themelis FIG. 1, RGB camera 30) aligned with a first filter (Themelis FIG. 1, white-light recording filter 60) configured to pass first image light, received in response to the first illumination light, to the first image sensor and to block the excitation light (Themelis paragraph [0080], “white-light recording filter 60 is preferably a band-stop filter having at least one stop band which coincides with the fluorescence excitation pass band 50”);
a second image sensor (Themelis FIG. 1, fluorescence camera 32) aligned with a second filter (Themelis FIG. 1, optical fluorescence emissions filter 52) configured to pass fluorescence light, emitted in response to the excitation light, to the second image sensor (Themelis paragraph [0077], “optical filter 52 is preferably a narrow-band pass-band filter having a fluorescence emission pass band 54 which is located in the fluorescence spectrum 18 of the at least one fluorophore 8”); and
a controller (Themelis FIG. 1, image-processing device 64) coupled to the first and second image sensors and to the illumination system (Themelis FIG. 1, camera system 2 including and light sources 34,35 and cameras 30,32; Themelis FIG. 1, input section 68; Themelis paragraph [0084], “image-processing device 64 may comprise an input section 68 to which a camera system 2…may be connected”), the controller including logic that, when executed, causes the imaging system to perform operations including:
simultaneously illuminating a scene with the first illumination light and the excitation light emitted from the endoscope tube; and capturing first image data with the first image sensor and fluorescence image data with the second image sensor in response to the simultaneous illuminating (Themelis paragraph [0050], “The fluorescence observation device may, in particular, comprise a camera system being configured to record the at least one digital white-light color image and the at least one digital fluorescence image, preferably simultaneously. The fluorescence observation device may comprise separate cameras for recording the at least one digital white-light color image and the at least one digital fluorescence image”);
adjusting an image acquisition characteristic of one or more of the first or second image sensors independently of each other (Themelis FIG. 2, step 103 and digital attenuation filters 131 in type 63 of endoscope 1; Themelis paragraph [0108], “In another step 130, the relative intensity and/or brightness of the at least one digital white-light color image 4 and the at least one digital fluorescence image 20 is adjusted to match the relative intensity and/or brightness of the background and the fluorescence in the type 82, or the digital color output image 84 produced by type 82, respectively. If the relative intensity and/or brightness is adjusted in step 130, a luminosity function, such as a CIE luminosity function, may be used. For adjusting the relative intensity and/or brightness, a digital attenuation filter 131 or a library 122 containing a plurality of digital attenuation filters 131 may be used”).
Although it is well known that every endoscope has an endoscope tube, for purposes of compact prosecution Examiner further cites Blumzvig as teaching an endoscope tube (Blumzvig FIG. 1, insertion tube 23 in endoscope 21).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to include Blumzvig’s insertion tube with the imaging system disclosed by Themelis. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of an endoscope that is controllably inserted into a small opening of a patient during endoscopic surgery.
Examiner believes that adjusting an image acquisition characteristic of one or more of the first or second image sensors independently of each other includes post-production of images taken by an image sensor, as described above in the citation of Themelis paragraph [0108]. However, Themelis in view of Blumzvig does not explicitly teach adjusting an image acquisition characteristic of one or both of the first or second image sensors independently image sensor during pre-production (at the time of image capture), nor does Themelis in view of Blumzvig explicitly teach wherein the image acquisition characteristic is selected from the group consisting of at least one of a frame rate, an exposure time, gain, and a duty cycle.
Hasegawa is analogous art in the field of imaging that teaches adjusting an image acquisition characteristic of one or more of the first or second image sensors (Hasegawa FIG. 2, imaging element 122 and imaging element 123; Hasegawa paragraph [0072], “imaging elements 122 and 123 are configured with, for example, color CCDs (Charge Coupled Device)”) independently of each other (Hasegawa FIG. 2, timing generator (TG) 148 and TG 149); Hasegawa paragraph [0072], “imaging elements 122 and 123 are controlled by charge drain pulses inputted from the TGs 148 and 149, respectively.”), wherein the image acquisition characteristic is selected from the group consisting of at least one of a frame rate, an exposure time (Hasegawa paragraph [0072], “electronic shutter speed (photo charge accumulation time) in the imaging elements 122 and 123 are controlled by charge drain pulses inputted from the TGs 148 and 149, respectively“), gain, and a duty cycle.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Hasegawa’s independent control of image sensors with the imaging system taught by Themelis in view of Blumzvig. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield predictable result of a system that controls an imaging sensor’s shutter speed by selectively controlling the timing (e.g., based on intensity) for when capacitors in the CCD are drained (see Hasegawa paragraph [0073]).
As (1) the combination of the prior art includes each element claimed, although not necessarily in a single prior art reference, as described above; (2) the record shows no evidence of each element taught by the cited prior art performing differently in combination as it does separately (i.e., either alone or in combination the system for focusing performs in a same manner); and (3) a finding that one of ordinary skill in the art would have recognized that the results of the combination were predictable (i.e., nothing in the record supports anything other than the conclusion using Hasegawa’s pulses to selective choose when drain a CCD would create a predicable result of selectively controlling the frame rate/exposure time in two independent image sensors, as described in Themelis in view of Blumzvig). As such, no additional findings based on the Graham (Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966) factual inquiries are necessary, in view of the facts of the case under consideration, to explain a conclusion of obviousness.
Regarding Claim 4, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 1, as described above.
Themelis further discloses wherein the first illumination light comprises visible illumination light (Themelis FIG. 1, illumination white light 36) and the first image light comprises visible image light (Themelis FIG. 1, reflected light 38).
Regarding Claim 7, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 4, as described above.
Themelis further discloses wherein the controller includes further logic that, when executed, causes the imaging system to perform further operations, including adjusting, while contemporaneously emitting the visible illumination light and the excitation light (Themelis FIG. 1, showing fluorescence excitation light 40 and illumination white light 36 being contemporaneously emitted) a relative intensity between the visible illumination light and the excitation light (Themelis paragraph [0040], “To adjust the relative intensities, or the relative brightness, of background and fluorescence in the second type of fluorescence observation device to correspond to the first type of fluorescence observation device, the type-emulation module may comprise a digital attenuation filter for adjusting at least one of the mean intensity or brightness of the digital white-light input image and the mean intensity or brightness of the digital fluorescence input image relative to one another in order to e.g. obtain a predetermined ratio”.).
Blumzvig further teaches independently controlling drive current or duty cycle of at least two respective light sources of the illumination system that generate the visible illumination light and the excitation light (Blumzvig FIG. 2, light sources 30; Blumzvig paragraph [0093], light sources 30 comprise white-light LEDs, but alternatively, miniature light source of different types may be used, including LEDs of other colors or infrared LEDs“; Blumzvig paragraph [0063], “a controller, which is adapted to drive the LEDs to emit the optical radiation at different, respective intensities”).
Regarding Claim 8, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 4, as described above.
Themelis further discloses wherein the second filter is configured to pass the fluorescence light while blocking at least a majority of both the visible image light and the excitation light (Themelis paragraph [0077], “An optical fluorescence emission filter 52 is located between the observation area 48 and the camera system 2. The optical filter 52 is preferably a narrow-band pass-band filter having a fluorescence emission pass band 54 which is located in the fluorescence spectrum 18 of the at least one fluorophore 8…Thus, the light recorded in the at least one digital fluorescence image 20 is restricted to wavelengths contained within the fluorescence emission pass band 54.”; Themelis paragraph [0023], “fluorescence recording filter may have a pass band which allows only the fluorescence wavelengths to pass and blocks all other wavelengths. Thus, the at least one digital fluorescence image of the second type contains only fluorescent light, in particular if illumination containing fluorescence wavelengths has been blocked before reaching the object.”).
Regarding Claim 10, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 1, as described above.
Blumzvig further teaches wherein the first and second image sensors (Blumzvig FIG. 14, image sensors 128 and 130) are disposed at a distal end of the endoscope tube (Blumzvig FIG. 14, assembly 115; Blumzvig paragraph [0118], “assembly 115 is based on the principles of assembly 27, as described above in detail, and is thus suitable for use within the distal end of an endoscope”).
Regarding Claim 11, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 10, as described above.
Blumzvig further teaches:
wherein the first and second image sensors are oriented back-to-back at the distal end of the endoscope tube, the imaging apparatus further comprising:
first (Blumzvig FIG. 14, image sensor 128) and second reflectors (Blumzvig FIG. 14, image sensor 130) disposed at the distal end of the endoscope tube (Blumzvig FIG. 14, assembly 115; Blumzvig paragraph [0118], “assembly 115 is based on the principles of assembly 27, as described above in detail, and is thus suitable for use within the distal end of an endoscope”), wherein the first reflector (Blumzvig FIG. 14, prism 118) is configured to direct the first image light onto the first image sensor and the second reflector (Blumzvig FIG. 14, prism 120) is configured to direct the fluorescence light onto the second image sensor.
Regarding Claim 13, Themelis discloses:
A method of operation of an endoscope, the method comprising:
simultaneously illuminating a scene with visible illumination light (Themelis FIG. 1, white light source 34) and excitation light (Themelis FIG. 1, fluorescence light source 35) emitted from an endoscope tube, wherein the excitation light is outside of a visible wavelength profile of the visible illumination light (Themelis paragraph [0073], “fluorescence excitation light 40 preferably only contains wavelengths λ. which are in the absorption spectrum 16 of the at least fluorophore 8”);
filtering scene light received from the scene in response to the simultaneous illuminating with a first filter (Themelis FIG. 1, white-light recording filter 60) configured to pass visible image light (Themelis paragraph [0080], “white-light recording filter 60 is preferably a band-stop filter having at least one stop band which coincides with the fluorescence excitation pass band 50”) and a second filter (Themelis FIG. 1, optical fluorescence emissions filter 52) configured to pass fluorescence light (Themelis paragraph [0077], “optical filter 52 is preferably a narrow-band pass-band filter having a fluorescence emission pass band 54 which is located in the fluorescence spectrum 18 of the at least one fluorophore 8”);
adjusting an image acquisition characteristic of one or both of the first or second image sensors independently of each other (Themelis FIG. 2, step 103 and digital attenuation filters 131 in type 63 of endoscope 1; Themelis paragraph [0108], “In another step 130, the relative intensity and/or brightness of the at least one digital white-light color image 4 and the at least one digital fluorescence image 20 is adjusted to match the relative intensity and/or brightness of the background and the fluorescence in the type 82, or the digital color output image 84 produced by type 82, respectively. If the relative intensity and/or brightness is adjusted in step 130, a luminosity function, such as a CIE luminosity function, may be used. For adjusting the relative intensity and/or brightness, a digital attenuation filter 131 or a library 122 containing a plurality of digital attenuation filters 131 may be used”); and
contemporaneously capturing visible image data in response to the visible image light incident upon a first image sensor and fluorescence image data in response to the fluorescence light incident upon a second image sensor (Themelis paragraph [0050], “The fluorescence observation device may, in particular, comprise a camera system being configured to record the at least one digital white-light color image and the at least one digital fluorescence image, preferably simultaneously. The fluorescence observation device may comprise separate cameras for recording the at least one digital white-light color image and the at least one digital fluorescence image”).
Although it is well known that every endoscope has an endoscope tube, for purposes of compact prosecution Examiner further cites Blumzvig as teaching an endoscope tube (Blumzvig FIG. 1, insertion tube 23 in endoscope 21).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to include Blumzvig’s insertion tube with the imaging system disclosed by Themelis. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of an endoscope that is controllably inserted into a small opening of a patient during endoscopic surgery.
Examiner believes that adjusting an image acquisition characteristic of one or more of the first or second image sensors independently of each other includes post-production of images taken by an image sensor, as described above in the citation of Themelis paragraph [0108]. However, Themelis in view of Blumzvig does not explicitly teach adjusting an image acquisition characteristic of one or both of the first or second image sensors independently image sensor during pre-production (at the time of image capture), nor does Themelis in view of Blumzvig explicitly teach wherein the image acquisition characteristic is selected from the group consisting of at least one of a frame rate, an exposure time, gain, and a duty cycle.
Hasegawa is analogous art in the field of imaging that teaches adjusting an image acquisition characteristic of one or more of the first or second image sensors (Hasegawa FIG. 2, imaging element 122 and imaging element 123; Hasegawa paragraph [0072], “imaging elements 122 and 123 are configured with, for example, color CCDs (Charge Coupled Device)”) independently of each other (Hasegawa FIG. 2, timing generator (TG) 148 and TG 149); Hasegawa paragraph [0072], “imaging elements 122 and 123 are controlled by charge drain pulses inputted from the TGs 148 and 149, respectively.”), wherein the image acquisition characteristic is selected from the group consisting of at least one of a frame rate, an exposure time (Hasegawa paragraph [0072], “electronic shutter speed (photo charge accumulation time) in the imaging elements 122 and 123 are controlled by charge drain pulses inputted from the TGs 148 and 149, respectively“), gain, and a duty cycle.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Hasegawa’s independent control of image sensors with the imaging system taught by Themelis in view of Blumzvig. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield predictable result of a system that controls an imaging sensor’s shutter speed by selectively controlling the timing (e.g., based on intensity) for when capacitors in the CCD are drained (see Hasegawa paragraph [0073]).
As (1) the combination of the prior art includes each element claimed, although not necessarily in a single prior art reference, as described above; (2) the record shows no evidence of each element taught by the cited prior art performing differently in combination as it does separately (i.e., either alone or in combination the system for focusing performs in a same manner); and (3) a finding that one of ordinary skill in the art would have recognized that the results of the combination were predictable (i.e., nothing in the record supports anything other than the conclusion using Hasegawa’s pulses to selective choose when drain a CCD would create a predicable result of selectively controlling the frame rate/exposure time in two independent image sensors, as described in Themelis in view of Blumzvig). As such, no additional findings based on the Graham (Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966) factual inquiries are necessary, in view of the facts of the case under consideration, to explain a conclusion of obviousness.
Regarding Claim 22, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 1, as described above.
Themelis further discloses wherein, during the simultaneous illuminating, the controller contemporaneously captures the first image data and the fluorescence image data while operating the first image sensor with a first image acquisition characteristic and operating the second image sensor with a second image acquisition characteristic that differs from the first image acquisition characteristic, and wherein the endoscope tube receives the scene light at a distal end and provides the first image light and the fluorescence light to the first and second image sensors through the first and second filters, respectively (Themelis Fig. 1, showing white light source 34 and excitation light source 35 simultaneously illuminating observation area 48 while white light sensor/filter 30/60 and fluorescence sensor/filter 32/52 simultaneously capture the image).
Regarding Claim 23, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 1, as described above.
Blumzvig further teaches wherein the first and second image sensors are disposed back-to-back at a distal end of the endoscope tube with respective light-sensitive array normals oriented substantially perpendicular to a longitudinal axis of the endoscope tube (Blumzvig FIG. 14, image sensor 128 image sensor 130 back-to-back and positioned perpendicular to the longitudinal axis of distal end 115 of an endoscope tube), and wherein the endoscope tube further comprises first and second reflectors (Blumzvig FIG. 15, prisms 118 and 120) configured to bend received scene light toward the first and second image sensors, respectively.
Themelis further discloses received scene light passes through the first and second filters (Themelis FIG. 1, filters 52 and 60) aligned with the first and second image sensors (Themelis FIG. 1, cameras 30 and 32).
Claims 5-6, 9 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Themelis (US PGPUB 2020/0204776 – “Themelis”) in view of Blumzvig et al. (US PGPUB 2005/0267328 – “Blumzvig”), Hasegawa (US PGPUB 2011/0018971 – “Hasegawa”), and Frangioni (US PGPUB 2019/0079011 – “Frangioni”).
Regarding Claim 5, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 4, as described above.
Themelis in view of Blumzvig and Hasegawa does not explicitly teach:
wherein the controller includes further logic that, when executed, causes the imaging system to perform further operations including:
adjusting at least one of a first frame rate, a first exposure time, or a first gain of the first image sensor while holding a second frame rate, a second exposure time, and a second gain of the second image sensor constant during acquisition of a series of visible images and fluorescence images with the first and second image sensors, respectively.
Frangioni is analogous art in the field of image capture/processing that teaches:
wherein the controller includes further logic that, when executed, causes the imaging system to perform further operations including:
adjusting at least one of a first frame rate, a first exposure time, or a first gain of the first image sensor while holding a second frame rate, a second exposure time, and a second gain of the second image sensor constant during acquisition of a series of visible images and fluorescence images with the first and second image sensors, respectively (Frangioni paragraph [0079], “Where the images are superimposed, a frame rate adjustment may be required. That is, if the visible light sensor 106 is capturing images at the conventional rate of thirty frames per second and the NIR sensor 104 is taking still pictures with an effective frame rate of fifteen frames per second, some additional processing may be required to render the superimposed images concurrently. This may entail either reducing the frame rate of the visible light sensor 106 to the frame rate of the NIR sensor 104 either by using every other frame of video data…”. Examiner interprets the characteristics of the images from the NIR sensor 104 as being unchanged.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Frangioni’s image adjustment system with the imaging system taught by Themelis in view of Blumzvig and Hasegawa. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of a system in which both white light images and excitation images are visible when overlaid (see Frangioni paragraphs [0066] and [0084] and Frangioni FIG. 3).
Regarding Claim 6, Themelis in view of Blumzvig, Hasegawa, and Frangioni teaches the features of Claim 5, as described above.
Frangioni further teaches wherein the controller includes further logic that, when executed, causes the imaging system to perform further operations including adjusting at least one of the second frame rate, the second exposure time, or the second gain of the second image sensor while holding the first frame rate, the first exposure time, and the first gain of the first image sensor constant while acquiring a series of visible images and fluorescence images, with the first and second image sensors, respectively (Frangioni paragraph [0079], “Where the images are superimposed, a frame rate adjustment may be required…This may instead entail increasing the frame rate of the near-infrared image data.” Examiner interprets this alternating adjustment of the frame rate of the visible light image while keeping the frame rate of the NIR image fixed, and adjustment of the frame rate of the NIR image while keeping the visible light image fixed.).
Regarding Claim 9, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 4, as described above.
Themelis in view of Blumzvig and Hasegawa does not explicitly teach wherein the second filter comprises a near-infrared long-pass filter having a cutoff wavelength longer than the excitation wavelength profile.
Frangioni teaches wherein the second filter (Frangioni FIG. 1B, splitter 108) comprises a near-infrared long-pass filter having a cutoff wavelength longer than the excitation wavelength profile (Frangioni paragraph [0072], “splitter 108 may be operable to separate and direct the NIR and visible light received from the illuminated subject 116. For example, splitter 108 may include any number of filters and/or dichroic mirrors, to direct the fluorescence wavelengths towards NIR sensor 104 and the visible wavelengths towards visible light sensor 106 for capture”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to utilize Frangioni’s filters with the imaging system taught by Themelis in view of Blumzvig and Hasegawa. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of an imaging system having an excitation/NIR light sensor that is not affected by “bloom” or other visual artifacts caused by visible light.
Regarding Claim 16, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 13, as described above.
Themelis in view of Blumzvig and Hasegawa does not explicitly teach:
adjusting at least a first frame rate, a first exposure time, or a first gain of the first image sensor while holding a second frame rate, a second exposure time, and a second gain of the second image sensor constant while acquiring a first series of visible images and fluorescence images with the first and second image sensors, respectively.
Frangioni teaches adjusting at least a first frame rate, a first exposure time, or a first gain of the first image sensor while holding a second frame rate, a second exposure time, and a second gain of the second image sensor constant while acquiring a first series of visible images and fluorescence images with the first and second image sensors, respectively (Frangioni paragraph [0079], “Where the images are superimposed, a frame rate adjustment may be required. That is, if the visible light sensor 106 is capturing images at the conventional rate of thirty frames per second and the NIR sensor 104 is taking still pictures with an effective frame rate of fifteen frames per second, some additional processing may be required to render the superimposed images concurrently. This may entail either reducing the frame rate of the visible light sensor 106 to the frame rate of the NIR sensor 104 either by using every other frame of video data…”. Examiner interprets the characteristics of the images from the NIR sensor 104 as being unchanged.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to utilize Frangioni’s frame rates with the imaging system taught by Themelis in view of Blumzvig and Hasegawa. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of an imaging system having the ability to superimpose images captured by lights having different wavelengths and different frame rates (see Frangioni paragraph [0079]).
Regarding Claim 17, Themelis in view of Blumzvig, Hasegawa, and Frangioni teaches the features of Claim 16, as described above.
Frangioni further teaches adjusting at least the second frame rate, the second exposure time, or the second gain of the second image sensor while holding the first frame rate, the first exposure time, and the first gain of the first image sensor constant while acquiring a second series of visible images and fluorescence images with the first and second image sensors, respectively (Frangioni paragraph [0079], “Where the images are superimposed, a frame rate adjustment may be required…This may instead entail increasing the frame rate of the near-infrared image data.” Examiner interprets this alternating adjustment of the frame rate of the visible light image while keeping the frame rate of the NIR image fixed, and adjustment of the frame rate of the NIR image while keeping the visible light image fixed.).
Regarding Claim 18, Themelis in view of Blumzvig, Hasegawa and Frangioni teaches the features of Claim 16, as described above.
Themelis further discloses adjusting a relative intensity between the visible illumination light and the excitation light while contemporaneously emitting both (Themelis paragraph [0040], “To adjust the relative intensities, or the relative brightness, of background and fluorescence in the second type of fluorescence observation device to correspond to the first type of fluorescence observation device, the type-emulation module may comprise a digital attenuation filter for adjusting at least one of the mean intensity or brightness of the digital white-light input image and the mean intensity or brightness of the digital fluorescence input image relative to one another in order to e.g. obtain a predetermined ratio”.).
Regarding Claim 19, Themelis in view of Blumzvig, Hasegawa, and Frangioni teaches the features of Claim 16, as described above.
Themelis further discloses wherein filtering the scene light comprises passing fluorescence light with the second filter while blocking the both the first illumination light and the excitation light (Themelis paragraph [0077], “An optical fluorescence emission filter 52 is located between the observation area 48 and the camera system 2. The optical filter 52 is preferably a narrow-band pass-band filter having a fluorescence emission pass band 54 which is located in the fluorescence spectrum 18 of the at least one fluorophore 8…Thus, the light recorded in the at least one digital fluorescence image 20 is restricted to wavelengths contained within the fluorescence emission pass band 54.”).
Regarding Claim 20, Themelis in view of Blumzvig, Hasegawa, and Frangioni teaches the features of Claim 16, as described above.
Frangioni further teaches wherein the second filter comprises a near- infrared long-pass filter having a cutoff wavelength longer than an excitation wavelength profile (Frangioni paragraph [0072], “splitter 108 may be operable to separate and direct the NIR and visible light received from the illuminated subject 116. For example, splitter 108 may include any number of filters and/or dichroic mirrors, to direct the fluorescence wavelengths towards NIR sensor 104 and the visible wavelengths towards visible light sensor 106 for capture”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to utilize Frangioni’s filters with the imaging system taught by Themelis in view of Blumzvig, Hasegawa, and Frangioni. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of an imaging system having an excitation/NIR light sensor that is not affected by “bloom” or other visual artifacts caused by visible light.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Themelis (US PGPUB 2020/0204776 – “Themelis”) in view of Blumzvig et al. (US PGPUB 2005/0267328 – “Blumzvig”), Hasegawa (US PGPUB 2011/0018971 – “Hasegawa”), and Kang et al. (US PGPUB 2003/0078477 – “Kang”).
Regarding Claim 12, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 1, as described above.
Themelis in view of Blumzvig and Hasegawa does not explicitly teach:
wherein the first and second image sensors are disposed at a proximal end of the endoscope tube, the imaging apparatus further comprising:
a beam splitter configured to receive the first image light and the fluorescence light from the endoscope tube and to direct the first image light to the first image sensor and the fluorescence light to the second image sensor,
wherein the first image sensor has a first sensor array normal that is oriented substantially perpendicular to a longitudinal axis of the endoscope tube and the second image sensor has a second sensor array normal that is oriented substantially perpendicular to the first sensor array normal and parallel to the longitudinal axis, and wherein the beam splitter is positioned between the first image sensor and the second image sensor to direct the first image light to the first image sensor and pass the fluorescence light to the second image sensor.
Kang teaches:
wherein the first (Kang FIG. 1, color camera 60) and second image sensors (Kang FIG. 1, monochromatic camera 70) are disposed at a proximal end of the endoscope tube (Kang FIG. 1, proximal end 30b of endoscope probe 30), the imaging apparatus further comprising:
a beam splitter (Kang FIG. 1, dichroic mirror 50) configured to receive the first image light and the fluorescence light from the endoscope tube and to direct the first image light to the first image sensor and the fluorescence light to the second image sensor (Kang paragraph [0042], “The fluorescence light and the excitation light are to be transmitted into the dichroic mirror 50 by going through the optical fiber bundle 32, an image transmission path, then through the ocular lens 40. Among the light transmitted into the dichroic mirror 50, the reflective light of the excitation light gets passed through the dichroic mirror 50, whereas the fluorescence light gets divided into separate paths after reflected off from the same.”; Kang paragraph [0043], “The reflective light gets inputted into the color CCD camera 60 through the objective lens 101. The color CCD camera 60, then, produces color images according to the inputted reflective light; and at the same, the fluorescence light gets inputted into the high sensitive monochromatic CCD camera 70 through the objective lens 102. The monochromatic CCD camera 70, then, produces high sensitive monochromatic images according to the inputted fluorescence light.”),
wherein the beam splitter is positioned between the first image sensor and the second image sensor to direct the first image light to the first image sensor and pass the fluorescence light to the second image sensor (Kang FIG. 1, showing dichroic mirror 50 selecting directing the first image light to the first color CCD camera 60 and the fluorescence light to the second monochromatic CCD camera 70)..
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Kang’s proximal beam splitter and dedicated light image sensor and fluorescence image sensor with the imaging system taught by Themelis in view of Blumzvig and Hasegawa. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of an imaging system that displays both visible light images and fluorescence images simultaneous for optimal image information to a user (see Kang paragraph [0044]).
Kang further teaches wherein the first image sensor has a first sensor array (Kang FIG. 1, color CCD camera 60) normal that is oriented substantially perpendicular to a longitudinal axis of the endoscope tube (Kang FIG. 1, rigid endoscope probe 30) and the second image sensor has a second sensor array (Kang FIG. 1, monochromatic CCD camera 70) normal that is oriented substantially perpendicular to the first sensor array normal and parallel to the longitudinal axis. Examiner notes that it is well known to a person having ordinary skill in the art of image capture/processing that CCD cameras comprise an array of pixel sensors. (See Murayama (US PGPUB 2009/0015689 – “Murayama”), cited below in the rejection of Claim 21, which described CCD camera in Murayama paragraph [0033].).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Themelis (US PGPUB 2020/0204776 – “Themelis”) in view of Blumzvig et al. (US PGPUB 2005/0267328 – “Blumzvig”), Hasegawa (US PGPUB 2011/0018971 – “Hasegawa”), Murayama (US PGPUB 2009/0015689 – “Murayama”), and Kang et al. (US PGPUB 2003/0078477 – “Kang”).
Regarding Claim 21, Themelis in view of Blumzvig and Hasegawa teaches the features of Claim 1, as described above.
Themelis discloses simultaneous illumination by white light source 34 and fluorescence excitation light source 35 (“during simultaneous illuminating”), and an optical path that receives scene light and provides (i) first image light to the first image sensor (Themelis FIG. 1, camera 30) through the first filter (Themelis FIG. 1, filter 60) and (ii) fluorescence light to the second image sensor (Themelis FIG. 1, camera 32) through the second filter (Themelis FIG. 1, filter 52).
While Hasegawa teaches independent control of separate image sensors, as described in the rejection of Claim 1 above, Themelis in view of Blumzvig and Hasegawa does not explicitly teach wherein the first image sensor and the second image sensor are separately addressable by the controller such that, during the simultaneous illuminating, the controller independently sets at least one image acquisition characteristic of the first image sensor independently of the second image sensor while contemporaneously capturing the first image data and the fluorescence image data.
Murayama teaches wherein the first image sensor and the second image sensor are separately addressable by the controller such that, the controller independently sets at least one image acquisition characteristic of the first image sensor independently of the second image sensor while contemporaneously capturing the first image data and the fluorescence image data (Murayama FIG. 1, first electronic shutter speed generator 23 for CCD 6 and second electronic shutter speed generator 53 for CCD 36).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to utilize Murayama’s separately controlled image sensors with the imaging system taught by Themelis in view of Blumzvig and Hasegawa. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of an imaging that coordinates imaging capture parameters between multiple image sensors.
Themelis in view of Blumzvig, Hasegawa, and Murayama does not explicitly teach wherein the endoscope tube includes an optical path that receives scene light at a distal end and provides (i) first image light to the first image sensor and (ii) fluorescence light to the second image sensor.
Kang teaches wherein the endoscope tube (Kang FIG. 1, rigid endoscope probe 30) includes an optical path that receives scene light at a distal end and provides (i) first image light to the first image sensor (Kang FIG. 1, color CCDE camera 60) and (ii) fluorescence light to the second image sensor (Kang FIG. 1, monochromatic CCD camera 70).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Kang’s endoscope tube with the imaging system taught by Themelis in view of Blumzvig, Hasegawa, and Murayama. A person having ordinary skill in the art would be motivated to combine these prior art elements according to known methods to yield the predictable result of an imaging system that displays both visible light images and fluorescence images simultaneous for optimal image information to a user (see Kang paragraph [0044]).
Response to Arguments
Applicant’s arguments, see pages 8-10, filed July 7, 2026, with respect to the rejection(s) of Claims 1 and 13 under 35 U.S.C. 103 in view of Murayama (US PGPUB 2009/0015689 – “Murayama”) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Hasegawa (US PGPUB 2011/00118971 – “Hasegawa”), which addresses above in the rejection of Claims 1 and 13 under 35 U.S.C. 103 the previously claimed feature (“adjusting an image acquisition characteristic of one or more of the first or second image sensors independently of each other”) and the newly-claimed feature (“wherein the image acquisition characteristic is selected from the group consisting of at least one of a frame rate, an exposure time, gain, and a duty cycle”). That is, Murayama is no longer relied upon in the rejection of Claims 1 and 13, and thus the arguments related to the teachings of Murayama are now moot.
As such, the rejection of Claims 1, 4-13, and 16-23 under 35 U.S.C. 103 are maintained.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIM BOICE whose telephone number is (571)272-6565. The examiner can normally be reached Monday-Friday 9:00am - 5:00pm Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571)272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JIM BOICE
Examiner
Art Unit 3795
/JAMES EDWARD BOICE/Examiner, Art Unit 3795
