DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is responsive to the correspondence filled on 08/04/2025.
Claims 1 are presented for examination.
IDS Considerations
The information disclosure statement (IDS) submitted on 09/24/2025 is/are being considered by the examiner as the submission is in compliance with the provisions of 37 CFR 1.97.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO internet Web site contains terminal disclaimer forms which may be used. The filing date of the application will determine what form should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission.
Claims 1 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1 of US Pat. 12402786 B2.
Even though instant application does not claim “and to interrupt the white light illumination and at least two differently illuminated narrowband interspersed frames to be assembled into a multispectral or hyperspectral data cube at a second frame rate”, however not claiming this does not provide instant application a patentable distinction. Because lack of limitation makes the claim broad obvious variation of US Pat. 12402786 B2.
Even though US Pat. 12402786 B2 does not claim a light source configured to provide a broad-band, an endoscope comprising an optical system to collect an image light from the subject scene and relay it to an image sensor configured to capture white light illuminated frames of the subject scene. However, this is well known in the art as an example given in prior art Talbert (U.S. Pub. No. 20200400828 A1) teach Talbert [0087] FIG. 1 illustrates an implementation wherein the emitter 102 emits four different partitions of electromagnetic radiation, including red 104, green 106, blue 108 wavelengths, and a specialty 110 emission. Talbert [0099] FIG. 2 is a system 200 for providing illumination to a light deficient environment, such as for endoscopic imaging. The system 200 may be used in combination with any of the systems, methods, or devices disclosed herein. The system 200 includes an emitter 202, a controller 204, a jumper waveguide 206, a waveguide connector 208, a lumen waveguide 210, a lumen 212, and an image sensor 214 with accompanying optical components (such as a lens). The emitter 202 (may be generically referred to as a “light source”) generates light that travels through the jumper waveguide 206 and the lumen waveguide 210 to illuminate a scene at a distal end of the lumen 212. The emitter 202 may be used to emit any wavelength of electromagnetic energy including visible wavelengths, infrared, ultraviolet, hyperspectral, fluorescence excitation, or other wavelengths. The lumen 212 may be inserted into a patient's body for imaging, such as during a procedure or examination. The light is output as illustrated by dashed lines 216. A scene illuminated by the light may be captured using the image sensor 214 and displayed for a doctor or some other medical personnel. The controller 204 may provide control signals to the emitter 202 to control when illumination is provided to a scene. In one embodiment, the emitter 202 and controller 204 are located within a camera control unit (CCU) or external console to which an endoscope is connected. If the image sensor 214 includes a CMOS sensor, light may be periodically provided to the scene in a series of illumination pulses between readout periods of the image sensor 214 during what is known as a blanking period. Thus, the light may be pulsed in a controlled manner to avoid overlapping into readout periods of the image pixels in a pixel array of the image sensor 214.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine US Pat. 12402786 B2 and Talbert (U.S. Pub. No. 20200400828 A1) to accommodate broad band. The combined teaching will increase efficiency with predictable results.
Instant Application 19/289,560
US Pat. 12402786 B2
1. An endoscopic video system for multispectral imaging comprising:
a light source configured to provide a broad-band,
white light illumination to a subject scene
and configured to provide a narrowband illumination to the subject scene;
an endoscope comprising an optical system to collect an image light from the subject scene and relay it to an image sensor configured to capture white light illuminated frames of the subject scene
and narrowband illuminated frames of the subject scene;
a processor configured to receive an image signal from the image sensor
and to generate a video feed for display, the video feed for display comprising only white light illuminated frames, wherein the narrowband illuminated frames are replaced with white light illuminated frames from one or more time-adjacent white light illuminated frames,
and to process multispectral image data and/or generate a hyperspectral data cube at a second frame rate based on the narrowband illuminated frames.
1. An endoscopic video system comprising: and to generate the multispectral or hyperspectral data cube;
a light source adapted to provide white light illumination at a first frame rate suitable for live video,
and to interrupt the white light illumination
and provide narrowband illumination for individual frames interspersed among the white light illuminated frames at regular or irregular intervals,
wherein the spectrum of the narrowband illumination varies amongst the interspersed frames sufficient for a series of at least two differently illuminated narrowband interspersed frames to be assembled into a multispectral or hyperspectral data cube at a second frame rate;
and a processor adapted to receive an image signal based on sensor data received from a focal plane array sensor
and create a live video feed based on the white light illuminated frames by replacing the narrowband illuminated frames with frames generated from one or more time-adjacent white light illuminated frames,
and to generate the multispectral or hyperspectral data cube at a second frame rate based on the narrowband illuminated frames.
Allowable subject matter
Talbert (U.S. Pub. No. 20200400828 A1) is the closest prior art.
Regarding to claim 1:
1. Talbert teach an endoscopic video system for multispectral imaging comprising: (Talbert [0017] FIG. 6A is a schematic diagram of a process for recording a video with full spectrum light over a period of time from t(0) to t(1); [0018] FIG. 6B is a schematic diagram of a process for recording a video by pulsing portioned spectrum light over a period of time from t(0) to t(1); [0070] Additionally, light pulsing patterns may be emitted for the generation of a specialty data such as hyperspectral, fluorescence, or laser mapping imaging data. [0071])
a light source configured to provide a broad-band, (Talbert [0087] FIG. 1 illustrates an implementation wherein the emitter 102 emits four different partitions of electromagnetic radiation, including red 104, green 106, blue 108 wavelengths, and a specialty 110 emission.)
white light illumination to a subject scene (Talbert [0099] FIG. 2 is a system 200 for providing illumination to a light deficient environment, such as for endoscopic imaging. The system 200 may be used in combination with any of the systems, methods, or devices disclosed herein. The system 200 includes an emitter 202, a controller 204, a jumper waveguide 206, a waveguide connector 208, a lumen waveguide 210, a lumen 212, and an image sensor 214 with accompanying optical components (such as a lens). The emitter 202 (may be generically referred to as a “light source”) generates light that travels through the jumper waveguide 206 and the lumen waveguide 210 to illuminate a scene at a distal end of the lumen 212.)
and configured to provide a narrowband illumination to the subject scene; (Talbert [0027] FIG. 15 is a schematic diagram of a process flow for applying correction algorithms and for applying frame reconstruction to a plurality of exposure frames for generating a YCbCr image frame with specialty data [narrowband] overlaid thereon; [0291] For example, if the combination of wavelengths results in a pixel having a value exceeding or falling below a threshold, that pixel may be classified as corresponding to a specific type of tissue. Each frame may be used to further narrow the type of tissue that is present at that pixel (e.g., and each pixel in the image) to provide a very specific classification of the tissue and/or a state of the tissue (diseased/healthy) based on a spectral response of the tissue and/or whether a fluorescent reagent is present at the tissue.) an endoscope comprising an optical system to collect an image light from the subject scene and relay it to an image sensor configured to capture white light illuminated frames of the subject scene and (Talbert [0099] FIG. 2 is a system 200 for providing illumination to a light deficient environment, such as for endoscopic imaging. The system 200 may be used in combination with any of the systems, methods, or devices disclosed herein. The system 200 includes an emitter 202, a controller 204, a jumper waveguide 206, a waveguide connector 208, a lumen waveguide 210, a lumen 212, and an image sensor 214 with accompanying optical components (such as a lens). The emitter 202 (may be generically referred to as a “light source”) generates light that travels through the jumper waveguide 206 and the lumen waveguide 210 to illuminate a scene at a distal end of the lumen 212. The emitter 202 may be used to emit any wavelength of electromagnetic energy including visible wavelengths, infrared, ultraviolet, hyperspectral, fluorescence excitation, or other wavelengths. The lumen 212 may be inserted into a patient's body for imaging, such as during a procedure or examination. The light is output as illustrated by dashed lines 216. A scene illuminated by the light may be captured using the image sensor 214 and displayed for a doctor or some other medical personnel. The controller 204 may provide control signals to the emitter 202 to control when illumination is provided to a scene. In one embodiment, the emitter 202 and controller 204 are located within a camera control unit (CCU) or external console to which an endoscope is connected. If the image sensor 214 includes a CMOS sensor, light may be periodically provided to the scene in a series of illumination pulses between readout periods of the image sensor 214 during what is known as a blanking period. Thus, the light may be pulsed in a controlled manner to avoid overlapping into readout periods of the image pixels in a pixel array of the image sensor 214.)
narrowband illuminated frames of the subject scene; (Talbert [0095] The pixel array 122 senses reflected electromagnetic radiation. Each of the sensed red 105, the sensed green 107, the sensed blue 109, and the sensed specialty 111 data can be referred to as an “exposure frame.” The sensed specialty 111 may result in multiple separate exposure frames that are separate and independent from one another.) a processor configured to receive an image signal from the image sensor and (Talbert [0082] As used herein, monochromatic sensor refers to an unfiltered imaging sensor. Since the pixels are color agnostic, the effective spatial resolution is appreciably higher than for their color (typically Bayer-pattern filtered) counterparts in conventional single-sensor cameras. Monochromatic sensors may also have higher quantum efficiency because fewer incident photons are wasted between individual pixels. [0247] The process flow 1500 includes receiving image data from an image sensor at 1502. Sensor correction 1504 is performed on the sensor data. The super resolution (SR) and color motion artifact correction (CMAC) algorithms are implemented at 1506. The SR and CMAC processes 1506 may be performed within the camera image signal processor on raw, captured sensor data.)
to generate a video feed for display, (Talbert [0309] the grid array 2706 may be overlaid on an image of the surface 2704 on a display. The grid array 2706 may be overlaid on a black-and-white or RGB image of the surface 2704 such that the grid array 2706 may be visible by a user during use of the system 2700. A user of the system 2700 may indicate whether the grid array 2706 should be overlaid on an image of the surface 2704 and/or whether the grid array 2706 should be visible to the user. The system 2700 may include a display that provides real-time measurements of a distance from the endoscope 2706 to the surface 2704 or another object within the light deficient environment. The display may further provide real-time surface area information about the surface 2704 and/or any objects, structures, or tools within the light deficient environment.) the video feed for display comprising only white light illuminated frames, (Talbert [0293] In one embodiment, dual image sensors may be used to obtain three-dimensional images or video feeds. A three-dimensional examination may allow for improved understanding of a three-dimensional structure of the examined region as well as a mapping of the different tissue or material types within the region.)
However prior art does not teach wherein the narrowband illuminated frames are replaced with white light illuminated frames from one or more time-adjacent white light illuminated frames, and to process multispectral image data and/or generate a hyperspectral data cube at a second frame rate based on the narrowband illuminated frames.
Closely related prior art
Examiner notes teaching of U.S. Pub. No. 20180020965 A1 and U.S. Pub. No. 20220257101 A1 is/are pertinent to the independent claim(s), because these teach multispectral endoscope. However, is not used because these also do not teach allowable subject matter.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NASIM N NIRJHAR whose telephone number is (571) 272-3792. The examiner can normally be reached on Monday - Friday, 8 am to 5 pm ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William F Kraig can be reached on (571) 272-8660. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NASIM N NIRJHAR/Primary Examiner, Art Unit 2896
Prosecution Insights