Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, the “control system” (claim 9) must be shown or the feature(s) canceled from the claim(s). No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-2 and 4-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
A) Capturing different wavelength ranges at the distal end of the endoscope
Applicant’s disclosure places a dichroic mirror or prism (spec: [0009]), which is described as a “thin, partially reflective, and partially transmissive surface” (spec: [0041]), in front of a lens of a fiber-optic imaging system of an endoscope to spatially separate incoming light from a target area into two distinct wavelength ranges (spec: [0038]). Since the end result is a composite image created from combining image data from both wavelength ranges (spec: [0014]), various questions are raised that the current disclosure fails to answer. How is the beamsplitter arranged so as to capture light and separate the light while transmitting such light into a lens of a fiber-optic imaging system? As shown in arrangement 1 below, visible (V) and infrared (IR) light can be collected from two different directions, wherein the beamsplitter reflects visible light into the lens and transmits (passes) infrared light into the lens, but such lens recombines the visible and infrared wavelengths for transmission through the fiber-optic imaging system. Thus, the separation of the wavelength ranges is not maintained and image data from two different targets is obtained, one image in the visible wavelength range and a different image in the infrared range. With this arrangement, it is not clear how the images are extracted by the cameras from the fiber optic imaging system so as to provide separate imaging and analysis. As shown in arrangement 2 below, both visible and infrared light from the same target can enter the beamsplitter, but since the beamsplitter transmits one wavelength range (e.g. V) and reflects the other (e.g. IR) (or vice versa), only one wavelength range will enter the fiberoptic imaging system.
PNG
media_image1.png
517
856
media_image1.png
Greyscale
Again, even with this arrangement, it is not clear how the images in both wavelength ranges are extracted by the cameras at the proximal end of the fiber optic imaging system so as to provide separate imaging and analysis.
Applicant’s disclosure fails to disclose if the images in the different wavelength ranges are of the same target (arrangement 2 above) or different targets (arrangement 1 above), if only one image or both are being transmitted through the lens/fiber-optic imaging system, singly, alternatively, or simultaneously, how each image in the different wavelength range is captured by its own camera (e.g. no additional splitting at the proximal end of the endoscope is mentioned or shown), or the nature of the composite image (e.g. image data from each image being overlaid (for the instance of images of the same target) or merely displayed side by side in a single display image (for the instance of images of different targets).
B) Adjusting of beamsplitter settings
Claim 9 recites that the endoscope comprises a control system configured to automatically adjust beam splitter settings. The disclosure fails to describe any particular detail for how such “control system” would be configured, how beam splitter settings could be adjusted automatically by such “control system”, the nature of adjustability being “automatic”, or which “settings” the beamsplitter is provided with. The beamsplitter is not described in a way so as to provide it with “adjustable” settings and it is not clear how such settings would be adjusted, except at time of manufacture.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4 and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As to claim 4, this claim redundantly recites “a visible light imaging sensor or camera”, making it unclear whether an additional image sensor or camera is being claimed or if this is referring to the previously claimed image sensor or camera. It appears that this claim was intended to be canceled, similar to claim 3, and that an additional image sensor/camera is not being claimed.
As to claim 20, term “internal structure” lacks antecedence. It appears this should more accurately be “internal structures” to be consistent with claim 19.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4, 8 and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe et al. (US 2010/0053312, hereinafter “Watanabe”) in view of Perry et al. (US 2024/0280490, hereinafter “Perry”).
As to claim 1, Watanabe discloses an endoscope (endoscope 1, Fig.1) having a light source (light source 3, Fig.1), a first lumen (space in endoscope shaft 2 that accommodates light guide 10, Fig.1) configured for delivering illumination from the light source to a target area ([0072]), and a second lumen (space in endoscope shaft 2 that accommodates lens system 11 and image guiding fiber 9, Fig.1) and including a lens (lens system 11) and a fiber-optic imaging system (image guiding fiber 9) configured for providing real-time images from the target area ([0071]); a sleeve (distal cylindrical portion of shaft including windows 7 and 6, Fig.1) attached to an end of the endoscope (as shown in Fig.1); a beam splitter (dichroic mirror 8, Figs.1,7) positioned within the sleeve in front of the lens (Figs.1,7), wherein the beam splitter is configured to separate incoming light into two or more distinct wave length ranges whereby to separate imaging or analysis of different regions in the electromagnetic spectrum (splits incoming light into light L2 (reflected) and light L4 (transmitted), Fig.1, [0077]-[0078], light L2 in first wavelength band and light L4 of second wavelength band, [0079]); a visible light imaging sensor or camera configured to capture images or data in the visible wavelength range (CCD 17, configured to receive visible reflection light L4, [0028], making it a visible light imaging sensor); an NIR or SWIR imaging sensor or camera configured to capture images or data in the NIR or SWIR wavelength range (CCD 16, configured to receive the fluorescence light L2, which, as modified below with respect to Perry, is in the IR/NIR/SWIR wavelength range, making it a NIR or SWIR imaging sensor); and a processor (image combining unit 20, Fig.1) configured to process and analyze light data captured by the endoscope and provide a composite real-time image of the target area including real time images in the visible spectrum L4 and real time images in the fluorescence spectrum L2 (combines image data into a composite image G1/G2, Fig.3, [0083]-[0084]; processed CCD images provide real time images).
Watanabe discloses that the beamsplitter separates the incoming light into two distinct wavelength ranges (Watanabe exemplifies one range as being for reflected visible light and the other range for fluorescence light from a substance within the body, [0027]-[0028]) but does not disclose the particulars of such wavelength ranges, and specifically, does not disclose that the wavelength ranges are the visible light spectrum and the near infrared/shortwave infrared (NIR/SWIR) wavelengths. Perry evidences that it is known in the medical imaging art, including endoscopes ([0008]), to simultaneously obtain and combine visible light image data and fluorescent light data in the IR/SWIR/NIR range ([0038],[0149], infrared can include wavelengths from 700 nm to 3000 nm, [0453]) using beamsplitters to separate the wavelength ranges ([0005]), such wavelength ranges being diagnostically beneficial ([0001]-[0003]). In view of these wavelength ranges being diagnostically beneficial, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have configured the two distinct wavelength ranges in Watanabe as being the visible light spectrum and NIR/SWIR light spectrum. Accordingly, in makings such modification, the composite image generated by Watanabe would include real time images in the visible spectrum and real time images in the NIR or SWIR wavelength range.
As to claim 2, the beam splitter comprises a dichroic mirror or a prism (dichroic mirror/prism 8, [0068], [0090], Figs.1,7).
As to claim 4, (assumed to be redundant, see 112(b) rejection above) Watanabe further discloses a visible light imaging sensor or camera configured to capture images or data in the visible wavelength range (CCD 17, configured to receive visible reflection light L4, [0028]).
As to claim 8, Watanabe further discloses a user interface configured to allow users to view and analyze the composite image, and/or adjust settings of the endoscope (display 5, Fig.1, constitutes a user interface configured to allow a user to view and analyze the composite image).
As to claim 19, Watanabe discloses a method for inspecting internal structures comprising providing an endoscope device (endoscope 1, Figs.1,7) that utilizes a beam splitter (dichroic mirror 8, Fig.1,7) configured to separate incoming light into two or more distinct wavelength ranges (splits incoming light into light L2 (reflected) and light L4 (transmitted), Fig.1, [0077]-[0078]), comprising the steps of: attaching a sleeve to the end of an endoscope (distal cylindrical portion of shaft including windows 6,7 attached to distal end of endoscope 1, Fig.1), positioning a beam splitter within the sleeve in front of the endoscope's lens (shown positioned in front of lens system 11 in Fig.1), and capturing real time images or data by using the beam splitter to separate incoming light into two or more distinct wavelength ranges (light L2 in first wavelength band and light L4 of second wavelength band, [0079], are transmitted to CCD image sensors 16 and 17, [0081], Fig.1; CCD imagers provide real time images).
Watanabe discloses that the beamsplitter separates the incoming light into two distinct wavelength ranges (Watanabe exemplifies one range as being for reflected visible light and the other range for fluorescence light from a substance within the body, [0027]-[0028]) but does not disclose the particulars of such wavelength ranges, and specifically, does not disclose that the wavelength ranges are the visible light spectrum and the near infrared/shortwave infrared (NIR/SWIR) wavelengths. Perry evidences that it is known in the medical imaging art, including endoscopes ([0008]), to simultaneously obtain and combine visible light image data and IR/SWIR/NIR light image data ([0038],[0149], infrared can include wavelengths from 700 nm to 3000 nm, [0453]) using beamsplitters to separate the wavelength ranges ([0005]), such wavelength ranges being diagnostically beneficial ([0001]-[0003]). In view of these wavelength ranges being diagnostically beneficial, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have configured the two distinct wavelength ranges in Watanabe as being the visible light spectrum and NIR/SWIR light spectrum.
As to claim 20, wherein the internal structures comprises an animal internal structure (body cavity, [0067]).
Claim(s) 5-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe et al. (US 2010/0053312, hereinafter “Watanabe”) in view of Perry et al. (US 2024/0280490, hereinafter “Perry”), as set forth above with respect to claim 1, and further in view of Chen (US 2022/0361738).
As to claims 5-7, Watanabe, as set forth above with respect to claim 1, discloses a generic image processing arrangement (e.g. imaging generating units 18,19, image combining unit 20) that processes and combines the obtained images from the different wavelength bands for comparison in a composite image, but fails to disclose the specific processor arrangement for doing this, including wherein a hyper-spectral fusion Artificial Intelligence (AI) system to combine data captured in different wavelength ranges by the beam splitter and to create a composite image. However, Chen teaches, in a similar endoscope system which obtains and combines images of different wavelength bands (Chen: Fig.10, [0001]-[0003]), that a AI driven image processing algorithms can be used to combine and compare the different images for analysis ([0049]). Since Watanabe does not disclose the particulars of the image processing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used image processing techniques known in the art for combining and comparing images, including AI driven image processing algorithms (e.g. AI hyper-spectral fusion), as taught by Chen.
Claim(s) 10-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe et al. (US 2010/0053312, hereinafter “Watanabe”) in view of Perry et al. (US 2024/0280490, hereinafter “Perry”) and Chen (US 2022/0361738), as set forth with respect to claim 5, and further in view of He et al. (US 2023/0242912, hereinafter “He”).
As to claims 10-12, Watanabe, as modified by Perry and Chen as set forth above with respect to claim 5, disclose an AI system for aiding in image processing, but fails to disclose that the AI system is further configured to analyze the images to provide a diagnosis by detecting abnormalities or disease. However, He teaches, in an endoscopic imaging system, to employ a trained AI system (neural models, [0004] that are trained by image datasets, [0116]) to provide a diagnosis and/or risk assessment by detecting abnormalities (e.g.[0117]-[0119]). He teaches that use of such AI detection in endoscopy is beneficial because it can improve detection rate ([0002]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have provided the AI system with the further ability to detect and diagnose abnormalities in the images of Watanabe to improve detection rate, as taught by He.
Claim(s) 13-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe et al. (US 2010/0053312, hereinafter “Watanabe”) in view of Perry et al. (US 2024/0280490, hereinafter “Perry”), as set forth above with respect to claim 1, and further in view of Hale et al. (US 2005/0113643).
As to claim 13-15, Watanabe, as set forth above with respect to claim 1, fails to disclose that the image comprises a topographical image. However, Hale teaches, in the endoscope art, that it is known to use image processing software on a computer (processor) ([0017]) to process the images to form topographical images (Fig.4B, [0022]) to generate a 3D representation of the target image (Fig.4D, [0022]). Hale teaches that doing so provides an enhanced, versatile, and more realistic representation of structures viewed by the endoscope ([0027]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have provided the processing arrangement of Watanabe with the ability to form topographical images for the desirable reasons taught by Hale.
As to claim 16, Watanabe, as modified by Hale above, further discloses a user interface configured to allow users to view the 3D representation (display unit 5 used to display images for a user).
As to claim 17, the processor is configured to employ a mathematical model method selected from the group consisting of triangulation, surface reconstruction, and volumetric representation, to create topographical images (as modified by Hale, surface reconstruction or volumetric representation can be used to create the images, Hale: [0022]).
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe et al. (US 2010/0053312, hereinafter “Watanabe”) in view of Perry et al. (US 2024/0280490, hereinafter “Perry”), as set forth above with respect to claim 1, and further in view of Fouts et al. (US 2022/0183760, hereinafter “Fouts”).
As to claim 18, Watanabe, as set forth above with respect to claim 1, generates 2D images of the anatomy and fails to disclose that the processor is configured to employ machine loading algorithms selected from the group consisting of deep neural networks and convolutional neural networks that are mathematically described and analyzed using optimization and gradient descent algorithms, to create 3D modeling images. However, Fouts teaches in the medical imaging art that it is known to generate 3D model images of an anatomical target from 2D images ([0054]) using machine learning algorithms such as deep neural networks and convolutional neural networks ([0073]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used known machine learning algorithms to create 3D model images from 2D images in order to provide the predictable result of aiding a viewer of the images by providing enhanced and more realistic representation being imaged ([0006]).
Response to Arguments
Rejections and objections from the previous Office Action that have not been repeated in this Office Action should be considered as addressed or corrected, and thus hereby withdrawn.
Applicant's arguments filed April 6, 2026 have been fully considered but they are not persuasive.
Regarding the 112(a) rejection of claim 1, Applicant provides no explanation that would answer any of the questions raised by the Examiner in the 112(a) rejection, which has been maintained above. This is not unexpected since Applicant’s disclosure does not provide enough information to answer these questions. Although Applicant appears to explain how the different wavelength range images are obtained by the cameras, this explanation fails to explain why the wavelength ranges need to be split and separated at the distal end of the endoscope, only to be recombined when transmitted through the fiber-optic imaging system (see Arrangement 1 figure appearing in the 112(a) rejection above), where, at the proximal end of such fiber-optic imaging system, it would appear that such wavelength range images would have to be split once again for directing into the respective cameras. No part of the disclosure teaches or suggests how light travels from the proximal end of the fiber-optic imaging system to two separate cameras. Furthermore, although Applicant fails to explain whether or not the “target area” comprises the SAME overlapping region for each obtained wavelength range image or DIFFERENT regions within the target area, if it is assumed that the wavelength range images (i.e. visible image and IR image) are of the SAME overlapping region of the target area (i.e. images of the same object, just each image of a different wavelength range), it is still not clear how light from both wavelength ranges, which has been split by the beamsplitter at the distal end of the endoscope (see Arrangement 2 figure appearing in the 112(a) rejection above), enters the lens of the fiber-optic imaging system, given that the beamsplitter directs both light beams in different directions. Applicant’s disclosure fails to provide adequate information as to what exactly Applicant contemplated in order to achieve the desired results.
Regarding the 112(a) rejection of claim 9, Applicant points to the specification “beginning at paragraph [0041]” to support the missing information. Neither paragraph [0041] nor the remaining specification offers any insight as to what “beam splitter settings” are, and how the “beam splitter settings” are “automatically controlled” by the processor. Thus, such rejection is being maintained above.
As for the 103 rejection over Watanabe in view of Perry, Applicant initially argues that Watanabe doesn’t teach “a visible light imaging sensor or camera configured to capture images or data in the visible wavelength range; an NIR or SWIR imaging sensor or camera configured to capture images or data in the NIR or SWIR wavelength range; a processor configured to process and analyze light data captured by the endoscope and provide a composite real time image of the target area including real time images in the visible spectrum and real time images in the NIR or SWIR wavelength range”. The Examiner respectfully disagrees. Watanabe does provide for a visible light imaging sensor/camera (CCD 17) and a fluorescence light imaging sensor/camera (CCD 16) which, in view of Perry, will obtain images in the NIR or SWIR wavelength range as the fluorescent light, making it a NIR or SWIR imaging sensor/camera. In addition, Watanabe immediately (e.g. no intermediate storage) processes and displays a composite of the visible spectrum image and the fluorescent image (e.g. NIR or SWIR, as per Perry), making them real-time images (see Fig.3, images G1 and G2). Thus, Watanable in view of Perry does teach this.
Applicant further argues that Watanabe uses a beam splitter for an entirely different purpose (assume different from Applicant’s purpose), namely to increase brightness. As the Examiner is not sure what this as to do with Watanabe’s structure meeting the claim limitations, this argument is deemed irrelevant. In addition, Applicant contends that the Examiner acknowledges that Watanabe operates only in a visible light wavelength range. This is false. As acknowledged by the Examiner in the previous rejection, Watanabe separates out TWO different wavelength ranges, one being visible wavelengths and one being of a fluorescence wavelength range, which is not defined by Watanabe and thus is not necessarily in the visible wavelength band. Perry teaches that fluorescence wavelengths can occur in the NIR or SWIR wavelength range, depending on the target composition or desired image characteristics. Thus, the Watanabe/Perry combination teaches a structure and functionality substantially similar to that of Applicants.
Applicant further argues that modifying Watanabe with Perry “would require a complete reconstruction of Watanabe”. This Examiner respectfully disagrees. Applicant provides no evidence as to what in Watanabe requires reconstruction. It appears that modifying the wavelength ranges obtained by the beamsplitter of Watanabe is merely an academic selection of a specific material to provide the desired characteristics.
As to the rejections of claims 5-7 and 10-18 over various references, Applicant relies the arguments with respect to Watanabe and Perry, and does not separately argue the rejections of these claims, or the references applied therein with any reasonable particularity. Thus, they stand or fall together with the claim with which they are grouped. In re Nielson, 816 F.2d 1567, 2 USPQ2d 1525 (Fed. Cir. 1987); In re Kaslow, 707 F.2d 1366, 1376, 201 USPQ 67, 70 (CCPA 1979).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN P LEUBECKER whose telephone number is (571)272-4769. The examiner can normally be reached Generally, M-F, 5:30-2:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan T Nguyen can be reached at 571-272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN P LEUBECKER/Primary Examiner, Art Unit 3795
Prosecution Insights