Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/30/2026 has been entered.
Response to Amendment
Claims 1-21, 24, are pending. Claim 25 is allowed.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-10, 24 is/are rejected under 35 U.S.C. 102 as being anticipated by Christian (US 20220079415).
Regarding claim 1, Christian discloses an imaging device (FIGS. 1, 2) for performing image correction and/or adjustment, the device comprising:
one or more processors (image processing unit 134; para [0127]) that operate to:
receive a navigation and/or viewing controls or orientations data profile for a first imaging device (endoscope 120) and a navigation and/or viewing controls or orientations data profile for a second imaging device (microscope 101; FIG.1 annotated), the first imaging device and the second imaging device having different navigation and/or viewing controls or orientations (Image recording device 124, microscope 101; FIG.2; The images taken endoscope is rotated to align with the image taken by the microscope 101; Para [0124]);
receive a captured image or images (image 142; Para [0127]) captured by the first imaging device (endoscope 120) at a first position (FIG. 1);
determine an estimated image or images (Image 141 is the image that can be acquired when the microscope 101 is brought near the endoscope 101. In use, the microscope 101 can be moved in different position along a vertical line. ) that match, are based on, or represent an image or images that the second imaging device captures in a case where the second imaging device is used to image at the first position (Second imaging device is not positively claimed;), the estimated image or images having a different alignment or orientation than the alignment or orientation of the captured image or images (The unrotated image 142 and image 141 have different orientation. The processor 134 receives the images from the endoscope 120 and the microscope 101, and image from the microscope 101 is use to rotate the image from the endoscope 120 to align the image from the endoscope 120 with the image from the microscope 101.);
adjust or correct the captured image or images based on the estimated image or images such that the adjusted or corrected image or images correspond to, and have an alignment or orientation that matches an alignment or orientation of, the estimated image or images at the first position (The second image 142 captured by the endoscope 120, is rotated in such a way that the second viewing axis Y2 (of the endoscope) corresponds to the first viewing axis Y1 (of the microscope 101). The second image 142 is rotated by an angle in such a way that the second viewing axis Y2 is arranged vertically. The image captured by the endoscope 120 is rotated to align with the axis Y1; Para [0149]; also see [0192], [0192], and [0201], for similar embodiments.) where the adjustment or correction is based on a comparison of the data profiles of the first imaging device and the second imaging device to transform the alignment or orientation of the captured image or images of the first imaging device to match the respective estimated image or images (The processor 134 receives the images from the endoscope 120 and the microscope 101, and image from the microscope 101 is use to rotate the image from the endoscope 120 to align the image from the endoscope 120 with the image from the microscope 101; Navigation data profile or viewing control (image angle or orientation) of each of the device are used to align the images.); and
control, navigate, and/or orient the first imaging device using the adjusted or corrected image or images, and/or display the adjusted or corrected image or images based on the adjustment or correction (Image control; Display 140; Para [0127]).
PNG
media_image1.png
504
895
media_image1.png
Greyscale
Regarding claim 2, Christian discloses an including a display to display (FIG. 1; display [0140]) the adjusted or corrected image or images.
Regarding claim 3, Christian discloses an a receiver (control 130; FIG.1) that operates to receive the captured image or images captured by the first imaging device at the first position, the first imaging device being a first endoscope (endoscope 120), and transmit the capture image or images to the one or more processors (an image processing unit 134; para [0127]) such that the one or more processors receive the captured image or images;
a controller as being part of the one or more processors, the controller operating to obtain or determine the estimated image or images that would have been captured by the second imaging device at the first position (image captured by microscope 101 when it is moved to the position of endoscope 120, the captured image is captured at the position of the second imaging device; FIG. 1, annotated), the second imaging device being a second endoscope; and
a display controller (control 130) as being part of the one or more processors, the display controller or the one or more processors operating to provide an image or images for display, the image or images for display being based on the captured image or images and information for the estimated image or images (FIG. 1).
Regarding claim 4, Christian discloses wherein the second endoscope has a lower degree of freedom from a bendable degree of freedom of the first endoscope (Microscope 101 has a lower degree of freedom; Second endoscope is not positively claimed. The second endoscope having a lower degree of freedom from a bendable degree of freedom of the first endoscope does not add any further limitation regarding how the estimated image is obtained. This feature is not positively recited, and does not affect the capability of the recited processor.).
Regarding claim 5, Christian discloses wherein the first endoscope is a camera deployed at a tip of a steerable catheter (FIG. 1; endoscope 120; The first endoscope is not positively recited.) and is bent with the steerable catheter, and/or the camera is detachably attached to, or removably inserted into, the steerable catheter (These features are not positively recited, and do not affect the capability of the recited one or more processors.).
Regarding claim 6, Christian discloses a wherein the second endoscope is a virtual endoscope (The surgical microscope 101 can also be configured as a purely digital surgical microscope; Para [0129]; FIG. 1) or is represented by a preset or predetermined data profile (The second endoscope is also not positively recited and does not affect the capability of the recited one or more processors of the imaging device.).
Regarding 7, Christian discloses wherein the first endoscope has a bending section that operates to bend three-dimensionally and/or to bend on two or more planes (The first endoscope is not positively recited and does not affect the capability of the one or more processors of the imaging device that is recited.) and/or wherein the imaging device further comprises the first imaging device being the first endoscope, or the first imaging device being the first endoscope and the second imaging device being the second endoscope.
Regarding 8, Christian discloses wherein the second endoscope has a bending section that can bend only on one plane (The second endoscope is also not positively recited and does not affect the capability of the one or more processors of the imaging device that is recited.) and/or wherein the imaging device further comprises the first imaging device being the first endoscope, or the first imaging device being the first endoscope and the second imaging device being the second endoscope.
Regarding claim 9, Christian discloses an wherein the image or images for display are the captured image or images that are adjusted or corrected by rotation so that the orientation of the adjusted or corrected image or images for display corresponds to the orientation of the estimated image or images (The second image 142 captured by the endoscope 120, is rotated in such a way that the second viewing axis Y2 (of the endoscope) corresponds to the first viewing axis Y1 (of the microscope 101). The second image 142 is rotated by an angle in such a way that the second viewing axis Y2 is arranged vertically. The image captured by the endoscope 120 is rotated to align with the axis Y1; Para [0149]; also see [0192], [0192], and [0201] for similar embodiments), and wherein the display controller or the one or more processors display the image or images for display on a display (FIG. 1).
Regarding claim 10, Christian discloses an wherein the image or images for display comprise the captured image or images and an additional image or images, the additional image or images being the captured image or images that are rotated based on the information for the estimated image or images (The second image 142 captured by the endoscope 120, is rotated in such a way that the second viewing axis Y2 (of the endoscope) corresponds to the first viewing axis Y1 (of the microscope 101). The second image 142 is rotated by an angle in such a way that the second viewing axis Y2 is arranged vertically. The image captured by the endoscope 120 is rotated to align with the axis Y1; Para [0149]; also see [0192], [0192], and [0201] for similar embodiments).
Regarding claim 24, Christian discloses a non-transitory computer-readable storage medium storing at least one program for causing a computer to execute a method for performing image correction and/or adjustment (FIGS. 1-2), the method comprising:
receiving a navigation and/or viewing controls or orientations data profile for a first imaging device and a navigation and/or viewing controls or orientations data profile for a second imaging device, the first imaging device and the second imaging device having different navigation and/or viewing controls or orientations (endoscope 120, Image recording device 124; FIG. 2; Their images are rotated to align each other; para [0124]);
receiving a captured image or images (image 142; Para [0127]) captured by the first imaging device (endoscope 120) at a first position (FIG. 1);
determining an estimated image or images (image 141 is the image that can be acquired when the microscope is brought close to the endoscope. In use, the microscope 101 can be moved in different positions along a vertical line.) that match, are based on, or represent an image or images that the second imaging device captures in a case where the second imaging device is used to image at the first position (Second imaging device is not positively claimed;), the estimated image or images having a different alignment or orientation than the alignment or orientation of the captured image or images (The unrotated image 142 and image 141 have different orientation.);
adjusting or correcting the captured image or images based on the estimated image or images such that the adjusted or corrected image or images correspond to, and have an alignment or orientation that matches an alignment or orientation of, the estimated image or images at the first position (The second image 142 captured by the endoscope 120, is rotated in such a way that the second viewing axis Y2 (of the endoscope) corresponds to the first viewing axis Y1 (of the microscope 101). The second image 142 is rotated by an angle in such a way that the second viewing axis Y2 is arranged vertically. The image captured by the endoscope 120 is rotated to align with the axis Y1; Para [0149]; also see [0192], [0192], and [0201] for similar embodiments) where the adjusting or correcting is based on a comparison of the data profiles of the first imaging device and the second imaging device to transform the alignment or orientation of the captured image or images of the first imaging device to match the respective estimated image or images (The processor 134 receives the images from the endoscope 120 and the microscope 101, and image from the microscope 101 is use to rotate the image from the endoscope 120 to align the image from the endoscope 120 with the image from the microscope 101; Navigation data profile or viewing control (image angle or orientation) of each of the device are used to align the images); and
controlling, navigating, and/or orienting the first imaging device using the adjusted or corrected image or images, and/or displaying the adjusted or corrected image or images on a display based on the adjustment or correction (Image control; Display 140; Para [0127]).
PNG
media_image1.png
504
895
media_image1.png
Greyscale
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 11-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Christian (US 20220079415) in view of Kitamura (US 20120327186).
Regarding claim 11, Christian does not expressly disclose wherein the captured image or images and the additional image or images are displayed at the same time.
Kitamura is directed to a virtual endoscopic image generating unit for generating a virtual 3D medical image (abstract) and teaches wherein the captured image or images and the additional image or images are displayed at the same time (virtual endoscopic image and real endoscopic image is displayed side by side. Para [0068]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Christian to have images displayed side by side in accordance with the teaching of Kitamura so that both images could be compared and referenced conveniently.
Regarding claim 12, Christian, as modified, teaches wherein the display controller or the one or more processors further operate to display the additional image or images in accordance with a user instruction to display the additional image or images on the display (virtual endoscopic image and real endoscopic image is displayed side by side. Para [0068]).
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Christian (US 20220079415) in view of Buharin (US 20230316639).
Regarding claim 13, Christian does not expressly disclose wherein the display controller or the one or more processors further operate to switch the image or images for display from the captured image or images to the additional image or images.
Buharin is directed to systems and methods disclosed herein may be used to enhance medical imaging (para [0003]) and teaches wherein the display controller or the one or more processors further operate to switch the image or images for display from the captured image or images to the additional image or images (The one or more interfaces may permit the operator or medical worker to view the first image, the second image, and/or both the first and second images, and to switch between any of these views as desired. Para [0191]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Christian’s display system so that it can switch the image or images for display so that surgeon could view the images as needed.
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Christian (US 20220079415) in view of Ishiguro (US 5682895) and further in view of Ishikawa (US 20190008367).
Regarding claim 14, Christian does not expressly disclose an interface configured to receive a command to bend the first endoscope, wherein the display controller or the one or more processors further operate to temporarily suspend the rotation of the captured image or images while the interface is receiving the command.
Ishiguro is directed to an endoscope system including an endoscope, a moving device that moves the endoscope, a storage unit, and a processor (abstract) and teaches an interface configured to receive a command to bend the first endoscope (Rotation command is transmitted; Col. 3, line 64-col. 4, lines 40; FIGS. 2, 4; the first endoscope is not positively claimed.), wherein the display controller or the one or more processors further operate to temporarily suspend the rotation of the captured image or images (By way of an image rotation command menu which is indicated on the monitor screen, the operator can select either an "IMAGE ROTATION" or "NO IMAGE ROTATION" command (Step 6). Upon selecting an "IMAGE ROTATION" command, the original 3D image begins to rotate about Z-axis (Step 7), and stops its rotation as soon as a "STOP" command is entered (Step 8). Thus, the rotation of the original 3D image can be stopped when an image side of particular interest comes into view. col. 10, line 55-col 11, line 30).
Ishikawa is directed to a surgical system (abstract) and teaches wherein the images obtained during the stopped bending state of the endoscope are high quality as compared to the images obtained during the bending state of the endoscope (para [0079]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Christian to temporarily suspend the rotation of the captured image while the endoscope is in bending state so that image quality would be enhanced by way of not having to process the image signal during the bending of the endoscope.
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Christian (US 20220079415) in view of Alvarez (US 20100280449).
Regarding claim 16, Christian does not expressly disclose an interface that operates to receive a command to bend the first endoscope, wherein the interface receives the command corresponding to a bending direction or a twisting amount of the second endoscope.
Alvarez is directed to an instrument having a flexible and elongated body includes at least a lumen and a flex member disposed within the lumen (abstract) and teaches an interface (The wired connection (112) may transmit manipulation, articulation, and control commands from an operator or surgeon (116) who is working at the operator control station (102) and who may be providing the necessary input to the instrument driver (106) by way of one or more input devices, such as an instinctive Motion.TM. controller (118), joystick, keyboard (120), trackball, data gloves, exoskeletal gloves, or the like, for operating the instrument assembly (108) to perform various operations, such as minimally invasive procedures, on the patient who is lying on the operating table (104). The wired connection (112) may also transmit information (e.g., visual. Para [0055]) configured to receive a command to bend the first endoscope, wherein the interface receives the command corresponding to a bending direction or a twisting amount of the second endoscope (FIG. 1; Para [0072]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Christian to include commands for bending the endoscope so that automated bending could be achieved.
Claim(s) 17-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Christian (US 20220079415) in view of Alvarez (US 20100280449).
Regarding claim 17, Christian discloses at least one bending section and an endoscope camera (tilting of the camera of the endoscope 120); and receive one or more endoscopic images and display the one or more endoscopic images on a display (FIG.1).
Christian does not expressly disclose a steerable catheter; an actuation unit or a driver that operates to bend the bending section, wherein the controller or the one or more processors further operate to: receive one or more control commands or instructions for a bending amount and a bending plane orientation; and send the one or more commands or instructions to the actuation unit or the driver to bend the at least one bending section.
Alvarez is directed to an instrument having a flexible and elongated body includes at least a lumen and a flex member disposed within the lumen (abstract) and teaches a steerable catheter (catheter 204); an actuation unit or a driver that operates to bend the bending section (control unit, 202 may be configured to steer an elongate instrument or catheter 204; para [0056]), wherein the controller or the one or more processors further operate to: receive one or more control commands or instructions for a bending amount and a bending plane orientation (controlled bending in planes; para [0073]); and send the one or more commands or instructions to the actuation unit or the driver to bend the bending section (FIG. 1; para [0088], [0055]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Christian to include commands for bending the endoscope so that automated and controlled bending could be achieved.
Regarding claim 18, Christian, as modified, teaches one or more of the following: (i) the imaging device further comprises an operational controller or joystick (Alvarez: The wired connection (112) may transmit manipulation, articulation, and control commands from an operator or surgeon (116) who is working at the operator control station (102) and who may be providing the necessary input to the instrument driver (106) by way of one or more input devices, such as an instinctive Motion.TM. controller (118), joystick, keyboard (120), trackball, data gloves, exoskeletal gloves, or the like, for operating the instrument assembly (108) to perform various operations, such as minimally invasive procedures, on the patient who is lying on the operating table (104). The wired connection (112) may also transmit information (e.g., visual). Para [0055]; FIG. 1; Para [0072) that operates to issue or input the one or more commands or instructions to the controller or the one or more processors; (ii) the imaging device further includes a display to display the one or more endoscopic images, or the imaging device further includes a display to display the one or more endoscopic images where the display has a reference direction; (iii) the controller or the one or more processors further operate to store the bending plane orientation in the one or more control commands or instructions in relation to the one or more endoscopic images during navigation; (iv) the controller or the one or more processors further operate to rotate a current endoscopic image to an orientation where a bending plane orientation stored at the last moment or stored last is aligned to a reference direction of a display of the imaging device; and/or (v) the imaging device further comprises an operational controller or joystick that operates to issue or input the one or more commands or instructions to the controller or the one or more processors, and the operational controller or joystick operates to be controlled by a user of the imaging device.
Regarding claim 19, Christian, as modified, teaches an operational controller or joystick (Alvarez: The wired connection (112) may transmit manipulation, articulation, and control commands from an operator or surgeon (116) who is working at the operator control station (102) and who may be providing the necessary input to the instrument driver (106) by way of one or more input devices, such as an instinctive Motion.TM. controller (118), joystick, keyboard (120), trackball, data gloves, exoskeletal gloves, or the like, for operating the instrument assembly (108) to perform various operations, such as minimally invasive procedures, on the patient who is lying on the operating table (104). The wired connection (112) may also transmit information (e.g., visual). Para [0055]; FIG. 1; Para [0072), the operational controller or joystick having a rotation controller and a bending controller, wherein one or more of the following: the rotation controller operates to issue a control command or instruction of or for the bending plane orientation; the bending controller operates to issue a control command or instruction of or for the bending amount; and/or in a case where the controller or the one or more processors further operate to rotate a current endoscopic image to an orientation where a bending plane orientation stored at the last moment or stored last is aligned to a reference direction of a display of the imaging device, the controller or the one or more processors further operate to rotate the current endoscopic image as the rotation controller issues the control command or instruction of or for the bending plane orientation.
Regarding claim 20, Christian discloses at least one bending section and an endoscope camera (FIG. 1; Para [0210]) and receive one or more endoscopic images and display the one or more endoscopic images on a display (FIG. 1).
Christian does not expressly disclose a catheter with at least one bending section; an operational controller or joystick that operates to issue or input one or more commands or instructions of a bending amount and a bending plane orientation into the imaging device; and a tracking device that operates to track a real-time bending plane orientation, wherein the controller or the one or more processors operate to receive one or more endoscopic images and the one or more commands or instructions.
Alvarez is directed to an instrument having a flexible and elongated body includes at least a lumen and a flex member disposed within the lumen (abstract) and teaches a steerable catheter (catheter 204) with at least one bending section; an operational controller or joystick (The wired connection (112) may transmit manipulation, articulation, and control commands from an operator or surgeon (116) who is working at the operator control station (102) and who may be providing the necessary input to the instrument driver (106) by way of one or more input devices, such as an instinctive Motion.TM. controller (118), joystick, keyboard (120), trackball, data gloves, exoskeletal gloves, or the like, for operating the instrument assembly (108) to perform various operations, such as minimally invasive procedures, on the patient who is lying on the operating table (104). The wired connection (112) may also transmit information (e.g., visual). Para [0055]; FIG. 1; Para [0072) that operates to issue or input one or more commands or instructions of a bending amount and a bending plane orientation into the imaging device (controlled bending in a plane; para [0073]); and a tracking device that operates to track a real-time bending plane orientation (controller (118), joystick, track real time track; Para [0055]), wherein the controller or the one or more processors operate to receive one or more endoscopic images and the one or more commands or instructions.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Christian to include an operational controller or joystick that operates to input one or more commands or instructions for bending in accordance with the teaching of Alvarez so that so that automated and controlled bending could be achieved.
Regarding claim 21, Christian discloses the imaging device further comprises a display that operates to display the one or more endoscopic images (FIG. 1).
Response to Arguments
Applicant’s arguments submitted on 04/30/2026 have been fully considered and are persuasive. Therefore, the rejection dated 02/04/2026 have been withdrawn. However, upon further consideration, a new rejection has been made in view of amendment. See rejection set forth above.
Allowable Subject Matter
Claim 15 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 15
The following is an examiner’s statement of reasons for allowable subject matter:
The prior art of record fails to explicitly teach or fairly suggest, alone or in combination, an interface that operates to receive a command to bend the first endoscope, wherein the display controller or the one or more processors further operate to restrict an amount of the rotation of the captured image or images captured by the first endoscope in a case where an angle of the rotation of the captured image or images in accordance with the received command is larger than a predetermined angle of rotation, the amount of the rotation of the captured image or images corresponding to the bend of the first endoscope caused by the received command, along with the remaining features of claims 15, 9, 3, and 1.
The closest art Mizutani (US 20230180998) teaches controlling vertical direction of the image-by-image rotation but does not teach restricting an amount of the rotation of the captured image corresponding to the bend of the first endoscope caused by the received command.
Teranuma (US 20190183321) teaches the rotation of the image corresponding a position of a surgeon but does not teach associating this rotation of the image with the bending command received for bending of an endoscope to restrict an amount of the rotation of the captured image.
Accordingly, claim 15 is allowed.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO – 892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHANKAR R GHIMIRE whose telephone number is (571)272-0515. The examiner can normally be reached 8 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at 571-272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHANKAR RAJ GHIMIRE/Examiner, Art Unit 3795
/ANH TUAN T NGUYEN/Supervisory Patent Examiner, Art Unit 3795
5/26/26