DETAILED ACTION
This action is responsive to the Amendments and Remarks received 03/30/2026 in which no claims are cancelled, claims 1 and 5–12 are amended, and claims 13–19 are added as new claims.
Response to Arguments
On page 11 of the Remarks, Applicant contends Examiner’s interpretation of the digital tracking feature of claim 1 is in error. However, Applicant’s explanation is unclear. To argue, as Applicant does, that “the digital tracking processing involves changing the position of the display area within the larger captured image…” is exactly the same as Examiner’s interpretation of centering a region of interest within a view. The argument is merely one of semantics and does not represent a real, technological difference. For example, one cannot center a region of interest within a displayed image without the image being larger than what is displayed. Centering means adding content to one side and subtracting content from the other side, meaning that the actual image is larger than what is displayed so as to have content to add to and subtract from. Applicant’s argument is unreasonable in view of the level of skill in the art. Therefore, Examiner is unpersuaded of error.
On page 11 of the Remarks, Applicant contends the prior art is silent regarding detection of positional relationships between first and second target regions and prescribed regions of interest. Examiner disagrees. The mere term, “tracking,” teaches the averred features to one of ordinary skill in the art. A digitally tracked image feature, such as an instrument tip, is found within the image and its position is compared to a center position of the displayed image. Based on the difference between the instrument tip, typically defined by a bounding box, and the centerpoint of the image, offset coordinates are obtained so that the system knows how much (x, y) positional distance is required to center the salient feature within the image. Then, those coordinates are sent to a physical platform capable of making physical adjustments commensurate with the digital coordinates and physical adjustments are made to correspond to the digital adjustments called for. Therefore, contrary to Applicant’s arguments, the prior art teaches digital and physical positional relationships between regions of interest and target regions as claimed. Accordingly, Examiner is unpersuaded of error.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1–8, and 10–19 are rejected under 35 U.S.C. 103 as being unpatentable over Bihlmaier A. (2016). Learning Dynamic Spatial Relations, 221 (herein “Bihlmaier”) and Zinchenko K., Song K.-T. (2021). Autonomous Endoscope Robot Positioning Using Instrument Segmentation with Virtual Reality Visualization. IEEE Access 9, 72614–72623 (herein “Zinchenko”).
Regarding claim 1, the combination of Bihlmaier and Zinchenko teaches or suggests an endoscope system comprising: an endoscope that comprises an imaging element and that acquires an image of a visual field of the imaging element as a captured image; a drive mechanism configured to move the visual field; and a processor configured to control the drive mechanism (Zinchenko, Section 1: describes prior art control algorithms for an endoscope “to maintain a centered camera position, focusing on the instruments within the view field.”; Zinchenko, Section 2: describes maintaining a region of interest in the field of view of an endoscope using tool location and surgeon’s gaze (RoA) wherein the “the system calculates the next RoA and desired camera view position for robot control, knowing the surgeon’s attention focus and the detected instruments’ locations, and the current center of the endoscope image.”), wherein the processor is configured to: detect a prescribed region of interest in the captured image; select a portion of the captured image as a display area; detect a positional relationship between a first target region in the display area and the prescribed region of interest (Zinchenko, Section 2: describes maintaining a region of interest in the field of view of an endoscope using tool location and surgeon’s gaze (RoA) wherein the “the system calculates the next RoA and desired camera view position for robot control, knowing the surgeon’s attention focus and the detected instruments’ locations, and the current center of the endoscope image.”; Examiner interprets the positional relationship to be the difference between the region of interest and the current center of the image and that such a vector is necessary to make adjustments to the field of view of the scope); detect a positional relationship between a second target region in the captured image and the prescribed region of interest (Zinchenko, Section 2: describes maintaining a region of interest in the field of view of an endoscope using tool location and surgeon’s gaze (RoA) wherein the “the system calculates the next RoA and desired camera view position for robot control, knowing the surgeon’s attention focus and the detected instruments’ locations, and the current center of the endoscope image.”; Examiner interprets the positional relationship to be the difference between the region of interest and the current center of the image and that such a vector is necessary to make adjustments to the field of view of the scope); and execute a combination of digital tracking processing and physical tracking processing(Examiner maintains the struck-through language of the original claim because it added important context Applicant apparently thought was too enlightening; Applicant’s shift away from subject matter drawn to centering and tracking in favor of more obscure language is noteworthy; Zinchenko, Section 1: describes prior art control algorithms for a tracking endoscope “to maintain a centered camera position, focusing on the instruments within the view field.”; Zinchenko, Section 2: describes maintaining a region of interest in the field of view of an endoscope using tool location and surgeon’s gaze (RoA) wherein the “the system calculates the next RoA and desired camera view position for robot control, knowing the surgeon’s attention focus and the detected instruments’ locations, and the current center of the endoscope image.”; The combination of digital and physical tracking can be referred to as “visual servoing”; Bihlmaier, Section 5.2.1: teaches digital tracking processing to track instrument tips within digital images utilizing machine vision techniques and listing the work of Uecker, for example, which describes visual servoing (see listing of Uecker under Conclusion Section of this Office Action)), wherein the digital tracking processing comprises changing, based on the positional relationship between the first target and the prescribed region of interest, a position of the display area in the captured image in a direction in which the first target region approaches the prescribed region of interest (Examiner maintains the struck-through language of the original claim because it added important context Applicant apparently thought was too enlightening; Applicant’s shift away from subject matter drawn to centering and tracking in favor of more obscure language is noteworthy; Examiner interprets the digital tracking as digitally calculating the center of the region of interest and comparing it to the digitally calculated current center of the field of view of the camera to find the difference therebetween; Zinchenko, Section 2: describes maintaining a region of interest in the field of view of an endoscope using tool location and surgeon’s gaze (RoA) wherein the “the system calculates the next RoA and desired camera view position for robot control, knowing the surgeon’s attention focus and the detected instruments’ locations, and the current center of the endoscope image.”; Bihlmaier, Section 5.2.1: teaches digital tracking processing to track instrument tips within digital images utilizing machine vision techniques and listing the work of Uecker, for example, which describes visual servoing (see listing of Uecker under Conclusion Section of this Office Action)), and wherein the physical tracking processing comprises moving, based on the positional relationship between the second target region and the prescribed region of interest, the visual field in a direction in which the second target region approaches the prescribed region of interest by controlling the drive mechanism (Examiner maintains the struck-through language of the original claim because it added important context Applicant apparently thought was too enlightening; Applicant’s shift away from subject matter drawn to centering and tracking in favor of more obscure language is noteworthy; Examiner interprets the physical tracking as robotically controlling the center of field of view of the endoscopic camera by articulating the distal end of the endoscope to move the camera’s field of view to a desired field of view defined by the centerpoint of the field of view; Zinchenko, Section 2: describes maintaining a region of interest in the field of view of an endoscope using tool location and surgeon’s gaze (RoA) wherein the “the system calculates the next RoA and desired camera view position for robot control, knowing the surgeon’s attention focus and the detected instruments’ locations, and the current center of the endoscope image.”; Bihlmaier, Section 5.2.1: teaches digital tracking processing to track instrument tips within digital images utilizing machine vision techniques and listing the work of Uecker, for example, which describes visual servoing (see listing of Uecker under Conclusion Section of this Office Action)).
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Bihlmaier, with those of Zinchenko, because both references are drawn to the same field of endeavor such that one wishing to practice Autonomous Robotic Endoscope Positioning (and tracking) would have been led to their relevant teachings and because Zinchenko’s control algorithm to maintain a centered camera position benefits from Bihlmaier’s teaching of controlling unnecessary or disturbing continuous movement (e.g. Bihlmaier, Chapter 2 and Section 6.2.2: teaches motion hysteresis wherein deadzones control whether tracking is disabled so that, when the region of interest is sufficiently centered, smooth motions that are not disturbing to the user are obtained). Thus, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Bihlmaier and Zinchenko used in this Office Action unless otherwise noted.
Regarding claim 2, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein the processor is configured to execute the digital tracking processing and the physical tracking processing by means of parallel processing (Examiner notes parallel processing is an obvious way to speed up computational tasks; Bihlmaier, Section 5.3.4: teaches shared-memory parallelism for performing algorithms to speed up processing; Bihlmaier, Fig. 6.11: teaches steps of the tracking algorithm may be implemented in parallel; Zinchenko, Section IV: teaches using parallel pipelines for running the tracking algorithms).
Regarding claim 3, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 2, wherein the processor is configured to start the digital tracking processing and subsequently start the physical tracking processing (Original claims 3 and 4 demonstrate the choice of order of the initialization is arbitrary; Examiner finds the processes are closed-loop in the sense that the digital tracking informs the physical movement and the physical movement informs the digital tracking; Thus, in such a feedback loop there is no technological advantage to which process starts first; see Zinchenko, Introduction: teaching, “To correctly position the endoscope, the robot must acknowledge the surgeon's region of attention (RoA). The surgeon's viewing direction and the objects of interest inside the viewing area can help to define such a region. Typically objects of interest are surgical instruments. Therefore, the surgical instruments' position detection is one of the critical tasks for robotic system automation. In the fully robotized operation, when the robot handles all surgical tools, it is possible to rely on robot kinematic to estimate the instruments' position and relate them to the current camera location.”; In other words, knowledge of kinematics can help track the object digitally and digital tracking can help the kinematics track the object/region of interest).
Regarding claim 4, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 2, wherein the processor is configured to start the physical tracking processing and subsequently start the digital tracking processing (Original claims 3 and 4 demonstrate the choice of order of the initialization is arbitrary; Examiner finds the processes are closed-loop in the sense that the digital tracking informs the physical movement and the physical movement informs the digital tracking; Thus, in such a feedback loop there is no technological advantage to which process starts first; see Zinchenko, Introduction: teaching, “To correctly position the endoscope, the robot must acknowledge the surgeon's region of attention (RoA). The surgeon's viewing direction and the objects of interest inside the viewing area can help to define such a region. Typically objects of interest are surgical instruments. Therefore, the surgical instruments' position detection is one of the critical tasks for robotic system automation. In the fully robotized operation, when the robot handles all surgical tools, it is possible to rely on robot kinematic to estimate the instruments' position and relate them to the current camera location.”; In other words, knowledge of kinematics can help track the object digitally and digital tracking can help the kinematics track the object/region of interest).
Regarding claim 5, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein, in the digital tracking processing, changing the position of the display area is restricted within a prescribed area of the captured image (Examiner interprets this limitation as conveying the concept of motion hysteresis and dead zones or buffer zones; Zinchenko, Section II.B and Fig. 3: teaches a buffer zone to distinguish between deliberate and aimless motion to add stability to the user’s field of view; Bihlmaier, Chapter 2 and Section 6.2.2: teaches motion hysteresis wherein dead zones control whether tracking is disabled so that, when the region of interest is sufficiently centered, smooth motions that are not disturbing to the user are obtained).
Regarding claim 6, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein, after the first target region reaches the prescribed region of interest, the processor is configured to reduce a moving speed of the visual field in the physical tracking processing (Bihlmaier, Section 6.2.2: teaches motion hysteresis wherein dead zones control whether tracking is disabled so that, when the region of interest is sufficiently centered, smooth motions that are not disturbing to the user are obtained and further teaches slowing down the movement using a time-decaying motion hysteresis; see also Bihlmaier, pgs. 83–84: teaching velocity-based control of robotics; see also Bihlmaier, 6.1.4: teaching maximum velocity at which no motion blur occurs).
Regarding claim 7, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein: a dead zone including the second target region is set in a central portion of the captured image; and the processor is configured to end the physical tracking processing when the prescribed region of interest enters the dead zone (Examiner interprets this limitation as conveying the concept of motion hysteresis and dead zones or buffer zones; Zinchenko, Section II.B and Fig. 3: teaches a buffer zone to distinguish between deliberate and aimless motion to add stability to the user’s field of view; Bihlmaier, Chapter 2 and Section 6.2.2: teaches motion hysteresis wherein dead zones control whether tracking is disabled so that, when the region of interest is sufficiently centered, smooth motions that are not disturbing to the user are obtained).
Regarding claim 8, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein, in the physical tracking processing, the processor is configured to cause the visual field to move until the prescribed region of interest passes across the second target region (Examiner interprets this limitation as conveying the concept of motion hysteresis and dead zones or buffer zones; Zinchenko, Section II.B and Fig. 3: teaches a buffer zone to distinguish between deliberate and aimless motion to add stability to the user’s field of view wherein the system moves or stops moving according to when the area crosses the buffer zone; Bihlmaier, Chapter 2 and Section 6.2.2: teaches motion hysteresis wherein dead zones control whether tracking is disabled so that, when the region of interest is sufficiently centered, smooth motions that are not disturbing to the user are obtained).
Regarding claim 10, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein the drive mechanism includes a moving device that holds and moves the endoscope (Examiner interprets this limitation as a “robot endoscope holder” or similar as used in the art; Bihlmaier, page 103: teaches the technology relates to a “motorized endoscope holder”; Zinchenko, Fig. 2 and Section 2.C: teaches a “robotic endoscope holder”).
Claim 11 lists the same elements as claim 1, but in method form rather than apparatus form. Therefore, the rationale for the rejection of claim 1 applies to the instant claim.
Claim 12 lists the same elements as claim 1, but in CRM form rather than apparatus form. Therefore, the rationale for the rejection of claim 1 applies to the instant claim.
Regarding claim 13, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein the processor is configured to stop the digital tracking processing when the first target region reaches the prescribed region of interest, and execute only the physical tracking processing thereafter (Examiner notes this is conventional motion hysteresis behavior known in this art because otherwise if the tracking system were to continuously track, the view would never come to a standstill and be disturbing for the user; Therefore, zones are created so that the tracking system only becomes active when the view is too far from ideal; Bihlmaier, Chapter 2 and Section 6.2.2: teaches motion hysteresis wherein deadzones control whether tracking is disabled so that, when the region of interest is sufficiently centered, smooth motions that are not disturbing to the user are obtained; Zinchenko, Section II.B and Fig. 3: teaches a buffer zone to distinguish between deliberate and aimless motion to add stability to the user’s field of view).
Regarding claim 14, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein the processor is configured to execute the combination of the digital tracking processing and the physical tracking processing when a distance from the second target region to the prescribed region of interest exceeds a prescribed threshold (Examiner notes this is conventional tracking and motion hysteresis behavior; If the tracking system were to continuously track, the view would never come to a standstill and be disturbing for the user; Therefore, zones are created so that the tracking system only becomes active when the view is too far from ideal; Bihlmaier, Chapter 2 and Section 6.2.2: teaches motion hysteresis wherein deadzones control whether tracking is disabled so that, when the region of interest is sufficiently centered, smooth motions that are not disturbing to the user are obtained; Zinchenko, Section II.B and Fig. 3: teaches a buffer zone to distinguish between deliberate and aimless motion to add stability to the user’s field of view).
Regarding claim 15, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein the processor is configured to execute the combination of the digital tracking processing and the physical tracking processing until both the first target region and the second target region reach the prescribed region of interest (Examiner notes this is conventional tracking and motion hysteresis behavior; If the tracking system were to continuously track, the view would never come to a standstill and be disturbing for the user; Therefore, zones are created so that the tracking system only becomes active when the view is too far from ideal; Bihlmaier, Chapter 2 and Section 6.2.2: teaches motion hysteresis wherein deadzones control whether tracking is disabled so that, when the region of interest is sufficiently centered, smooth motions that are not disturbing to the user are obtained; Zinchenko, Section II.B and Fig. 3: teaches a buffer zone to distinguish between deliberate and aimless motion to add stability to the user’s field of view).
Regarding claim 16, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein the first target region is located at a center of the display area (Examiner interprets the digital tracking (defining the first target region) as digitally calculating the center of the region of interest and comparing it to the digitally calculated current center of the field of view of the camera to find the difference therebetween; Zinchenko, Section 2: describes maintaining a region of interest in the field of view of an endoscope using tool location and surgeon’s gaze (RoA) wherein the “the system calculates the next RoA and desired camera view position for robot control, knowing the surgeon’s attention focus and the detected instruments’ locations, and the current center of the endoscope image.”; Bihlmaier, Section 5.2.1: teaches digital tracking processing to track instrument tips within digital images utilizing machine vision techniques and listing the work of Uecker, for example, which describes visual servoing (see listing of Uecker under Conclusion Section of this Office Action)).
Regarding claim 17, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein the first target region is located in a vicinity of a center of the display area (Examiner interprets this claim in view of original claim 16 and as combining the centering feature with the motion hysteresis feature such that pure active centering would create jostling and be disturbing to the viewer; Examiner interprets the digital tracking (defining the first target region) as digitally calculating the center of the region of interest and comparing it to the digitally calculated current center of the field of view of the camera to find the difference therebetween; Zinchenko, Section 2: describes maintaining a region of interest in the field of view of an endoscope using tool location and surgeon’s gaze (RoA) wherein the “the system calculates the next RoA and desired camera view position for robot control, knowing the surgeon’s attention focus and the detected instruments’ locations, and the current center of the endoscope image.”; Bihlmaier, Section 5.2.1: teaches digital tracking processing to track instrument tips within digital images utilizing machine vision techniques and listing the work of Uecker, for example, which describes visual servoing (see listing of Uecker under Conclusion Section of this Office Action); If the tracking system were to continuously track, the view would never come to a standstill and be disturbing for the user; Therefore, zones are created so that the tracking system only becomes active when the view is too far from ideal; Bihlmaier, Chapter 2 and Section 6.2.2: teaches motion hysteresis wherein deadzones control whether tracking is disabled so that, when the region of interest is sufficiently centered, smooth motions that are not disturbing to the user are obtained; Zinchenko, Section II.B and Fig. 3: teaches a buffer zone to distinguish between deliberate and aimless motion to add stability to the user’s field of view).
Regarding claim 18, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein the second target region is located at a center of the captured image (Examiner interprets the physical tracking (defining the second target region) as robotically controlling the center of field of view of the endoscopic camera by articulating the distal end of the endoscope to move the camera’s field of view to a desired field of view defined by the centerpoint of the field of view; Zinchenko, Section 2: describes maintaining a region of interest in the field of view of an endoscope using tool location and surgeon’s gaze (RoA) wherein the “the system calculates the next RoA and desired camera view position for robot control, knowing the surgeon’s attention focus and the detected instruments’ locations, and the current center of the endoscope image.”; Bihlmaier, Section 5.2.1: teaches digital tracking processing to track instrument tips within digital images utilizing machine vision techniques and listing the work of Uecker, for example, which describes visual servoing (see listing of Uecker under Conclusion Section of this Office Action)).
Regarding claim 19, the combination of Bihlmaier and Zinchenko teaches or suggests the endoscope system according to claim 1, wherein the second target region is located in a vicinity of a center of the captured image (Examiner interprets this claim in view of original claim 18 and as combining the centering feature with the motion hysteresis feature such that pure active centering would create jostling and be disturbing to the viewer; Examiner interprets the physical tracking (defining the second target region) as robotically controlling the center of field of view of the endoscopic camera by articulating the distal end of the endoscope to move the camera’s field of view to a desired field of view defined by the centerpoint of the field of view; Zinchenko, Section 2: describes maintaining a region of interest in the field of view of an endoscope using tool location and surgeon’s gaze (RoA) wherein the “the system calculates the next RoA and desired camera view position for robot control, knowing the surgeon’s attention focus and the detected instruments’ locations, and the current center of the endoscope image.”; Bihlmaier, Section 5.2.1: teaches digital tracking processing to track instrument tips within digital images utilizing machine vision techniques and listing the work of Uecker, for example, which describes visual servoing (see listing of Uecker under Conclusion Section of this Office Action); If the tracking system were to continuously track, the view would never come to a standstill and be disturbing for the user; Therefore, zones are created so that the tracking system only becomes active when the view is too far from ideal; Bihlmaier, Chapter 2 and Section 6.2.2: teaches motion hysteresis wherein deadzones control whether tracking is disabled so that, when the region of interest is sufficiently centered, smooth motions that are not disturbing to the user are obtained; Zinchenko, Section II.B and Fig. 3: teaches a buffer zone to distinguish between deliberate and aimless motion to add stability to the user’s field of view).
Claims 9 is rejected under 35 U.S.C. 103 as being unpatentable over Bihlmaier, Zinchenko, Sramek (US 2022/0061927 A1), and Morishima (US 2020/0100655 A1).
Regarding claim 9, the combination of Bihlmaier, Zinchenko, Sramek, and Morishima teaches or suggests the endoscope system according to claim 1, wherein the imaging element is provided at the distal end of the endoscope (Morishima, Abstract: teaches an imaging element included in the distal end of the endoscope), and the drive mechanism comprises a bending portion provided in the endoscope and configured to move the distal end of the endoscope (Examiner notes this is typical of endoscopes; Sramek, ¶ 0117: teaches a distal bending portion of an endoscope).
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Bihlmaier and Zinchenko, with those of Sramek, because all three references are drawn to the same field of endeavor such that one wishing to practice Autonomous Robotic Endoscope Positioning would have been led to their relevant teachings and because Sramek’s distal end control for bending, which is a common feature of today’s endoscopes, would help control the position of the camera in Bilmaier’s and Zinchenko’s tracking system. Thus, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Bihlmaier, Zinchenko, and Sramek used in this Office Action unless otherwise noted.
One of ordinary skill in the art, before the effective filing date of the claimed invention, would have been motivated to combine the elements taught by Bihlmaier, Zinchenko, and Sramek, with those of Morishima, because all four references are drawn to the same field of endeavor such that one wishing to practice Autonomous Robotic Endoscope Positioning would have been led to their relevant teachings and because Morishima’s imaging element at a distal end of the endoscope is a common feature of today’s endoscopes such that it would have been obvious to include in Bilmaier’s and Zinchenko’s endoscope tracking systems. Thus, the combination is a mere combination of prior art elements, according to known methods, to yield a predictable result. This rationale applies to all combinations of Bihlmaier, Zinchenko, Sramek, and Morishima used in this Office Action unless otherwise noted.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Gruijthuijsen, Caspar et al. “Robotic Endoscope Control Via Autonomous Instrument Tracking.” Frontiers in robotics and AI vol. 9 832208. 11 Apr. 2022.
Uecker D. R., Lee C., Wang Y. F., Wang Y. (1995). Automated Instrument Tracking in Robotically Assisted Laparoscopic Surgery. J. Image Guid. Surg. 1, 308–325. This publication teaches visual servoing of a robot manipulator to apply robotic control of an endoscope given the current location of a tracked feature in the image plane and teaches parallelization to increase processing speed. For example, Uecker teaches, “Visual servoing, as applied to robotic control of a laparoscope, can be defined as ‘given the current location of the tracked feature (e.g., the tip of an instrument) in the image plane, how do we manipulate the scope so that the feature appears at the desired image location (e.g., center of the image)?’ The general form of the visual servoing algorithm drives the camera motion based on error between the desired and current feature locations in the image plane. Hence, the control law used by the servoing algorithm must relate the error in feature position to an appropriate camera command, which is then implemented by the robot.”
Osa T., Staub C., Knoll A. (2010). “Framework of Automatic Robot Surgery System Using Visual Servoing,” in 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE; ), 1837–1842.
Agustinos A., Wolf R., Long J. A., Cinquin P., Voros S. (2014). “Visual Servoing of a Robotic Endoscope Holder Based on Surgical Instrument Tracking,” in 5th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (Sao Paulo, Brazil: IEEE; ), 13–18. This publication teaches turning off tracking if the tip of the instrument stays within a zone centered around the center of the field of view (Section II.C.1).
J. Funda, R. H. Taylor, B. Eldridge, S. Gomory, and K. G. Gruben, ‘‘Constrained Cartesian motion control for teleoperated surgical robots,’’ IEEE Trans. Robot. Autom., vol. 12, no. 3, pp. 453–465, Jun. 1996. Section III discusses the autonomous repositioning of an endoscopic camera “so as to center a particular anatomical feature within the camera’s field of view.”
Eslamian S., Reisner L. A., Pandya A. K. (2020). Development and evaluation of an autonomous camera control algorithm on the da vinci surgical system. Int. J. Med. Robot 16, e2036.
Kai-Tai Song and Chun-Ju Chen. Autonomous and stable tracking of endoscope instrument tools with monocular camera. In Advanced Intelligent Mechatronics (AIM), 2012 IEEE/ASME International Conference on, pages 39–44, July 2012. This publication teaches buffer zones and motion hysteresis.
A Casals, J Amat, and E Laporte. Automatic guidance of an assistant robot in laparoscopic surgery. In Robotics and Automation, 1996. Proceedings., 1996 IEEE International Conference on, volume 1, pages 895–900. IEEE, 1996. This publication teaches deadzones and hysteresis to effectuate smooth motion for the user.
Tezuka (US 2021/0000329 A1), assigned to Applicant-Olympus, teaches an imaging element in a distal portion of an endoscope (¶ 0003).
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael J Hess whose telephone number is (571)270-7933. The examiner can normally be reached Mon - Fri 9:00am-5:30pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached on (571)272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8933.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J HESS/Examiner, Art Unit 2481
Prosecution Insights