Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of claims
The following claims have been rejected or allowed for the following reasons:
Claim(s) 1-23 is rejected under 35 USC § 103
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. 62577020, filed on 10/25/2017.
Information Disclosure Statement
The information disclosure statement/statements (IDS) were filed on 10/1/25, 8/21/24. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 7-11, 13, 17-20, 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over as applied to Itkowitz (US 20170000574 A1), in further view of Nowlin (US 20070013336 A1); in further view of Brandon (US 20110118748 A1);
Regarding claim 1 Itkowitz teaches An operator workstation for controlling a computer-assisted device, the operator workstation comprising: a repositionable arm; (Itkowitz [0028] reads “As shown in FIG. 1, computer-assisted system 100 includes a device 110 with one or more movable or articulated arms 120. Each of the one or more articulated arms 120 may support one or more end effectors 125. In some examples, device 110 may be consistent with a computer-assisted surgical device. The one or more end effectors 125 may include surgical instruments, imaging devices, and/or the like. In some examples, the surgical instruments may include clamps, grippers, retractors, cautery tools, suction tools, suturing devices, and/or the like. In some examples, the imaging devices may include endoscopes, cameras, stereoscopic devices, and/or the like.”);
an input control device physically coupled to the repositionable arm, the input control device configured to be moved by an operator to teleoperate an end effector that is physically separate from the operator workstation; (Itkowitz [0032] reads “Operator workstation 170 may further include a console workspace with one or more input or master controls 195 that may be used for operating the device 110, the articulated arms 120, and/or the end effectors 125. Each of the input controls 195 may be coupled to the distal end of their own articulated arms so that movements of the input controls 195 may be detected by the operator workstation 170 and communicated to control unit 130.” Itkowitz Figure 1 also depicts that the input control device could be used to control a robotic arm could be physically separated from the robotic manipulator.);
PNG
media_image1.png
601
543
media_image1.png
Greyscale
Itkowitz Figure 1
one or more actuators coupled to drive the repositionable arm and move the input control device; (Itkowitz [0040] reads “One or more movement commands are sent to one or more actuators of the articulated arm coupled to the imaging device to command and/or direct the imaging device to execute the view recentering move. One or more movement commands are also sent to one or more actuators of the articulated arms coupled to the input controls to command and/or direct the input controls to execute the input control recentering moves.”);
and a controller comprising processor circuitry, the controller coupled with the input control device and the one or more actuators, the controller being configured to: (Itkowitz [0029] reads “Operation of control unit 130 is controlled by processor 140. And although control unit 130 is shown with only one processor 140, it is understood that processor 140 may be representative of one or more central processing units, multi-core processors, microprocessors, microcontrollers, digital signal processors, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and/or the like in control unit 130.”);
receive sensor data from the first sensor; make an interaction determination based on the sensor data, the interaction determination indicative of whether the input control device has received an operator interaction for controlling a position or an orientation or a motion of the input control device; and in response to the interaction determination being indicative of the input control device not having received the operator interaction: determine a trajectory for the repositionable arm that moves the input control device from a current position or orientation toward a desired position or orientation; (Itkowitz [0101] reads “At a process 1015, it is determined whether one or more of the motion input controls is being used. In some examples, upon entering the imaging device motion mode during process 1010, a timeout period may begin. During the timeout period the one or more motion input controls may be monitored to determine whether the operator is attempting to manually control the position and/or orientation of the imaging device using the one or more motion input controls. … When the timeout period ends with no use of the one or more motion input controls, recentering begins starting with a process 1020. When use of the one or more input controls is detected during the timeout period, manual control of the imaging device begins with a process 1040.” And [0009] reads “determining one or more input control recentering moves to provide positional and orientational harmony between each of the input controls and a corresponding one of the end effectors, executing the view and input control recentering moves, and reinstating teleoperated control of the end effectors by the input controls.”);
and cause the one or more actuators to drive the repositionable arm and move the input control device based on the trajectory. (Itkowitz [0040] reads “At the process 260, the view and input control recentering moves are coordinated. One or more movement commands are sent to one or more actuators of the articulated arm coupled to the imaging device to command and/or direct the imaging device to execute the view recentering move. One or more movement commands are also sent to one or more actuators of the articulated arms coupled to the input controls to command and/or direct the input controls to execute the input control recentering moves.”);
Itkowitz does not teach a sensor system physically coupled to the input control device or another part of the operator workstation, the sensor system comprising a first sensor, the first sensor being a contact or proximity sensor;
Nowlin in analogous art, teaches the first sensor being a contact or proximity sensor; (Nowlin [0186] reads “The sensors may comprise contact and/or proximity sensors, and may be monitored on a real time or near real time basis by the controller.”);
It would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention to have modified the teachings of Itkowitz with that of Nowlin to provide a surgical system that encompasses a robotic arm with advanced sensor capabilities. This would allow the system to make more precise movements that could result in better patient outcomes. (Nowlin [0004] reads “For these and other reasons, it would be advantageous to provide improved devices, systems, and methods for surgery, robotic surgery, and other robotic applications. It would be particularly beneficial if these improved technologies provided a faster and easier set-up, and/or inhibited collisions of the robotic devices during use. Ideally, these improvements would be provided without significantly increasing the size, mechanical complexity, or costs of these systems, and while maintaining or improving their dexterity.”);
Nowlin Itkowitz does not teach a sensor system physically coupled to the input control device or another part of the operator workstation, the sensor system.
Brandon in analogous art, teaches a sensor system physically coupled to the input control device or another part of the operator workstation, the sensor system (Brandon [0091] reads “In addition, display device 160 includes a surgeon presence switch. When surgeon 101 is facing display device 160 and is within a range of the surgeon presence switch, the switch provides a signal to control system 190 that allows control system 190 to enter and stay in the following mode. When surgeon 101 is either not facing display device 160 or not within the range of the switch, the surgeon presence switch provides a signal to control system 190 that inhibits control system 190 from being in the following mode. In one aspect, one or more infrared (IR) range sensors are used for sensing close-range distances of surgeon 101 from display 160 or alternatively from surgeon's viewer 361.”);
It would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention to have modified the teachings of Itkowitz/Nowlin with that of Brandon to include a method for better sensing the operators position and attention. This would allow for the robotic system to more precisely move in a manner that would be closer to that that the operator intended. (Brandon [0004-0005] reads “The da Vinci® Surgical System, manufactured by Intuitive Surgical, Inc., Sunnyvale, Calif., is a minimally invasive, teleoperated robotic surgical system that offers patients many benefits, such as reduced trauma to the body, faster recovery and shorter hospital stay. The da Vinci® Surgical System provides intuitive and ergonomic control of minimally invasive slave surgical instruments, which provides telepresence for the surgeon. This system incorporates a dedicated surgeon console, which provides a three-dimensional stereo viewer, two master tool manipulators, foot pedals for controlling modes of operation, and an ergonomic head and arm rest for prolonged seated use. While using such a teleoperated robotic surgical system, the surgeon is typically physically separated from the sterile surgical field. Thus, the surgeon relies on assistants in the operating room to perform some tasks at the patient side, which can not be robotically controlled.”);
Regarding claim 7 Itkowitz/Nowlin/Brandon teaches The operator workstation of claim 1, wherein to make the interaction determination, the controller is configured to: make the interaction determination indicative of the lack of the operator interaction in response to the sensor data indicating the lack of the operator interaction for a threshold period of time. (Itkowitz [0101] reads “At a process 1015, it is determined whether one or more of the motion input controls is being used. In some examples, upon entering the imaging device motion mode during process 1010, a timeout period may begin. During the timeout period the one or more motion input controls may be monitored to determine whether the operator is attempting to manually control the position and/or orientation of the imaging device using the one or more motion input controls. In some examples, the timeout period may be of a configurable length, such as 0.5 seconds or so. In some examples, use of the one or more motion input controls may be determined based on whether the operator moves one or more of the motion input controls more than a threshold distance, rotates one or more of the motion input controls through more than a threshold angle, and/or some aggregate combination of both.“);
Regarding claim 8 Itkowitz/Nowlin/Brandon teaches The operator workstation of claim 1, wherein the controller is further configured to: cause the one or more actuators to stop driving the repositionable arm and moving the input control device in response to detecting a stopping condition. (Itkowitz [0043] reads “According to some embodiments, additional conditions may result in premature termination of method 200 such as by returning operator control being using process 280 and/or by suspension of device operation. In some examples, the additional conditions may include manual intervention or override from an operator using one or more controls on the operator workstation or the articulated arms, detection of operator disengagement with the operator workstation using one or more safety interlocks, position tracking errors in the articulated arms and/or input controls, system faults, and/or the like.”);
Regarding claim 9 Itkowitz/Nowlin/Brandon teaches The operator workstation of claim 8, wherein the stopping condition comprises an event selected from the group consisting of: a change in a state of the operator workstation; a change in a state of the computer-assisted device; an occurrence of the operator interaction with the input control device; and a difficulty in causing the one or more actuators to drive the repositionable arm and move the input control device based on the trajectory. (Itkowitz [0043] reads “According to some embodiments, additional conditions may result in premature termination of method 200 such as by returning operator control being using process 280 and/or by suspension of device operation. In some examples, the additional conditions may include manual intervention or override from an operator using one or more controls on the operator workstation or the articulated arms, detection of operator disengagement with the operator workstation using one or more safety interlocks, position tracking errors in the articulated arms and/or input controls, system faults, and/or the like.”);
Regarding claim 10 Itkowitz/Nowlin/Brandon teaches The operator workstation of claim 9, wherein the stopping condition comprises the difficulty, and wherein to detect the difficulty, the controller is configured to detect one or more conditions selected from group consisting of: a magnitude of force applied by an actuator of the input control device being above a force threshold; a magnitude of torque applied by the actuator being above a torque threshold; a magnitude of electrical current of the actuator being above a current threshold; a magnitude of change in the force applied by the actuator being above a change in force threshold; a magnitude of change in the torque applied by the actuator being above a change in torque threshold; a magnitude of change in the electrical current of the actuator being above a change in current threshold; a position error in a distal portion of the input control device being above a first position error threshold; a position error in a joint of the input control device being above a second position error threshold; a speed of motion of the distal portion of the input control device being outside a first range; a speed of motion of a joint of the input control device being outside a second range; a magnitude of an aggregation of forces applied by a plurality of actuators of the input control device being above an aggregate force threshold; a magnitude of an aggregation of torques applied by the plurality of actuators being above an aggregate torque threshold; a magnitude of a change in the aggregation of forces applied by the plurality of actuators being above an aggregate change in force threshold; a magnitude of a change in the aggregation of torques applied by the plurality of actuators being above an aggregate change in torque threshold; a magnitude of an aggregation of changes in currents of the plurality of actuators being above an aggregate change in current threshold; and an aggregation of position errors in a plurality of joints of the input control device being above an aggregation position error threshold. (Itkowitz [0043] reads “According to some embodiments, additional conditions may result in premature termination of method 200 such as by returning operator control being using process 280 and/or by suspension of device operation. In some examples, the additional conditions may include manual intervention or override from an operator using one or more controls on the operator workstation or the articulated arms, detection of operator disengagement with the operator workstation using one or more safety interlocks, position tracking errors in the articulated arms and/or input controls, system faults, and/or the like.” And [0067] reads “In some examples, force and/or torque on one or more of the joints used to manipulate the imaging device may be monitored using suitable sensors to determine whether unexpected forces and/or torques may indicate that the imaging device is in unacceptable contact with the anatomy of the patient and/or other obstacles. In some examples, errors between the commanded positions and/or velocities and actual positions and/or velocities of the imaging device and/or the joints used to manipulate the imaging device may be monitored to determine whether the errors exceed a configurable threshold. In some examples, the configurable threshold may be different for each of the joints. In some examples, the errors may be low-pass filtered and/or smoothed to avoid false positive detections that may be due to otherwise acceptable transient conditions.”););
Regarding claim 11 Itkowitz/Nowlin/Brandon teaches The operator workstation of claim 8, further comprising; a second input control device, wherein the stopping condition comprises a detection of an operator interaction with the second input control device. (Itkowitz [0043] reads “According to some embodiments, additional conditions may result in premature termination of method 200 such as by returning operator control being using process 280 and/or by suspension of device operation. In some examples, the additional conditions may include manual intervention or override from an operator using one or more controls on the operator workstation or the articulated arms, detection of operator disengagement with the operator workstation using one or more safety interlocks, position tracking errors in the articulated arms and/or input controls, system faults, and/or the like.” It would be appreciated by one with ordinary skill in the art that manual intervention would comprise a secondary emergency stop feature that is common in many different situations.);
Regarding claim 13 Itkowitz teaches A method of controlling motion of an input control device of an operator workstation, the method comprising: (Itkowitz [0028] reads “As shown in FIG. 1, computer-assisted system 100 includes a device 110 with one or more movable or articulated arms 120. Each of the one or more articulated arms 120 may support one or more end effectors 125. In some examples, device 110 may be consistent with a computer-assisted surgical device. The one or more end effectors 125 may include surgical instruments, imaging devices, and/or the like. In some examples, the surgical instruments may include clamps, grippers, retractors, cautery tools, suction tools, suturing devices, and/or the like. In some examples, the imaging devices may include endoscopes, cameras, stereoscopic devices, and/or the like.”);
, and the input control device being configured to be moved by an operator to teleoperate an end effector that is physically separate from the operator workstation; (Itkowitz [0032] reads “Operator workstation 170 may further include a console workspace with one or more input or master controls 195 that may be used for operating the device 110, the articulated arms 120, and/or the end effectors 125. Each of the input controls 195 may be coupled to the distal end of their own articulated arms so that movements of the input controls 195 may be detected by the operator workstation 170 and communicated to control unit 130.” Itkowitz Figure 1 also depicts that the input control device could be used to control a robotic arm could be physically separated from the robotic manipulator.);
PNG
media_image1.png
601
543
media_image1.png
Greyscale
Itkowitz Figure 1
making, by the controller, an interaction determination based on the sensor data, the interaction determination indicative of whether the input control device has received an operator interaction for controlling a position or an orientation or a motion of the input control device; and in response to the interaction determination being indicative of the input control device not having received the operator interaction: determining, by the controller, a trajectory for a repositionable arm that moves the input control device from a current position or orientation toward a desired position or orientation; (Itkowitz [0101] reads “At a process 1015, it is determined whether one or more of the motion input controls is being used. In some examples, upon entering the imaging device motion mode during process 1010, a timeout period may begin. During the timeout period the one or more motion input controls may be monitored to determine whether the operator is attempting to manually control the position and/or orientation of the imaging device using the one or more motion input controls. … When the timeout period ends with no use of the one or more motion input controls, recentering begins starting with a process 1020. When use of the one or more input controls is detected during the timeout period, manual control of the imaging device begins with a process 1040.” And [0009] reads “determining one or more input control recentering moves to provide positional and orientational harmony between each of the input controls and a corresponding one of the end effectors, executing the view and input control recentering moves, and reinstating teleoperated control of the end effectors by the input controls.”);
and causing, by the controller, one or more actuators to drive a repositionable arm to move the input control device based on the trajectory. (Itkowitz [0040] reads “At the process 260, the view and input control recentering moves are coordinated. One or more movement commands are sent to one or more actuators of the articulated arm coupled to the imaging device to command and/or direct the imaging device to execute the view recentering move. One or more movement commands are also sent to one or more actuators of the articulated arms coupled to the input controls to command and/or direct the input controls to execute the input control recentering moves.”);
Itkowitz does not teach receiving, by a controller comprising processing circuity, sensor data from a first sensor of a sensor system, the sensor system being physically coupled to operator workstation or physically coupled to the input control device, the first sensor being a contact or proximity sensor.
Nowlin in analogous art, teaches receiving, by a controller comprising processing circuity … the first sensor being a contact or proximity sensor; (Nowlin [0186] reads “The sensors may comprise contact and/or proximity sensors, and may be monitored on a real time or near real time basis by the controller.”);
It would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention to have modified the teachings of Itkowitz with that of Nowlin to provide a surgical system that encompasses a robotic arm with advanced sensor capabilities. This would allow the system to make more precise movements that could result in better patient outcomes. (Nowlin [0004] reads “For these and other reasons, it would be advantageous to provide improved devices, systems, and methods for surgery, robotic surgery, and other robotic applications. It would be particularly beneficial if these improved technologies provided a faster and easier set-up, and/or inhibited collisions of the robotic devices during use. Ideally, these improvements would be provided without significantly increasing the size, mechanical complexity, or costs of these systems, and while maintaining or improving their dexterity.”);
Itkowitz/Nowlin does not teach sensor data from a first sensor of a sensor system, the sensor system being physically coupled to or physically coupled to the input control device,
Brandon in analogous art, teaches sensor data from a first sensor of a sensor system, the sensor system being physically coupled to or physically coupled to the input control device, (Brandon [0091] reads “In addition, display device 160 includes a surgeon presence switch. When surgeon 101 is facing display device 160 and is within a range of the surgeon presence switch, the switch provides a signal to control system 190 that allows control system 190 to enter and stay in the following mode. When surgeon 101 is either not facing display device 160 or not within the range of the switch, the surgeon presence switch provides a signal to control system 190 that inhibits control system 190 from being in the following mode. In one aspect, one or more infrared (IR) range sensors are used for sensing close-range distances of surgeon 101 from display 160 or alternatively from surgeon's viewer 361.”);
It would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention to have modified the teachings of Itkowitz/Nowlin with that of Brandon to include a method for better sensing the operators position and attention. This would allow for the robotic system to more precisely move in a manner that would be closer to that that the operator intended. (Brandon [0004-0005] reads “The da Vinci® Surgical System, manufactured by Intuitive Surgical, Inc., Sunnyvale, Calif., is a minimally invasive, teleoperated robotic surgical system that offers patients many benefits, such as reduced trauma to the body, faster recovery and shorter hospital stay. The da Vinci® Surgical System provides intuitive and ergonomic control of minimally invasive slave surgical instruments, which provides telepresence for the surgeon. This system incorporates a dedicated surgeon console, which provides a three-dimensional stereo viewer, two master tool manipulators, foot pedals for controlling modes of operation, and an ergonomic head and arm rest for prolonged seated use. While using such a teleoperated robotic surgical system, the surgeon is typically physically separated from the sterile surgical field. Thus, the surgeon relies on assistants in the operating room to perform some tasks at the patient side, which can not be robotically controlled.”);
Regarding claim 17 Itkowitz/Nowlin/Brandon teaches The method of claim 13, wherein making the interaction determination comprises: making the interaction determination indicative of the lack of the operator interaction in response to the sensor data indicating the lack of the operator interaction for a threshold period of time. (Itkowitz [0101] reads “At a process 1015, it is determined whether one or more of the motion input controls is being used. In some examples, upon entering the imaging device motion mode during process 1010, a timeout period may begin. During the timeout period the one or more motion input controls may be monitored to determine whether the operator is attempting to manually control the position and/or orientation of the imaging device using the one or more motion input controls. In some examples, the timeout period may be of a configurable length, such as 0.5 seconds or so. In some examples, use of the one or more motion input controls may be determined based on whether the operator moves one or more of the motion input controls more than a threshold distance, rotates one or more of the motion input controls through more than a threshold angle, and/or some aggregate combination of both.“);
Regarding claim 18 Itkowitz/Nowlin/Brandon teaches The method of claim 13, further comprising: causing, by the controller, the one or more actuators to stop driving the repositionable arm and moving the input control device in response to detecting a stopping condition. (Itkowitz [0043] reads “According to some embodiments, additional conditions may result in premature termination of method 200 such as by returning operator control being using process 280 and/or by suspension of device operation. In some examples, the additional conditions may include manual intervention or override from an operator using one or more controls on the operator workstation or the articulated arms, detection of operator disengagement with the operator workstation using one or more safety interlocks, position tracking errors in the articulated arms and/or input controls, system faults, and/or the like.”);
Regarding claim 19 Itkowitz/Nowlin/Brandon teaches The method of claim 18, wherein the stopping condition comprises a detection of an operator interaction with a second input control device of the operator workstation. (Itkowitz [0043] reads “According to some embodiments, additional conditions may result in premature termination of method 200 such as by returning operator control being using process 280 and/or by suspension of device operation. In some examples, the additional conditions may include manual intervention or override from an operator using one or more controls on the operator workstation or the articulated arms, detection of operator disengagement with the operator workstation using one or more safety interlocks, position tracking errors in the articulated arms and/or input controls, system faults, and/or the like.” It would be appreciated by one with ordinary skill in the art that manual intervention would comprise a secondary emergency stop feature that is common in many different situations.);
Regarding claim 20 Itkowitz teaches A non-transitory machine-readable medium comprising a plurality of machine-readable instructions which when executed by processing circuity associated with an operator workstation are adapted to cause the processing circuitry to perform a method comprising: receiving sensor data from a first sensor of a sensor system, (Itkowitz [0028] reads “As shown in FIG. 1, computer-assisted system 100 includes a device 110 with one or more movable or articulated arms 120. Each of the one or more articulated arms 120 may support one or more end effectors 125. In some examples, device 110 may be consistent with a computer-assisted surgical device. The one or more end effectors 125 may include surgical instruments, imaging devices, and/or the like. In some examples, the surgical instruments may include clamps, grippers, retractors, cautery tools, suction tools, suturing devices, and/or the like. In some examples, the imaging devices may include endoscopes, cameras, stereoscopic devices, and/or the like.”);
and the input control device being configured to be moved by an operator to teleoperate an end effector that is physically separate from the operator workstation; (Itkowitz [0032] reads “Operator workstation 170 may further include a console workspace with one or more input or master controls 195 that may be used for operating the device 110, the articulated arms 120, and/or the end effectors 125. Each of the input controls 195 may be coupled to the distal end of their own articulated arms so that movements of the input controls 195 may be detected by the operator workstation 170 and communicated to control unit 130.” Itkowitz Figure 1 also depicts that the input control device could be used to control a robotic arm could be physically separated from the robotic manipulator.);
PNG
media_image2.png
249
225
media_image2.png
Greyscale
Itkowitz Figure 1
making an interaction determination based on the sensor data, the interaction determination indicative of whether an input control device (Itkowitz [0101] reads “At a process 1015, it is determined whether one or more of the motion input controls is being used. In some examples, upon entering the imaging device motion mode during process 1010, a timeout period may begin. During the timeout period the one or more motion input controls may be monitored to determine whether the operator is attempting to manually control the position and/or orientation of the imaging device using the one or more motion input controls. … When the timeout period ends with no use of the one or more motion input controls, recentering begins starting with a process 1020. When use of the one or more input controls is detected during the timeout period, manual control of the imaging device begins with a process 1040.” And [0009] reads “determining one or more input control recentering moves to provide positional and orientational harmony between each of the input controls and a corresponding one of the end effectors, executing the view and input control recentering moves, and reinstating teleoperated control of the end effectors by the input controls.”);
and causing one or more actuators to drive a repositionable arm to move the input control device based on the trajectory. (Itkowitz [0040] reads “At the process 260, the view and input control recentering moves are coordinated. One or more movement commands are sent to one or more actuators of the articulated arm coupled to the imaging device to command and/or direct the imaging device to execute the view recentering move. One or more movement commands are also sent to one or more actuators of the articulated arms coupled to the input controls to command and/or direct the input controls to execute the input control recentering moves.”);
Itkowitz does not teach the sensor system being physically coupled to or physically coupled to an input control device of the operator workstation, the first sensor being a contact or proximity sensor.
Nowlin in analogous art, teaches the first sensor being a contact or proximity sensor; (Nowlin [0186] reads “The sensors may comprise contact and/or proximity sensors, and may be monitored on a real time or near real time basis by the controller.”);
It would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention to have modified the teachings of Itkowitz with that of Nowlin to provide a surgical system that encompasses a robotic arm with advanced sensor capabilities. This would allow the system to make more precise movements that could result in better patient outcomes. (Nowlin [0004] reads “For these and other reasons, it would be advantageous to provide improved devices, systems, and methods for surgery, robotic surgery, and other robotic applications. It would be particularly beneficial if these improved technologies provided a faster and easier set-up, and/or inhibited collisions of the robotic devices during use. Ideally, these improvements would be provided without significantly increasing the size, mechanical complexity, or costs of these systems, and while maintaining or improving their dexterity.”);
Itkowitz/Nowlin does not teach the sensor system being physically coupled to or physically coupled to an input control device of the operator workstation, the first sensor being a contact or proximity sensor.
Brandon in analogous art, teaches the sensor system being physically coupled to or physically coupled to an input control device of the operator workstation, the first sensor being a contact or proximity sensor. (Brandon [0091] reads “In addition, display device 160 includes a surgeon presence switch. When surgeon 101 is facing display device 160 and is within a range of the surgeon presence switch, the switch provides a signal to control system 190 that allows control system 190 to enter and stay in the following mode. When surgeon 101 is either not facing display device 160 or not within the range of the switch, the surgeon presence switch provides a signal to control system 190 that inhibits control system 190 from being in the following mode. In one aspect, one or more infrared (IR) range sensors are used for sensing close-range distances of surgeon 101 from display 160 or alternatively from surgeon's viewer 361.”);
It would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention to have modified the teachings of Itkowitz/Nowlin with that of Brandon to include a method for better sensing the operators position and attention. This would allow for the robotic system to more precisely move in a manner that would be closer to that that the operator intended. (Brandon [0004-0005] reads “The da Vinci® Surgical System, manufactured by Intuitive Surgical, Inc., Sunnyvale, Calif., is a minimally invasive, teleoperated robotic surgical system that offers patients many benefits, such as reduced trauma to the body, faster recovery and shorter hospital stay. The da Vinci® Surgical System provides intuitive and ergonomic control of minimally invasive slave surgical instruments, which provides telepresence for the surgeon. This system incorporates a dedicated surgeon console, which provides a three-dimensional stereo viewer, two master tool manipulators, foot pedals for controlling modes of operation, and an ergonomic head and arm rest for prolonged seated use. While using such a teleoperated robotic surgical system, the surgeon is typically physically separated from the sterile surgical field. Thus, the surgeon relies on assistants in the operating room to perform some tasks at the patient side, which can not be robotically controlled.”);
Regarding claim 23 Itkowitz/Nowlin/Brandon teaches The non-transitory machine-readable medium of claim 20, wherein the method further comprises: causing the one or more actuators to stop driving the repositionable arm and moving the input control device in response to detecting an operator interaction with a second input control device of the operator workstation. (Itkowitz [0043] reads “According to some embodiments, additional conditions may result in premature termination of method 200 such as by returning operator control being using process 280 and/or by suspension of device operation. In some examples, the additional conditions may include manual intervention or override from an operator using one or more controls on the operator workstation or the articulated arms, detection of operator disengagement with the operator workstation using one or more safety interlocks, position tracking errors in the articulated arms and/or input controls, system faults, and/or the like.” It would be appreciated by one with ordinary skill in the art that manual intervention would comprise a secondary emergency stop feature that is common in many different situations.);
Claim(s) 2-6, 12, 14-16, 21-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over as applied to Itkowitz/Nowlin/Brandon, in further view of Hoffman (US 20060258938 A1).
Regarding claim 2 Itkowitz/Nowlin/Brandon teaches The operator workstation of claim 1, wherein: the sensor system further comprises a second sensor configured to detect the operator interactions, (Nowlin [0006] reads “The RDOF surgical robotic system comprises a manipulator assembly, an input device, and a processor. In response to a command to effect a desired movement of the end effector as received by the input device, the manipulator assembly manipulates a distal end effector relative to a proximal base.”);
the second sensor being a second contact or proximity sensor; (Nowlin [0186] reads “The sensors may comprise contact and/or proximity sensors, and may be monitored on a real time or near real time basis by the controller.”);
Itkowitz/Nowlin/Brandon does not teach and the controller is further coupled with the second sensor, wherein the sensor data further comprises data from the second sensor, and wherein to make the interaction determination, the controller is configured to: combine a first indication with a second indication, the first indication being determined using the data from the first sensor and the second indication being determined using the data from the second sensor.
Hoffman in analogous art, teaches and the controller is further coupled with the second sensor, wherein the sensor data further comprises data from the second sensor, and wherein to make the interaction determination, the controller is configured to: combine a first indication with a second indication, the first indication being determined using the data from the first sensor and the second indication being determined using the data from the second sensor. (Hoffman [0089] reads “The tool tracking method in this case employs an Extended Kalman Filter (“EKF”), which has the purpose of producing an optimal estimate of the state of the tool being tracked, {circumflex over (x)}k, by combining one or more non-endoscopically derived tool state information, e.g., zNV1-k and zNV2-k (respectively generated in blocks 501 and 502, by processing sensor and/or external camera data associated with the tool) with one or more endoscopically derived tool state information, such as zV1-k, zV2-k, and zV3-k (respectively generated in blocks 511, 512, and 513, by processing image data of the tool received from a stereoscopic endoscope using corresponding vision algorithms), and using a model of the system dynamics.”);
It would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention to have modified the teachings of Itkowitz/Nowlin/Brandon with that of Hoffman to provide a system that would allow of the combination of information from multiple sensors. This would allow for the operator to better control the surgical robot and would lead to better patient outcomes. (Hoffman [0011] reads “To make each of multiple tools easily distinguishable to the surgeon on the workstation display screen and also to patient-side staff, a number of computer assisted techniques may be employed such as: predicting the position and orientation of the tool, and overlaying a computer aided design (“CAD”) model of the tool, or other virtual mark or indicator, over the predicted position and orientation of the tool in the display screen; predicting the position and orientation of the tool, and uniquely marking each tool at its predicted position and orientation to make it distinguishable from other tools on the display screen; and predicting the position and orientation of the tool, and erasing or brushing out the shaft of the tool while highlighting its end effector in some fashion so that the end effector seemingly floats in and stands out on the display screen.”);
Regarding claim 3 Itkowitz/Nowlin/Brandon/Hoffman teaches The operator workstation of claim 2, wherein to combine the first indication with the second indication, the controller is configured to: compute a weighted sum of the first indication and the second indication. (Hoffman [103] reads “The Kalman gain is then computed. The Kalman gain, in essence, weights the contributions from one or more measurements, such that their impact on the new state estimate reflects a current estimate of their reliability.”);
Regarding claim 4 Itkowitz/Nowlin/Brandon/Hoffman teaches The operator workstation of claim 2, wherein: the first sensor is a hand or finger sensor; (Itkowitz [0081] reads “In general, this provides for intuitive operation of the end effectors 310 and/or 330 during teleoperation as operator hand movements of the input controls may be translated to corresponding movements of the end effectors 310 and/or 330.”);
and the second sensor is a head or body sensor. (Itkowitz [0068] reads “In some examples, one or more input controls, such as a head-in sensor may be used to determine that an operator is present at an operator console and in position to view images from the imaging device.”);
Regarding claim 5 Itkowitz/Nowlin/Brandon/Hoffman teaches The operator workstation of claim 4, wherein a first weight for the first indication is larger than a second weight for the second indication. (Hoffman [0103] reads “Additionally, it allows weighting of the reliance on the model vs. the measurements. In other words, the contribution from a reliable measurement may be weighted more, and an unreliable measurement less.“);
Regarding claim 6 Itkowitz/Nowlin/Brandon/Hoffman teaches The operator workstation of claim 2, wherein: in response to the first indication indicating the operator interaction contemporaneously with the second indication indicating a lack of the operator interaction, (Hoffman [0089] reads “The tool tracking method in this case employs an Extended Kalman Filter (“EKF”), which has the purpose of producing an optimal estimate of the state of the tool being tracked, {circumflex over (x)}k, by combining one or more non-endoscopically derived tool state information, e.g., zNV1-k and zNV2-k (respectively generated in blocks 501 and 502, by processing sensor and/or external camera data associated with the tool) with one or more endoscopically derived tool state information, such as zV1-k, zV2-k, and zV3-k (respectively generated in blocks 511, 512, and 513, by processing image data of the tool received from a stereoscopic endoscope using corresponding vision algorithms), and using a model of the system dynamics.”);
Regarding claim 12 Itkowitz/Nowlin/Brandon teaches The operator workstation of claim 1, wherein the repositionable arm comprises a plurality of links coupled by a plurality of joints, (Nowlin figure 2 depicts a surgical robot that consists of arms with a plurality of end effectors.);
PNG
media_image3.png
270
392
media_image3.png
Greyscale
Nowlin figure 2
Itkowitz/Nowlin/Brandon does not teach and wherein to determine the trajectory, the controller is configured to: determine, using a Jacobian of the repositionable arm, motions of the plurality of joints that maintains an orientation of the input control device based on an orientation of the end effector.
Hoffman in analogous art, teaches and wherein to determine the trajectory, the controller is configured to: determine, using a Jacobian of the repositionable arm, motions of the plurality of joints that maintains an orientation of the input control device based on an orientation of the end effector. (Hoffman [0136] reads “As previously described, the kinematics measurement is of the state directly. Therefore, the measurement Jacobian for the kinematics measurement is the 13×13 identity matrix, (22). HNV1-k=I13x13 (22)”);
It would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention to have modified the teachings of Itkowitz/Nowlin/Brandon with that of Hoffman to provide a system that would allow of the combination of information from multiple sensors. This would allow for the operator to better control the surgical robot and would lead to better patient outcomes. (Hoffman [0011] reads “To make each of multiple tools easily distinguishable to the surgeon on the workstation display screen and also to patient-side staff, a number of computer assisted techniques may be employed such as: predicting the position and orientation of the tool, and overlaying a computer aided design (“CAD”) model of the tool, or other virtual mark or indicator, over the predicted position and orientation of the tool in the display screen; predicting the position and orientation of the tool, and uniquely marking each tool at its predicted position and orientation to make it distinguishable from other tools on the display screen; and predicting the position and orientation of the tool, and erasing or brushing out the shaft of the tool while highlighting its end effector in some fashion so that the end effector seemingly floats in and stands out on the display screen.”);
Regarding claim 14 Itkowitz/Nowlin/Brandon teaches The method of claim 13, wherein: the sensor system further comprises a second sensor, the second sensor being a second contact or proximity sensor; (Nowlin [0186] reads “The sensors may comprise contact and/or proximity sensors, and may be monitored on a real time or near real time basis by the controller.”);
Itkowitz/Nowlin/Brandon does not teach and making the interaction determination comprises combining a first indication with a second indication, the first indication being determined using the data from the first sensor and the second indication being determined using the data from the second sensor.
Hoffman in analogous art, teaches and making the interaction determination comprises combining a first indication with a second indication, the first indication being determined using the data from the first sensor and the second indication being determined using the data from the second sensor. (Hoffman [0089] reads “The tool tracking method in this case employs an Extended Kalman Filter (“EKF”), which has the purpose of producing an optimal estimate of the state of the tool being tracked, {circumflex over (x)}k, by combining one or more non-endoscopically derived tool state information, e.g., zNV1-k and zNV2-k (respectively generated in blocks 501 and 502, by processing sensor and/or external camera data associated with the tool) with one or more endoscopically derived tool state information, such as zV1-k, zV2-k, and zV3-k (respectively generated in blocks 511, 512, and 513, by processing image data of the tool received from a stereoscopic endoscope using corresponding vision algorithms), and using a model of the system dynamics.”);
It would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention to have modified the teachings of Itkowitz/Nowlin/Brandon with that of Hoffman to provide a system that would allow of the combination of information from multiple sensors. This would allow for the operator to better control the surgical robot and would lead to better patient outcomes. (Hoffman [0011] reads “To make each of multiple tools easily distinguishable to the surgeon on the workstation display screen and also to patient-side staff, a number of computer assisted techniques may be employed such as: predicting the position and orientation of the tool, and overlaying a computer aided design (“CAD”) model of the tool, or other virtual mark or indicator, over the predicted position and orientation of the tool in the display screen; predicting the position and orientation of the tool, and uniquely marking each tool at its predicted position and orientation to make it distinguishable from other tools on the display screen; and predicting the position and orientation of the tool, and erasing or brushing out the shaft of the tool while highlighting its end effector in some fashion so that the end effector seemingly floats in and stands out on the display screen.”);
Regarding claim 15 Itkowitz/Nowlin/Brandon/Hoffman teaches The method of claim 14, wherein combining the first indication with the second indication comprises: computing a weighted sum of the first indication and the second indication. (Hoffman [103] reads “The Kalman gain is then computed. The Kalman gain, in essence, weights the contributions from one or more measurements, such that their impact on the new state estimate reflects a current estimate of their reliability.”);
Regarding claim 16 Itkowitz/Nowlin/Brandon/Hoffman teaches The method of claim 14, wherein: the first sensor is a hand or finger sensor; (Itkowitz [0081] reads “In general, this provides for intuitive operation of the end effectors 310 and/or 330 during teleoperation as operator hand movements of the input controls may be translated to corresponding movements of the end effectors 310 and/or 330.”);
Regarding claim 21 Itkowitz/Nowlin/Brandon teaches The non-transitory machine-readable medium of claim 20.
Itkowitz/Nowlin/Brandon does not teach wherein: the sensor system further comprises a second sensor, the second sensor being a second contact or proximity sensor; and making the interaction determination comprises combining a first indication with a second indication, the first indication being determined using the data from the first sensor and the second indication being determined using the data from the second sensor.
Hoffman in analogous art, teaches wherein: the sensor system further comprises a second sensor, the second sensor being a second contact or proximity sensor; and making the interaction determination comprises combining a first indication with a second indication, the first indication being determined using the data from the first sensor and the second indication being determined using the data from the second sensor. (Hoffman [0089] reads “The tool tracking method in this case employs an Extended Kalman Filter (“EKF”), which has the purpose of producing an optimal estimate of the state of the tool being tracked, {circumflex over (x)}k, by combining one or more non-endoscopically derived tool state information, e.g., zNV1-k and zNV2-k (respectively generated in blocks 501 and 502, by processing sensor and/or external camera data associated with the tool) with one or more endoscopically derived tool state information, such as zV1-k, zV2-k, and zV3-k (respectively generated in blocks 511, 512, and 513, by processing image data of the tool received from a stereoscopic endoscope using corresponding vision algorithms), and using a model of the system dynamics.”);
It would have been obvious to one with ordinary skill in the art, before the effective filing date of the claimed invention to have modified the teachings of Itkowitz/Nowlin/Brandon with that of Hoffman to provide a system that would allow of the combination of information from multiple sensors. This would allow for the operator to better control the surgical robot and would lead to better patient outcomes. (Hoffman [0011] reads “To make each of multiple tools easily distinguishable to the surgeon on the workstation display screen and also to patient-side staff, a number of computer assisted techniques may be employed such as: predicting the position and orientation of the tool, and overlaying a computer aided design (“CAD”) model of the tool, or other virtual mark or indicator, over the predicted position and orientation of the tool in the display screen; predicting the position and orientation of the tool, and uniquely marking each tool at its predicted position and orientation to make it distinguishable from other tools on the display screen; and predicting the position and orientation of the tool, and erasing or brushing out the shaft of the tool while highlighting its end effector in some fashion so that the end effector seemingly floats in and stands out on the display screen.”);
Regarding claim 22 Itkowitz/Nowlin/Brandon/Hoffman teaches The non-transitory machine-readable medium of claim 21, wherein: the first sensor is a hand or finger sensor; (Itkowitz [0081] reads “In general, this provides for intuitive operation of the end effectors 310 and/or 330 during teleoperation as operator hand movements of the input controls may be translated to corresponding movements of the end effectors 310 and/or 330.”);
the second sensor is a head or body sensor; (Itkowitz [0068] reads “In some examples, one or more input controls, such as a head-in sensor may be used to determine that an operator is present at an operator console and in position to view images from the imaging device.”);
and making the interaction determination based on the combining comprises: in response to the first indication indicating the operator interaction contemporaneously with the second indication indicating a lack of the operator interaction, (Hoffman [0089] reads “The tool tracking method in this case employs an Extended Kalman Filter (“EKF”), which has the purpose of producing an optimal estimate of the state of the tool being tracked, {circumflex over (x)}k, by combining one or more non-endoscopically derived tool state information, e.g., zNV1-k and zNV2-k (respectively generated in blocks 501 and 502, by processing sensor and/or external camera data associated with the tool) with one or more endoscopically derived tool state information, such as zV1-k, zV2-k, and zV3-k (respectively generated in blocks 511, 512, and 513, by processing image data of the tool received from a stereoscopic endoscope using corresponding vision algorithms), and using a model of the system dynamics.”);
making the interaction determination as being indicative of the input control device having received the operator interaction. (Itkowitz [0073] reads “At a process 630, it is determined whether sufficient motion is detected in the imaging device. Using the current position and/or orientation values recorded during processes 610 and 620, the amount of motion of the imaging device may be determined. In some examples, the amount of motion may be a distance, such as a Euclidean distance, between the starting and ending positions. In some examples, the amount of motion may further be based on the angular changes between the starting and ending orientations. In some examples, the angular changes may be converted to distances by determining a sine and/or a cosine of the angular changes and multiplying one of them by a distance related to a working distance of the imaging device from before the start of motion was detected during process 610. When the amount of motion exceeds a minimum threshold, such as 0.5 cm or so, a new preferred working distance is determined beginning with a process 640. When the amount of motion does not exceed the minimum threshold, method 600 may return to process 610 to detect future motion in the imaging device.”);
Response to arguments
Applicant argues < In the rejections, the Examiner acknowledges that the cited portions of the Itkowitz reference fail to disclose the prior version of the above claim limitations and, instead, relies on the Nowlin reference in the rejections. See Office Action at p. 6. Applicant respectfully traverses with respect to the amended claim language. The cited portions of the Nowlin reference disclose the general idea of using a device with multiple different manipulator assembly arms that hold end effectors. In the cited portions of Nowlin, contact or proximity sensors can be placed on each manipulator assembly arm to detect contact or proximity with other arms in order to prevent arm-to-arm collisions. See Nowlin at [0186]; Fig. 18G.> [Arguments page 10, second and third paragraphs ]. The examiner respectfully disagrees. The newly amended claims change the broadest reasonable interpretation of the claimed invention. As currently written the office relies upon the use of Brandon to teach that the input control system is physically connected to a sensor system that could monitor the operator in a number of different ways. While Nowlin is still used to teach that each of the given sensors on the input controls may be proximity sensors that could be used to determine if the controls would have contract with each other. Therefore, the combination teaches the claimed invention.
Other references not Cited
Throughout examination other references were found that could read onto the prior art. Though these references were not used in this examination they could be used in future examination and could read on the contents of the current disclosure. These references are, Diolaiti (US 20070287992 A1); Stahler (US 20090138025 A1); Arkenbout, E. A., De Winter, J. C. F., & Breedveld, P. (2015). Robust Hand Motion Tracking through Data Fusion of 5DT Data Glove and Nimble VR Kinect Camera Measurements. Sensors, 15(12), 31644-31671. https://doi.org/10.3390/s151229868
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN MARTIN O'MALLEY whose telephone number is (571)272-6228. The examiner can normally be reached Mon - Fri 9 am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramon Mercado can be reached at (571) 270 - 5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN MARTIN O'MALLEY/Examiner, Art Unit 3658
/Ramon A. Mercado/Supervisory Patent Examiner, Art Unit 3658
Prosecution Insights