DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 6, 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over TABANDEH (US 20170245946) in view of Sexson et al. (WO2020243631A1).
Regarding claim 1, TABANDEH discloses a registration method for an automatic registration of a patient for a surgical navigation (¶12 The tracking hardware may store, process and/or be programmed with various software applications, data and utilities that may include image processing, filtering, triangulation algorithms, registration algorithms, and coordinate transformation processing; ¶36-39 a system and process for the optical tracking of an object), the registration method comprising the steps of:
detecting a position and/or orientation of the patient by a tracking camera of a surgical navigation system (¶45 General tracking systems may include at least one receiver to detect at least one fiducial marker. The receiver(s) are coupled to at least one processor for processing the receiver output. The processing may then determine the position and/or orientation of the fiducial marker and/or fiducial marker array; ¶49 A tracking system 121 with at least one optical receiver 109 may be in communication with tracking hardware 110 also shown in FIG. 2);
detecting a pose of a robot head with a robot camera of a medical robot by the tracking camera of the surgical navigation system and following the robot camera (¶10 Robots or computer assisted surgical devices 101 have an end effector 106, usually a drill or burr, for preparing the femur and tibia to receive an implant. The end effector 106 is tracked or navigated relative to the bone 112 using tracking arrays 107, 113 and an optical tracking system 108; ¶57 the relative POSE and transformations between the tool 106, the fiducial marker array 107 and the movable joint 119 are all known relative to one another and with respect to the robot coordinates and/or tracking system coordinates. The POSE of the fiducial marker array 107 at the initial optimal location may then be recorded and stored within the hardware/software with respect to the robot base coordinates (Block 401) and used as the future optimal LOS reference orientation);
controlling and moving a robot arm such that the robot camera in a first pose is directed to a region of interest of the patient (¶57 the coordinate frames/systems of the tool 106, fiducial marker array 107, movable joint 119, robot 123, and optical receiver/tracking system 121 may be calibrated relative to one another at this position and orientation using techniques well known in the art; Then, as the tool 106 and various joint/links (102, 103) (Block 403) of the robot 123 articulate, either autonomously and/or moved manually by a user, the POSE of the fiducial marker array 107 may be calculated from the forward kinematics of the robot (Block 405). The forward kinematics may be determined by the encoder values incorporated with each robot joint 102, 103);
checking whether there is a line of sight between the tracking camera and the robot head with the robot camera in the first pose (¶68 the hardware/software may periodically monitor the LOS of the fiducial marker array 107 to determine if the LOS may be compromised (Block 605); If no such command, threshold, or any other trigger signaling that the LOS may be compromised, the tracking resumes (Block 607).);
detecting, when there is a line of sight, a pose of the robot head with the robot camera via the tracking camera (¶65 the fiducial marker array 107 may remain in the LOS of the optical receivers 109 and the POSE of the tool 106 may be accurately updated and known with respect to the tracking system coordinates), or detecting, when the line of sight is missing or interrupted or reduced, a pose of the robot head with the robot camera via a robot-kinematics tracking system; and
executing a registration of the patient and/or of the image with respect to the surgical navigation system (¶54 One or more tracking arrays 113 are attached to a patient's anatomy 112 and registered with a 3-D model of the bone to fix the POSE of the coordinate system of the bone 112 with the tracking array 113 and with respect to the coordinate frame of the optical receivers 109).
While TABANDEH teaches the configuration of a tracking camera and a robot with a robot head, TABANDEH does not teach where Sexson teaches a robot head with a robot camera (¶57 As shown in FIG. 3, the surgical robot 200 with the camera 230 mounted to the flange 204, may be positioned with the camera 230 facing the surgical site of the patient 214).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to provide the optically tracked robotic end effector of Tabandeh with an imaging camera as taught by Sexson. Tabandeh teaches optically tracking a robotic end effector relative to patient anatomy using an external optical tracking system and monitoring line of sight to maintain tracking. Sexson teaches mounting a camera on a surgical robot end effector and using the camera to acquire surgical-site information for patient registration. Providing Tabandeh's optically tracked robotic end effector with Sexson's imaging camera would enable the end effector to acquire surgical-site images while its position and orientation are tracked by the existing optical tracking system. This modification would combine the surgical imaging functionality expressly disclosed by Sexson with the optical tracking and visibility-monitoring functionality expressly disclosed by Tabandeh, producing a predictable arrangement in which the pose of the camera-carrying robotic end effector is available for surgical navigation and registration.
Regarding claim 6, the combination of TABANDEH and Sexson disclose the registration method according to claim 1, wherein the step of registering the patient is performed when images have been created and the robot camera is again in a field of view of the tracking camera (TABANDEH ¶54 The movable joint 119 may articulate the fiducial marker array 107 such that the LOS is maintained or the field of view is optimized. One or more tracking arrays 113 are attached to a patient's anatomy 112 and registered with a 3-D model of the bone to fix the POSE of the coordinate system of the bone 112 with the tracking array 113 and with respect to the coordinate frame of the optical receivers 109).
Regarding claim 14, the combination of TABANDEH and Sexson disclose a non-transitory computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform steps of the registration method according to claim 1 (Sexson ¶46 computer readable medium may be readable by a processor, processing unit, processing circuit, or the like, to obtain data (e.g. instructions) from the medium for execution).
Claim(s) 4-5 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of TABANDEH and Sexson as applied to claim 1 above, and further in view of Coiseur et al (US 20210244485).
Regarding claim 4, the combination of TABANDEH and Sexson disclose the registration method according to claim 1, wherein the registration method further comprises the step of: detecting a relative pose of the robot head with the robot camera to the pose of a head of the patient via the tracking camera, in order to provide the medical robot with an orientation for automatic registration (TABANDEH ¶45 General tracking systems may include at least one receiver to detect at least one fiducial marker. The receiver(s) are coupled to at least one processor for processing the receiver output. The processing may then determine the position and/or orientation of the fiducial marker and/or fiducial marker array; ¶49 A tracking system 121 with at least one optical receiver 109 may be in communication with tracking hardware 110 also shown in FIG. 2).
The combination of TABANDEH and Sexson fails to teach a pose of a head of the patient. Coiseur teaches a pose of a head of the patient (¶34 For a brain surgery, a face of a patient may be what is scanned, and the surgeon or other operator may place the camera 200 above the patient. The camera 200 or laser rangefinder may then be used to map features of the patient's face.).
Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to have implemented the teaching of a pose of a head of the patient from Coiseur into the registration method as disclosed by the combination of TABANDEH and Sexson. Both references are directed to acquiring image data of patient for purposes of determining relationship and performing registration within a surgical navigation context. The substitution merely involves using one known anatomical target (e.g. face of head) in place of another known anatomical target. Such substation represents the use of a known technique applied to another know device in a predictable manner, yielding no more than predictable results. As such, the motivation for doing this is to improve registration, calibration, or tracking of patient anatomy using a camera and a robotic surgical system.
Regarding claim 5, the combination of TABANDEH and Sexson disclose the registration method according to claim 1, but fail to teach where Coiseur teaches wherein the registration method further comprises the step of: using a 3D camera with depth information as a robot camera and scanning a face of the patient from multiple sides in order to obtain an overall 3D surface image of the face (¶32 the camera 200 may be any camera capable of capturing depth information, such as a depth camera, a time of flight camera, a 3D camera, two cameras, a structured light camera, light-field camera, plenoptic camera, or the like; ¶34 a face of a patient may be what is scanned, and the surgeon or other operator may place the camera 200 above the patient. The camera 200 or laser rangefinder may then be used to map features of the patient's face.).
Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to have implemented the teaching of wherein the registration method further comprises the step of: using a 3D camera with depth information as a robot camera and scanning a face of the patient from multiple sides in order to obtain an overall 3D surface image of the face from Coiseur into the registration method as disclosed by the combination of TABANDEH and Sexson. The motivation for doing this is to improve registration, calibration, or tracking of patient anatomy using a camera and a robotic surgical system.
Claim(s) 7, 9, 11, 12, 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over TABANDEH (US 20170245946) in view of Sexson et al. (WO2020243631A1) and further in view of Foster et al (US 20170245943).
Regarding claim 7, TABANDEH discloses a surgical navigation system for a surgical intervention on a patient, the surgical navigation system (¶46 computer-assisted surgical systems which are to be considered synonymous with computer-aided surgical system, robotic surgical systems, navigation assisted surgical system, image-guided surgical systems and the like) comprising:
a robot with a robot base (Fig. 2 base 105), a robot arm connected to the robot base (Fig. 2 various joints and links 102, 103) and a robot head connected to the robot arm (¶10 Robots or computer assisted surgical devices 101 have an end effector 106, usually a drill or burr, for preparing the femur and tibia to receive an implant. The end effector 106 is tracked or navigated relative to the bone 112 using tracking arrays 107, 113 and an optical tracking system 108) with a robot camera, so that a pose of the robot camera relative to the robot base is adjustable (¶57 Then, as the tool 106 and various joint/links (102, 103) (Block 403) of the robot 123 articulate, either autonomously and/or moved manually by a user, the POSE of the fiducial marker array 107 may be calculated from the forward kinematics of the robot (Block 405). The forward kinematics may be determined by the encoder values incorporated with each robot joint 102, 103)
a tracking system with a tracking camera adapted for following a position and/or orientation of a patient and for following a pose of the robot camera ((¶57 the coordinate frames/systems of the tool 106, fiducial marker array 107, movable joint 119, robot 123, and optical receiver/tracking system 121 may be calibrated relative to one another at this position and orientation using techniques well known in the art; Then, as the tool 106 and various joint/links (102, 103) (Block 403) of the robot 123 articulate, either autonomously and/or moved manually by a user, the POSE of the fiducial marker array 107 may be calculated from the forward kinematics of the robot (Block 405). The forward kinematics may be determined by the encoder values incorporated with each robot joint 102, 103));
a robot-kinematics tracking system adapted to determine a relative pose of the robot head with the robot camera with respect to its robot base (¶54 One or more tracking arrays 113 are attached to a patient's anatomy 112 and registered with a 3-D model of the bone to fix the POSE of the coordinate system of the bone 112 with the tracking array 113 and with respect to the coordinate frame of the optical receivers 109; ¶57 the relative POSE and transformations between the tool 106, the fiducial marker array 107 and the movable joint 119 are all known relative to one another and with respect to the robot coordinates and/or tracking system coordinate); and
a control unit adapted to control the robot and to perform registration of the patient (¶54 One or more tracking arrays 113 are attached to a patient's anatomy 112 and registered with a 3-D model of the bone to fix the POSE of the coordinate system of the bone 112 with the tracking array 113 and with respect to the coordinate frame of the optical receivers 109),
wherein the control unit is configured:
when there is a line of sight between the tracking camera and the robot head with the robot camera to detect the pose of the robot camera by the tracking camera (¶65 the fiducial marker array 107 may remain in the LOS of the optical receivers 109 and the POSE of the tool 106 may be accurately updated and known with respect to the tracking system coordinates) or, when the line of sight is missing or interrupted or reduced, to detect the pose of the robot camera via the robot-kinematics tracking system; and
to perform a registration of the patient and/or of the image with respect to the surgical navigation system based on the image and the pose (¶54 One or more tracking arrays 113 are attached to a patient's anatomy 112 and registered with a 3-D model of the bone to fix the POSE of the coordinate system of the bone 112 with the tracking array 113 and with respect to the coordinate frame of the optical receivers 109).
While TABANDEH teaches the configuration of a tracking camera and a robot with a robot head, TABANDEH does not teach where Sexson teaches a robot head with a robot camera (¶57 As shown in FIG. 3, the surgical robot 200 with the camera 230 mounted to the flange 204, may be positioned with the camera 230 facing the surgical site of the patient 214).
Furthermore, TABANDEH fails to teach where Foster teaches wherein the control unit is configured:
in a first pose of the robot camera, to create an image by the robot camera to align the robot camera with a face of the patient and to create an image of the face of the patient (¶97 During the procedure, the clinician can align the markers in the computer generated image 1742 with the markers 1744 on the patient's face. In some implementations, the display system 1700 can include a camera that captures the location of the alignment markers 1744 during the actual procedure and automatically aligns the markers in the computer generated image 1742 with the markers 1744 on the patient's face).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to provide the optically tracked robotic end effector of Tabandeh with an imaging camera as taught by Sexson. Tabandeh teaches optically tracking a robotic end effector relative to patient anatomy using an external optical tracking system and monitoring line of sight to maintain tracking. Sexson teaches mounting a camera on a surgical robot end effector and using the camera to acquire surgical-site information for patient registration. Providing Tabandeh's optically tracked robotic end effector with Sexson's imaging camera would enable the end effector to acquire surgical-site images while its position and orientation are tracked by the existing optical tracking system. This modification would combine the surgical imaging functionality expressly disclosed by Sexson with the optical tracking and visibility-monitoring functionality expressly disclosed by Tabandeh, producing a predictable arrangement in which the pose of the camera-carrying robotic end effector is available for surgical navigation and registration.
Furthermore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to have implemented the teaching of the teaching of in a first pose of the robot camera, to create an image by the robot camera to align the robot camera with a face of the patient and to create an image of the face of the patient from Foster into the registration method as disclosed by the combination of TABANDEH and Sexson. The motivation for doing this is to improve medical outcomes and patient safety.
Regarding claim 9, the combination of Tabandeh, Sexson, and Foster disclose the surgical navigation system according to claim 7, wherein the surgical navigation system comprises an infrared-based optical tracking system (Tabandeh ¶45 The receiver may detect the fiducial marker through a variety of mechanisms illustratively including, but not limited to, optical energy, electromagnetic energy, infrared energy, ultraviolet energy, magnetic energy, acoustic energy, targeted visible light, as well as shape, pattern, line, sequence or character recognition).
Regarding claim 11, the combination of Tabandeh, Sexson, and Foster disclose the surgical navigation system according to claim 7, wherein the surgical navigation system comprises an EM-based tracking system (Tabandeh ¶45 The receiver may detect the fiducial marker through a variety of mechanisms illustratively including, but not limited to, optical energy, electromagnetic energy, infrared energy, ultraviolet energy, magnetic energy, acoustic energy, targeted visible light, as well as shape, pattern, line, sequence or character recognition).
Regarding claim 12, the combination Tabandeh, Sexson, and Foster disclose the surgical navigation system according to claim 7, wherein the surgical navigation system comprises a machine vision-based tracking system (Tabandeh ¶97 machine vision may be utilized by way of a camera 1601 to survey the field of view that may be used with the hardware/software and/or active optical receiver controller to aid in controlling the movable joint(s) 1603 to re-position and/or re-orient the optical receivers 109).
Regarding claim 13, the combination of Tabandeh, Sexson, and Foster disclose the surgical navigation system according to claim 7, the surgical navigation system comprises: an infrared-based optical tracking system; and/or an EM-based tracking system; and/or a machine vision-based tracking system (Tabandeh ¶97 machine vision may be utilized by way of a camera 1601 to survey the field of view that may be used with the hardware/software and/or active optical receiver controller to aid in controlling the movable joint(s) 1603 to re-position and/or re-orient the optical receivers 109).
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Tabandeh, Sexson, and Foster as applied to claim 7 above, and further in view of Terry et al (US 20220392012).
Regarding claim 8, the combination of Tabandeh, Sexson, and Foster disclose the surgical navigation system according to claim 7, but fail to teach where Terry teaches wherein the surgical navigation system has a surgical microscope that is robot-guided and a microscope head of the surgical microscope that functions as the robot head with the robot camera (¶17 The serial robot may be optionally embodied as a six-axis ophthalmic surgical robot, with the digital camera connected to or integral with an ophthalmic microscope coupled to an end-effector of such a robot).
Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to have implemented the teaching of wherein the surgical navigation system has a surgical microscope that is robot-guided and a microscope head of the surgical microscope that functions as the robot head with the robot camera from Terry into the surgical navigation system as disclosed by the combination of Tabandeh, Sexson, and Foster. The motivation for doing this is to improve calibration of a robotic camera system.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 and 4-9, and 11-14 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Allowable Subject Matter
Claim 2 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 2, the prior art of record, alone or in combination, fails to teach at least “wherein the registration method further comprises the steps of: controlling and moving the robot arm such that the robot camera in a second pose is directed to a further region of interest of the patient and creating a second image by the robot camera; checking whether in the second pose there is a line of sight between the tracking camera and the robot head with the robot camera; detecting, when there is a line of sight, the second pose of the robot head with the robot camera via the tracking camera, or detecting, when the line of sight is missing or interrupted or reduced, a pose of the robot head with the robot camera via the robot-kinematics tracking system.”
At best, Zhao et al (US 20090088634) teaches in ¶82 If a robotic instrument is not visible (e.g., tools inside an organ or occluded by other tools) in the surgical site, the tool tracking system is at a kinematics-tracking state 618 for estimating tool pose.
At best, TABANDEH (US 20170245946) teaches in ¶68 the hardware/software may periodically monitor the LOS of the fiducial marker array 107 to determine if the LOS may be compromised (Block 605); If no such command, threshold, or any other trigger signaling that the LOS may be compromised, the tracking resumes (Block 607).
Claim 3 is allowed.
The following is an examiner’s statement of reasons for allowance:
Regarding claim 3, the prior art of record, alone or in combination, does not teach the entirety of “wherein, in the step of detecting the pose, when a visual connection is missing or interrupted or reduced, a detection of the pose of the robot head with the robot camera is performed via the robot-kinematics tracking system, wherein a transformation between a robot base of the medical robot and the tracking camera is used in order to determine a transformation between a local coordinate system of the tracking camera and the local coordinate system of the robot camera and thus its pose via the transformation from the tracking camera to the robot base and a transformation of the robot-kinematics tracking system from the robot base to the robot head of the robot camera.”.
At best, Zhao et al (US 20090088634) teaches in ¶82 If a robotic instrument is not visible (e.g., tools inside an organ or occluded by other tools) in the surgical site, the tool tracking system is at a kinematics-tracking state 618 for estimating tool pose.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN KY whose telephone number is (571)272-7648. The examiner can normally be reached Monday-Friday 9-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEVIN KY/ Primary Examiner, Art Unit 2671
