DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 3, & 5-8 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1,3, & 5-8 are rejected under 35 U.S.C. 103 as being unpatentable over Dagnino et al (G. Dagnino et al., “Navigation System for robot-assisted intra-articular lower-limb fracture surgery,” International Journal of Computer Assisted Radiology and Surgery, vol. 11, no. 10, pp. 1831–1843, May 2016.; hereinafter referred to as Dagnino ) in view of Crawford et al (US20190021795A1; hereinafter referred to as Crawford) and further in view of Aghdasi et al (US20220215532A1; hereinafter referred to as Aghdasi).
Regarding Claim 1, Dagnino discloses a control system for navigation and reduction operation comprising a master control apparatus, a tracing apparatus, and an operation apparatus (“This system, introduced in [25], consists of a reduction software, an optical tracking system, and a user controller. The reduction software receives pre-operative CT scan data of the fracture and generates the 3D models of the bone fragments. The GUI displays the 3D models and allows the surgeon to interact with them by using a controller for pre- and intra-operative planning of fracture reduction, i.e. virtual reduction. The optical tracking system (Polaris Spectra, NDI Inc.) provides a real-time (25 Hz) pose update of the optical tools (0.25 mm accuracy) connected to the bone fragments and the RFM. The optical tools have different and unique geometries to enable real-time tracking.” [Clinical requirements and surgical system configuration]),
wherein the master control apparatus comprises a host and an optical tracker (“It employs a host–target structure composed by a PC (host) and a real-time controller with FPGA (target), and a low-level motor controller. The host PC runs the graphical user interface (GUI) and the configuration interface (CI) (Fig. 1b). It creates the link between the surgical team and the robotic system.” [Clinical requirements and surgical system configuration], “The optical tracking system (Polaris Spectra, NDI Inc.) provides a real-time (25 Hz) pose update of the optical tools (0.25 mm accuracy) connected to the bone fragments and the RFM. The optical tools have different and unique geometries to enable real-time tracking.” [Clinical requirements and surgical system configuration]);
the tracing apparatus comprises a target body tracer arranged on a target body (“Orthopaedic manipulation pins are inserted into the bone fragments and tracked using a commercially available optical tracker (Polaris, NDI) through the attached optical tools (see Fig. 5).” [Introduction], “The main components of the intra-operative procedure are the reduction software, the optical tracker, the robotic system, and the patient (i.e. the fracture). One optical tool (T1) is placed on the orthopaedic pin (P1) inserted in fragment 1 (F1), and a second optical tool (T2) is placed on the orthopaedic pin (P2) inserted in the reference bone (F2)… The orthopaedic pins P1 and P2 were designed to be connected in a unique way to the optical tools T1 and T2 (Fig. 5), having their coordinate frames coincident” [Intra-operative procedure]);
the optical tracker is configured to obtain a geometric feature of the target body tracer in an actual working space (“The optical tracking system (Polaris Spectra, NDI Inc.) provides a real-time (25 Hz) pose update of the optical tools (0.25 mm accuracy) connected to the bone fragments and the RFM. The optical tools have different and unique geometries to enable real-time tracking.” [Clinical requirements and surgical system configuration], “The orthopaedic pins P1 and P2 were designed to be connected in a unique way to the optical tools T1 and T2 (Fig. 5), having their coordinate frames coincident, i.e. CFP1≡CFT1, and CFP2≡CFP1. Therefore, assuming that P1𝑻F1 and P2𝑻F2 are constant during the operation, the optical tracker provides the actual poses of F1 (by tracking P1) and F2 (by tracking P2). This establishes a direct correspondence between the image space (reduction software, virtual models) and the task space (real fracture) by using the optical tracker, which enables the intra-operative imaging. This is described by the transformations IMG𝑻F1 and IMG𝑻F2.” [Intra-operative procedure]);
the host is configured to further convert the preliminary image into the actual working space (“A pre-operative CT scan of the fracture and inserted pins is taken, and the resulting data set segmented to generate 3D models (STL format) of each bone fragment and the inserted pins using the ImageSim commercial software (Fig. 3b) [26]. These models are imported in the reduction software, and reference frames are defined as shown in Fig… Results of the pre-operative procedure are stored in the system and used for intra-operative navigation, robot motion command calculation, and for the evaluation of the reduction results, as described in the next subsection.” [Pre-operative planning], “The orthopaedic pins P1 and P2 were designed to be connected in a unique way to the optical tools T1 and T2 (Fig. 5), having their coordinate frames coincident, i.e. CFP1≡CFT1, and CFP2≡CFP1. Therefore, assuming that P1𝑻F1 and P2𝑻F2 are constant during the operation, the optical tracker provides the actual poses of F1 (by tracking P1) and F2 (by tracking P2). This establishes a direct correspondence between the image space (reduction software, virtual models) and the task space (real fracture) by using the optical tracker, which enables the intra-operative imaging. This is described by the transformations IMG𝑻F1 and IMG𝑻F2.” [Intra-operative procedure]);
the optical tracker is further configured to obtain a pose of the target body tracer in the actual working space in real time; the host is further configured to obtain a pose of the target body in the actual working space (“the optical tracker provides the actual poses of F1 (by tracking P1) and F2 (by tracking P2). This establishes a direct correspondence between the image space (reduction software, virtual models) and the task space (real fracture) by using the optical tracker, which enables the intra-operative imaging. ” [Intra-operative procedure], “The real-time imaging updates the actual pose of the fragments in real time, and the surgeon can check intra-operatively the reduction in 3D without the use of any other intra-operative imaging device.“ [Intra-operative procedure]).
the tracing apparatus further comprises an operation apparatus tracer arranged on the operation apparatus; the optical tracker is further configured to obtain a pose of the operation apparatus tracer in the actual working space in real time; and the host is further configured to obtain a pose of the operation apparatus in the actual working space (“A further optical tool (TR) is placed on the RFM (see Fig. 8a).” [Intra-operative procedure], “Robotic fracture manipulator (RFM) This device (Fig. 1a), introduced in [24], is designed to be connected to the bone fragment through an orthopaedic pin for fragment manipulation.” [[Clinical requirements and surgical system configuration]]),
and the host is further configured to obtain a target pose of the operation apparatus in the actual working space according to a target pose of the target body in the actual working space and a connection relationship between the target body and the operation apparatus (“An optical tracker TR is mounted on the RFM end- effector. The coordinate frame of TR is coincident with the coordinate frame of the robot end-effector, i.e. CFROT≡CFEE. A surgeon’s assistant rigidly connects P1 to the RFM, and the reduction software—based on the relative position of P1 with respect to the RFM (by tracking TR)—calculates the transformation RFM𝑻P1 between the robot and the orthopaedic pin P1. Results of the pre-operative planning, i.e. the virtual reduction parameters, are uploaded into the intra-operative procedure, and the corresponding desired pose for the RFM to achieve the fracture reduction” [Intra-operative procedure]),
and to control the operation apparatus to move to the target pose of the operation apparatus for reduction (“Finally, the RFM executes the desired movement for F1 to achieve the physical reduction in the fracture, while reference bone F2 remains fixed. The real-time imaging updates the actual pose of the fragments in real time, and the surgeon can check intra-operatively the reduction in 3D without the use of any other intra-operative imaging device. If the reduction is acceptable, then the surgeon proceeds with the fixation of the fracture by using plate and screws or intramedullary nail, and the surgery ends.” [Intra-operative procedure]);
wherein the operation apparatus is connected to the target body by a tool and a fixation pin (“One optical tool (T1) is placed on the orthopaedic pin (P1) inserted in fragment 1 (F1), and a second optical tool (T2) is placed on the orthopaedic pin (P2) inserted in the reference bone (F2). A further optical tool (TR) is placed on the RFM (see Fig. 8a)…. An optical tracker TR is mounted on the RFM end- effector. The coordinate frame of TR is coincident with the coordinate frame of the robot end-effector, i.e. CFROT≡CFEE. A surgeon’s assistant rigidly connects P1 to the RFM, and the reduction software—based on the relative position of P1 with respect to the RFM (by tracking TR)—calculates the transformation RFM𝑻P1 between the robot and the orthopaedic pin P1.” [Intra-operative procedure]).
Dagnino does not specifically teach that the host is configured to convert a preliminary image into an intermediate image by matching the preliminary image with the intermediate image through point cloud registration, and to obtain a conversion relationship between the intermediate image and the actual working space by comparing the geometric feature of the target body tracer in the intermediate image with the geometric feature of the target body tracer in the actual working space obtained by the optical tracker.
However, in a similar field of endeavor, Crawford teaches a surgical implant planning computer is connectable to a fluoroscopy imager, a marker tracking camera, and a robot having a robot base coupled to a robot arm that is movable by motors relative to the robot base [Abstract].
Crawford also teaches the host is configured to convert a preliminary image into an intermediate image (“The robotic computer system is a Robotic Positioning System that includes a computer controlled robotic arm, hardware, and software that enables real time surgical navigation and robotic guidance using radiological patient images (pre-operative CT, intra-operative CT and fluoroscopy), using a dynamic reference base and positioning camera. The navigation and guidance system determines the registration or mapping between the virtual patient (points on the patient images) and the physical patient (corresponding points on the patient's anatomy).” [0094])
and to obtain a conversion relationship between the intermediate image and the actual working space by comparing the geometric feature of the target body tracer in the intermediate image with the geometric feature of the target body tracer in the actual working space obtained by the optical tracker (“The intra-op registration fixture is placed onto a patient attachment instrument by clamping the compression clamp onto the shaft of the attachment instrument, allowing the fixture to hover over the surgical site. The fiducials are detected automatically in the intra-operative scan and are used to register the patient's anatomy during the scan to the DRB, which is tracked by the camera throughout the procedure. The reflective markers are detected by the camera. Once the registration is transferred to the DRB, the intra-op registration fixture is removed to provide access to the surgical site. FIG. 27 illustrates the intra-op registration fixture 2712 and pivoting arm 2708. FIG. 27 further illustrates the compression clamp 2602, the DRB knob 2604, a starburst connection 2406, a gear tooth joint 2710, and a set of seven fiducials 2714.” [0174].
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Dagnino as outlined above with the host is configured to convert a preliminary image into an intermediate image by matching the preliminary image with the intermediate image through point cloud registration, and to obtain a conversion relationship between the intermediate image and the actual working space by comparing the geometric feature of the target body tracer in the intermediate image with the geometric feature of the target body tracer in the actual working space obtained by the optical tracker as taught by Crawford, because the visualization can help guide the surgeon's planning and approach [0094].
Dagnino in view of Crawford does not specifically teach matching the preliminary image with the intermediate image through point cloud registration.
However, in a similar field of endeavor, Aghdasi methods and systems for generating a real-time or near-real-time three-dimensional (3D) virtual perspective of a scene for a mediated-reality viewer [0002].
Aghdasi also teaches matching the preliminary image with the intermediate image through point cloud registration (“The processing device of the imaging system can implement a method for registering the preoperative medical scan data to the intraoperative image data that includes processing intraoperative depth data of the scene. More specifically, the method can include processing the intraoperative image data to generate a point cloud depth map of the scene. Then, the method can utilize a registration algorithm that maps the point cloud depth map to the preoperative medical scan data. In some embodiments, the processing device of the imaging system can generate a 3D mesh based on the point cloud depth map that can be used in, for example, generating the 3D virtual image of the scene. Accordingly, the registration algorithm can be initiated based on the point cloud depth map rather than the 3D mesh. In some aspects of the present technology, utilizing the point cloud depth map allows the registration to be run in parallel to the generation of the 3D mesh and subsequent synthesis of the 3D virtual image, thereby increasing the processing speed of the imaging system.” [0021]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Dagnino in view of Crawford as outlined above with matching the preliminary image with the intermediate image through point cloud registration as taught by Aghdasi, because it can increase the processing speed of the imaging system [0021].
Regarding Claim 3, Dagnino discloses that the tracing apparatus further comprises a tool tracer arranged on the tool; the optical tracker is further configured to obtain a pose of the tool tracer in the actual working space in real time; and the host is further configured to obtain a pose of the tool in the actual working space (“The robotic system is controlled by software according to the results of the pre- and intra-operative image analysis. The main components of the intra-operative procedure are the reduction software, the optical tracker, the robotic system, and the patient (i.e. the fracture). One optical tool (T1) is placed on the orthopaedic pin (P1) inserted in fragment 1 (F1), and a second optical tool (T2) is placed on the orthopaedic pin (P2) inserted in the reference bone (F2). A further optical tool (TR) is placed on the RFM (see Fig. 8a). The poses of the optical tools are measured in the optical tracking system (CFC), and the corresponding homogeneous transformations C𝑻TR, C𝑻P1, and C𝑻P2 can be calculated. The orthopaedic pins P1 and P2 were designed to be connected in a unique way to the optical tools T1 and T2 (Fig. 5), having their coordinate frames coincident, i.e. CFP1≡CFT1, and CFP2≡CFP1. Therefore, assuming that P1𝑻F1 and P2𝑻F2 are constant during the operation, the optical tracker provides the actual poses of F1 (by tracking P1) and F2 (by tracking P2). This establishes a direct correspondence between the image space (reduction software, virtual models) and the task space (real fracture) by using the optical tracker, which enables the intra-operative imaging.” [Intra-operative procedure])
Regarding Claim 5, Liu discloses t that the operation apparatus comprises a robotic arm controller and a robotic arm having six or more degrees of freedom (“Robotic fracture manipulator (RFM) This device (Fig. 1a), introduced in [24], is designed to be connected to the bone fragment through an orthopaedic pin for fragment manipulation. This component, based on parallel robot configuration with 6-DOF, has 6 motorized linear actuators fully computer-controlled and is able to realize accurate positioning within its workspace (±10.25mm along x, y, ±15mm along z and rotational limits of ±17∘ around each axis). It provides a 0.03±0.01mm translational accuracy and a 0.12∘±0.01∘ rotational accuracy [23].” [Surgical system configuration];
the robotic arm is connected to the robotic arm controller; the robotic arm controller is connected to the host; and the operation apparatus tracer is arranged on the robotic arm (“System workstation It employs a host–target structure composed by a PC (host) and a real-time controller with FPGA (target), and a low-level motor controller. The host PC runs the graphical user interface (GUI) and the configuration interface (CI) (Fig. 1b)… The host PC communicates with the target controller via ethernet. The target controller (NI-compactRIO 9068, National Instruments) processes users’ commands and sends the motion commands to the low-level motor controller (EPOS 2 24/3, Maxon Motor) that executes the movement of the robotic system.” [Surgical system configuration]).
Regarding Claim 6, Dagnino discloses a control system for navigation and reduction operation comprising a master control apparatus, a tracing apparatus, and an operation apparatus (“This system, introduced in [25], consists of a reduction software, an optical tracking system, and a user controller. The reduction software receives pre-operative CT scan data of the fracture and generates the 3D models of the bone fragments. The GUI displays the 3D models and allows the surgeon to interact with them by using a controller for pre- and intra-operative planning of fracture reduction, i.e. virtual reduction. The optical tracking system (Polaris Spectra, NDI Inc.) provides a real-time (25 Hz) pose update of the optical tools (0.25 mm accuracy) connected to the bone fragments and the RFM. The optical tools have different and unique geometries to enable real-time tracking.” [Clinical requirements and surgical system configuration]),
wherein the master control apparatus comprises a host and an optical tracker (“It employs a host–target structure composed by a PC (host) and a real-time controller with FPGA (target), and a low-level motor controller. The host PC runs the graphical user interface (GUI) and the configuration interface (CI) (Fig. 1b). It creates the link between the surgical team and the robotic system.” [Clinical requirements and surgical system configuration], “The optical tracking system (Polaris Spectra, NDI Inc.) provides a real-time (25 Hz) pose update of the optical tools (0.25 mm accuracy) connected to the bone fragments and the RFM. The optical tools have different and unique geometries to enable real-time tracking.” [Clinical requirements and surgical system configuration]);
the tracing apparatus comprises a target body tracer arranged on a target body (“Orthopaedic manipulation pins are inserted into the bone fragments and tracked using a commercially available optical tracker (Polaris, NDI) through the attached optical tools (see Fig. 5).” [Introduction], “The main components of the intra-operative procedure are the reduction software, the optical tracker, the robotic system, and the patient (i.e. the fracture). One optical tool (T1) is placed on the orthopaedic pin (P1) inserted in fragment 1 (F1), and a second optical tool (T2) is placed on the orthopaedic pin (P2) inserted in the reference bone (F2)… The orthopaedic pins P1 and P2 were designed to be connected in a unique way to the optical tools T1 and T2 (Fig. 5), having their coordinate frames coincident” [Intra-operative procedure]);
the optical tracker is configured to obtain a geometric feature of the target body tracer in an actual working space (“The optical tracking system (Polaris Spectra, NDI Inc.) provides a real-time (25 Hz) pose update of the optical tools (0.25 mm accuracy) connected to the bone fragments and the RFM. The optical tools have different and unique geometries to enable real-time tracking.” [Clinical requirements and surgical system configuration], “The orthopaedic pins P1 and P2 were designed to be connected in a unique way to the optical tools T1 and T2 (Fig. 5), having their coordinate frames coincident, i.e. CFP1≡CFT1, and CFP2≡CFP1. Therefore, assuming that P1𝑻F1 and P2𝑻F2 are constant during the operation, the optical tracker provides the actual poses of F1 (by tracking P1) and F2 (by tracking P2). This establishes a direct correspondence between the image space (reduction software, virtual models) and the task space (real fracture) by using the optical tracker, which enables the intra-operative imaging. This is described by the transformations IMG𝑻F1 and IMG𝑻F2.” [Intra-operative procedure]);
the host is configured to convert the preliminary image into the actual working space (“A pre-operative CT scan of the fracture and inserted pins is taken, and the resulting data set segmented to generate 3D models (STL format) of each bone fragment and the inserted pins using the ImageSim commercial software (Fig. 3b) [26]. These models are imported in the reduction software, and reference frames are defined as shown in Fig… Results of the pre-operative procedure are stored in the system and used for intra-operative navigation, robot motion command calculation, and for the evaluation of the reduction results, as described in the next subsection.” [Pre-operative planning], “The orthopaedic pins P1 and P2 were designed to be connected in a unique way to the optical tools T1 and T2 (Fig. 5), having their coordinate frames coincident, i.e. CFP1≡CFT1, and CFP2≡CFP1. Therefore, assuming that P1𝑻F1 and P2𝑻F2 are constant during the operation, the optical tracker provides the actual poses of F1 (by tracking P1) and F2 (by tracking P2). This establishes a direct correspondence between the image space (reduction software, virtual models) and the task space (real fracture) by using the optical tracker, which enables the intra-operative imaging. This is described by the transformations IMG𝑻F1 and IMG𝑻F2.” [Intra-operative procedure]);
the optical tracker is further configured to obtain a pose of the target body tracer in the actual working space in real time; the host is further configured to obtain a pose of the target body in the actual working space (“the optical tracker provides the actual poses of F1 (by tracking P1) and F2 (by tracking P2). This establishes a direct correspondence between the image space (reduction software, virtual models) and the task space (real fracture) by using the optical tracker, which enables the intra-operative imaging. ” [Intra-operative procedure], “The real-time imaging updates the actual pose of the fragments in real time, and the surgeon can check intra-operatively the reduction in 3D without the use of any other intra-operative imaging device.“ [Intra-operative procedure]).
the tracing apparatus further comprises an operation apparatus tracer arranged on the operation apparatus; the optical tracker is further configured to obtain a pose of the operation apparatus tracer in the actual working space in real time; and the host is further configured to obtain a pose of the operation apparatus in the actual working space (“A further optical tool (TR) is placed on the RFM (see Fig. 8a).” [Intra-operative procedure], “Robotic fracture manipulator (RFM) This device (Fig. 1a), introduced in [24], is designed to be connected to the bone fragment through an orthopaedic pin for fragment manipulation.” [[Clinical requirements and surgical system configuration]]),
and the host is further configured to obtain a target pose of the operation apparatus in the actual working space according to a target pose of the target body in the actual working space and a connection relationship between the target body and the operation apparatus (“An optical tracker TR is mounted on the RFM end- effector. The coordinate frame of TR is coincident with the coordinate frame of the robot end-effector, i.e. CFROT≡CFEE. A surgeon’s assistant rigidly connects P1 to the RFM, and the reduction software—based on the relative position of P1 with respect to the RFM (by tracking TR)—calculates the transformation RFM𝑻P1 between the robot and the orthopaedic pin P1. Results of the pre-operative planning, i.e. the virtual reduction parameters, are uploaded into the intra-operative procedure, and the corresponding desired pose for the RFM to achieve the fracture reduction” [Intra-operative procedure]),
and to control the operation apparatus to move to the target pose of the operation apparatus for reduction (“Finally, the RFM executes the desired movement for F1 to achieve the physical reduction in the fracture, while reference bone F2 remains fixed. The real-time imaging updates the actual pose of the fragments in real time, and the surgeon can check intra-operatively the reduction in 3D without the use of any other intra-operative imaging device. If the reduction is acceptable, then the surgeon proceeds with the fixation of the fracture by using plate and screws or intramedullary nail, and the surgery ends.” [Intra-operative procedure]);
wherein the operation apparatus is connected to the target body by a tool and a fixation pin (“One optical tool (T1) is placed on the orthopaedic pin (P1) inserted in fragment 1 (F1), and a second optical tool (T2) is placed on the orthopaedic pin (P2) inserted in the reference bone (F2). A further optical tool (TR) is placed on the RFM (see Fig. 8a)…. An optical tracker TR is mounted on the RFM end- effector. The coordinate frame of TR is coincident with the coordinate frame of the robot end-effector, i.e. CFROT≡CFEE. A surgeon’s assistant rigidly connects P1 to the RFM, and the reduction software—based on the relative position of P1 with respect to the RFM (by tracking TR)—calculates the transformation RFM𝑻P1 between the robot and the orthopaedic pin P1.” [Intra-operative procedure]) .
the method comprising: obtaining a preliminary image, and further converting the preliminary image into the actual working space (“A pre-operative CT scan of the fracture and inserted pins is taken, and the resulting data set segmented to generate 3D models (STL format) of each bone fragment and the inserted pins using the ImageSim commercial software (Fig. 3b) [26]. These models are imported in the reduction software, and reference frames are defined as shown in Fig… Results of the pre-operative procedure are stored in the system and used for intra-operative navigation, robot motion command calculation, and for the evaluation of the reduction results, as described in the next subsection.” [Pre-operative planning], “The orthopaedic pins P1 and P2 were designed to be connected in a unique way to the optical tools T1 and T2 (Fig. 5), having their coordinate frames coincident, i.e. CFP1≡CFT1, and CFP2≡CFP1. Therefore, assuming that P1𝑻F1 and P2𝑻F2 are constant during the operation, the optical tracker provides the actual poses of F1 (by tracking P1) and F2 (by tracking P2). This establishes a direct correspondence between the image space (reduction software, virtual models) and the task space (real fracture) by using the optical tracker, which enables the intra-operative imaging. This is described by the transformations IMG𝑻F1 and IMG𝑻F2.” [Intra-operative procedure]);
obtaining a real-time pose of an operation apparatus tracer; and obtaining a pose of an operation apparatus in the actual working space according to a relative pose of the operation apparatus tracer and the operation apparatus and the real-time pose of the operation apparatus tracer (“the optical tracker provides the actual poses of F1 (by tracking P1) and F2 (by tracking P2). This establishes a direct correspondence between the image space (reduction software, virtual models) and the task space (real fracture) by using the optical tracker, which enables the intra-operative imaging. ” [Intra-operative procedure], “The real-time imaging updates the actual pose of the fragments in real time, and the surgeon can check intra-operatively the reduction in 3D without the use of any other intra-operative imaging device.“ [Intra-operative procedure]);
obtaining a target pose of the target body in the actual working space; obtaining a target pose of the operation apparatus in the actual working space according to a reduction plan, the target pose of the target body in the actual working space and a connection relationship between the target body and the operation apparatus (“An optical tracker TR is mounted on the RFM end- effector. The coordinate frame of TR is coincident with the coordinate frame of the robot end-effector, i.e. CFROT≡CFEE. A surgeon’s assistant rigidly connects P1 to the RFM, and the reduction software—based on the relative position of P1 with respect to the RFM (by tracking TR)—calculates the transformation RFM𝑻P1 between the robot and the orthopaedic pin P1. Results of the pre-operative planning, i.e. the virtual reduction parameters, are uploaded into the intra-operative procedure, and the corresponding desired pose for the RFM to achieve the fracture reduction” [Intra-operative procedure]);
and controlling the operation apparatus to move to the target pose of the operation apparatus for reduction (“Finally, the RFM executes the desired movement for F1 to achieve the physical reduction in the fracture, while reference bone F2 remains fixed. The real-time imaging updates the actual pose of the fragments in real time, and the surgeon can check intra-operatively the reduction in 3D without the use of any other intra-operative imaging device. If the reduction is acceptable, then the surgeon proceeds with the fixation of the fracture by using plate and screws or intramedullary nail, and the surgery ends.” [Intra-operative procedure]).
Dagnino does not specifically teach the host is configured to convert a preliminary image into an intermediate image by matching the preliminary image with the intermediate image through point cloud registration, and to obtain a conversion relationship between the intermediate image and the actual working space by comparing the geometric feature of the target body tracer in the intermediate image with the geometric feature of the target body tracer in the actual working space obtained by the optical tracker, and obtaining an intermediate image; converting the preliminary image into the intermediate image according to by matching the preliminary image and the intermediate image; obtaining a geometric feature of a target body tracer in an actual working space; obtaining a conversion relationship between the intermediate image and the actual working space by comparing the geometric feature of the target body tracer in the intermediate image with the geometric feature of the target body tracer in the actual working space.
However, in a similar field of endeavor, Crawford teaches a surgical implant planning computer is connectable to a fluoroscopy imager, a marker tracking camera, and a robot having a robot base coupled to a robot arm that is movable by motors relative to the robot base [Abstract].
Crawford also teaches the host is configured to convert a preliminary image into an intermediate image and obtaining an intermediate image; converting the preliminary image into the intermediate image according to by matching the preliminary image and the intermediate image; (“The robotic computer system is a Robotic Positioning System that includes a computer controlled robotic arm, hardware, and software that enables real time surgical navigation and robotic guidance using radiological patient images (pre-operative CT, intra-operative CT and fluoroscopy), using a dynamic reference base and positioning camera. The navigation and guidance system determines the registration or mapping between the virtual patient (points on the patient images) and the physical patient (corresponding points on the patient's anatomy).” [0094])
and to obtain a conversion relationship between the intermediate image and the actual working space by comparing the geometric feature of the target body tracer in the intermediate image with the geometric feature of the target body tracer in the actual working space obtained by the optical tracker; obtaining a geometric feature of a target body tracer in an actual working space; obtaining a conversion relationship between the intermediate image and the actual working space by comparing the geometric feature of the target body tracer in the intermediate image with the geometric feature of the target body tracer in the actual working space (“The intra-op registration fixture is placed onto a patient attachment instrument by clamping the compression clamp onto the shaft of the attachment instrument, allowing the fixture to hover over the surgical site. The fiducials are detected automatically in the intra-operative scan and are used to register the patient's anatomy during the scan to the DRB, which is tracked by the camera throughout the procedure. The reflective markers are detected by the camera. Once the registration is transferred to the DRB, the intra-op registration fixture is removed to provide access to the surgical site. FIG. 27 illustrates the intra-op registration fixture 2712 and pivoting arm 2708. FIG. 27 further illustrates the compression clamp 2602, the DRB knob 2604, a starburst connection 2406, a gear tooth joint 2710, and a set of seven fiducials 2714.” [0174].
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Dagnino as outlined above with the host is configured to convert a preliminary image into an intermediate image, and to obtain a conversion relationship between the intermediate image and the actual working space by comparing the geometric feature of the target body tracer in the intermediate image with the geometric feature of the target body tracer in the actual working space obtained by the optical tracker, and obtaining an intermediate image; converting the preliminary image into the intermediate image according to by matching the preliminary image and the intermediate image; obtaining a geometric feature of a target body tracer in an actual working space; obtaining a conversion relationship between the intermediate image and the actual working space by comparing the geometric feature of the target body tracer in the intermediate image with the geometric feature of the target body tracer in the actual working space as taught by Crawford, because the visualization can help guide the surgeon's planning and approach [0094].
Dagnino in view of Crawford does not specifically teach matching the preliminary image with the intermediate image through point cloud registration.
However, in a similar field of endeavor, Aghdasi methods and systems for generating a real-time or near-real-time three-dimensional (3D) virtual perspective of a scene for a mediated-reality viewer [0002].
Aghdasi also teaches matching the preliminary image with the intermediate image through point cloud registration (“The processing device of the imaging system can implement a method for registering the preoperative medical scan data to the intraoperative image data that includes processing intraoperative depth data of the scene. More specifically, the method can include processing the intraoperative image data to generate a point cloud depth map of the scene. Then, the method can utilize a registration algorithm that maps the point cloud depth map to the preoperative medical scan data. In some embodiments, the processing device of the imaging system can generate a 3D mesh based on the point cloud depth map that can be used in, for example, generating the 3D virtual image of the scene. Accordingly, the registration algorithm can be initiated based on the point cloud depth map rather than the 3D mesh. In some aspects of the present technology, utilizing the point cloud depth map allows the registration to be run in parallel to the generation of the 3D mesh and subsequent synthesis of the 3D virtual image, thereby increasing the processing speed of the imaging system.” [0021]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Dagnino in view of Crawford as outlined above with matching the preliminary image with the intermediate image through point cloud registration as taught by Aghdasi, because it can increase the processing speed of the imaging system [0021].
Regarding Claim 7, Dagnino discloses that further comprising: obtaining a real-time pose of the target body tracer; and obtaining a three-dimensional model for a target body on the basis of converting the preliminary image into the actual working space (“ The reduction software receives pre-operative CT scan data of the fracture and generates the 3D models of the bone fragments. The GUI displays the 3D models and allows the surgeon to interact with them by using a controller for pre- and intra-operative planning of fracture reduction, i.e. virtual reduction. The optical tracking system (Polaris Spectra, NDI Inc.) provides a real-time (25 Hz) pose update of the optical tools (0.25 mm accuracy) connected to the bone fragments and the RFM. The optical tools have different and unique geometries to enable real-time tracking. ” [Surgical system configuration]).
Regarding Claim 8, Dagnino discloses that further comprising: obtaining a real-time pose of a tool tracer; and obtaining a pose of the tool in the actual working space and a three-dimensional model for the tool according to a relative pose of the tool tracer and the tool and the real-time pose of the tool tracer (“ The reduction software receives pre-operative CT scan data of the fracture and generates the 3D models of the bone fragments. The GUI displays the 3D models and allows the surgeon to interact with them by using a controller for pre- and intra-operative planning of fracture reduction, i.e. virtual reduction. The optical tracking system (Polaris Spectra, NDI Inc.) provides a real-time (25 Hz) pose update of the optical tools (0.25 mm accuracy) connected to the bone fragments and the RFM. The optical tools have different and unique geometries to enable real-time tracking. ” [Surgical system configuration], “The robotic system is controlled by software according to the results of the pre- and intra-operative image analysis. The main components of the intra-operative procedure are the reduction software, the optical tracker, the robotic system, and the patient (i.e. the fracture). One optical tool (T1) is placed on the orthopaedic pin (P1) inserted in fragment 1 (F1), and a second optical tool (T2) is placed on the orthopaedic pin (P2) inserted in the reference bone (F2). A further optical tool (TR) is placed on the RFM (see Fig. 8a). The poses of the optical tools are measured in the optical tracking system (CFC), and the corresponding homogeneous transformations C𝑻TR, C𝑻P1, and C𝑻P2 can be calculated… This establishes a direct correspondence between the image space (reduction software, virtual models) and the task space (real fracture) by using the optical tracker, which enables the intra-operative imaging.” [Intra-operative procedure]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's
disclosure (CN112603538A; G. Dagnino et al., “Intra-operative fiducial-based CT/Fluoroscope Image Registration Framework for image-guided robot-assisted Joint Fracture surgery,” International Journal of Computer Assisted Radiology and Surgery, vol. 12, no. 8, pp. 1383–1397, May 2017.).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN MALDONADO whose telephone number is 703-756-1421. The examiner can normally be reached 8:00 am-4:00 pm PST M-Th Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Koharski can be reached on (571) 272-7230. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Steven Maldonado/
Patent Examiner, Art Unit 3797
/CHRISTOPHER KOHARSKI/Supervisory Patent Examiner, Art Unit 3797
Prosecution Insights