DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
This application claims benefit of provisional application 63/622,670 filed 01/19/2024.
Information Disclosure Statement
The information disclosure statements (IDS) submitted were filed on 04/04/2025 and 04/14/2025. The submissions are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4 and 19-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Stawiaski (US20220175467).
Regarding claim 1, Stawiaski teaches a surgical navigation system comprising:
a tracker (48) coupled to a surgical object (22) and comprising tracking elements arranged in a tracker geometry (Fig. 1, [0039-0040], [0043]);
a localizer (34, 36, 41) configured to track the tracker (48) and to detect parameters of the surgical object (22) ([0033], [0037], “The software converts the signals received from the camera unit 36 or the optical sensors 40 into data representative of the position and orientation of the objects being tracked”, [0043], “The optical sensors 40 of the localizer 34 receive signals from the trackers 44, 46, 48”, [0046], [0048], [0050], “the camera unit 36 receives optical signals 53 from the LEDs 50 of the trackers 44, 46, 48 and outputs to the processor 52 signals relating to the position of the LEDs 50 of the trackers 44, 46, 48 relative to the localizer 34”); and
a controller (26) in communication with the localizer (34) ([0033], [0037]) and configured to:
detect, with the localizer (34), a pose of the tracker geometry ([0043], [0070], “Therefore, the navigation computer may apply known pattern recognition algorithms to the short exposure image to discern the arrangement of tracker markers present in the target space”, [0078], [0090], “The navigation computer 26 relates the pattern of tracker markers to the virtual tracker data for each short exposure image to estimate a pose of the tracker in the target space”, [0094]),
receive a first parameter (shape) of the surgical object detected by the localizer ([0072], [0092], wherein the surfaces, ridges, and edges (shape) of the tracked object, i.e. surgical tool, detected during long exposure images comprises receiving a first parameter),
create, based on the first parameter, a temporary descriptor describing an appearance of the surgical object ([0072], [0092], wherein creating a model based on resolving the surfaces/shape of the tracked surgical tool comprises creating a temporary descriptor, [0071], wherein “rapidly update the virtual models of the tracked objects” means the model/descriptor is temporary),
receive a second parameter (pose) of the surgical object detected by the localizer ([0070], [0071], “The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects in order to keep the surgeon advised of the present positions and orientation with minimal or no latency”, [0074], “Throughout the specification, the terminology will be used to designate the short exposure as sufficient to detect the tracker markers and the long exposure as sufficient to detect, identify, or recognize aspects, characteristics, or geometry of the tracked object”, wherein the first parameter is received from long exposure images and the second parameter is received from short exposure images, [0078], [0090], [0094], wherein detecting the tracker marker arrangement and/or pose from short exposure images comprises receiving a second parameter of the surgical object),
update, based on the second parameter, the temporary descriptor to create an updated temporary descriptor describing the appearance of the surgical object ([0071], “The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects in order to keep the surgeon advised of the present positions and orientation with minimal or no latency”, wherein tracking the pose (position and orientation) of the surgical tool using short exposure images and updating the model of the tracked object (surgical tool) to reflect the tracked pose comprises updating, based on the second parameter (pose) the temporary descriptor (model) to create an updated temporary descriptor (updated model) describing the appearance (position and orientation (pose)) of the surgical object), and
track the surgical object (22) based on a combination of the updated temporary descriptor and the pose of the tracker geometry ([0039], [0070], “Therefore, the navigation computer may apply known pattern recognition algorithms to the short exposure image to discern the arrangement of tracker markers present in the target space”, [0090], “The navigation computer 26 relates the pattern of tracker markers to the virtual tracker data for each short exposure image to estimate a pose of the tracker in the target space”, wherein the arrangement of the tracker markers comprises a pose of the tracker geometry, [0071], “The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects in order to keep the surgeon advised of the present positions and orientation with minimal or no latency”, [0093], “…thereby providing tracking of the tool, and specifically the tool tip, by detecting and tracking the movement of the tracker's markers”, [0051], “Based on the position and orientation of the trackers 44, 46, 48 and the previously loaded data, such as virtual object data representing the geometry of the object to which the tracker is attached, navigation processor 52 determines the position of the working end of the surgical instrument 22 (e.g., the centroid of a surgical bur) and the orientation of the surgical instrument 22”).
Regarding claim 2, Stawiaski teaches the invention as claimed above in claim 1.
Stawiaski further teaches wherein:
the first parameter is detected by the localizer at a first time (Fig. 7, steps 212 & 214, [0089], [0092], wherein the first parameter comprising shape of the surgical tool is detected during calibration);
the second parameter is detected by the localizer at a second time ([0071], “During the surgical operation, other than during a calibration….The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects”, [0090], wherein the second parameter comprising pose of the tracker/surgical tool is detected after calibration for updating the model); and
the first time is prior to the second time (Fig. 7, [0089], [0092], [0071], wherein the system continually detects the second parameter comprising the pose of the tracker/tool for updating the model after detecting the first parameter comprising the shape of the tool for calibration for creating the model).
Regarding claim 3, Stawiaski teaches the invention as claimed above in claim 1.
Stawiaski further teaches wherein:
at least one of the first and second parameters is a physical parameter of the surgical object (Fig. 7, Claim 7, [0072], [0092], “The navigation computer 26 retrieves virtual object data representing the object to be tracked. The virtual object data corresponds to the tool geometry and is used to generate a 3D model of the tool as a virtual object…applies a detection algorithm to recognize ridges and edges to discern the shape of the surgical tool in the two stereo images to geometrically align with the 3D model of the virtual object data”); and
the physical parameter includes at least one of a geometry or shape of the surgical object, a contour of the surgical object, a color of the surgical object, an envelope of the surgical object, a surface roughness of the surgical object, a surface marking of the surgical object, or a color or shading of the surgical object ([0072], [0092], wherein the surface, geometry, and shape of the surgical object is detected).
Regarding claim 4, Stawiaski teaches the invention as claimed above in claim 1.
Stawiaski further teaches wherein:
at least one of the first and second parameters is a motion parameter of the surgical object ([0063-0064], wherein the pose of the instrument tracker is being tracked comprises the second parameter, [0046], “The navigation system 20 is able to detect movement of these objects by continuously taking images, reviewing the images, and detecting movement of the groups of pixels associated with the objects”, [0071], “…short exposure images, such as those described, may be used by the navigation computer to track movement of the objects in the target space over time. The short exposure images being limited to highlight and focus on the trackers…”); and
the motion parameter includes at least one of a speed or velocity, an acceleration of the surgical object, a rotation of the surgical object, and a displacement of the surgical object ([0063-0064], [0046], [0071], wherein tracking the pose of the surgical tool over time to determine movement includes a rotation and/or displacement of the surgical object)
Regarding claim 19, Stawiaski teaches a method of tracking a surgical object (22) using a localizer (34), the surgical object (22) being coupled to a tracker (48) comprising tracking elements arranged in a tracker geometry (Figs. 1 & 7, [0010-0011], [0016], [0031], [0033], [0039-0040], [0043]), the method comprising:
detecting, with the localizer (34), a pose of the tracker geometry ([0043], [0070], “Therefore, the navigation computer may apply known pattern recognition algorithms to the short exposure image to discern the arrangement of tracker markers present in the target space”, [0078], [0090], “The navigation computer 26 relates the pattern of tracker markers to the virtual tracker data for each short exposure image to estimate a pose of the tracker in the target space”, [0094]),
receiving a first parameter (shape) of the surgical object (22) detected by the localizer (34) ([0072], [0092], wherein the surfaces, ridges, and edges (shape) of the tracked object, i.e. surgical tool, detected during long exposure images comprises receiving a first parameter),
creating, based on the first parameter, a temporary descriptor (model) describing an appearance of the surgical object (22) ([0072], [0092], wherein creating a model based on resolving the surfaces/shape of the tracked surgical tool comprises creating a temporary descriptor, [0071], wherein “rapidly update the virtual models of the tracked objects” means the model/descriptor is temporary),
receiving a second parameter (pose) of the surgical object (22) detected by the localizer (34) ([0070], [0071], “The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects in order to keep the surgeon advised of the present positions and orientation with minimal or no latency”, [0074], “Throughout the specification, the terminology will be used to designate the short exposure as sufficient to detect the tracker markers and the long exposure as sufficient to detect, identify, or recognize aspects, characteristics, or geometry of the tracked object”, wherein the first parameter is received from long exposure images and the second parameter is received from short exposure images, [0078], [0090], [0094], wherein detecting the tracker marker arrangement and/or pose from short exposure images comprises receiving a second parameter of the surgical object),
updating, based on the second parameter, the temporary descriptor to create an updated temporary descriptor (updated model) describing the appearance of the surgical object (22) ([0071], “The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects in order to keep the surgeon advised of the present positions and orientation with minimal or no latency”, wherein tracking the pose (position and orientation) of the surgical tool using short exposure images and updating the model of the tracked object (surgical tool) to reflect the tracked pose comprises updating, based on the second parameter (pose) the temporary descriptor (model) to create an updated temporary descriptor (updated model) describing the appearance (position and orientation (pose)) of the surgical object), and
tracking the surgical object (22) based on a combination of the updated temporary descriptor and the pose of the tracker geometry ([0039], [0070], “Therefore, the navigation computer may apply known pattern recognition algorithms to the short exposure image to discern the arrangement of tracker markers present in the target space”, [0090], “The navigation computer 26 relates the pattern of tracker markers to the virtual tracker data for each short exposure image to estimate a pose of the tracker in the target space”, wherein the arrangement of the tracker markers comprises a pose of the tracker geometry, [0071], “The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects in order to keep the surgeon advised of the present positions and orientation with minimal or no latency”, [0093], “…thereby providing tracking of the tool, and specifically the tool tip, by detecting and tracking the movement of the tracker's markers”, [0051], “Based on the position and orientation of the trackers 44, 46, 48 and the previously loaded data, such as virtual object data representing the geometry of the object to which the tracker is attached, navigation processor 52 determines the position of the working end of the surgical instrument 22 (e.g., the centroid of a surgical bur) and the orientation of the surgical instrument 22”).
Regarding claim 20, Stawiaski teaches a non-transitory computer readable medium having instructions stored thereon ([0018], “The control and processing system includes a memory device operable to store virtual object data, and virtual tracker data. The control and processing system includes instructions that when executed cause the system to…”, [0037], “Navigation computer 26 has the displays 28, 29, central processing unit (CPU) and/or other processors, memory (not shown), and storage (not shown). The navigation computer 26 is loaded with software…”), the instructions configured to be executed by a controller (26) connected to a localizer (34) to cause the controller (26) to perform operations comprising ([0031-0033], [0037], [0049], [0062]):
detecting, with the localizer (34), a pose of the tracker geometry ([0043], [0070], “Therefore, the navigation computer may apply known pattern recognition algorithms to the short exposure image to discern the arrangement of tracker markers present in the target space”, [0078], [0090], “The navigation computer 26 relates the pattern of tracker markers to the virtual tracker data for each short exposure image to estimate a pose of the tracker in the target space”, [0094]),
receiving a first parameter (shape) of the surgical object (22) detected by the localizer (34) ([0072], [0092], wherein the surfaces, ridges, and edges (shape) of the tracked object, i.e. surgical tool, detected during long exposure images comprises receiving a first parameter),
creating, based on the first parameter, a temporary descriptor (model) describing an appearance of the surgical object (22) ([0072], [0092], wherein creating a model based on resolving the surfaces/shape of the tracked surgical tool comprises creating a temporary descriptor, [0071], wherein “rapidly update the virtual models of the tracked objects” means the model/descriptor is temporary),
receiving a second parameter (pose) of the surgical object (22) detected by the localizer (34) ([0070], [0071], “The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects in order to keep the surgeon advised of the present positions and orientation with minimal or no latency”, [0074], “Throughout the specification, the terminology will be used to designate the short exposure as sufficient to detect the tracker markers and the long exposure as sufficient to detect, identify, or recognize aspects, characteristics, or geometry of the tracked object”, wherein the first parameter is received from long exposure images and the second parameter is received from short exposure images, [0078], [0090], [0094], wherein detecting the tracker marker arrangement and/or pose from short exposure images comprises receiving a second parameter of the surgical object),
updating, based on the second parameter, the temporary descriptor to create an updated temporary descriptor describing the appearance of the surgical object (22) ([0071], “The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects in order to keep the surgeon advised of the present positions and orientation with minimal or no latency”, wherein tracking the pose (position and orientation) of the surgical tool using short exposure images and updating the model of the tracked object (surgical tool) to reflect the tracked pose comprises updating, based on the second parameter (pose) the temporary descriptor (model) to create an updated temporary descriptor (updated model) describing the appearance (position and orientation (pose)) of the surgical object), and
tracking the surgical object (22) based on a combination of the updated temporary descriptor and the pose of the tracker geometry (([0039], [0070], “Therefore, the navigation computer may apply known pattern recognition algorithms to the short exposure image to discern the arrangement of tracker markers present in the target space”, [0090], “The navigation computer 26 relates the pattern of tracker markers to the virtual tracker data for each short exposure image to estimate a pose of the tracker in the target space”, wherein the arrangement of the tracker markers comprises a pose of the tracker geometry, [0071], “The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects in order to keep the surgeon advised of the present positions and orientation with minimal or no latency”, [0093], “…thereby providing tracking of the tool, and specifically the tool tip, by detecting and tracking the movement of the tracker's markers”, [0051], “Based on the position and orientation of the trackers 44, 46, 48 and the previously loaded data, such as virtual object data representing the geometry of the object to which the tracker is attached, navigation processor 52 determines the position of the working end of the surgical instrument 22 (e.g., the centroid of a surgical bur) and the orientation of the surgical instrument 22”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Zhao (US20090088897) and Jones (US20180168490).
Regarding claim 5, Stawiaski teaches the invention as claimed above in claim 1.
However, Stawiaski fails to teach wherein the controller is configured to detect an environmental condition.
In an analogous surgical tool tracking field of endeavor, Zhao teaches such a feature. Zhao teaches imaging with a camera to track robotic surgical instruments (Abstract, [0061-0062]). Zhao teaches illumination variation is a natural challenge in tool tracking ([0092-0093]). Zhao teaches appearance learning techniques (724) may be used to handle image or appearance changes such as from illumination variation ([0101]). Zhao teaches appearance learning techniques have been used extensively in object tracking to handle appearance changes due to illumination variations and teaches parametric models have been built to handle the illumination variations ([0101]). Zhao teaches training tool tracking systems on image samples of the same tool under different viewing conditions in order to handle illumination variation ([0121]).
However, the modified combination noted above fails to specifically teach to normalize the detected parameters according to the environmental condition (i.e. illumination).
In an analogous image analysis field of endeavor, Jones teaches such a feature. Jones teaches correcting for non-uniform illumination ([1308], [1321], [1328], [1336], [1436]). Jones teaches normalizing raw images against a camera baseline to account for illumination ([1308], [1321], [1328], [1336], [1436]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to detect an environmental condition such as illumination as taught by Zhao ([0092-0093], [0101]) or Jones ([1336], [1346]) and to normalize the raw images according to the environmental condition as taught by Jones ([1308], [1321], [1328], [1336], [1436]). Illumination variation may image-based processing and tool tracking less reliable as recognized by Zhao ([0092-0093]). By normalizing the raw images against a camera baseline to account for illumination, like taught by Jones, the effects of illumination variation on tool tracking may predictably be reduced or eliminated. Modifying the invention of Stawiaski to detect illumination (i.e. a baseline camera illumination) and to normalize raw images against the camera baseline illumination would predictably result in the parameters being detected from the images to be normalized according to the environmental condition (i.e. illumination).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Feilkas (US20080135733).
Regarding claim 6, Stawiaski teaches the invention as claimed above in claim 1.
Stawiaski further teaches wherein:
the localizer (34) includes a first sensor (41) configured to detect visible light and a second sensor (40) configured to detect infrared or near-infrared light ([0033-0034]);
the pose of the tracker geometry is detected by the second sensor ([0040], [0043], [0074], [0089], “short exposure images are acquired by the left optical sensor 40 of the camera unit 36 and by the right optical sensor 40 of the camera unit 36”,[0090], “The navigation computer 26 relates the pattern of tracker markers to the virtual tracker data for each short exposure image to estimate a pose of the tracker in the target space”).
However, Stawiaski fails to teach the first parameter is detected by the first sensor.
In an analogous surgical instrument tracking field of endeavor, Feilkas teaches such a feature. Feilkas teaches calibrating and tracking a surgical instrument ([0002], [0021], [0035], [0039]). Feilkas teaches a navigation system (1) including cameras (4a, 4b) which can be used as video cameras in which visible light is detected ([0055]). Feilkas teaches the cameras (4a, 4b) may be in the visible range for detecting the shape or geometry of the surgical instrument ([0056]). Feilkas therefore teaches a first sensor configured to detect visible light and wherein a first parameter (i.e. shape of the surgical instrument/object) is detected by the first sensor (visible light video camera).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Feilkas to use the video cameras configured to detect visible light to detect the first parameter comprising the shape of the instrument as taught by Feilkas ([0055-0056]). The shape of the instrument may suitably be detected using video cameras as recognized by Feilkas ([0056]), and by detecting the shape, an instrument may be calibrated/verified, and deviations in shape may be detected as further recognized by Feilkas ([0037-0039], [0056]).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Feilkas (US20080135733) as applied to claim 6 above, and further in view of Roessler (US20170333137).
Regarding claim 7, Stawiaski in view of Feilkas teaches the invention as claimed above in claim 6.
Stawiaski further teaches wherein:
the controller (26) detect the pose of the tracker geometry (48) in a first coordinate system (LCLZ) (Abstract, [0037], [0056], “tracking of objects is generally conducted with reference to a localizer coordinate system LCLZ”, [0062-0063], “Based on the same signals received for the instrument tracker 48, the localization engine 100 determines the pose of the instrument tracker coordinate system TLTR in the localizer coordinate system LCLZ”, [0078], [0090]); and
the controller is configured to register one of the first (LCLZ) and second coordinate systems to one of the first coordinate system, the second coordinate system, and a third coordinate system (other trackers/objects) ([0065], [0067], “other trackers may be coupled to any other suitable object to be tracked within the operating room, and each object and associated tracker may be registered to the localizer coordinate system LCLZ as described above”, wherein each tracker has its own coordinate system as described in [0057] and thus registering each tracker to the localizer coordinate system comprises registering the first coordinate system to a third coordinate system).
However, Stawiaski fails to teach wherein the controller detects the first parameter in a second coordinate system.
In an analogous surgical tracking field of endeavor, Roessler teaches such a feature. Roessler teaches tracking physical objects near a target site during a surgical procedure (Abstract, [0002]). Roessler teaches detecting the pose of trackers in a localizer coordinate system, i.e. poses of tracker geometry in a first coordinate system ([0050], [0052]). Roessler further teaches a vision device (72) comprising a camera (160) configured to provide vision data sets, the vision data sets being a set of data points in the vision coordinate system (VIS) ([0054-0055]). Roessler teaches the camera (160) may identify and track physical objects (surgical objects) near a target site by identifying features of the physical objects that define a feature set (170) ([0056-0057]). Roessler teaches the features may be parameters of the physical objects such as coatings or surface roughness ([0058]). Roessler teaches the vision data set includes the features of the feature set (170) ([0059]) and thus teaches wherein a first parameter (features of a surgical object) is detected in a second coordinate system (vision coordinate system).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to detect the features of the surgical object (first parameter) in the second coordinate system as taught by Roessler ([0050], [0052], [0054-0059]). Different sensing modalities have their own native frame of reference and thus when you have a localizer which detects tracker pose and a camera which detects object features, whatever the localizer detects is in its own coordinate system and the same goes for the camera. By detecting object features with a separate vision camera, and thus a second coordinate system, features such as surface roughness or coatings may be detected and be used to describe the object as recognized by Roessler ([0058]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Zhang (US20130245429).
Regarding claim 8, Stawiaski teaches the invention as claimed above in claim 1.
Stawiaski further teaches wherein the controller is configured to:
track the pose of the surgical object (22) based on a combination of the updated temporary descriptor and the pose of the tracker geometry ([0071], “The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects in order to keep the surgeon advised of the present positions and orientation with minimal or no latency”, [0051], “Based on the position and orientation of the trackers 44, 46, 48 and the previously loaded data, such as virtual object data representing the geometry of the object to which the tracker is attached, navigation processor 52 determines the position of the working end of the surgical instrument 22 (e.g., the centroid of a surgical bur) and the orientation of the surgical instrument 22”).
However, Stawiaski fails to teach wherein the controller is configured to: detect a third parameter of the surgical object, and update the temporary descriptor based on the third parameter.
In an analogous tracking of surgical objects field of endeavor, Zhang teaches such a feature. Zhang teaches tracking one or more objects such as catheters in a sequence of images (Abstract, [0002], [0028]). Zhang teaches updating a dictionary to represent changes in appearance of a target object ([0013]). Zhang therefore teaches wherein the dictionary is akin to a model of a target object or temporary descriptor. Zhang teaches receiving new image frames, showing a change in the appearance of objects over time ([0055]). Zhang teaches if a new feature is identified in the new image frames, the dictionary is updated with the new appearance information of the object by applying semi-supervised learning algorithms ([0055]). Zhang therefore teaches detecting a third parameter (new feature/appearance information) of a surgical object and updating a temporary descriptor (i.e. dictionary) based on the third parameter.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to detect new features from new image frames received and to update the temporary descriptor of the surgical object based on new appearance information of the object as taught by Zhang ([0055]). By updating the temporary descriptor describing the target object based on new parameters or features identified from the target object, tracking of the object may predictably be made more reliable and accurate.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Zhang (US20130245429) as applied to claim 8 above, and further in view of Ahmed (US20210142492).
Regarding claim 9, Stawiaski in view of Zhang teaches the invention as claimed above in claim 8.
However, Stawiaski fails to teach wherein the controller is configured to: determine that the pose of the tracker geometry is not detectable by the localizer, and track the pose of the surgical object based solely on the updated temporary descriptor in response to determining that the pose of the tracker geometry is not detectable by the localizer.
In an analogous tracking of objects field of endeavor, Ahmed teaches such a feature. Ahmed teaches tracking an object (404) with a camera and losing the object (404) due to it leaving the field of view of the camera ([0095]). Ahmed teaches the processor can relocate the object if it re-enters the field of view by using a CAD model or an updated 3D object map ([0095]). Ahmed teaches the pose of the object (404) may be determined using the CAD model ([0093]). Ahmed therefore teaches determining that an object is not detectable (i.e. leaves field of view) and tracking the pose of the object based on an updated descriptor or model in response.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to detect the object is not detectable and to track the object using the updated model as taught by Ahmed ([0093], [0095]). The model may be used to re-locate the object when it re-enters the field of view and also track the pose of the object as recognized by Ahmed ([0093], [0095]). Ahmed teaches detecting that the object is lost ([0095]). Stawiaski modified by the teachings of Ahmed would result in determining that the pose of the tracker geometry is not detectable by the localizer as this would cause the object to be “lost” by the tracker system of Stawiaski. Moreover, since Ahmed teaches using a model or updated 3D object map to restore tracking, Stawiaski modified by the teachings of Ahmed would predictably result in using the model of the object that is continually being updated, i.e. the updated temporary descriptor.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Zhang (US20130245429) and Ahmed (US20210142492) as applied to claim 9 above, and further in view of Calloway (US20210338337).
Regarding claim 10, Stawiaski in view of Zhang and Ahmed teaches the invention as claimed above in claim 9.
However, Stawiaski fails to teach wherein the controller is configured to: determine that the pose of the tracker geometry is detectable by the localizer after determining that the pose of the tracker geometry is not detectable by the localizer, and track the pose of the surgical object based on at least one of the updated temporary descriptor and the pose of the tracker geometry in response to determining that the pose of the tracker geometry is detectable by the localizer.
In an analogous tracking system field of endeavor, Calloway teaches such a feature. Calloway teaches a camera tracking system for computer assisted navigation during surgery (Abstract, [0001]). Calloway teaches tracking the poses of surgical tools within the field-of-view of a set of tracking cameras ([0162]). Calloway teaches the tracking cameras may resume tracking when the surgical tools are moved from outside of field-of-view to back inside field-of-view ([0162]). Moreover, Calloway teaches wherein the pose of the surgical tools is determined by determining the pose of attached DRAs (dynamic reference arrays, i.e. tracker geometry) ([0133]). Calloway therefore teaches determining that a pose of an surgical object is detectable (i.e. in field-of-view) after determining that it is not detectable (i.e. out of field-of-view) and also teaches resuming tracking of pose of the surgical object based at least on the pose of tracker geometry in response to determining that the pose of the tracker geometry is detectable by the localizer.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to resume pose tracking of the surgical object when the surgical object and its tracker geometry is back inside field-of-view as taught by Calloway ([0133], [0162]). By resuming pose tracking of the surgical object via the tracker geometry, tracking of the object may predictably be made more reliable or robust when the tracked objects move in and out of field of view.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Zhang (US20130245429), Ahmed (US20210142492), and Calloway (US20210338337) as applied to claim 10 above, and further in view of Kheradpir (US20200281659).
Regarding claim 11, Stawiaski in view of Zhang, Ahmed, and Calloway teaches the invention as claimed above in claim 10.
However, Stawiaski fails to teach wherein the controller is configured to register or re-register the tracker to the surgical object by comparing a new parameter of the surgical object to the updated temporary descriptor.
In an analogous surgical navigation system field of endeavor, Kheradpir teaches such a feature. Kheradpir teaches calibrating a medical tool with a frame and frame tracking marker attached to the frame (e.g. tracker geometry) ([0011]). Kheradpir teaches the medical navigation system (205) computes registration to reference markers, allowing for the medical navigation system (205) to know the relationship between the tip of a tracked instrument relative to tracking markers of the tracked instrument ([0089]). Kheradpir teaches wherein a calibration apparatus (600) calibrates a medical tool having a tool tracking marker ([0099]). Kheradpir teaches wherein calibration involves calculating an expected spatial relationship between a tool tracking marker (e.g. the tool) and frame tracking markers (e.g. the tracker geometry) ([0122]). Kheradpir teaches re-calibrating (e.g. re-registering) the tool if at least one dimension of the tool is altered beyond a threshold value in relation to an expected relationship (i.e. comparing a new parameter (dimension) of a surgical object to a descriptor (expected relationship)).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to re-register or re-calibrate the surgical tool if one of its dimensions is altered compared to an expected relationship as taught by Kheradpir ([0122]). The new parameter, i.e. altered dimension, may predictably cause inaccuracies in tracking and thus re-registering the surgical object with its new parameter (altered dimension) may help maintain accuracy of object tracking. Stawiaski modified by the teachings of Kheradpir to re-register the surgical object when one of its dimensions has been altered would predictably result in re-registering the surgical object to its tracker by comparing the altered dimension (new parameter) to the expected dimension (updated model or updated temporary descriptor) and to subsequently update the model with the altered dimension.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Fuerst (US20210290311).
Regarding claim 12, Stawiaski teaches the invention as claimed above in claim 1.
However, Stawiaski fails to teach wherein the controller provides the first and second parameters to a machine learning module, and the machine learning module outputs a calculated parameter of the surgical object based on the first and second parameters.
In an analogous object tracking field of endeavor, Fuerst teaches such a feature. Fuerst teaches sensors (55, 57) configured to measure a magnetic field associated with a trocar (63), the tracked object ([0042-0043]). Fuerst teaches a machine learning model comprising a convolutional neural network takes, as input, data from the two sensors (55, 57), therefore teaching providing of first and second parameters to a machine learning module ([0074]). Fuerst teaches the input data is not limited to magnetic sensors and may instead be, for example, images produced by cameras ([0074]). Fuerst teaches the machine learning model outputs the 3D position and orientation of the tracked object based on the two inputs received from the sensors (Fig. 15, [0077-0078], see figure 15). Fuerst therefore teaches providing first and second parameters (data from two sensors) to a machine learning module, and the machine learning module outputs a calculated parameter (3D position and orientation) of the surgical object based on the first and second parameters. Fuerst further teaches wherein the machine-learning or neural network based solution for pose estimation may be deployed in parallel with a deterministic model-based solution ([0062]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to additionally provide inputs to a machine learning module for it to output a calculated parameter such as pose as taught by Fuerst (Fig. 15, [0062], [0074], [0077-0078]). By deploying a machine-learning or neural network based solution in parallel with other solutions for pose estimation, robustness of pose estimation may be increased as recognized by Fuerst ([0062], last sentence).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Fuerst (US20210290311) as applied to claim 12 above, and further in view of Neubauer (US20060264742).
Regarding claim 13, Stawiaski in view of Fuerst teaches the invention as claimed above in claim 12.
However, Stawiaski fails to teach wherein the controller is configured to: update the updated temporary descriptor based on the calculated parameter to create a learned temporary descriptor describing the association between the surgical object and the detected tracker geometry, and track the pose of the surgical object based on a combination of the learned temporary descriptor and the pose of the tracker geometry.
In an analogous surgical instrument tracking field of endeavor, Neubauer teaches such a feature. Neubauer teaches calibrating a surgical instrument for use in a navigation system and computer-assisted surgery (Abstract, [0008-0009]). Neubauer teaches by using geometric data determined from a scan, e.g. calculated parameters, a 3D model of a surgical instrument may be calculated ([0029-0030]). Neubauer teaches a complete model of the instrument includes the reference system, the reference system comprising a tracker attached to the instrument ([0002], [0032]). Neubauer therefore teaches a learned descriptor (model) describing the association between the surgical object (instrument) and a detected tracker geometry (reference system). Neubauer teaches the model contains all the information necessary for navigation, information concerning geometry of the instrument’s functional elements and reference system, and their relation to one another ([0011]). Neubauer teaches the geometric information (e.g. model) concerning the instrument can be transferred to the navigation system in order to enable the surgeon to use the instrument in computer-assisted surgery (i.e. tracking the instrument’s pose) ([0011]). Neubauer teaches the position and orientation (pose) of surgical instruments with reference to the patient’s anatomical structures are represented to the surgeon with aid of the navigation system ([0003]). Neubauer therefore further teaches tracking the pose of the surgical object based on a combination of the learned temporary descriptor (model) and the pose of the tracker geometry (relation between the instrument and reference system).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to further update the model to include the tracker geometry and its relation to the surgical instrument for tracking as taught by Neubauer ([0002-0003], [0011], [0029-0030], [0032]). Because the surgical instrument includes a fixed tracker attached thereto, the model of the instrument may be made more accurate by including said tracker since it would result in a more accurate physical representation of the instrument. Moreover, Stawiaski already teaches tracking the pose of the surgical object based on a model of the object and the tracked pose of the tracker geometry and thus Stawiaski modified by the teachings of Neubauer would result in the same but with the updated model including the tracker.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Fuerst (US20210290311) and Neubauer (US20060264742) as applied to claim 13 above, and further in view of Zhang (US20130245429).
Regarding claim 14, Stawiaski in view of Fuerst and Neubauer teaches the invention as claimed above in claim 13.
Stawiaski further teaches wherein the controller is configured to:
track the pose of the surgical object (22) based on a combination of the updated learned temporary descriptor and the pose of the tracker geometry ([0071], “The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects in order to keep the surgeon advised of the present positions and orientation with minimal or no latency”, [0051], “Based on the position and orientation of the trackers 44, 46, 48 and the previously loaded data, such as virtual object data representing the geometry of the object to which the tracker is attached, navigation processor 52 determines the position of the working end of the surgical instrument 22 (e.g., the centroid of a surgical bur) and the orientation of the surgical instrument 22”; the virtual object data would include any updates to the model, including an updated learned temporary descriptor).
However, Stawiaski fails to teach wherein the controller is configured to: receive a third parameter of the surgical object, compare the calculated parameter to the third parameter, update the learned temporary descriptor based on the comparison to create an updated learned descriptor describing the association between the surgical object and the detected tracker geometry.
In an analogous tracking of surgical objects field of endeavor, Zhang teaches such a feature. Zhang teaches tracking one or more objects such as catheters in a sequence of images (Abstract, [0002], [0028]). Zhang teaches updating a dictionary to represent changes in appearance of a target object ([0013]). Zhang therefore teaches wherein the dictionary is akin to a model of a target object or temporary descriptor. Zhang teaches receiving new image frames, showing a change in the appearance of objects over time ([0055]). Zhang teaches labeling each image frame as either “update” or “non-update” based on features present in the image frame, thereby teaching comparing of any new parameters/features to a previous model (i.e. calculated parameter). Zhang teaches if a new feature is identified in the new image frames, the dictionary is updated with the new appearance information of the object by applying semi-supervised learning algorithms ([0055]). Zhang therefore teaches receiving a third parameter (new feature/appearance information) of a surgical object, comparing the third parameter/new feature with a previous model/dictionary, and updating a temporary descriptor (i.e. dictionary) based on the third parameter if the third parameter is new to create an updated learned descriptor.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to detect new features from new image frames received and to update the learned temporary descriptor of the surgical object based on new appearance information of the object as taught by Zhang ([0055]). By updating the learned temporary descriptor describing the target object based on new parameters or features identified from the target object, tracking of the object may predictably be made more reliable and accurate. Stawiaski in view of Neubauer above teaches wherein the temporary descriptor is a learned temporary descriptor which describes the association between the surgical object and the detected tracker geometry. Therefore Stawiaski in view of Neubauer further modified by the teachings of Zhang would result in updating the learned temporary descriptor to create an updated learned temporary descriptor which describes an association between the surgical object and the detected tracker geometry
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Morgan (US20190357986) and Zehavi (US20240225752).
Regarding claim 15, Stawiaski teaches the invention as claimed above in claim 1.
However, Stawiaski fails to teach wherein the second parameter is realized as a motion parameter, and the controller is configured to: detect a pose change of the tracker geometry, and create the updated temporary descriptor.
In an analogous tracking of an object field of endeavor, Morgan teaches such a feature. Morgan teaches determining a displaced posed of a tracker ([0124], wherein a displaced pose comprises a motion parameter). Morgan teaches comparing the displaced pose to the original pose at time of registration to update the geometry of the tracker setup, i.e. update a temporary descriptor ([0124]). Morgan therefore teaches detecting a pose change of tracker geometry and creating an updated temporary descriptor.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to detect displacement of the pose of the tracker and to correspondingly update the geometry of the tracker setup to account for the difference as taught by Morgan ([0124]). By updating the geometry of the tracker setup in response to detecting a displacement of pose of the tracker, tracker registration may be recovered or maintained as recognized by Morgan ([0124]).
However, the modified combination noted above fails to teach wherein creating the updated temporary descriptor is based on a determination that the motion parameter is sufficiently aligned with the pose change of the tracker geometry.
In an analogous pose determination field of endeavor, Zehavi teaches such a feature. Zehavi teaches validating a pose of a tool and/or robotic arm based on a determined pose of the tool and/or robotic arm ([0120]). Zehavi teaches the pose of the tool/robotic arm may be validated by comparing the determined pose of the tool/robotic arm to pose information received from the tool/robotic arm, i.e. pose information received from a sensor ([0120]). Zehavi teaches when the pose information received matches, i.e. aligns, with the determined pose determined from the tracking device, the pose of the tool/robotic arm is validated ([0120]). Zehavi therefore teaches checking whether the pose determined via the tracker is consistent or aligns with the pose determined by other means.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski in view of Morgan to first determine whether the determined pose is valid or aligns with pose information received from other means as taught by Zehavi ([0120]). By cross-checking the tracker-determined pose with pose measured by other means and validating the pose determination, the determined pose may be made more reliable. Modifying Stawiaski in view of Morgan which teaches updating a temporary descriptor (i.e. geometry of the tracker) with the teaches of Zehavi to ensure that the determined pose is aligned with or matches pose determined by other means (e.g. the detected motion parameter being displacement of the pose) would predictably result in the created updated temporary descriptor to be based on a determination that the motion parameter (pose displacement) is sufficiently aligned with or matches the detected pose change of the tracker geometry.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Morgan (US20190357986).
Regarding claim 16, Stawiaski teaches the invention as claimed above in claim 1.
However, Stawiaski fails to teach wherein the controller is configured to replace the first parameter with the second parameter in response to a determination that the second parameter is inconsistent with the first parameter.
In an analogous tracking of an object field of endeavor, Morgan teaches such a feature. Morgan teaches an instrument tracker (48) is coupled to a surgical instrument (22) ([0035]). Morgan therefore teaches the instrument tracker (48) is a parameter of the surgical instrument (22) or surgical object. Morgan teaches determining a displaced pose of a tracker ([0124]). Morgan teaches comparing the displaced pose of the tracker (i.e. second parameter) with the original pose of the tracker at the time of registration (i.e. first parameter) ([0124]). Morgan teaches updating the geometry of the tracker setup to account for the difference; Morgan teaches generating a new geometry of the tracker setup (second parameter) that replaces the old geometry (first parameter) ([0124]). Morgan therefore teaches replacing a first parameter with a second parameter in response to a determination that the second parameter is inconsistent with the first parameter.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to replace the geometry of the tracker setup when its pose is displaced as recognized by Morgan ([0124]). Registration may be fully recovered or maintained by correcting the geometry as recognized by Morgan ([0124]).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of Schipper (US20220331971).
Regarding claim 17, Stawiaski teaches the invention as claimed above in claim 1.
Stawiaski further teaches wherein the tracker (48) is realized as a first tracker (48) and the surgical object (22) is realized as a first surgical object (22) (Fig. 3, [0038-0040]), and further comprising:
a second tracker (44) coupled to a second surgical object (femur F) and comprising tracking elements arranged in a tracker geometry (Fig. 3, [0038-0040]); and
wherein the localizer (34) is configured to track the first (48) and second trackers (44) and to detect parameters associated with the first surgical object (22) and second surgical object (femur F) ([0003-0004], [0043-0044], [0050], [0058], [0064]); and
wherein the controller (26) is configured to:
detect, with the localizer (34), a pose of the first tracker (48) and a first parameter associated with the first surgical object (22) ([0037], “the software converts the signals received from the camera unit 36 or the optical sensors 40 into data representative of the position and orientation of the objects being tracked”, [0050-0051], “Based on the position and orientation of the trackers 44, 46, 48…. navigation processor 52 determines the position of the working end of the surgical instrument 22 (e.g., the centroid of a surgical bur) and the orientation of the surgical instrument 22”, wherein the pose of the surgical instrument 22 comprises a first parameter associated with the first surgical object),
create a first tracking entity (first model) based on the pose of the first tracker and the first parameter ([0071], wherein tracking the pose of the trackers allows for virtual models of the tracked objects to be updated, the virtual models comprise a first tracking entity, [0072], [0092], “generate a 3D model of the tool as a virtual object… discern the shape of the surgical tool in the two stereo images to geometrically align with the 3D model of the virtual object data”, [0093], “The rigid relationship between the tracker and the tool geometry is captured and reflected in the virtual environment of the navigation computer 26, thereby providing tracking of the tool, and specifically the tool tip, by detecting and tracking the movement of the tracker's markers”, wherein a 3D model of the tool is generated based on tracking of pose of the markers and the tool’s pose in the two stereo images),
detect, with the localizer (34), a pose of the second tracker (44) and a second parameter associated with the second surgical object ([0037], [0048], [0050-0051], [0058], [0060], wherein the pose of the femur F comprises a second parameter associated with the second surgical object),
create a second tracking entity (second model) based on the pose of the second tracker and the second parameter ([0004], “From the detected position of the trackers, the surgical navigation system can determine the position and/or orientation of the surgical tool or patient anatomy, and monitor and track the position and/or orientation for changes over time”, [0006], [0010], [0046], [0071-0072], wherein models of tracked objects are created and updated based on tracking of the trackers, i.e. a model of the femur comprises a second tracking entity based on the pose of the second tracker and the second parameter, [0086], “constructs a depth map for matching similar features between the two stereo images using, for example, block matching methods so that the depth map obtained is then geometrically aligned with the 3D model of the virtual object data”, wherein a model of patient anatomy may be generated, i.e. the tracked femur, based on the pose of the anatomy in two stereo images), and
track poses of the first and second surgical objects based on movement of the first and second tracking entities, respectively ([0004], [0071], “…track movement of the objects in the target space over time. The short exposure images being limited to highlight and focus on the trackers themselves allow the rapid image acquisition and image processing necessary to rapidly update the virtual models of the tracked objects”, [0093], “The rigid relationship between the tracker and the tool geometry is captured and reflected in the virtual environment of the navigation computer 26, thereby providing tracking of the tool, and specifically the tool tip, by detecting and tracking the movement of the tracker's markers”, wherein the updated virtual models in virtual space reflect or indicate the pose of the surgical objects, thus allowing for the poses of the objects to be tracked based on movement of the models (first and second tracking entities) in the virtual space).
However, Stawiaski fails to teach wherein the tracking elements arranged in a tracker geometry of the second tracker is identical to the tracker geometry of the first tracker.
In an analogous optical tracking system field of endeavor, Schipper teaches such a feature. Schipper teaches an image sensor (122) configured to capture images to determine poses of objects being tracked ([0052]). Schipper teaches a target (112) (tracker) attached to a femur (104) and another target (114) (tracker) attached to a tool (142) (Fig. 1, [0052-0053]). Schipper teaches the target tracker (114) can be identical to target tracker (112), i.e. have the same geometry (Fig. 1, [0053]). Schipper also teaches wherein the target trackers could be different, having different geometries ([0053]). As shown in figure 1, the target trackers 112 and 114 have the same geometry in which the tracking elements are arranged the same or identical. Schipper therefore teaches wherein the tracking elements arranged in a tracker geometry of a second tracker (112) may be identical to the tracker geometry of a first tracker (114).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to have the trackers have the same tracker geometry as taught by Schipper (Fig. 1, [0053]). Schipper shows objects may sufficiently be tracked by using the same tracker geometry and thus costs or complexity may be lowered as only one type of tracker geometry is needed or required for tracking.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Stawiaski (US20220175467) in view of DeLand (US20130113929).
Regarding claim 18, Stawiaski teaches the invention as claimed above in claim 1.
Stawiaski teaches the invention further comprising:
a database containing descriptors of surgical objects ([0016], “querying a database of surgical tool models; and retrieving virtual object data based on querying the database of surgical tool models”, [0044]); and
wherein the controller (26) is configured to:
compare the first parameter (shape) of the surgical object to the descriptors (model, virtual object data) stored in the database ([0092], “The navigation computer 26 retrieves virtual object data representing the object to be tracked. The virtual object data corresponds to the tool geometry and is used to generate a 3D model of the tool as a virtual object… applies a detection algorithm to recognize ridges and edges to discern the shape of the surgical tool in the two stereo images to geometrically align with the 3D model of the virtual object data”).
However, Stawiaski fails to teach wherein the controller is further configured to: based on the comparison, determine that there are no descriptors in the database describing the appearance of the surgical object, and, in response, create the temporary descriptor describing the surgical object based on the detected parameter of the surgical object.
In an analogous tracking of surgical instruments field of endeavor, DeLand teaches such a feature. Deland teaches a recognizing surgical instruments via images captured by a camera and tracking said instruments (Abstract, [0001]). DeLand teaches the identities, i.e. the sizes, shapes, color, and texture, of the surgical instruments are stored in a database ([0035], [0049], [0051]). DeLand teaches an instrument tracker (255) comprising multiple cameras configured to track surgical instruments ([0027-0028]). DeLand teaches if the instrument tracker cannot identify a surgical instrument or found on the menus (i.e. database) within the surgical instrument tracker, the surgical instrument may be added to the database within the surgical instrument tracker ([0056]). DeLand teaches recording images of the surgical instrument from different angles in order to update the database with the newly identified surgical instrument ([0056]). DeLand therefore teaches importing the shape or appearance of the surgical instrument into the database to add the unknown/unidentified surgical instrument to the database. The model or appearance of the surgical instrument in the database comprises a temporary descriptor describing the surgical object based on the detected parameter (shape/appearance) of the surgical object.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the invention of Stawiaski to have the controller be configured to add a surgical instrument to the database if it was not found in the database as taught by DeLand ([0056]). By allowing such, new surgical instruments not previously known or stored in the database may predictably be used and tracked by the system.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TOMMY T LY whose telephone number is (571) 272-6404. The examiner can normally be reached M-F 12:00pm-8:00pm eastern time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at 571-272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TOMMY T LY/ Examiner, Art Unit 3797
/SERKAN AKAR/ Primary Examiner, Art Unit 3797
Prosecution Insights