DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-20 are currently pending in the present application, with claims 1, 10, and 18 being independent.
Response to Amendments / Arguments
Applicant’s arguments, see Pg. 7-9, filed 07/28/2026, with respect to the rejection(s) of claim(s) 1-20 under 35 U.S.C. §102 and §103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of newly found prior art.
Regarding the remaining arguments: Applicant argues with respect to the amended claim language, which is fully addressed in the prior art rejections set forth below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8, and 10-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tako et al. (WO 2017066373), hereinafter referred to as “Tako”, in view of Noble et al. (US 20220395171), hereinafter referred to as “Noble”, and in further view of Weitzman et al. (US 20190374236), hereinafter referred to as “Weitzman”.
Regarding claim 1, Tako discloses an augmented reality (AR) display system for a surgical environment (Par. 0114; Augmented Reality Surgical Navigation System ("ARS") 100), the AR display system comprising:
, wherein the end of the shaft is configured to be inserted into a patient (Par. 0111; Real Time Tracking and Feedback to track a real surgery instrument during the surgery. Par. 0114; surgeon 102 can inspect specific patient 104 anatomy…step into the MD6DM model and physically move the user's body and look around at specific structures. The user can look 360 around a targeted anatomy or pathology);
a wearable AR display device configured to be worn by a surgeon and provide AR imagery to the surgeon (Fig. 1; Head-Mounted Display (HMD) 160' and Fig. 24. Par. 0114; the surgeon 102 no longer needs to take his eyes off the microscope or endoscope 106 to look at patient scans, planning software or a MD6DM model because the virtual image and the real optical image are projected side by side, or fused together and displayed either on a display monitor 108 or on the Head-Mounted Display (HMD) 160'); and
processing circuitry configured to (Fig. 1; ARS computing device 112):
obtain real-time image data from the scope (Par. 0115-0116; FIG. 31A-31C modified to incorporate the features of MD6DM), enables the ARS system 100 by obtaining patient DICOM 2D images and building virtual models in real time or obtaining prebuilt models…The ARS computing device 112 is further configured to integrate the prebuilt images with a live video feed received from the endoscope 106…The ARS computing device 112 is further configured to receive, as a second input, a live video feed 204, such as a video feed from an endoscope), X-ray image data of the patient (Par. 0088-0089; The SNAP allows surgeons to analyze and plan a specific patient's case before surgery, and then take that plan into the operating room (OR) and use it in conjunction with the navigation system during surgery… medical imaging device that present real time and dynamic multiple line of sights views (from different/multiple perspectives) of the surgery procedure. The image includes the scanned medical image (based on scan such as CT, MRI, Ultrasound, X-ray etc.) and the surgery instruments), and navigation data for a surgical procedure of the patient (Par. 0115-0116; The ARS computing device 112 is further configured to integrate the prebuilt images with a live video feed received from the endoscope 106 based on navigation data also received from the tracking tip of the endoscope 106…The ARS computing device 112 is further configured to receive, as a third input, navigation system tracking data 206. In one example, the tracking data 206 is indicative of the position and orientation of a surgeon 102 relative to the patient 104);
generate an AR environment that includes the real-time image data, the X-ray image data, and the navigation data (Par. 0114-0116; Augmented Reality Surgical Navigation System ("ARS") 100 described herein and illustrated in Figure 1, which gives the surgeon 102 or another user (such as assistants, nurses, or other doctors) an integrated view of the optic and scanned views on the same screen (as illustrated in Figure 3)… planning software or a MD6DM model because the virtual image and the real optical image are projected side by side, or fused together and displayed either on a display monitor 108 or on the Head-Mounted Display (HMD) 160'…The ARS computing device 112 is further configured to integrate the prebuilt images with a live video feed received from the endoscope 106 based on navigation data also received from the tracking tip of the endoscope 106 to form an augmented reality view of the patient 104 with tracking capabilities…); and
operate the wearable AR display device to provide the AR environment including the real-time image data, the X-ray data, and the navigation data as the AR imagery (Par. 0117; the ARS computing device 112 processes the three inputs 202, 204, and 206, and integrates them to form an augmented reality view of the patient 104 with tracking and navigating capabilities. The ARS computing device 112 is further configured to output the augmented reality view of the patient to a device 208 for viewing. The device 208 may be, for example, a display monitor 108 or a HMD 160' (or both) …).
Tako does not disclose a scope comprising a rigid shaft and a distal camera disposed at an end of the shaft, the shaft including a single bend.
In the same art of surgical instruments and camera systems, Noble discloses a scope comprising a rigid shaft and a distal camera disposed at an end of the shaft (Par. 0004; a camera device that provides for direct visualization during spinal surgery that allows a user to better assess a patient's anatomy without unroofing. In an implementation, a camera device has a handpiece; a shaft further comprising: a proximal end coupled to the handpiece; and a distal end. The camera device also has a probe coupled to the distal end of the shaft…the distal portion having an image sensor assembly. Par. 0034; The diameter of the distal portion 50 allows for a wall thickness of greater than about 1.7 mm around the shaft 30 which creates a rigid structure that can be used to move tissue out of the field of view) the shaft including a single bend (Fig. 8 and Par. 0039; as shown in FIGS. 8 and 9 the shaft 30 has one or more bends 70 to allow a user to more easily access different parts of a patient's anatomy. As shown in FIG. 8, the shaft 30 may have a single bend 70…Each bend may be between 30 and about 150 degrees).
Tao teaches a scope, but does not teach the scope having a rigid shaft that includes a single bend. On the other hand, Noble teaches a scope that has a shaft with a single bend. It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the scope of Tako to incorporate a rigid shaft including a single bend and a distal camera as taught by Noble. Doing so provides direct visualization during spinal surgery while allowing access to different portions of the patient’s anatomy (Par. 0004; provides for direct visualization during spinal surgery that allows a user to better assess a patient's anatomy without unroofing…Par. 0038; allows a user to better assess the anatomy before determining the need to destabilize the spine by unroofing the affected anatomy. This can lead to better surgical outcomes by minimizing the need for unneeded fusion procedures…), yielding predictable results in improved positioning and maneuverability of the scope within a confined surgical environment.
Tako in view of Noble does not appear to explicitly disclose the single bend sized to provide clearance between a base section of the scope and the end of the shaft such that a site at which the scope is inserted into a patient is accessible by a disc preparation instrument.
In the same art of spinal surgical instruments, Weitzman discloses the single bend sized to provide clearance between a base section of the scope and the end of the shaft such that a site at which the scope is inserted into a patient is accessible by a disc preparation instrument (Par. 0020; whereas the distal bend is configured to enable access and maneuverability of the device at hidden anatomies, e.g. between or underneath vertebral discs, the proximal bend(s) is/are configured to enable an unhindered line of site of the device's distal tip. Fig. 1A-1C, 2A-2B, and Par. 0136; single proximal bend 150…Proximal bend 150 serves to ensure that part of elongated hollow member 130 distal to proximal bend 150 is offset of handle 120, thus ensuring a surgeon's unhindered line of site 101 to rotatable cutting tip 110, when holding handle 120. Advantageously, proximal bend 150 enables utilizing surgical tools…).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to further configure the single bend of the modified scope to include the offset configuration as taught by Weitzman. Doing so provides unobstructed access for surgical instruments and line of sight during visualization of the operative site (Par. 0017; a shaft with a single proximal bend may be of particular advantage when utilizing surgical microscopes and/or endoscopes that have short focal length, and that preferably would be located parallel to the shaft and close to the surgical site, this, since the handle is prevented from interfering with the illumination path and the line of sight of the endoscope), yielding predictable results in improved instrument clearance, allow simultaneous use of disc preparation tools, enhanced maneuverability, and visibility during a surgical procedure.
Regarding claim 2, Tako in view of Noble in further view of Weitzman discloses the AR display system of claim 1, and Tako further discloses wherein the processing circuitry is further configured to:
obtain a user input (Fig. 29 and Par. 0059; Controller - A device which includes buttons and a direction controller) indicating a desired location, size, and orientation of at least one of the real-time image data, the X-ray image data, or the navigation data (Par. 103-104; the display 160 may be a touchscreen display, the display 160 may act as the user interface 130, along with other devices such as a keyboard, mouse, or other input device…The SNAP tool also provides an external system interface 150 that allows the system to interface to other external systems that provide navigation or other functions to the surgeon, so that the SNAP tool can generate images that are consistent with the outputs of the external systems, e.g., mirror the navigation or update the imaging based on the operation of the external system(s). The SNAP tool can then update its display images appropriately for providing an integrated view to the surgeon in high resolution, 3D images that interact with the graphical tools); and
generate the AR environment according to the desired location, size, and orientation of the at least one of the real-time image data, the X-ray image data, or the navigation data (Par. 0117; the ARS computing device 112 is able to calculate and pinpoint the angle of view in addition to location and therefore offer a more unique and distinct experience for the surgeon in the operating room. In particular, the ARS computing device 112 processes the three inputs 202, 204, and 206, and integrates them to form an augmented reality view of the patient 104 with tracking and navigating capabilities. The ARS computing device 112 is further configured to output the augmented reality view of the patient to a device 208 for viewing. The device 208 may be, for example, a display monitor 108 or a HMD 160' (or both). It should be appreciated that, although only three (3) inputs are illustrated, the ARS computing device 112 may be configured to receive any suitable number of inputs from a variety of suitable input sources in order to provide the surgeon 102 more complete information that may be helpful for performing a particular surgery. For example, the ARS computing device 112 may be configured to receive additional live video feeds or additional tracking and sensor information).
Tako, Noble, and Weitzman are combined for the reasons set forth above with respect to claim 1.
Regarding claim 3, Tako in view of Noble in further view of Weitzman discloses the AR display system of claim 1, and Tako further discloses wherein the X-ray image data comprises X-ray imagery of one or more locations of the patient at which the surgical procedure is being performed (Par. 0089; medical imaging device that present real time and dynamic multiple line of sights views (from different/multiple perspectives) of the surgery procedure. The image includes the scanned medical image (based on scan such as CT, MRI, Ultrasound, X-ray etc.) and the surgery instruments).
Tako, Noble, and Weitzman are combined for the reasons set forth above with respect to claim 1.
Regarding claim 4, Tako in view of Noble in further view of Weitzman discloses the AR display system of claim 1, and Tako further discloses wherein the navigation data comprises instructions or guidelines for one or more steps for performing the surgical procedure (Par. 0075; The SNAP provides visual graphic guidance to the surgeon…Par. 0087-0088; The SNAP provides surgeons (neurosurgeons etc.), doctors, and their aids a unique virtual-reality guidance to determine the safest and most efficient pathway to remove tumors (cerebral etc.) and treat anomalies as vascular anomalies…The SNAP allows surgeons to analyze and plan a specific patient's case before surgery, and then take that plan into the operating room (OR) and use it in conjunction with the navigation system during surgery. The SNAP then presents the navigation data into the advanced inter-active, high quality 3D image, with multiple points of view…).
Tako, Noble, and Weitzman are combined for the reasons set forth above with respect to claim 1.
Regarding claim 5, Tako in view of Noble in further view of Weitzman discloses the AR display system of claim 1, but Tako does not explicitly disclose wherein the shaft comprises an elongated tubular member including the camera and a light positioned at the end of the elongated tubular member.
Noble discloses wherein the shaft comprises an elongated tubular member including the camera and a light positioned at the end of the elongated tubular member (Fig. 8 and Par. 0004; Par. 0004; a camera device that provides for direct visualization during spinal surgery that allows a user to better assess a patient's anatomy without unroofing. In an implementation, a camera device has a handpiece; a shaft further comprising: a proximal end coupled to the handpiece; and a distal end. The camera device also has a probe coupled to the distal end of the shaft…the distal portion having an image sensor assembly. Par. 0028; The optical fibers 46 transmit light from the light source 28 out of the distal end 40 of the probe 36 to illuminate a surgical scene. In alternative implementations, a light source or plurality of light sources may be positioned proximal to the distal end 40 of the probe 36. Power may be provided to the light source or plurality of light sources using a wire extending from the handpiece 16 along the shaft 30)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Noble’s light source into the endoscope of Tako. It is understood that endoscopes used for internal imaging conventionally include a light source at or near the end of the insertion shaft to illuminate internal anatomy for image capture, therefore, providing distal illumination is a known and routine design feature in surgical endoscopes in order to visualize low-light internal body cavities, yielding predictable results of improved image quality and visibility.
Regarding claim 6, Tako in view of Noble in further view of Weitzman discloses the AR display system of claim 5, but Tako in view of Noble does not explicitly disclose wherein the single bend is defined between a first portion and a second portion of the shaft, the first portion coupled to a base section of the scope, and the second portion defining the end of the shaft, the first portion being shorter than the second portion, wherein the single bend is positioned along the shaft and sized to provide clearance between the base section and the second portion such that a site at which the scope is inserted into the patient is accessible by the disc preparation instrument without obstruction by the base section.
Weitzman discloses wherein the single bend is defined between a first portion and a second portion of the shaft, the first portion coupled to a base section of the scope, and the second portion defining the end of the shaft, the first portion being shorter than the second portion (Fig. 1A-1C and Par. 136; proximal bend 150), wherein the single bend is positioned along the shaft and sized to provide clearance between the base section and the second portion such that the site at which the scope is inserted into the patient is accessible by the disc preparation instrument without obstruction by the base section (Par. 0020; whereas the distal bend is configured to enable access and maneuverability of the device at hidden anatomies, e.g. between or underneath vertebral discs, the proximal bend(s) is/are configured to enable an unhindered line of site of the device's distal tip. Par. 0136; single proximal bend 150…Proximal bend 150 serves to ensure that part of elongated hollow member 130 distal to proximal bend 150 is offset of handle 120, thus ensuring a surgeon's unhindered line of site 101 to rotatable cutting tip 110, when holding handle 120. Advantageously, proximal bend 150 enables utilizing surgical tools…).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Tako and Noble’s combined surgical display system to incorporate the bent shaft geometry of Weitzman. The motivation lies in the advantage of offsetting the proximal base section away from the insertion site to improve instrument clearance, provide simultaneous use of disc preparation tools, enhance maneuverability, and visibility during a surgical procedure.
Regarding claim 7, Tako in view of Noble in further view of Weitzman discloses the AR display system of claim 6, and Tako further discloses wherein the base section configured to provide a surface for the surgeon to hold the scope (Tako Fig. 1; endoscope 106).
Tako, Noble, and Weitzman are combined for the reasons set forth above with respect to claim 6.
Regarding claim 8, Tako in view of Noble in further view of Weitzman discloses the AR display system of claim 1, but Tako does not appear to explicitly disclose wherein the scope comprises a wire extending through the shaft from the camera.
Noble discloses wherein the scope comprises a wire extending through the shaft from the camera (Fig. 1, Fig. 3, and Par. 0023; The handpiece 16 and camera control unit 18 may be connected via wire or wirelessly. Par. 0027; The image sensor assembly 42 is electrically coupled to the handpiece. In an implementation, a printed wiring assembly (PWA) 44 extends from the image sensor assembly 42, through the shaft 30 to a printed circuit board in the handpiece 16).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the scope of Tako to incorporate the explicit wired electrical connection, as taught by Noble. The motivation lies in the advantage of providing a conventional and reliable communication path between the distal image sensor and the proximal camera for transmitting live image data from an endoscopic camera to associated processing devices.
Regarding claim 10, Tako discloses a display system for a surgical environment (Fig. 1), the display system comprising:
wherein the end of the shaft is configured to be inserted into a patient (Par. 0111; Real Time Tracking and Feedback to track a real surgery instrument during the surgery. Par. 0114; surgeon 102 can inspect specific patient 104 anatomy…step into the MD6DM model and physically move the user's body and look around at specific structures. The user can look 360 around a targeted anatomy or pathology),
processing circuitry configured to (Fig. 1; ARS computing device 112):
obtain real-time image data from the camera of the scope (Par. 0115-0116; FIG. 31A-31C modified to incorporate the features of MD6DM), enables the ARS system 100 by obtaining patient DICOM 2D images and building virtual models in real time or obtaining prebuilt models…The ARS computing device 112 is further configured to integrate the prebuilt images with a live video feed received from the endoscope 106…The ARS computing device 112 is further configured to receive, as a second input, a live video feed 204, such as a video feed from an endoscope), X-ray image data of the patient (Par. 0088-0089; The SNAP allows surgeons to analyze and plan a specific patient's case before surgery, and then take that plan into the operating room (OR) and use it in conjunction with the navigation system during surgery… medical imaging device that present real time and dynamic multiple line of sights views (from different/multiple perspectives) of the surgery procedure. The image includes the scanned medical image (based on scan such as CT, MRI, Ultrasound, X-ray etc.) and the surgery instruments) from an X-ray database (Fig. 1; database 114) and navigation data providing step-by-step instructions and illustrations (Par. 0088-0089; SNAP. Par. 0111; Feedback is provided to the surgeon based on the patient specific simulation and the instruments' location and orientation…the system generates feedback to the surgeons for the type of tissue he is dissecting and alarming the surgeon in case that he dissects healthy brain tissue instead of a tumor….the system allows the surgeon to rotate the simulated image/model that is princely oriented as the real anatomical structure based on the tracking, and observe and evaluate the location and efficacy of the placed implant…Par. 0128; Voice - Speaking keywords or phrases that are received by a sensor such as a microphone activates a specific functionality…) from a navigation database (Fig. 1; database 114) for a surgical procedure of the patient (Par. 0115-0116; The ARS computing device 112 is further configured to integrate the prebuilt images with a live video feed received from the endoscope 106 based on navigation data also received from the tracking tip of the endoscope 106…The ARS computing device 112 is further configured to receive, as a third input, navigation system tracking data 206. In one example, the tracking data 206 is indicative of the position and orientation of a surgeon 102 relative to the patient 104);
generate an AR environment that includes the real-time image data, the X-ray image data, and the navigation data (Par. 0114-0116; Augmented Reality Surgical Navigation System ("ARS") 100 described herein and illustrated in Figure 1, which gives the surgeon 102 or another user (such as assistants, nurses, or other doctors) an integrated view of the optic and scanned views on the same screen (as illustrated in Figure 3)… planning software or a MD6DM model because the virtual image and the real optical image are projected side by side, or fused together and displayed either on a display monitor 108 or on the Head-Mounted Display (HMD) 160'…The ARS computing device 112 is further configured to integrate the prebuilt images with a live video feed received from the endoscope 106 based on navigation data also received from the tracking tip of the endoscope 106 to form an augmented reality view of the patient 104 with tracking capabilities…), the real time image data, and the navigation data displayed simultaneously in separate window panes at locations in an operating room (Par. 0096; The SNAP can present models on one or multiply windows on the same screen or on multiply screens. Examples for the features and applications of the SNAP, multiple features can be activated side by side on the screen. Par. 0115; …It should be appreciated that the multiple views (i.e. the live feed and the retrieved images) can also be viewed separately or in parallel either on the display monitor or through the HMD 160); and
operate the wearable AR display device to provide the AR environment including the real-time image data, the X-ray data, and the navigation data as the AR imagery (Par. 0117; the ARS computing device 112 processes the three inputs 202, 204, and 206, and integrates them to form an augmented reality view of the patient 104 with tracking and navigating capabilities. The ARS computing device 112 is further configured to output the augmented reality view of the patient to a device 208 for viewing. The device 208 may be, for example, a display monitor 108 or a HMD 160' (or both) …).
Tako does not disclose a scope comprising a rigid shaft and a distal camera disposed at an end of the shaft, the shaft including a single bend.
In the same art of surgical instruments and camera systems, Noble discloses a scope comprising a rigid shaft and a distal camera disposed at an end of the shaft (Par. 0004; a camera device that provides for direct visualization during spinal surgery that allows a user to better assess a patient's anatomy without unroofing. In an implementation, a camera device has a handpiece; a shaft further comprising: a proximal end coupled to the handpiece; and a distal end. The camera device also has a probe coupled to the distal end of the shaft…the distal portion having an image sensor assembly. Par. 0034; The diameter of the distal portion 50 allows for a wall thickness of greater than about 1.7 mm around the shaft 30 which creates a rigid structure that can be used to move tissue out of the field of view) the shaft including a single bend (Fig. 8 and Par. 0039; as shown in FIGS. 8 and 9 the shaft 30 has one or more bends 70 to allow a user to more easily access different parts of a patient's anatomy. As shown in FIG. 8, the shaft 30 may have a single bend 70…Each bend may be between 30 and about 150 degrees).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the scope of Tako to incorporate a rigid shaft including a single bend and a distal camera as taught by Noble. Doing so provides direct visualization during spinal surgery while allowing access to different portions of the patient’s anatomy (Par. 0004; provides for direct visualization during spinal surgery that allows a user to better assess a patient's anatomy without unroofing…Par. 0038; allows a user to better assess the anatomy before determining the need to destabilize the spine by unroofing the affected anatomy. This can lead to better surgical outcomes by minimizing the need for unneeded fusion procedures…), yielding predictable results in improved positioning and maneuverability of the scope within a confined surgical environment.
Tako in view of Noble does not appear to explicitly disclose the single bend sized to provide clearance between a base section of the scope and the end of the shaft such that a site at which the scope is inserted into a patient is accessible by a disc preparation instrument.
In the same art of spinal surgical instruments, Weitzman discloses the single bend sized to provide clearance between a base section of the scope and the end of the shaft such that a site at which the scope is inserted into a patient is accessible by a disc preparation instrument (Par. 0020; whereas the distal bend is configured to enable access and maneuverability of the device at hidden anatomies, e.g. between or underneath vertebral discs, the proximal bend(s) is/are configured to enable an unhindered line of site of the device's distal tip. Fig. 1A-1C, 2A-2B, and Par. 0136; single proximal bend 150…Proximal bend 150 serves to ensure that part of elongated hollow member 130 distal to proximal bend 150 is offset of handle 120, thus ensuring a surgeon's unhindered line of site 101 to rotatable cutting tip 110, when holding handle 120. Advantageously, proximal bend 150 enables utilizing surgical tools…).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to further configure the single bend of the modified scope to include the offset configuration as taught by Weitzman. Doing so provides unobstructed access for surgical instruments and line of sight during visualization of the operative site (Par. 0017; a shaft with a single proximal bend may be of particular advantage when utilizing surgical microscopes and/or endoscopes that have short focal length, and that preferably would be located parallel to the shaft and close to the surgical site, this, since the handle is prevented from interfering with the illumination path and the line of sight of the endoscope), yielding predictable results in improved instrument clearance, allow simultaneous use of disc preparation tools, enhanced maneuverability, and visibility during a surgical procedure.
Regarding claim 11, Tako in view of Noble in further view of Weitzman discloses the display system of claim 10, and Tako further discloses wherein the processing circuitry is further configured to:
obtain a user input (Fig. 29 and Par. 0059; Controller - A device which includes buttons and a direction controller) indicating a desired location, size, and orientation of at least one of the real-time image data, the X-ray image data, or the navigation data (Par. 103-104; the display 160 may be a touchscreen display, the display 160 may act as the user interface 130, along with other devices such as a keyboard, mouse, or other input device…The SNAP tool also provides an external system interface 150 that allows the system to interface to other external systems that provide navigation or other functions to the surgeon, so that the SNAP tool can generate images that are consistent with the outputs of the external systems, e.g., mirror the navigation or update the imaging based on the operation of the external system(s). The SNAP tool can then update its display images appropriately for providing an integrated view to the surgeon in high resolution, 3D images that interact with the graphical tools); and
generate the AR environment according to the desired location, size, and orientation of the at least one of the real-time image data, the X-ray image data, or the navigation data (Par. 0117; the ARS computing device 112 is able to calculate and pinpoint the angle of view in addition to location and therefore offer a more unique and distinct experience for the surgeon in the operating room. In particular, the ARS computing device 112 processes the three inputs 202, 204, and 206, and integrates them to form an augmented reality view of the patient 104 with tracking and navigating capabilities. The ARS computing device 112 is further configured to output the augmented reality view of the patient to a device 208 for viewing. The device 208 may be, for example, a display monitor 108 or a HMD 160' (or both). It should be appreciated that, although only three (3) inputs are illustrated, the ARS computing device 112 may be configured to receive any suitable number of inputs from a variety of suitable input sources in order to provide the surgeon 102 more complete information that may be helpful for performing a particular surgery. For example, the ARS computing device 112 may be configured to receive additional live video feeds or additional tracking and sensor information).
Regarding claim 12, Tako in view of Noble in further view of Weitzman discloses the display system of claim 10, and Tako further discloses wherein the X-ray image data comprises X-ray imagery of one or more locations of the patient at which the surgical procedure is being performed (Par. 0089; medical imaging device that present real time and dynamic multiple line of sights views (from different/multiple perspectives) of the surgery procedure. The image includes the scanned medical image (based on scan such as CT, MRI, Ultrasound, X-ray etc.) and the surgery instruments).
Tako, Noble, and Weitzman are combined for the reasons set forth above with respect to claim 1.
Regarding claim 13, Tako in view of Noble in further view of Weitzman discloses the display system of claim 10, and Tako further discloses wherein the navigation data comprises instructions or guidelines for one or more steps for performing the surgical procedure (Par. 0075; The SNAP provides visual graphic guidance to the surgeon…Par. 0087-0088; The SNAP provides surgeons (neurosurgeons etc.), doctors, and their aids a unique virtual-reality guidance to determine the safest and most efficient pathway to remove tumors (cerebral etc.) and treat anomalies as vascular anomalies…The SNAP allows surgeons to analyze and plan a specific patient's case before surgery, and then take that plan into the operating room (OR) and use it in conjunction with the navigation system during surgery. The SNAP then presents the navigation data into the advanced inter-active, high quality 3D image, with multiple points of view…).
Tako, Noble, and Weitzman are combined for the reasons set forth above with respect to claim 1.
Regarding claim 14, Tako in view of Noble in further view of Weitzman discloses the display system of claim 10, but Tako does not explicitly disclose wherein the shaft comprises an elongated tubular member including the camera and a light positioned at the end of the elongated tubular member.
Noble discloses wherein the shaft comprises an elongated tubular member including the camera and a light positioned at the end of the elongated tubular member (Fig. 8 and Par. 0004; Par. 0004; a camera device that provides for direct visualization during spinal surgery that allows a user to better assess a patient's anatomy without unroofing. In an implementation, a camera device has a handpiece; a shaft further comprising: a proximal end coupled to the handpiece; and a distal end. The camera device also has a probe coupled to the distal end of the shaft…the distal portion having an image sensor assembly. Par. 0028; The optical fibers 46 transmit light from the light source 28 out of the distal end 40 of the probe 36 to illuminate a surgical scene. In alternative implementations, a light source or plurality of light sources may be positioned proximal to the distal end 40 of the probe 36. Power may be provided to the light source or plurality of light sources using a wire extending from the handpiece 16 along the shaft 30)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Noble’s light source into the endoscope of Tako. It is understood that endoscopes used for internal imaging conventionally include a light source at or near the end of the insertion shaft to illuminate internal anatomy for image capture, therefore, providing distal illumination is a known and routine design feature in surgical endoscopes in order to visualize low-light internal body cavities, yielding predictable results of improved image quality and visibility.
Regarding claim 15, Tako in view of Noble in further view of Weitzman discloses the display system of claim 14, but Tako in view of Noble does not explicitly disclose wherein the single bend is defined between a first portion and a second portion of the shaft, the first portion coupled to a base section of the scope, and the second portion defining the end of the shaft, the first portion being shorter than the second portion, wherein the single bend is positioned along the shaft and sized to provide clearance between the base section and the second portion such that a site at which the scope is inserted into the patient is accessible by the disc preparation instrument without obstruction by the base section.
Weitzman discloses wherein the single bend is defined between a first portion and a second portion of the shaft, the first portion coupled to a base section of the scope, and the second portion defining the end of the shaft, the first portion being shorter than the second portion (Fig. 1A-1C and Par. 136; proximal bend 150), wherein the single bend is positioned along the shaft and sized to provide clearance between the base section and the second portion such that the site at which the scope is inserted into the patient is accessible by the disc preparation instrument without obstruction by the base section (Par. 0020; whereas the distal bend is configured to enable access and maneuverability of the device at hidden anatomies, e.g. between or underneath vertebral discs, the proximal bend(s) is/are configured to enable an unhindered line of site of the device's distal tip. Par. 0136; single proximal bend 150…Proximal bend 150 serves to ensure that part of elongated hollow member 130 distal to proximal bend 150 is offset of handle 120, thus ensuring a surgeon's unhindered line of site 101 to rotatable cutting tip 110, when holding handle 120. Advantageously, proximal bend 150 enables utilizing surgical tools…).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Tako and Noble’s combined surgical display system to incorporate the bent shaft geometry of Weitzman. The motivation lies in the advantage of offsetting the proximal base section away from the insertion site to improve instrument clearance, provide simultaneous use of disc preparation tools, enhance maneuverability, and visibility during a surgical procedure.
Regarding claim 16, Tako in view of Noble in further view of Weitzman discloses the display system of claim 15, and Tako further discloses wherein the base section configured to provide a surface for the surgeon to hold the scope (Fig. 1; endoscope 106).
Tako and Weitzman are combined for the reasons set forth above with respect to claim 15.
Regarding claim 17, Tako in view of Noble in further view of Weitzman discloses the display system of claim 10, but Tako does not appear to explicitly disclose wherein the scope comprises a wire extending through the shaft from the camera.
Noble discloses wherein the scope comprises a wire extending through the shaft from the camera (Fig. 1, Fig. 3, and Par. 0023; The handpiece 16 and camera control unit 18 may be connected via wire or wirelessly. Par. 0027; The image sensor assembly 42 is electrically coupled to the handpiece. In an implementation, a printed wiring assembly (PWA) 44 extends from the image sensor assembly 42, through the shaft 30 to a printed circuit board in the handpiece 16).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the scope of Tako to incorporate the explicit wired electrical connection, as taught by Noble. The motivation lies in the advantage of providing a conventional and reliable communication path between the distal image sensor and the proximal camera for transmitting live image data from an endoscopic camera to associated processing devices.
Regarding claim 18, Tako discloses a method of providing augmented reality (AR) for a surgical environment (Par. 0114; Augmented Reality Surgical Navigation System ("ARS") 100), the method comprising:
wherein the end of the shaft is configured to be inserted into a patient (Par. 0111; Real Time Tracking and Feedback to track a real surgery instrument during the surgery. Par. 0114; surgeon 102 can inspect specific patient 104 anatomy…step into the MD6DM model and physically move the user's body and look around at specific structures. The user can look 360 around a targeted anatomy or pathology),
obtaining real-time image data from the camera of the scope during a surgical procedure (Par. 0115-0116; FIG. 31A-31C modified to incorporate the features of MD6DM), enables the ARS system 100 by obtaining patient DICOM 2D images and building virtual models in real time or obtaining prebuilt models…The ARS computing device 112 is further configured to integrate the prebuilt images with a live video feed received from the endoscope 106…The ARS computing device 112 is further configured to receive, as a second input, a live video feed 204, such as a video feed from an endoscope), navigation data providing step-by-step instructions and illustrations for the surgical procedure (Par. 0088-0089; SNAP. Par. 0111; Feedback is provided to the surgeon based on the patient specific simulation and the instruments' location and orientation…the system generates feedback to the surgeons for the type of tissue he is dissecting and alarming the surgeon in case that he dissects healthy brain tissue instead of a tumor….the system allows the surgeon to rotate the simulated image/model that is princely oriented as the real anatomical structure based on the tracking, and observe and evaluate the location and efficacy of the placed implant…Par. 0128; Voice - Speaking keywords or phrases that are received by a sensor such as a microphone activates a specific functionality…), and X-ray image data of the patient (Par. 0088-0089; The SNAP allows surgeons to analyze and plan a specific patient's case before surgery, and then take that plan into the operating room (OR) and use it in conjunction with the navigation system during surgery… medical imaging device that present real time and dynamic multiple line of sights views (from different/multiple perspectives) of the surgery procedure. The image includes the scanned medical image (based on scan such as CT, MRI, Ultrasound, X-ray etc.) and the surgery instruments);
generating an AR environment based on the real-time image data, the navigation data, and the X-ray image data (Par. 0114-0116; Augmented Reality Surgical Navigation System ("ARS") 100 described herein and illustrated in Figure 1, which gives the surgeon 102 or another user (such as assistants, nurses, or other doctors) an integrated view of the optic and scanned views on the same screen (as illustrated in Figure 3)… planning software or a MD6DM model because the virtual image and the real optical image are projected side by side, or fused together and displayed either on a display monitor 108 or on the Head-Mounted Display (HMD) 160'…The ARS computing device 112 is further configured to integrate the prebuilt images with a live video feed received from the endoscope 106 based on navigation data also received from the tracking tip of the endoscope 106 to form an augmented reality view of the patient 104 with tracking capabilities…), the real time image data, the navigation data, and the X-ray image data displayed simultaneously in separate window panes at locations in an operating room (Par. 0096; The SNAP can present models on one or multiply windows on the same screen or on multiply screens. Examples for the features and applications of the SNAP, multiple features can be activated side by side on the screen. Par. 0115; …It should be appreciated that the multiple views (i.e. the live feed and the retrieved images) can also be viewed separately or in parallel either on the display monitor or through the HMD 160), the AR environment comprising a size and position of the real-time image data, the navigation data, and the X-ray image data (Par. 0091; The patient's anatomical structures in 2D or 3D and the position and orientation of the surgery instruments are synchronized in real time and are presented to the surgeon with a real time location and orientation of the instruments and markers in space in relation to the anatomical structures. Par. 0115; enables the ARS system 100 by obtaining patient DICOM 2D images and building virtual models in real time or obtaining prebuilt models. Data is obtained from a database 114, for example. The ARS computing device 112 is further configured to integrate the prebuilt images with a live video feed received from the endoscope 106 based on navigation data also received from the tracking tip of the endoscope 106 to form an augmented reality view of the patient 104 with tracking capabilities); and
operating an AR display device to provide the AR environment comprising the real-time image data, the navigation data, and the X-ray image data (Par. 0117; the ARS computing device 112 processes the three inputs 202, 204, and 206, and integrates them to form an augmented reality view of the patient 104 with tracking and navigating capabilities. The ARS computing device 112 is further configured to output the augmented reality view of the patient to a device 208 for viewing. The device 208 may be, for example, a display monitor 108 or a HMD 160' (or both) …).
Tako does not disclose providing a scope comprising a rigid shaft and a distal camera disposed at an end of the shaft, the shaft including a single bend.
In the same art of surgical instruments and camera systems, Noble discloses providing a scope comprising a rigid shaft and a distal camera disposed at an end of the shaft (Par. 0004; a camera device that provides for direct visualization during spinal surgery that allows a user to better assess a patient's anatomy without unroofing. In an implementation, a camera device has a handpiece; a shaft further comprising: a proximal end coupled to the handpiece; and a distal end. The camera device also has a probe coupled to the distal end of the shaft…the distal portion having an image sensor assembly. Par. 0034; The diameter of the distal portion 50 allows for a wall thickness of greater than about 1.7 mm around the shaft 30 which creates a rigid structure that can be used to move tissue out of the field of view) the shaft including a single bend (Fig. 8 and Par. 0039; as shown in FIGS. 8 and 9 the shaft 30 has one or more bends 70 to allow a user to more easily access different parts of a patient's anatomy. As shown in FIG. 8, the shaft 30 may have a single bend 70…Each bend may be between 30 and about 150 degrees).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the scope of Tako to incorporate a rigid shaft including a single bend and a distal camera as taught by Noble. Doing so provides direct visualization during spinal surgery while allowing access to different portions of the patient’s anatomy (Par. 0004; provides for direct visualization during spinal surgery that allows a user to better assess a patient's anatomy without unroofing…Par. 0038; allows a user to better assess the anatomy before determining the need to destabilize the spine by unroofing the affected anatomy. This can lead to better surgical outcomes by minimizing the need for unneeded fusion procedures…), yielding predictable results in improved positioning and maneuverability of the scope within a confined surgical environment.
Tako in view of Noble does not appear to explicitly disclose the single bend sized to provide clearance between a base section of the scope and the end of the shaft such that a site at which the scope is inserted into a patient is accessible by a disc preparation instrument.
In the same art of spinal surgical instruments, Weitzman discloses the single bend sized to provide clearance between a base section of the scope and the end of the shaft such that a site at which the scope is inserted into a patient is accessible by a disc preparation instrument (Par. 0020; whereas the distal bend is configured to enable access and maneuverability of the device at hidden anatomies, e.g. between or underneath vertebral discs, the proximal bend(s) is/are configured to enable an unhindered line of site of the device's distal tip. Fig. 1A-1C, 2A-2B, and Par. 0136; single proximal bend 150…Proximal bend 150 serves to ensure that part of elongated hollow member 130 distal to proximal bend 150 is offset of handle 120, thus ensuring a surgeon's unhindered line of site 101 to rotatable cutting tip 110, when holding handle 120. Advantageously, proximal bend 150 enables utilizing surgical tools…).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to further configure the single bend of the modified scope to include the offset configuration as taught by Weitzman. Doing so provides unobstructed access for surgical instruments and line of sight during visualization of the operative site (Par. 0017; a shaft with a single proximal bend may be of particular advantage when utilizing surgical microscopes and/or endoscopes that have short focal length, and that preferably would be located parallel to the shaft and close to the surgical site, this, since the handle is prevented from interfering with the illumination path and the line of sight of the endoscope), yielding predictable results in improved instrument clearance, allow simultaneous use of disc preparation tools, enhanced maneuverability, and visibility during a surgical procedure.
Regarding claim 19, Tako in view of Noble in further view of Weitman discloses the method of claim 18, and Tako further discloses obtain a user input (Fig. 29 and Par. 0059; Controller - A device which includes buttons and a direction controller) to adjust the size or position of at least one of the real-time image data, the navigation data, or the X-ray image data (Par. 103-104; the display 160 may be a touchscreen display, the display 160 may act as the user interface 130, along with other devices such as a keyboard, mouse, or other input device…The SNAP tool also provides an external system interface 150 that allows the system to interface to other external systems that provide navigation or other functions to the surgeon, so that the SNAP tool can generate images that are consistent with the outputs of the external systems, e.g., mirror the navigation or update the imaging based on the operation of the external system(s). The SNAP tool can then update its display images appropriately for providing an integrated view to the surgeon in high resolution, 3D images that interact with the graphical tools); and
adjusting the AR environment based on the user input to adjust the size or position of the at least one of the real-time image data, the navigation data, or the X-ray image data (Par. 0114-0117; the ARS computing device 112 is able to calculate and pinpoint the angle of view in addition to location and therefore offer a more unique and distinct experience for the surgeon in the operating room. In particular, the ARS computing device 112 processes the three inputs 202, 204, and 206, and integrates them to form an augmented reality view of the patient 104 with tracking and navigating capabilities. The ARS computing device 112 is further configured to output the augmented reality view of the patient to a device 208 for viewing. The device 208 may be, for example, a display monitor 108 or a HMD 160' (or both). It should be appreciated that, although only three (3) inputs are illustrated, the ARS computing device 112 may be configured to receive any suitable number of inputs from a variety of suitable input sources in order to provide the surgeon 102 more complete information that may be helpful for performing a particular surgery. For example, the ARS computing device 112 may be configured to receive additional live video feeds or additional tracking and sensor information).
Tako, Noble, and Weitzman are combined for the reasons set forth above with respect to claim 18.
Regarding claim 20, Tako in view of Noble in further view of Weitzman discloses the method of claim 18, and Tako further discloses wherein the navigation data for the surgical procedure comprises instructions and images of the surgical procedure (Par. 0075; The SNAP provides visual graphic guidance to the surgeon…Par. 0087-0088; The SNAP provides surgeons (neurosurgeons etc.), doctors, and their aids a unique virtual-reality guidance to determine the safest and most efficient pathway to remove tumors (cerebral etc.) and treat anomalies as vascular anomalies…The SNAP allows surgeons to analyze and plan a specific patient's case before surgery, and then take that plan into the operating room (OR) and use it in conjunction with the navigation system during surgery. The SNAP then presents the navigation data into the advanced inter-active, high quality 3D image, with multiple points of view…).
Tako, Noble, and Weitzman are combined for the reasons set forth above with respect to claim 18.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tako in view of Noble, in view of Weitzman, and in further view of Shluzas (US 8366747 B2).
Regarding claim 9, Tako in view of Noble in further view of Weitzman discloses the AR system of claim 1, but does not disclose wherein the scope is configured to be inserted into an expander in the patient’s body the scope having an end that is 0.25 inches in diameter or less.
Shluzas discloses wherein the scope (Column 8, lines 33-36; The viewing element may be an endoscope and camera that captures images to be displayed on the monitor M whereby the physician D is able to view the surgical site as the procedure is being performed) is configured to be inserted into an expander in the patient’s body (Column 20, lines 46-52; The distal portion of the access device 20 may be adjusted by inserting the expander apparatus 200 into the access device 20, expanding the distal portions 210, and contacting the inner wall of the skirt portion 24 to move the skirt portion 24 to the desired location. This step may be performed while the endoscope 500 is positioned within the access device 20), the scope having an end that is 0.25 inches in diameter or less (Column 18, 62-66; the endoscope 500 has an elongated rod portion 502 and a body portion 504…the rod portion 502 of endoscope 500 has a diameter of about 4 mm).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Tako, Noble, and Weitzman’s combined AR surgical system to incorporate the compact endoscope and expander-access arrangement of Shluzas. The motivation lies in the advantage of providing reduced diameter scope capable of fitting within an expandable desired surgical access point, minimizing tissue disruption, and preserving workspace, yielding predictable results of improved visualization and accessibility during surgical procedures.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JENNY NGAN TRAN whose telephone number is (571) 272-6888. The examiner can normally be reached Mon-Thurs 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner' s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JENNY N TRAN/Examiner, Art Unit 2615
/YANNA WU/Primary Examiner, Art Unit 2615
Prosecution Insights