DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 18-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 18 recites the limitations “a first indicator on the image” and “a second indicator on the image”. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-9, 12, & 18-21 are rejected under 35 U.S.C. 103 as being unpatentable over Altman et al (;hereinafter referred to as Altman) in view Gliner et al (US20200118319A1; hereinafter referred to as Gliner) and further in view of Wu et al (C. Wu, S. G. Narasimhan, and B. Jaramaz, Shape-from-Shading under Near Point Lighting and Partial views for Orthopedic Endoscopy, Oct. 2007;hereinafter referred to as Wu).
Regarding Claim 1, Altmann discloses a system (“The present invention relates generally to medical imaging, and particularly to methods and systems for visualization of invasive medical procedures.” [0002]), comprising:
(a) field generator assembly operable to generate an electromagnetic field around an anatomical region of a patient (“the position sensing system includes an electromagnetic tracking system, which includes one or more magnetic field generators positioned around the part of the body and a magnetic field sensor at a distal end of the medical tool.” [0008]);
(b) an instrument including a position sensor and a distal portion, the distal portion being sized for insertion into a patient, the position sensor being configured to provide a signal indicating a real-time position and orientation of the distal portion of the instrument within the patient in response to the electromagnetic field (“the position sensing system includes an electromagnetic tracking system, which includes one or more magnetic field generators positioned around the part of the body and a magnetic field sensor at a distal end of the medical tool.” [0008], “The field generators create magnetic fields, which are detected by a position sensor 32, such as a suitable coil, at a distal end 30 of a tool 28, such as a guidewire. When tool 28 is inserted into the nasal sinuses of patient 22, the signals output by the position sensor are processed by a system processor 40 in order to find location and orientation coordinates of the distal end of the probe.” [0024]);
(c) a display screen; and(d) a processor (“A processor is configured to register the position sensing system and a three-dimensional (3D) computerized tomography (CT) image of at least a part of the body within a common frame of reference, to identify a location and orientation of at least one virtual camera within the common frame of reference, and to render and display on a display screen a virtual endoscopic image, based on the 3D CT image, of the passage in the body from the specified location and orientation including an animated representation of the medical tool positioned in the virtual endoscopic image in accordance with the tracked coordinates.” [0013]), the processor being configured to:
(i) drive the display screen to display one or more images of the anatomical region of the patient, the one or more images including a three-dimensional virtual representation of the anatomical region of the patient (“A processor is configured to register the position sensing system and a three-dimensional (3D) computerized tomography (CT) image of at least a part of the body within a common frame of reference, to identify a location and orientation of at least one virtual camera within the common frame of reference, and to render and display on a display screen a virtual endoscopic image, based on the 3D CT image, of the passage in the body from the specified location and orientation including an animated representation of the medical tool positioned in the virtual endoscopic image in accordance with the tracked coordinates.” [0013]),
(ii) process signals from the position sensor to determine the real-time position and orientation of the distal portion of the instrument within the patient (“The field generators create magnetic fields, which are detected by a position sensor 32, such as a suitable coil, at a distal end 30 of a tool 28, such as a guidewire. When tool 28 is inserted into the nasal sinuses of patient 22, the signals output by the position sensor are processed by a system processor 40 in order to find location and orientation coordinates of the distal end of the probe.” [0024]),
(iii) provide a first indicator on at least one of the one or more images, the first indicator indicating the real-time position of the distal portion of the instrument within the patient (“A processor is configured to register the position sensing system and a three-dimensional (3D) computerized tomography (CT) image of at least a part of the body within a common frame of reference, to identify a location and orientation of at least one virtual camera within the common frame of reference, and to render and display on a display screen a virtual endoscopic image, based on the 3D CT image, of the passage in the body from the specified location and orientation including an animated representation of the medical tool positioned in the virtual endoscopic image in accordance with the tracked coordinates.” [0013]),
Altmann does not specifically disclose (iv) provide a second indicator on the three-dimensional virtual representation, the second indicator indicating the real-time orientation of the distal portion of the instrument within the patient, the second indicator including virtual illumination of an anatomical structure within the three-dimensional virtual representation, the virtually illuminated anatomical structure including a structure toward which the distal portion of the instrument is oriented, such that the virtual illumination simulates illumination of the anatomical structure via incoherent light, such that the virtual illumination is substantially concentrated within a central illuminated region with the illumination diffusing away from that central illuminated region, the virtual illumination indicating contours and other surface features of the virtually illuminated anatomical structure.
However, in a similar field of endeavor, Gliner teaches a processor configured to modify an image slice by filling a portion, of the image slice, that corresponds to an anatomical cavity with a representation of a wall of the anatomical cavity, and to display the modified image slice on the display [0007].
Gliner also teaches iv) provide a second indicator on the three-dimensional virtual representation, the second indicator indicating the real-time orientation of the distal portion of the instrument within the patient, the second indicator including virtual illumination of an anatomical structure within the three-dimensional virtual representation (“the processor computes representation 44, by rendering the wall of the cavity. First, the processor ascertains the form of the wall from the three-dimensional image from which image slice 31 was derived. The processor then uses any suitable rendering technique to render the wall. For example, the processor may illuminate the wall with a virtual light source, and render the wall in accordance with the view of a virtual camera positioned near the virtual light source. The processor then replaces the void in image slice 31 with the rendering. Typically, the wall is rendered in color, to help the physician differentiate between the anatomical cavity and the surrounding tissue.” [0034]),
the virtually illuminated anatomical structure including a structure toward which the distal portion of the instrument is oriented (“enhance the image further, by overlaying, on the image, an icon that represents the catheter, along with a marker that indicates the distance of the catheter from the cavity wall. To produce the marker, a processor may project a virtual ray from the tip of the catheter, and show the marker, in the image, at the point at which the virtual ray hits the cavity wall.” [0023]),
the virtual illumination indicating contours and other surface features of the virtually illuminated anatomical structure (“a processor may “illuminate” the cavity with a virtual light source, such that the walls of the cavity, beneath the displayed 2D slice, are “visible” to a virtual camera positioned near the virtual light source. The cavity walls may then be rendered, in color, with varying shades of brightness, corresponding to the view of the virtual camera.” [0022]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann as outlined above with (iv) provide a second indicator on the three-dimensional virtual representation, the second indicator indicating the real-time orientation of the distal portion of the instrument within the patient, the second indicator including virtual illumination of an anatomical structure within the three-dimensional virtual representation, the virtually illuminated anatomical structure including a structure toward which the distal portion of the instrument is oriented, the virtual illumination indicating contours and other surface features of the virtually illuminated anatomical structure as taught by Gliner, because it helps the physician differentiate between the anatomical cavity and the surrounding tissue [0034].
Altmann in view of Gliner does not specifically disclose that the virtual illumination simulates illumination of the anatomical structure via incoherent light, such that the virtual illumination is substantially concentrated within a central illuminated region with the illumination diffusing away from that central illuminated region.
However, in a similar field of endeavor, Wu teaches a novel technique to reconstruct the surface of the bone by applying shape-from-shading to a sequence of endoscopic images [Abstract]
Wu also teaches that the virtual illumination simulates illumination of the anatomical structure via incoherent light such that the virtual illumination is substantially concentrated within a central illuminated region with the illumination diffusing away from that central illuminated region, (“The endoscope uses one or more point light sources placed at its tip for illumination. Since the sources, camera and the scene are so close to each other, the endoscopic images are very different from the images we experience of common scenes under distant lighting such as from the sun or the sky.” [Pg.1 Col. 2], “By modeling the endoscope imaging system using perspective projection with near point light sources, shape from-shading can be achieved by minimizing the error between the image brightness E(x,y) and the reflectance map R. R is computed based on the model illustrated in Fig.5 and the derivation is given in our technical report[27].The important difference compared to the classical shape-from shading[9] is that we take into account the intensity fall-off from a divergent source, so R is not only a function of normal but also a function of distance z” [Pg.4 Col.1]),
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann in view of Gliner as outlined above with (the virtual illumination simulates illumination of the anatomical structure via incoherent light such that the virtual illumination is substantially concentrated within a central illuminated region with the illumination diffusing away from that central illuminated region as taught by Wu, because there is an immediate need for explicit computer reconstruction of shapes from endoscopic images [Pg. 1 Col. 2].
Regarding Claim 2, Altmann discloses that the anatomical region of the patient including a head of the patient, the distal portion of the instrument being sized for insertion into the head of the patient (“For the actual procedure, a set of one or more magnetic field generators 24 is fixed to or around the head of the patient, for example by incorporating the generators into a frame 26, which is clamped to the patient's head. The field generators create magnetic fields, which are detected by a position sensor 32, such as a suitable coil, at a distal end 30 of a tool 28, such as a guidewire. When tool 28 is inserted into the nasal sinuses of patient 22, the signals output by the position sensor are processed by a system processor 40 in order to find location and orientation coordinates of the distal end of the probe.” [0024]).
Regarding Claim 3, Altmann discloses that the position sensor comprising one or more coils (“For the actual procedure, a set of one or more magnetic field generators 24 is fixed to or around the head of the patient, for example by incorporating the generators into a frame 26, which is clamped to the patient's head. The field generators create magnetic fields, which are detected by a position sensor 32, such as a suitable coil, at a distal end 30 of a tool 28, such as a guidewire. When tool 28 is inserted into the nasal sinuses of patient 22, the signals output by the position sensor are processed by a system processor 40 in order to find location and orientation coordinates of the distal end of the probe.” [0024]).
Regarding Claim 4, Altmann discloses that the distal portion of the instrument including an end effector, the end effector being operable to affect tissue (“These virtual endoscope views may be used, for example, in visualizing the location and orientation of a guidewire relative to the anatomy, as well as other tools, such as a suction tool or a shaving tool (debrider). “ [0019]).
Regarding Claim 5, Altmann discloses that the end effector comprising one or more features selected from the group consisting of: an expandable dilator, one or more electrodes, a microbur, a shaver blade, a drill bit, a microdebrider, an illumination element, an irrigation opening, a suction opening, and a channel configured to slidably receive another instrument (“These virtual endoscope views may be used, for example, in visualizing the location and orientation of a guidewire relative to the anatomy, as well as other tools, such as a suction tool or a shaving tool (debrider). “ [0019]).
Regarding Claim 6, Altmann discloses that the processor being further configured to provide the first indicator on the three-dimensional virtual representation of the anatomical region of the patient, (“A processor is configured to register the position sensing system and a three-dimensional (3D) computerized tomography (CT) image of at least a part of the body within a common frame of reference, to identify a location and orientation of at least one virtual camera within the common frame of reference, and to render and display on a display screen a virtual endoscopic image, based on the 3D CT image, of the passage in the body from the specified location and orientation including an animated representation of the medical tool positioned in the virtual endoscopic image in accordance with the tracked coordinates.” [0013]).
Regarding Claim 7, Altmann discloses that the processor being further configured to: (i) modify a viewing perspective of the three-dimensional virtual representation of the anatomical region of the patient in response to signals from the position sensor indicating repositioning of the distal portion of the instrument within the patient, (ii) provide the first indicator on the three-dimensional virtual representation of the anatomical region of the patient with the modified viewing perspective, (“rendering and displaying the virtual endoscopic image includes changing a viewing characteristic of the virtual endoscopic image responsively to a movement of the medical tool within the passage as indicated by the tracked coordinates. In a disclosed embodiment, changing the viewing characteristic includes changing at least one of the location and the orientation of the at least one virtual camera so as to keep the medical tool in a field of view of the at least one virtual camera as the medical tool moves through the passage” [0011]).
Altmann does not specifically teach (iii) provide the second indicator on the three-dimensional virtual representation of the anatomical region of the patient with the modified viewing perspective.
However, in a similar field of endeavor, Gliner teaches (iii) provide the second indicator on the three-dimensional virtual representation of the anatomical region of the patient with the modified viewing perspective (“the processor computes representation 44, by rendering the wall of the cavity. First, the processor ascertains the form of the wall from the three-dimensional image from which image slice 31 was derived. The processor then uses any suitable rendering technique to render the wall. For example, the processor may illuminate the wall with a virtual light source, and render the wall in accordance with the view of a virtual camera positioned near the virtual light source. The processor then replaces the void in image slice 31 with the rendering. Typically, the wall is rendered in color, to help the physician differentiate between the anatomical cavity and the surrounding tissue.” [0034]),
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann as outlined above with (iii) provide the second indicator on the three-dimensional virtual representation of the anatomical region of the patient with the modified viewing perspective as taught by Gliner, because it helps the physician differentiate between the anatomical cavity and the surrounding tissue [0034].
Regarding Claim 8, Altmann discloses all limitations noted above except that the one or more images including a cross-sectional image of the anatomical region of the patient, the processor being further configured to: (i) provide the first indicator on the cross-sectional image of the anatomical region of the patient, and (ii) provide the second indicator on the cross-sectional image of the anatomical region of the patient.
However, in a similar field of endeavor, Gliner teaches the one or more images including a cross-sectional image of the anatomical region of the patient, the processor being further configured to: (i) provide the first indicator on the cross-sectional image of the anatomical region of the patient, and (ii) provide the second indicator on the cross-sectional image of the anatomical region of the patient (“It is noted that the term “image slice,” as used in the present application (including the claims), refers to any two-dimensional image acquired by imaging a particular cross-section of a three-dimensional object, or by taking a particular cross-section of a three-dimensional image of the object.“ [0027], “the processor further overlays, on modified image slice 32, an icon 40 that represents intrabody tool 28 (in particular, the distal end thereof) on a portion of modified image slice 32 that corresponds to the location of the intrabody tool within the anatomical cavity. Typically, the processor also overlays a marker 42 on a portion of representation 44 that corresponds to a location at which the intrabody tool would meet the wall, were the intrabody tool to continue moving toward the wall in the direction in which the intrabody tool is pointing.” [0037])
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann as outlined above with the one or more images including a cross-sectional image of the anatomical region of the patient, the processor being further configured to: (i) provide the first indicator on the cross-sectional image of the anatomical region of the patient, and (ii) provide the second indicator on the cross-sectional image of the anatomical region of the patient as taught by Gliner, because it helps the physician differentiate between the anatomical cavity and the surrounding tissue [0034].
Regarding Claim 9, Altmann discloses the processor being further configured to: (i) drive the display screen to simultaneously display two or more images of the anatomical region of the patient, (ii) provide the first indicator simultaneously on the two or more images the anatomical region of the patient (“ Multiple virtual views of the tool can be generated simultaneously and can be displayed on a viewing screen, instead of or in addition to conventional CT slice views. Further additionally or alternatively, the imaging system can generate virtual endoscopic views as would be seen from the distal tip of the tool itself”).
Altmann does not specifically teach (iii) provide the second indicator simultaneously on the two or more images the anatomical region of the patient.
However, in a similar field of endeavor, Gliner teaches (iii) provide the second indicator simultaneously on the two or more images the anatomical region of the patient (“the processor computes representation 44, by rendering the wall of the cavity. First, the processor ascertains the form of the wall from the three-dimensional image from which image slice 31 was derived. The processor then uses any suitable rendering technique to render the wall. For example, the processor may illuminate the wall with a virtual light source, and render the wall in accordance with the view of a virtual camera positioned near the virtual light source. The processor then replaces the void in image slice 31 with the rendering. Typically, the wall is rendered in color, to help the physician differentiate between the anatomical cavity and the surrounding tissue.” [0034]),
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann as outlined above with (iii) provide the second indicator simultaneously on the two or more images the anatomical region of the patient as taught by Gliner, because it helps the physician differentiate between the anatomical cavity and the surrounding tissue [0034].
Regarding Claim 12, Altmann discloses that the distal portion of the instrument lacking a source of actual light (“Embodiments of the present invention that are described herein address this problem by generating virtual endoscopic views of the procedure, as would be seen by an actual endoscope positioned at a selected location within the nasal passages, or even by multiple endoscopes at different, respective locations.” [0019]).
Regarding Claim 18, Altmann discloses a method comprising (“The present invention relates generally to medical imaging, and particularly to methods and systems for visualization of invasive medical procedures.” [0002]):
(a) driving a display screen to display one or more images of the anatomical region of the patient, the one or more images including a three-dimensional virtual representation of the anatomical region of the patient (“A processor is configured to register the position sensing system and a three-dimensional (3D) computerized tomography (CT) image of at least a part of the body within a common frame of reference, to identify a location and orientation of at least one virtual camera within the common frame of reference, and to render and display on a display screen a virtual endoscopic image, based on the 3D CT image, of the passage in the body from the specified location and orientation including an animated representation of the medical tool positioned in the virtual endoscopic image in accordance with the tracked coordinates.” [0013]),
(b) processing signals from a position sensor of an instrument to determine the real-time position and orientation of the distal portion of the instrument within the patient (“The field generators create magnetic fields, which are detected by a position sensor 32, such as a suitable coil, at a distal end 30 of a tool 28, such as a guidewire. When tool 28 is inserted into the nasal sinuses of patient 22, the signals output by the position sensor are processed by a system processor 40 in order to find location and orientation coordinates of the distal end of the probe.” [0024]),
(c) providing a first indicator on the image, the first indicator indicating the real-time position of the distal portion of the instrument within the patient (“A processor is configured to register the position sensing system and a three-dimensional (3D) computerized tomography (CT) image of at least a part of the body within a common frame of reference, to identify a location and orientation of at least one virtual camera within the common frame of reference, and to render and display on a display screen a virtual endoscopic image, based on the 3D CT image, of the passage in the body from the specified location and orientation including an animated representation of the medical tool positioned in the virtual endoscopic image in accordance with the tracked coordinates.” [0013]),
Altmann does not specifically disclose (d) providing a second indicator on the image, the second indicator indicating the real-time orientation of the distal portion of the instrument within the patient, by virtually illuminating on the three-dimensional virtual representation an anatomical structure toward which the distal portion of the instrument is oriented, the virtual illumination being in the form of structured illumination providing an illumination pattern that indicates contours and other surface features of the virtually illuminated anatomical structure.
However, in a similar field of endeavor, Gliner teaches a processor configured to modify an image slice by filling a portion, of the image slice, that corresponds to an anatomical cavity with a representation of a wall of the anatomical cavity, and to display the modified image slice on the display [0007].
Gliner also teaches (d) providing a second indicator on the image, the second indicator indicating the real-time orientation of the distal portion of the instrument within the patient(“the processor computes representation 44, by rendering the wall of the cavity. First, the processor ascertains the form of the wall from the three-dimensional image from which image slice 31 was derived. The processor then uses any suitable rendering technique to render the wall. For example, the processor may illuminate the wall with a virtual light source, and render the wall in accordance with the view of a virtual camera positioned near the virtual light source. The processor then replaces the void in image slice 31 with the rendering. Typically, the wall is rendered in color, to help the physician differentiate between the anatomical cavity and the surrounding tissue.” [0034]),
by virtually illuminating on the three-dimensional virtual representation an anatomical structure toward which the distal portion of the instrument is oriented (“enhance the image further, by overlaying, on the image, an icon that represents the catheter, along with a marker that indicates the distance of the catheter from the cavity wall. To produce the marker, a processor may project a virtual ray from the tip of the catheter, and show the marker, in the image, at the point at which the virtual ray hits the cavity wall.” [0023]),
the virtual illumination being in the form of structured illumination providing an illumination pattern that indicates contours and other surface features of the virtually illuminated anatomical structure (“a processor may “illuminate” the cavity with a virtual light source, such that the walls of the cavity, beneath the displayed 2D slice, are “visible” to a virtual camera positioned near the virtual light source. The cavity walls may then be rendered, in color, with varying shades of brightness, corresponding to the view of the virtual camera.” [0022]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann as outlined above with (d) providing a second indicator on the image, the second indicator indicating the real-time orientation of the distal portion of the instrument within the patient, by virtually illuminating on the three-dimensional virtual representation an anatomical structure toward which the distal portion of the instrument is oriented, the virtual illumination being in the form of structured illumination providing an illumination pattern that indicates contours and other surface features of the virtually illuminated anatomical structure as taught by Gliner, because it helps the physician differentiate between the anatomical cavity and the surrounding tissue [0034].
Regarding Claim 19, Altmann discloses that the first indicator being provided on the three-dimensional virtual representation of the anatomical region of the patient, (“A processor is configured to register the position sensing system and a three-dimensional (3D) computerized tomography (CT) image of at least a part of the body within a common frame of reference, to identify a location and orientation of at least one virtual camera within the common frame of reference, and to render and display on a display screen a virtual endoscopic image, based on the 3D CT image, of the passage in the body from the specified location and orientation including an animated representation of the medical tool positioned in the virtual endoscopic image in accordance with the tracked coordinates.” [0013]).
the method further comprising:(a) modifying a viewing perspective of the three-dimensional virtual representation of the anatomical region of the patient in response to signals from the position sensor indicating repositioning of the distal portion of the instrument within the patient; (b) providing the first indicator on the three-dimensional virtual representation of the anatomical region of the patient with the modified viewing perspective, (“rendering and displaying the virtual endoscopic image includes changing a viewing characteristic of the virtual endoscopic image responsively to a movement of the medical tool within the passage as indicated by the tracked coordinates. In a disclosed embodiment, changing the viewing characteristic includes changing at least one of the location and the orientation of the at least one virtual camera so as to keep the medical tool in a field of view of the at least one virtual camera as the medical tool moves through the passage” [0011]).
Altmann does not specifically teach (c) providing the second indicator on the three-dimensional virtual representation of the anatomical region of the patient with the modified viewing perspective.
However, in a similar field of endeavor, Gliner teaches (c) providing the second indicator on the three-dimensional virtual representation of the anatomical region of the patient with the modified viewing perspective (“the processor computes representation 44, by rendering the wall of the cavity. First, the processor ascertains the form of the wall from the three-dimensional image from which image slice 31 was derived. The processor then uses any suitable rendering technique to render the wall. For example, the processor may illuminate the wall with a virtual light source, and render the wall in accordance with the view of a virtual camera positioned near the virtual light source. The processor then replaces the void in image slice 31 with the rendering. Typically, the wall is rendered in color, to help the physician differentiate between the anatomical cavity and the surrounding tissue.” [0034]),
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann as outlined above with (c) providing the second indicator on the three-dimensional virtual representation of the anatomical region of the patient with the modified viewing perspective as taught by Gliner, because it helps the physician differentiate between the anatomical cavity and the surrounding tissue [0034].
Regarding Claim 20, Altmann discloses the method further comprising:(a) modifying an image of the anatomical region of the patient in response to signals from the position sensor indicating repositioning of the distal portion of the instrument within the patient; (b) providing the first indicator on the of the anatomical region of the patient with the modified position (“rendering and displaying the virtual endoscopic image includes changing a viewing characteristic of the virtual endoscopic image responsively to a movement of the medical tool within the passage as indicated by the tracked coordinates. In a disclosed embodiment, changing the viewing characteristic includes changing at least one of the location and the orientation of the at least one virtual camera so as to keep the medical tool in a field of view of the at least one virtual camera as the medical tool moves through the passage” [0011]).
Altmann does not specifically disclose the displayed one or more images further comprising a cross-sectional image of the anatomical region of the patient, the first indicator being provided on the cross-sectional image of the anatomical region of the patient, the second indicator being further provided on the cross-sectional image of the anatomical region of the patient, the method further comprising:(a) modifying a cross-sectional plane position of the cross-sectional image of the anatomical region of the patient in response to signals from the position sensor indicating repositioning of the distal portion of the instrument within the patient;(b) providing the first indicator on the cross-sectional image of the anatomical region of the patient with the modified cross-sectional plane position; and(c) providing the second indicator on the cross-sectional image of the anatomical region of the patient with the modified cross-sectional plane position.
However, in a similar field of endeavor, Gliner teaches the displayed one or more images further comprising a cross-sectional image of the anatomical region of the patient, the first indicator being provided on the cross-sectional image of the anatomical region of the patient, the second indicator being further provided on the cross-sectional image of the anatomical region of the patient, and (c) providing the second indicator on the cross-sectional image of the anatomical region of the patient with the modified cross-sectional plane position (“It is noted that the term “image slice,” as used in the present application (including the claims), refers to any two-dimensional image acquired by imaging a particular cross-section of a three-dimensional object, or by taking a particular cross-section of a three-dimensional image of the object.“ [0027], “the processor further overlays, on modified image slice 32, an icon 40 that represents intrabody tool 28 (in particular, the distal end thereof) on a portion of modified image slice 32 that corresponds to the location of the intrabody tool within the anatomical cavity. Typically, the processor also overlays a marker 42 on a portion of representation 44 that corresponds to a location at which the intrabody tool would meet the wall, were the intrabody tool to continue moving toward the wall in the direction in which the intrabody tool is pointing.” [0037])
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann as outlined above with the displayed one or more images further comprising a cross-sectional image of the anatomical region of the patient, the first indicator being provided on the cross-sectional image of the anatomical region of the patient, the second indicator being further provided on the cross-sectional image of the anatomical region of the patient, and (c) providing the second indicator on the cross-sectional image of the anatomical region of the patient with the modified cross-sectional plane position as taught by Gliner, because it helps the physician differentiate between the anatomical cavity and the surrounding tissue [0034].
Regarding Claim 21, Altmann discloses a system (“The present invention relates generally to medical imaging, and particularly to methods and systems for visualization of invasive medical procedures.” [0002]), comprising:
(a) field generator assembly operable to generate an electromagnetic field around an anatomical region of a patient (“the position sensing system includes an electromagnetic tracking system, which includes one or more magnetic field generators positioned around the part of the body and a magnetic field sensor at a distal end of the medical tool.” [0008]);
(b) an instrument including a position sensor and a distal portion, the distal portion being sized for insertion into a patient, the position sensor being configured to provide a signal indicating a real-time position and orientation of the distal portion of the instrument within the patient in response to the electromagnetic field (“the position sensing system includes an electromagnetic tracking system, which includes one or more magnetic field generators positioned around the part of the body and a magnetic field sensor at a distal end of the medical tool.” [0008], “The field generators create magnetic fields, which are detected by a position sensor 32, such as a suitable coil, at a distal end 30 of a tool 28, such as a guidewire. When tool 28 is inserted into the nasal sinuses of patient 22, the signals output by the position sensor are processed by a system processor 40 in order to find location and orientation coordinates of the distal end of the probe.” [0024]);
(c) a display screen; and(d) a processor (“A processor is configured to register the position sensing system and a three-dimensional (3D) computerized tomography (CT) image of at least a part of the body within a common frame of reference, to identify a location and orientation of at least one virtual camera within the common frame of reference, and to render and display on a display screen a virtual endoscopic image, based on the 3D CT image, of the passage in the body from the specified location and orientation including an animated representation of the medical tool positioned in the virtual endoscopic image in accordance with the tracked coordinates.” [0013]), the processor being configured to:
(i) drive the display screen to display one or more images of the anatomical region of the patient, the one or more images including a three-dimensional virtual representation of the anatomical region of the patient (“A processor is configured to register the position sensing system and a three-dimensional (3D) computerized tomography (CT) image of at least a part of the body within a common frame of reference, to identify a location and orientation of at least one virtual camera within the common frame of reference, and to render and display on a display screen a virtual endoscopic image, based on the 3D CT image, of the passage in the body from the specified location and orientation including an animated representation of the medical tool positioned in the virtual endoscopic image in accordance with the tracked coordinates.” [0013]),
(ii) process signals from the position sensor to determine the real-time position and orientation of the distal portion of the instrument within the patient (“The field generators create magnetic fields, which are detected by a position sensor 32, such as a suitable coil, at a distal end 30 of a tool 28, such as a guidewire. When tool 28 is inserted into the nasal sinuses of patient 22, the signals output by the position sensor are processed by a system processor 40 in order to find location and orientation coordinates of the distal end of the probe.” [0024]),
(iii) provide a first indicator on at least one of the one or more images, the first indicator indicating the real-time position of the distal portion of the instrument within the patient (“A processor is configured to register the position sensing system and a three-dimensional (3D) computerized tomography (CT) image of at least a part of the body within a common frame of reference, to identify a location and orientation of at least one virtual camera within the common frame of reference, and to render and display on a display screen a virtual endoscopic image, based on the 3D CT image, of the passage in the body from the specified location and orientation including an animated representation of the medical tool positioned in the virtual endoscopic image in accordance with the tracked coordinates.” [0013]),
Altmann does not specifically disclose (iv) provide a second indicator on the three-dimensional virtual representation, the second indicator indicating the real-time orientation of the distal portion of the instrument within the patient, the second indicator including virtual illumination of an anatomical structure within the three-dimensional virtual representation, the virtually illuminated anatomical structure including a structure toward which the distal portion of the instrument is oriented, the virtual illumination including one or more of: (A) a simulation of incoherent light, such that the virtual illumination is substantially concentrated within a central illuminated region with the illumination diffusing away from that central illuminated region, the virtual illumination indicating contours and other surface features of the virtually illuminated anatomical structure, or (B) structured illumination providing an illumination pattern that indicates contours and other surface features of the virtually illuminated anatomical structure.
However, in a similar field of endeavor, Gliner teaches a processor configured to modify an image slice by filling a portion, of the image slice, that corresponds to an anatomical cavity with a representation of a wall of the anatomical cavity, and to display the modified image slice on the display [0007].
Gliner also teaches iv) provide a second indicator on the three-dimensional virtual representation, the second indicator indicating the real-time orientation of the distal portion of the instrument within the patient, the second indicator including virtual illumination of an anatomical structure within the three-dimensional virtual representation (“the processor computes representation 44, by rendering the wall of the cavity. First, the processor ascertains the form of the wall from the three-dimensional image from which image slice 31 was derived. The processor then uses any suitable rendering technique to render the wall. For example, the processor may illuminate the wall with a virtual light source, and render the wall in accordance with the view of a virtual camera positioned near the virtual light source. The processor then replaces the void in image slice 31 with the rendering. Typically, the wall is rendered in color, to help the physician differentiate between the anatomical cavity and the surrounding tissue.” [0034]),
the virtually illuminated anatomical structure including a structure toward which the distal portion of the instrument is oriented (“enhance the image further, by overlaying, on the image, an icon that represents the catheter, along with a marker that indicates the distance of the catheter from the cavity wall. To produce the marker, a processor may project a virtual ray from the tip of the catheter, and show the marker, in the image, at the point at which the virtual ray hits the cavity wall.” [0023]),
the virtual illumination including one or more of: (A) a simulation of incoherent light, such that the virtual illumination is substantially concentrated within a central illuminated region with the illumination diffusing away from that central illuminated region, the virtual illumination indicating contours and other surface features of the virtually illuminated anatomical structure, or (B) structured illumination providing an illumination pattern that indicates contours and other surface features of the virtually illuminated anatomical structure (“a processor may “illuminate” the cavity with a virtual light source, such that the walls of the cavity, beneath the displayed 2D slice, are “visible” to a virtual camera positioned near the virtual light source. The cavity walls may then be rendered, in color, with varying shades of brightness, corresponding to the view of the virtual camera.” [0022]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann as outlined above with (iv) provide a second indicator on the three-dimensional virtual representation, the second indicator indicating the real-time orientation of the distal portion of the instrument within the patient, the second indicator including virtual illumination of an anatomical structure within the three-dimensional virtual representation, the virtually illuminated anatomical structure including a structure toward which the distal portion of the instrument is oriented, the virtual illumination including one or more of: (A) a simulation of incoherent light, such that the virtual illumination is substantially concentrated within a central illuminated region with the illumination diffusing away from that central illuminated region, the virtual illumination indicating contours and other surface features of the virtually illuminated anatomical structure, or (B) structured illumination providing an illumination pattern that indicates contours and other surface features of the virtually illuminated anatomical structure as taught by Gliner, because it helps the physician differentiate between the anatomical cavity and the surrounding tissue [0034].
Claims 13-16 are rejected under 35 U.S.C. 103 as being unpatentable over Altmann in view of Gliner further in view of Wu as applied to Claim 1 above, and further in view of Larkin et al (US20200331147A1; hereinafter referred to as Larkin).
Regarding Claim 13, Altmann discloses the processor being further configured to determining that the distal portion of the instrument is outside a certain proximity of an identified anatomical structure (“Operator 34 positions a probe in proximity to the bony sections and thus measures positions on the surface of the head overlying the bony sections. Processor 40 computes the correspondence between these measured positions and the voxel subset in the CT image and thus registers the magnetic tracking system with the CT image.” [0031])
Altmann in view of Gliner further in view of Wu does not specifically teach (i) persistently illuminate a third indicator on at least one images of the one or more images with a first illumination characteristic in response to determining that the distal portion of the instrument is outside a certain proximity of an identified anatomical structure, and (ii) persistently illuminate the third indicator on at least one images of the one or more images with a second illumination characteristic in response to determining that the distal portion of the instrument is within a certain proximity of the identified anatomical structure.
However, in a similar field of endeavor, Larkin teaches a tool position and identification indicator displayed in a boundary area of a computer display screen [0002].
Larkin also teaches (i) persistently illuminate a third indicator on at least one images of the one or more images with a first illumination characteristic in response to determining that the distal portion of the instrument is outside a certain proximity of an identified anatomical structure, and (ii) persistently illuminate the third indicator on at least one images of the one or more images with a second illumination characteristic in response to determining that the distal portion of the instrument is within a certain proximity of the identified anatomical structure (“The distance that the out-of-view tool 138 is away from the viewing area 300, may be indicated in a number of ways, such as by the size, color, brightness/intensity, blinking frequency, or oscillating frequency of its symbol. Alternatively, the distance may be simply indicated by displaying a distance number (such as the distance in centimeters) over the symbol. For example, when the tool is in-view, such as the tool 139, then its symbol may be a maximum size, such as the symbol 420 of the in-view tool 139. When the tool is out-of-view, however, such as the tool 138, then the size of its symbol may indicate the distance that the out-of-view tool is away from the viewing area 300 so that it gets larger as the tool moves closer to entering the viewing area 300. Alternatively, the color of the symbol may indicate distance using a color spectrum, or the brightness/intensity of the symbol or the blinking frequency of the symbol may indicate distance by increasing as the tool moves closer to entering the viewing area 300, or an oscillation frequency of the symbol about its nominal position may reduce as the tool is brought closer to being in the viewing area 300.” [0043]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann in view of Gliner further in view of Wu as outlined above with (i) persistently illuminate a third indicator on at least one images of the one or more images with a first illumination characteristic in response to determining that the distal portion of the instrument is outside a certain proximity of an identified anatomical structure, and (ii) persistently illuminate the third indicator on at least one images of the one or more images with a second illumination characteristic in response to determining that the distal portion of the instrument is within a certain proximity of the identified anatomical structure as taught by Larkin, because it makes the the surgical operation less tedious and time consuming for the surgeon [0007].
Regarding Claim 14, Altmann in view of Gliner further in view of Wu discloses all limitations noted above except that the first illumination characteristic comprising a first color, the second illumination characteristic comprising a second color.
However, in a similar field of endeavor, Larkin teaches the first illumination characteristic comprising a first color, the second illumination characteristic comprising a second color (“The distance that the out-of-view tool 138 is away from the viewing area 300, may be indicated in a number of ways, such as by the size, color, brightness/intensity, blinking frequency, or oscillating frequency of its symbol. Alternatively, the distance may be simply indicated by displaying a distance number (such as the distance in centimeters) over the symbol. For example, when the tool is in-view, such as the tool 139, then its symbol may be a maximum size, such as the symbol 420 of the in-view tool 139. When the tool is out-of-view, however, such as the tool 138, then the size of its symbol may indicate the distance that the out-of-view tool is away from the viewing area 300 so that it gets larger as the tool moves closer to entering the viewing area 300. Alternatively, the color of the symbol may indicate distance using a color spectrum, or the brightness/intensity of the symbol or the blinking frequency of the symbol may indicate distance by increasing as the tool moves closer to entering the viewing area 300, or an oscillation frequency of the symbol about its nominal position may reduce as the tool is brought closer to being in the viewing area 300.” [0043]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann in view of Gliner further in view of Wu as outlined above with the first illumination characteristic comprising a first color, the second illumination characteristic comprising a second color as taught by Larkin, because it makes the surgical operation less tedious and time consuming for the surgeon [0007].
Regarding Claim 15, Altmann in view of Gliner further in view of Wu discloses all limitations noted above except that the first illumination characteristic comprising a first flash frequency, the second illumination characteristic comprising a second flash frequency.
However, in a similar field of endeavor, Larkin teaches the first illumination characteristic comprising a first flash frequency, the second illumination characteristic comprising a second flash frequency (“The distance that the out-of-view tool 138 is away from the viewing area 300, may be indicated in a number of ways, such as by the size, color, brightness/intensity, blinking frequency, or oscillating frequency of its symbol. Alternatively, the distance may be simply indicated by displaying a distance number (such as the distance in centimeters) over the symbol. For example, when the tool is in-view, such as the tool 139, then its symbol may be a maximum size, such as the symbol 420 of the in-view tool 139. When the tool is out-of-view, however, such as the tool 138, then the size of its symbol may indicate the distance that the out-of-view tool is away from the viewing area 300 so that it gets larger as the tool moves closer to entering the viewing area 300. Alternatively, the color of the symbol may indicate distance using a color spectrum, or the brightness/intensity of the symbol or the blinking frequency of the symbol may indicate distance by increasing as the tool moves closer to entering the viewing area 300, or an oscillation frequency of the symbol about its nominal position may reduce as the tool is brought closer to being in the viewing area 300.” [0043]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann in view of Gliner further in view of Wu as outlined above with the first illumination characteristic comprising a first flash frequency, the second illumination characteristic comprising a second flash frequency.as taught by Larkin, because it makes the surgical operation less tedious and time consuming for the surgeon [0007].
Regarding Claim 16, Altmann in view of Gliner further in view of Wu discloses all limitations noted above except that the first illumination characteristic comprising a first illumination intensity, the second illumination characteristic comprising a second illumination intensity.
However, in a similar field of endeavor, Larkin teaches the first illumination characteristic comprising a first illumination intensity, the second illumination characteristic comprising a second illumination intensity (“The distance that the out-of-view tool 138 is away from the viewing area 300, may be indicated in a number of ways, such as by the size, color, brightness/intensity, blinking frequency, or oscillating frequency of its symbol. Alternatively, the distance may be simply indicated by displaying a distance number (such as the distance in centimeters) over the symbol. For example, when the tool is in-view, such as the tool 139, then its symbol may be a maximum size, such as the symbol 420 of the in-view tool 139. When the tool is out-of-view, however, such as the tool 138, then the size of its symbol may indicate the distance that the out-of-view tool is away from the viewing area 300 so that it gets larger as the tool moves closer to entering the viewing area 300. Alternatively, the color of the symbol may indicate distance using a color spectrum, or the brightness/intensity of the symbol or the blinking frequency of the symbol may indicate distance by increasing as the tool moves closer to entering the viewing area 300, or an oscillation frequency of the symbol about its nominal position may reduce as the tool is brought closer to being in the viewing area 300.” [0043]).
It would have been obvious to an ordinary skilled person in the art before the effective filing
date of the claimed invention to modify the system of Altmann in view of Gliner further in view of Wu as outlined above with the first illumination characteristic comprising a first illumination intensity, the second illumination characteristic comprising a second illumination intensity as taught by Larkin, because it makes the surgical operation less tedious and time consuming for the surgeon [0007].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure (US20200054399A1, US20230363831A1).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN MALDONADO whose telephone number is 703-756-1421. The examiner can normally be reached 8:00 am-4:00 pm PST M-Th Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Koharski can be reached on (571) 272-7230. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Steven Maldonado/
Patent Examiner, Art Unit 3797
/CHRISTOPHER KOHARSKI/Supervisory Patent Examiner, Art Unit 3797
Prosecution Insights