DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 14 January 2026 have been fully considered but they are not persuasive.
Applicant argues that combination of prior arts would require one or more of the references to operate contrary to its function, because Graumann and Hananel use separate markers for optical and the X-ray device, as Hananel’s markers are on a calibration plate attached to the X-ray device and distinct radiopaque and optical markers are placed in distinct locations as in Fig. 2B (pages 10-11). By contrary, Tolkowsky teaches radiopaque markers placed on the patient that are visible to both x-ray and optical cameras, and combination of separate markers of Graumann and Hananel and multimodal markers of Tolkowsky would make one or another’s failure to operate according to its established function (pages 11-12).
However, the examiner respectfully disagrees.
In response to applicant's argument that combination of prior arts result in failure of the one or another’s established functions, the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981).
In previous office action, the examiner clearly established that combination is not combining separate markers with multimodal marker, rather stated that “replacement” of separate markers with “multimodal” markers of Tolkowsky, as it would have been obvious to make replacement(substitution) to result in clear motivation of “simplified types of markers into single markers, visible to both camera and X-ray images” ([0526]).
The examiner has incorporated previous office action’s motivation state that was used as follows:
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the optical markers and radiopaque markers as taught by Graumann and Hananel to incorporate teaching of Tolkowsky, since radiopaque markers visible to both camera and X-ray images was well known in the art as taught by Tolkowsky. One of ordinary skill in the art could have combined the elements as claimed by Graumann and Hananel with no change in their respective functions, replacing its separate optical and radiopaque markers into radiopaque markers visible to both X-ray and optical imaging system and method, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to simplify the types of markers into single radiopaque markers, which are visible to both camera and X-ray images ([0526]), and there was reasonable expectation of success.
Thus, the examiner do not agree with applicant’ argument that combination would result in failure of established functions. Rather, replacing two markers with single marker would have provide benefit of simplifying the markers that can be detected by multimodal imaging devices as stated in previous office action.
With respect to amendment, the applicant argues that cited combination of relying on Tolkowsky would use markers to identify changes in a pose of 2D imaging device or a position of the patient, providing a reference for general orientation to the surgeon throughout a procedure as cited in paragraphs [0508]-[0526] of Tolkowsky, and does not disclose “determine, based on a spatial positions of the subset of fiducial markers relative to the camera and the spatial positions of the subset of fiducial markers relative to the x-ray system (pages 11-12).
However, the examiner respectfully disagrees.
The argument is toward amended limitation, and the examiner submits that the examiner cited Hananel for teaching the limitations as follows:
determine, based on at least the spatial positions of the subset of fiducial markers relative to the camera and the spatial positions of the subset of fiducial markers relative to the x-ray system, a spatial position for the instrument relative to the x-ray system ([0014] and [0054] and [0134] position of the tool with respect to X-ray device based on determined position of calibration plate including markers with respect to X-ray device, position of camera with respect to calibration plate which include markers, and camera with respect to X-ray)
In addition, Tolkowsky’s cited paragraphs disclose determining changes in a pose of 2D imaging device and a position of the patient, if the position of subject relative to the 2D imaging device changes, then the images would be a visible change in the appearance of the markers relative to the anatomy within the image ([0528]-[0533]). Tolkowsky further teaches “identifying changes in relative pose of the imaging device and of the subject,” in paragraph [0537], thus reads on claimed “determine spatial positions of the subset of fiducial markers relative to the camera and/or x-ray system.”
With respect to dependent claims, the examiner submits that same reasoning applies and the rejection is maintained.
Thus, the examiner finds the rejection to be proper and maintained.
For amended limitation, the examiner provided modified rejection as set forth below under “Claim Rejections - 35 USC § 103.”
Applicant’s arguments with respect to amended claim 2 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant’s arguments, see pages 12-13, filed 14 January 2026, with respect to amended claim 19 have been fully considered and are persuasive. The 103 rejection of 14 July 2025 has been withdrawn.
Reasons for Allowance
Claims 19-21, 32, and 34-35 are allowed.
The following is an examiner’s statement of reasons for allowance:
The following prior art previously made of record is considered pertinent to the reasons of allowance:
Graumann (US 2014/0051994) teaches a system for surgical navigation, comprising:
an instrument configured for a medical procedure on a patient (instrument 4, to be introduced on the body surface of the patient [0030] Figures 2-3);
a camera attached to the instrument, wherein the instrument has a spatial position relative to the camera (the navigation device 28, a camera fastened directly on the instrument, in the case of an optical mark [0037])
an x-ray system configured to acquire x-ray images of the patient during the medical procedure (x-ray C-arm [0039]);
a plurality of fiducial markers positioned on the surface of the patient during the medical procedure, said fiducial markers being detectable by the camera and the x-ray system (marker film attached on the patient with navigation marker viewable by camera and site markers viewable by X-ray [0040]); and
a computer configured to receive an optical image acquired by the camera (camera image [0014]), receive an x-ray image acquired by the x-ray system (X-ray image [0016])
Tolkowsky (US 2021/0386480 ) teaches radiopaque markers comprising a radio-opaque material arranged as at least one of a line and a point (radiopaque element of different shapes, lines, notches, numbers, characters and shapes [0525]), and “multimodal fiducial markers” and “wherein each multimodal fiducial marker in the subset is visible in the optical image and is also visible in the X-ray image” (radiopaque markers placed in the vicinity of a skeletal portion, [0508], all markings are visible both in the x-ray images and to optical camera [0526], [0608] and [0611]) and discloses “identify a subset of the multimodal fiducial markers,” as Tolkowsky teaches markers are identifiable in the images including at least some of the radiopaque markers by image processing ([0022]; at least some of the markers appear in images.
Therefore, claim(s) 19 overcome(s) previously and currently cited prior art and is/are found to be allowable.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 14, 16-17, 37-39, 51, 53-54, 56-58, 60, 70, and 72-73 are rejected under 35 U.S.C. 103 as being unpatentable over “Graumann et al.,” US 2014/0051994 (hereinafter Graumann) and “Hananel et al.,” US 2022/0175461 (hereinafter Hananel) and in view of “Tolkowsky et al.,” US 2021/0386480 (hereinafter Tolkowsky).
Regarding to claim 1, Graumann teaches a system for surgical navigation, comprising:
an instrument configured for a medical procedure on a patient (instrument 4, to be introduced on the body surface of the patient [0030] Figures 2-3);
a camera attached to the instrument, wherein the instrument has a spatial position relative to the camera (the navigation device 28, a camera fastened directly on the instrument, in the case of an optical mark [0037])
an x-ray system configured to acquire x-ray images of the patient during the medical procedure (x-ray C-arm [0039]);
a plurality of fiducial markers positioned on the surface of the patient during the medical procedure, said fiducial markers being detectable by the camera and the x-ray system (marker film attached on the patient with navigation marker viewable by camera and site markers viewable by X-ray [0040]); and
a computer configured to receive an optical image acquired by the camera (camera image [0014]), receive an x-ray image acquired by the x-ray system (X-ray image [0016])
Graumann does not explicitly disclose said fiducial markers being detectable by both the camera and the x-ray system, said fiducial markers comprising a radio-opaque material arranged as at least one of a line and a point and further limitations, identifying a subset of fiducial markers that are visible in the optical image and are also visible in the x-ray image, and determining a position of instrument using the relative position of camera, instrument, markers, and X-ray imaging system.
However, in the analogous field of endeavor in X-ray and visible imaging guidance of a medical procedure, Hananel teaches a camera attachable to treatment tool ([0119]) and a computer (processing unit 130 [0131]) configured to:
receive an optical image acquired by the camera (camera image [0133] and [0232])
receive an x-ray image acquired by the x-ray system (X-ray image [0127])
determine, based on the optical image, a spatial position relative to the camera for each fiducial marker in the subset of fiducial markers (determine a position and orientation of the camera with respect to the calibration plate based on visual representation of the at least one optical marker in the camera image [0024]);
determine, based on the x-ray image, a spatial position relative to the x-ray system for each fiducial marker in the subset of fiducial markers (identify the visual representation of radiopaque markers in the X-ray image [0127]; determine the position of markers in the calibration plate with respect to X-ray device [0131]); and
determine, based on at least the spatial positions of the subset of fiducial markers relative to the camera and the spatial positions of the subset of fiducial markers relative to the x-ray system, a spatial position for the instrument relative to the x-ray system ([0014] and [0054] and [0134] position of the tool with respect to X-ray device based on determined position of calibration plate including markers with respect to X-ray device, position of camera with respect to calibration plate which include markers, and camera with respect to X-ray device, calibration plate attached to a treatment tool [0370])
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify computing device as taught by Graumann to incorporate teaching of Hananel, since both Graumann and Hananel are directed to configuration of camera mounted instrument tracking using X-ray and optical imaging devices, and determining position of instrument with respect to X-ray imaging device was well known in the art as taught by Hananel. One of ordinary skill in the art could have combined the elements as claimed by Graumann with no change in their respective functions, using its optical and camera images of markers, to determine relative position of markers, with respect to camera and X-ray imaging system, to determine relative position of the tool with respect to X-ray imaging device, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to significantly reduce exposure of X-ray to the patient by reducing the need of X-ray imaging during the aiming and/or aligning of the treatment tool ([0384]), and there was reasonable expectation of success.
Graumann and Hananel do not explicitly disclose said fiducial markers comprising a radio-opaque material arranged as at least one of a line and a point and visible in both optical and X-ray images.
However, in the analogous field of image guided medical procedure using X-ray and optical imaging system, Tolkowsky teaches radiopaque markers comprising a radio-opaque material arranged as at least one of a line and a point (radiopaque element of different shapes, lines, notches, numbers, characters and shapes [0525]), and are visible to both in the X-ray image and to optical camera (radiopaque markers placed in the vicinity of a skeletal portion, [0508], all markings are visible both in the x-ray images and to optical camera [0526], [0608] and [0611]).
Thus, Tolkowsky’s teaching reads on amended limitation of “multimodal fiducial markers” and “wherein each multimodal fiducial marker in the subset is visible in the optical image and is also visible in the X-ray image” (radiopaque markers placed in the vicinity of a skeletal portion, [0508], all markings are visible both in the x-ray images and to optical camera [0526], [0608] and [0611]).
Moreover, Tolkowsky discloses “identify a subset of the multimodal fiducial markers,” as Tolkowsky teaches markers are identifiable in the images including at least some of the radiopaque markers by image processing ([0022]; at least some of the markers appear in images [0509]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the optical markers and radiopaque markers as taught by Graumann and Hananel to incorporate teaching of Tolkowsky, since radiopaque markers visible to both camera and X-ray images was well known in the art as taught by Tolkowsky. One of ordinary skill in the art could have combined the elements as claimed by Graumann and Hananel with no change in their respective functions, replacing its separate optical and radiopaque markers into radiopaque markers visible to both X-ray and optical imaging system and method, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to simplify the types of markers into single radiopaque markers, which are visible to both camera and X-ray images ([0526]), and there was reasonable expectation of success.
Regarding to claims 3 and 17, Graumann, Hananel, and Tolkowsky together teach all limitations of claim 1 as discussed above.
Tolkowsky further teaches following limitations:
Of claim 3, wherein the computer is further configured to: receive a computed tomography (CT) image acquired from a CT system prior to the medical procedure (CT and/or MRI scan performed prior to surgery [0006]); perform a registration of the x-ray image to the CT image; based on the registration of the CT image to the x-ray image ([0570]), display the spatial position of the instrument relative to the x-ray system as an overlay on the CT image (tool seen in 2D X-ray image is registered with the 3D CT data [0816])
Of claim 17, wherein the computer is further configured to: receive a magnetic resonance imaging (MRI) image acquired from an MRI system prior to the medical procedure (CT and/or MRI scan performed prior to surgery [0006]); perform a registration of the x-ray image to the MRI image (MRI image data are registered to intraprocedural 2D X-ray images [0860]); based on the registration of the MRI image to the x-ray image, display the spatial position of the instrument relative to the x-ray system as an overlay on the MRI image ([0860])
Regarding to claim 14, Graumann, Hananel, and Tolkowsky together teach all limitations of claim 1 as discussed above.
Graumann teaches wherein the spatial position of the instrument relative to the x-ray system is determined on the x-ray images during the medical procedure in real-time (position of the instrument is determined in real time [0003]).
Regarding to claim 16, Graumann, Hananel, and Tolkowsky together teach all limitations of claim 1 as discussed above.
Hananel teaches wherein the computer is further configured to determine the spatial position relative to the camera for the subset of fiducial markers based on at least one camera calibration parameter (position of camera with respect to plate with markers, based on the known position of optical markers within the plate and based on parameters of camera [0133])
Regarding to claim 37, Graumann teaches a system for surgical navigation, comprising:
an instrument configured for a medical procedure on a patient (instrument 4, to be introduced on the body surface of the patient [0030] Figures 2-3);
a camera attached to the instrument, wherein the instrument has a spatial position relative to the camera (the navigation device 28, a camera fastened directly on the instrument, in the case of an optical mark [0037])
an x-ray system configured to acquire x-ray images of the patient during the medical procedure (x-ray C-arm [0039]);
a plurality of fiducial markers positioned on the surface of the patient during the medical procedure, said fiducial markers being detectable by the camera and the x-ray system (marker film attached on the patient with navigation marker viewable by camera and site markers viewable by X-ray [0040]); and
a computer configured to receive a 2D optical image acquired by the camera (2D imaging [0013] and camera image [0014]), receive an 2D x-ray image acquired by the x-ray system (2D x-ray images [0012] and X-ray image [0016])
Graumann does not explicitly disclose said fiducial markers comprising a radio-opaque material arranged as at least one of a line and a point and further limitations, identifying a subset of fiducial markers that are visible in the optical image and are also visible in the x-ray image, and determining a position of instrument using the relative position of camera, instrument, markers, and X-ray imaging system.
However, in the analogous field of endeavor in X-ray and visible imaging guidance of a medical procedure, Hananel teaches a camera attachable to treatment tool ([0119]) and a computer (processing unit 130 [0131]) configured to:
receive an optical image acquired by the camera (camera image [0133] and [0232])
receive an x-ray image acquired by the x-ray system (X-ray image [0127])
determine, based on the optical image, a spatial position relative to the camera for each fiducial marker in the subset of fiducial markers ([0024]);
determine, based on the x-ray image, a spatial position relative to the x-ray system for each fiducial marker in the subset of fiducial markers (identify the visual representation of radiopaque markers in the X-ray image [0127]; determine the position of markers in the calibration plate with respect to X-ray device [0131]); and
determine, based on at least the spatial positions of the subset of fiducial markers relative to the camera and the spatial positions of the subset of fiducial markers relative to the x-ray system, a spatial position for the instrument relative to the x-ray system ([0014] and [0054] and [0134] position of the tool with respect to X-ray device based on determined position of calibration plate including markers with respect to X-ray device, position of camera with respect to calibration plate which include markers, and camera with respect to X-ray)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify computing device as taught by Graumann to incorporate teaching of Hananel, since both Graumann and Hananel are directed to a configuration of a camera mounted instrument tracking using X-ray and optical imaging devices and determining position of instrument with respect to X-ray imaging device was well known in the art as taught by Hananel. One of ordinary skill in the art could have combined the elements as claimed by Graumann with no change in their respective functions, using its 2D optical and camera images of markers disclosed by Graumann, to determine relative position of markers, with respect to camera and X-ray imaging system, to determine relative position of the tool with respect to X-ray imaging device, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to significantly reduce exposure of X-ray to the patient by reducing the need of X-ray imaging during the aiming and/or aligning of the treatment tool ([0384]), and there was reasonable expectation of success.
Graumann and Hananel do not explicitly disclose said fiducial markers comprising a radio-opaque material arranged as at least one of a line and a point and visible in both optical and X-ray images.
However, in the analogous field of image guided medical procedure using X-ray and optical imaging system, Tolkowsky teaches radiopaque markers comprising a radio-opaque material arranged as at least one of a line and a point (radiopaque element of different shapes, lines, notches, numbers, characters and shapes [0525]), and are visible to both in the X-ray image and to optical camera (radiopaque markers placed in the vicinity of a skeletal portion, [0508], [0526], [0608] and [0611]).
Tolkowsky additionally teaches using 2D optical camera image and 2D x-ray image (Figure 10 [0455] and [0605]; 2D optical camera [0856]).
Thus, Tolkowsky’s teaching reads on amended limitation of “multimodal fiducial markers” and “wherein each multimodal fiducial marker in the subset is visible in the optical image and is also visible in the X-ray image” (radiopaque markers placed in the vicinity of a skeletal portion, [0508], all markings are visible both in the x-ray images and to optical camera [0526], [0608] and [0611]).
Moreover, Tolkowsky discloses “identify a subset of the multimodal fiducial markers,” as Tolkowsky teaches markers are identifiable in the images including at least some of the radiopaque markers by image processing ([0022]; at least some of the markers appear in images [0509]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the optical markers and radiopaque markers as taught by Graumann and Hananel to incorporate teaching of Tolkowsky, since radiopaque markers visible to both camera and X-ray images was well known in the art as taught by Tolkowsky. One of ordinary skill in the art could have combined the elements as claimed by Graumann and Hananel with no change in their respective functions, replacing its separate optical and radiopaque markers into radiopaque markers, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to simplify the types of markers into single radiopaque markers, which are visible to both camera and X-ray images ([0526]), and there was reasonable expectation of success.
Regarding to claims 38-39 and 54, Graumann, Hananel, and Tolkowsky together teach all limitations of claim 37 as discussed above.
Tolkowsky further teaches following limitations:
Of claim 38, wherein the computer is further configured to display the spatial position of the instrument relative to the x-ray system as an overlay on the 2D x-ray images (portion of the tool overlaid upon the 2D x-ray image [0491], [0680])
Of claim 39, wherein the computer is further configured to: receive a 3D computed tomography (CT) image acquired from a CT system prior to the medical procedure (3D scan CT and/or MRI scan performed prior to surgery [0006]); perform a 3D-2D registration of the 3D CT image to the 2D x-ray image (3D CT registration with 2D image [0570] and [0579]); based on the 3D-2D registration of the 3D CT image to the 2D x-ray image, display the spatial position of the instrument relative to the x-ray system as an overlay on the 3D CT image (tool seen in 2D X-ray image is registered with the 3D CT data [0816])
Of claim 54, wherein the computer is further configured to: receive a 3D magnetic resonance imaging (MRI) image acquired from an MRI system prior to the medical procedure (3D CT and/or MRI scan performed prior to surgery [0006]) ; perform a 3D-2D registration of the 3D MRI image to the 2D x-ray image; based on the 3D-2D registration of the 3D MRI image to the 2D x-ray image (3D MRI image data are registered to intraprocedural 2D X-ray images [0860]), display the spatial position of the instrument relative to the x-ray system as an overlay on the 3D MRI image ([0860]).
Regarding to claim 51, Graumann, Hananel, and Tolkowsky together teach all limitations of claim 37 as discussed above.
Graumann teaches wherein the 2D spatial position of the instrument relative to the x-ray system is determined on the x-ray images during the medical procedure in real- time (position of the instrument is determined in real time [0003]).
Regarding to claim 53, Graumann, Hananel, and Tolkowsky together teach all limitations of claim 37 as discussed above.
Hananel teaches wherein the computer is further configured to determine the spatial position relative to the camera for the subset of fiducial markers based on at least one camera calibration parameter (position of camera with respect to plate with markers, based on the known position of optical markers within the plate and based on parameters of camera [0133])
Regarding to claim 56, Graumann teaches a method for surgical navigation, comprising:
receiving a two-dimensional (2D) optical image acquired by a camera (, said camera attached to an instrument configured for a medical procedure on a patient, wherein the instrument has a spatial position relative to the camera (instrument 4, to be introduced on the body surface of the patient [0030] Figures 2-3, the navigation device 28, a camera fastened directly on the instrument, in the case of an optical mark [0037])
receiving a 2D x-ray image acquired by an x-ray system configured to acquire x-ray images of the patient during the medical procedure (2D radiographic images are x-ray images from X-ray device [0016])
for a plurality of fiducial markers positioned on the surface of the patient during the medical procedure and detectable by both the camera and the x-ray system ((marker film attached on the patient with navigation marker viewable by camera and site markers viewable by X-ray [0040]);
Graumann does not explicitly disclose identifying a subset of fiducial markers that are visible in the optical image and are also visible in the x-ray image, and determining a position of instrument using the relative position of camera, instrument, markers, and X-ray imaging system.
However, in the analogous field of endeavor in X-ray and visible imaging guidance of a medical procedure, Hananel teaches a camera attachable to treatment tool ([0119]) and a computer (processing unit 130 [0131]) configured to:
receive an optical image acquired by the camera (camera image [0133] and [0232])
receive an x-ray image acquired by the x-ray system (X-ray image [0127])
determine, based on the optical image, a spatial position relative to the camera for each fiducial marker in the subset of fiducial markers ([0024]);
determine, based on the x-ray image, a spatial position relative to the x-ray system for each fiducial marker in the subset of fiducial markers (identify the visual representation of radiopaque markers in the X-ray image [0127]; determine the position of markers in the calibration plate with respect to X-ray device [0131]); and
determine, based on at least the spatial positions of the subset of fiducial markers relative to the camera and the spatial positions of the subset of fiducial markers relative to the x-ray system, a spatial position for the instrument relative to the x-ray system ([0014] and [0054] and [0134] position of the tool with respect to X-ray device based on determined position of calibration plate including markers with respect to X-ray device, position of camera with respect to calibration plate which include markers, and camera with respect to X-ray)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify computing device as taught by Graumann to incorporate teaching of Hananel, since both are directed to camera mounted instrument tracking using X-ray and optical imaging devices and determining position of instrument with respect to X-ray imaging device was well known in the art as taught by Hananel. One of ordinary skill in the art could have combined the elements as claimed by Graumann with no change in their respective functions, using its 2D optical and camera images of markers, to determine relative position of markers, with respect to camera and X-ray imaging system, to determine relative position of the tool with respect to X-ray imaging device, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to significantly reduce exposure of X-ray to the patient by reducing the need of X-ray imaging during the aiming and/or aligning of the treatment tool ([0384]), and there was reasonable expectation of success.
Graumann and Hananel do not explicitly disclose said fiducial markers being detectable by both the camera and the x-ray system, said fiducial markers comprising a radio-opaque material arranged as at least one of a line and a point and visible in both optical and X-ray images.
However, in the analogous field of image guided medical procedure using X-ray and optical imaging system, Tolkowsky teaches radiopaque markers comprising a radio-opaque material arranged as at least one of a line and a point visible to both X-ray and optical camera (radiopaque element of different shapes, lines, notches, numbers, characters and shapes [0525]), and identifying a subset of the fiducial markers that are visible to both in the X-ray image and to optical camera (radiopaque markers placed in the vicinity of a skeletal portion, set of markers [0508], [0526], [0608] and [0611]).
Tolkowsky additionally teaches using 2D optical camera image and 2D x-ray image (Figure 10 [0455] and [0605]; 2D optical camera [0856]).
Thus, Tolkowsky’s teaching reads on amended limitation of “multimodal fiducial markers” and “wherein each multimodal fiducial marker in the subset is visible in the optical image and is also visible in the X-ray image” (radiopaque markers placed in the vicinity of a skeletal portion, [0508], all markings are visible both in the x-ray images and to optical camera [0526], [0608] and [0611]).
Moreover, Tolkowsky discloses “identify a subset of the multimodal fiducial markers,” as Tolkowsky teaches markers are identifiable in the images including at least some of the radiopaque markers by image processing ([0022]; at least some of the markers appear in images [0509]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the optical markers and radiopaque markers as taught by Graumann and Hananel to incorporate teaching of Tolkowsky, since radiopaque markers visible to both camera and X-ray images was well known in the art as taught by Tolkowsky. One of ordinary skill in the art could have combined the elements as claimed by Graumann and Hananel with no change in their respective functions, replacing its separate optical and radiopaque markers into radiopaque markers, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to simplify the types of markers into single radiopaque markers, which are visible to both camera and X-ray images ([0526]), and there was reasonable expectation of success.
Regarding to claims 57-58, 60, and 73, Graumann, Hananel, and Tolkowsky together teach all limitations of claim 56 as discussed above.
Tolkowsky teaches following limitations:
Of claim 57, further comprising displaying the spatial position of the instrument relative to the x-ray system as an overlay on the x-ray images (portion of the tool overlaid upon the 2D x-ray image [0491], [0680])
Of claim 58, further comprising: receiving a 3D computed tomography (CT) image acquired from a CT system prior to the medical procedure (3D scan CT and/or MRI scan performed prior to surgery [0006]); performing a 3D-2D registration of the 3D CT image to the 2D x-ray image (3D CT registration with 2D image [0570] and [0579]); based on the 3D-2D registration of the 3D CT image to the 2D x-ray image, displaying the spatial position of the instrument relative to the x-ray system as an overlay on the 3D CT image (tool seen in 2D X-ray image is registered with the 3D CT data [0816])
Of claim 60, wherein the fiducial markers comprise a radio- opaque material arranged as at least one of a line and a point (radiopaque element of different shapes, lines, notches, numbers, characters and shapes [0525]).
Of claim 73, further comprising receiving a 3D magnetic resonance imaging (MRI) image acquired from an MRI system prior to the medical procedure (3D CT and/or MRI scan performed prior to surgery [0006]) ; performing a 3D-2D registration of the 3D MRI image to the 2D x-ray image; based on the 3D-2D registration of the 3D MRI image to the 2D x-ray image (3D MRI image data are registered to intraprocedural 2D X-ray images [0860]), displaying the spatial position of the instrument relative to the x-ray system as an overlay on the 3D MRI image ([0860]).
Regarding to claim 70, Graumann, Hananel, and Tolkowsky together teach all limitations of claim 56 as discussed above.
Graumann teaches further teaches wherein the 2D spatial position of the instrument relative to the x-ray system is determined on the x-ray images during the medical procedure in real- time (position of the instrument is determined in real time [0003]).
Regarding to claim 72, Graumann, Hananel, and Tolkowsky together teach all limitations of claim 56 as discussed above.
Hananel further teaches wherein the computer is further configured to determine the spatial position relative to the camera for the subset of fiducial markers based on at least one camera calibration parameter (position of camera with respect to plate with markers, based on the known position of optical markers within the plate and based on parameters of camera [0133]).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Graumann, Hananel, and Tolkowsky as applied to claim 1 above, and further in view of “Gullotti et al.,” US 2020/0405395 (hereinafter Gullotti).
Regarding to claim 2, Graumann, Hananel, and Tolkowsky together teach all limitations of claim 1 as discussed above.
Tolkowsky further teaches further comprising displaying the spatial position of the instrument relative to the x-ray system as an overlay on the x-ray images (portion of the tool overlaid upon the 2D x-ray image [0491], [0680]), but does not further teach wherein the overlay is updated in response to changes in spatial position of the instrument as determined from subsequent optical images (optical sensor for current position [0808]), but does not disclose “without requiring additional x-ray images.”
However, in the analogous field of endeavor intraoperatively navigation of medical procedure, Gullotti teaches significantly reducing X-ray and radiation exposure during minimally invasive procedure, as tracked surgical tools are placed in the field of view of previously-acquired x-ray images, such that the projected outline can be displayed over anatomy visualized in a previously acquired x-ray image, using location of the tool relative to the x-ray, to accurately display a real-time overlay of the tool on the previously acquired x-ray image ([0897]) and tracking camera for tracking current position of the tool ([0726]) using markers tracked by camera and radiopaque features identifiable with x-ray imaging, visualized in both 3D tracking camera space and x-ray imaging space ([0749] and [0780]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the optical markers and radiopaque markers as taught by Graumann, Hananel, and Tolkowsky to incorporate teaching of Gullotti, since all are directed to using optical and x-ray images and tracking instrument without additional x-ray images was well known in the art as taught by Gullotti. One of ordinary skill in the art could have combined the elements as claimed by Graumann, Hananel, and Tolkowsky with no change in their respective functions, without additional x-ray image acquisition, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art before the effective filing date of the claimed invention. The motivation would have been to reduce x-ray radiation exposure ([0897]) and there was reasonable expectation of success.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICIA J PARK whose telephone number is (571)270-1788. The examiner can normally be reached Monday-Thursday 8 am - 3 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at 571-272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICIA J PARK/Primary Examiner, Art Unit 3798
Prosecution Insights