DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
Extensive mechanical and design details of an apparatus should not be included in the abstract. The abstract should be in narrative form and generally limited to a single paragraph within the range of 50 to 150 words in length.
See MPEP § 608.01(b) for guidelines for the preparation of patent abstracts.
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
Claim Objections
Claim 13 is objected to because of the following informalities:
The phrase “The method of claim 14” should read “The method of claim 12” because the claim does not refer to a previous claim. In order to avoid clarity issues and prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, the Examiner suggests Applicant as suggested by the Examiner.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 16 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 16 recite the limitation "the metal pins" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 9, 11-12 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Swayze et al (Pub No.: 20180168741) in view of Bouget et al (NPL titled: Detecting Surgical Tools by Modelling Local Appearance and Global Shape).
Regarding claim 1, Swayze teaches a method for rendering metal objects translucent in medical images (method for minimally invasive surgery, and in particular for providing an augmented reality display of a surgical environment and surgical instrument can be rendered transparent – see [p][0001][0038]), the method comprising: determining an imaging space (a patient-side portion 110 – see Fig 1) of an image capturing device (camera module 210 – see Fig 2), wherein the image capturing device is configured to generate a two-dimensional image of a subject from the imaging space (an image of the surgical environment is typically generated and displayed on a display to the surgeon, such as on a video monitor, a headset, glasses, or another accessory worn by the surgeon – see [p][0036]); registering a three-dimensional dataset of the subject to the imaging space of the image capturing device (the processor can use a three-dimensional (3D) image reconstruction algorithm to construct a 3D image of the entire surgical environment, or portions thereof, based on images captured by the first and second cameras 706, 708. The reconstruction algorithm takes into account the difference in perspective of the two cameras due to their separation in space – see [p][0036]), predicting at least part of the two-dimensional image of the subject using the registered three-dimensional dataset (this can be accomplished using chroma key technology where a portion of an image having a predetermined color is identified and altered (e.g., replaced with a different image). For example, the shaft 606 can be marked with a predetermined color (e.g., blue, green, etc.), which is typically one that does not naturally occur in the surgical environment. The processor 216 then identifies the image of the shaft based on its assigned color (e.g., the processor 216 can search the image dataset of the image 600 for the predetermined color). After the shaft 606 has been identified, the actual image is modified by replacing the image of the shaft with the images of one or more portions of the target tissue 604. This can be achieved by using multiple cameras (e.g., multiple camera modules 210) to capture multiple images of the surgical environment 600 from various vantage points. The processor 216 receives the multiple images, identifies the image of the shaft 606, and generates a modified image using a visual algorithm – see [p][0055]), wherein the at least part of the two-dimensional image of the subject includes one or more metal objects (the image of the surgical environment can also be modified to replace the images of selected portions of the surgical instrument with those of the surgical environment (e.g., target tissues, tissues surrounding the target tissue, etc.) – see [p][0036]), and wherein the prediction of the at least part of the two-dimensional image of the subject includes a first set of anatomical information of the subject that is blocked by the one or more metal objects (It can also be desirable to view tissues in the surgical environment whose view may be obstructed by portions of the surgical device. This can be achieved, for example, by replacing the image of the portions of the surgical device with that of tissues in the surgical environment and certain regions of the surgical instrument can be rendered transparent – see [p][0036][0038]); and generating an improved two-dimensional image of the subject based on the two-dimensional image and the prediction of the at least part of the two-dimensional image of the subject, wherein the improved two-dimensional image is unobscured by the one or more metal objects (this can be done, for example, by using chroma key technology in which the images of predetermined regions of the surgical instrument are identified and replaced with the images of target tissues that were obstructed by the surgical instrument. This can be desirable as it provides the surgeon with an unimpeded view of the target tissue during the surgical procedure – see [p][0038]); and presenting the improved two-dimensional image to a user (the modified image can be displayed, on a display 222, which can be wall or table-mounted or on an accessory (e.g., a head set or glasses) worn by the surgeon – see [p][0042] and the processor can use the image reconstruction algorithm to generate a modified image of the surgical environment 700 from the combined image 800 that renders the predetermined regions of the surgical instrument (or parts thereof) transparent. In other words, in the modified image the predetermined regions of the surgical instrument are replaced by portions of the combined image, the view of which is otherwise obstructed by the predetermined regions - see [p][0059]).
Swayze teaches thereby generating a registered dataset (the processor 216 identifies the markers 502, 504, 506, 508 (and therefore the relative positions of the markers) in the image and compares this information with data from a database of surgical instruments stored in database 218 – see [p][0053]); however, does not explicitly teach three-dimensional.
Bouget explicitly teaches three-dimensional (a robotic tool dataset focusing on surgery performed using the DaVinci robot. It contains a total of 1 950 frames from 12 different stereoscopic videos (average length of 4 seconds) – see section 1, subsection B).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Swayze a method for rendering metal objects translucent in medical images with the teachings of Bouget three-dimensional.
Wherein having Swayze three-dimensional.
The motivation behind the modification would have been for displaying a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while presenting a new surgical tool detection dataset and a method for joint tool detection and pose estimation in images since both Swayze and Bouget relate to processing images for surgery, wherein Swayze displays a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while Bouget presents a new surgical tool detection dataset and a method for joint tool detection and pose estimation in images (Swayze et al (Pub No.: 20180168741), see [p][0006] Bouget et al (NPL titled: Detecting Surgical Tools by Modelling Local Appearance and Global Shape, see Abstract).
Regarding claim 2, Swayze in view of Bouget teaches the method of claim 1, Swayze teaches wherein generating an improved two-dimensional image of the subject comprises identifying a plurality of pixels that at least partly contains the one or more metal objects (FIG. 9A represents an example of an image of a surgical environment 900 comprising the target tissue 902 and surgical instrument 904 and a portion of shaft 910, the images of which are captured using cameras 906 and 908. FIG. 9B illustrates a modified image 950 of the surgical environment in which shaft 910 is no longer visible as it has been replaced by images of the underlying tissue – see Fig 9A &9B and [p][0060]).
Regarding claim 3, Swayze in view of Bouget teaches the method of claim 2, Swayze teaches wherein generating an improved two-dimensional image of the subject comprises modifying data of the plurality of pixels (FIG. 9B illustrates a modified image 950 of the surgical environment in which shaft 910 is no longer visible as it has been replaced by images of the underlying tissue – see Fig9B and [p][0060]).
Regarding claim 4, Swayze in view of Bouget teaches the method of claim 3, Swayze teaches wherein generating an improved two-dimensional image of the subject comprises combining the prediction of the at least part of the two-dimensional image of the subject and the data of the plurality of pixels pixel by pixel (see [p][0060] and Fig 9B).
Regarding claim 5, Swayze in view of Bouget teaches the method of claim 1, Swayze teaches wherein the improved two-dimensional image includes the first set of anatomical information of the subject (note the surgery site contains a set of body structures - see Fig 9B and [p0060]).
Regarding claim 9, Swayze in view of Bouget teaches the method of claim 1, Swayze teaches wherein the two-dimensional image is generated using projection imaging (a benefit of having the image projected onto a headset or glasses is that the surgeon can simultaneously view both the projected image and the actual image – see [p][0042]), and wherein the one or more metal objects block the first set of anatomical information of the subject within the two-dimensional image (see Fig 9A).
Regarding claim 11, Swayze in view of Bouget teaches the method of claim 1, Swayze teaches wherein presenting the improved two-dimensional image comprises presenting the prediction of the first set of blocked anatomical information as an overlay superimposed on the two-dimensional image thereby facilitating the medical procedure (the image 600 that is displayed to the surgeon is modified to remove the image of a portion of the shaft 606 that obstructs a view of some of the tissue and replace the shaft image with an image of the tissue that was not visible in the original image. This can be accomplished using chroma key technology where a portion of an image having a predetermined color is identified and altered (e.g., replaced with a different image) and the portion of the shaft that obstructs the view of tissue is replaced with a view of the underlying tissue – see [p][0055]).
Regarding claim 12, Swayze in view of Bouget teaches the method of claim 1, Swayze teaches wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device (the processor can generate a stereoscopic display in which the operator's brain will combine separate 2D images to create the perception of 3D – see [p][0057]); the prediction of the at least part of the two-dimensional image of the subject using the registered three-dimensional dataset (there are also portions of the surgical environment that may not captured by either of the two cameras, such as region 808. The processor is able to generate a 3D image for region 802 and a 2D image for regions 804 and 806 – see [p][0059]); and generating an improved two-dimensional image are all completed in real-time (modify the image of a surgical environment (e.g., include surgical instruments, target tissue, and tissues surrounding the target tissue, etc.) in real-time – see [p][0036]).
Regarding claim 15, Swayze in view of Bouget teaches the method of claim 1, Swayze teaches wherein the three-dimensional dataset is acquired with metal items within a field of view thereof (for example, operating parameters related to a part of the surgical instrument can be located at or near the related part. As the surgical instrument moves, the operating parameters can track the motion of the surgical instruments. In some embodiments, an operating parameter or a change thereof can be visually represented (e.g., change of color, flashing images etc.) – see [p][0037]).
Claims 6-8 10 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Swayze et al (Pub No.: 20180168741) in view of Bouget et al (NPL titled: Detecting Surgical Tools by Modelling Local Appearance and Global Shape) as applied to claim 1 further in view of Thomas et al (Pub No.: US20170309069)
Regarding claim 6, Swayze in view of Bouget teaches the method of claim 2, Swayze in view of Bouget teaches wherein the one or more metal objects block the first set of anatomical information of the subject therewithin (see Fig 9A); however, Swayze in view of Bouget does not explicit teach wherein the two-dimensional image is an X-ray image.
Thomas explicit teaches wherein the two-dimensional image is an X-ray image (see [p][0048]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Swayze as modified by Bouget a method for rendering metal objects translucent in medical images with the teachings of Thomas wherein the two-dimensional image is an X-ray image
Wherein having Swayze wherein the two-dimensional image is an X-ray image
The motivation behind the modification would have been for displaying a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while producing a 3D image containing entry points to the tissue, computing and storing one or more surgical trajectory paths based on a surgical outcome criteria, and displaying a selected trajectory path at the user interface since both Swayze and Thomas relate to processing images for surgery, wherein Swayze displays a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while Thomas produces a 3D image containing entry points to the tissue, computing and storing one or more surgical trajectory paths based on a surgical outcome criteria, and displaying a selected trajectory path at the user interface (Swayze et al (Pub No.: 20180168741), see [p][0006], Thomas et al (Pub No.: US20170309069), [p][0022]).
Regarding claim 7, Swayze in view of Bouget teach the method of claim 1, Swayze teaches comprising adjusting registration of the three-dimensional dataset of the subject to the imaging space of the image capturing device until prediction of a second set of unblocked anatomical information of the two-dimensional image (the processor can generate a stereoscopic display in which the operator's brain will combine separate 2D images to create the perception of 3D. This can be accomplished in one various ways. In one example, each 2D image is processed such that the chroma keyed device components are first removed from each 2D image. The correlated pixels of the opposing 2D image are then be stitched into this space – see [p][0057]); however, Swayze in view of Bouget does not teach satisfies a pre-determined criterion.
Thomas explicitly teaches satisfies a pre-determined criterion (these values may correlate with the planning model of the brain, and indicate to the surgeon when they are dis-concordant or when a tolerance threshold that is pre-determined has been exceeded – see [p][0188]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Swayze as modified by Bouget a method for rendering metal objects translucent in medical images with the teachings of Thomas satisfies a pre-determined criterion.
Wherein having Swayze satisfies a pre-determined criterion.
The motivation behind the modification would have been for displaying a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while producing a 3D image containing entry points to the tissue, computing and storing one or more surgical trajectory paths based on a surgical outcome criteria, and displaying a selected trajectory path at the user interface since both Swayze and Thomas relate to processing images for surgery, wherein Swayze displays a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while Thomas produces a 3D image containing entry points to the tissue, computing and storing one or more surgical trajectory paths based on a surgical outcome criteria, and displaying a selected trajectory path at the user interface (Swayze et al (Pub No.: 20180168741), see [p][0006], Thomas et al (Pub No.: US20170309069), [p][0022]).
Regarding claim 8, Swayze in view of Bouget teach the method of claim 7, Swayze teaches wherein adjusting registration of the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises calculating a similarity of the prediction of the second set of unblocked anatomical information to the second set of unblocked anatomical information in the two-dimensional image (the processor can generate a stereoscopic display in which the operator's brain will combine separate 2D images to create the perception of 3D. This can be accomplished in one various ways. In one example, each 2D image is processed such that the chroma keyed device components are first removed from each 2D image. The correlated pixels of the opposing 2D image are then be stitched into this space – see [p][0057]).
Regarding claim 10, Swayze in view of Bouget teaches does not teach the method of claim 1, wherein the three-dimensional dataset includes one or more of a CT dataset, a Magnetic Resonance Imaging dataset, a Positron Emission Tomography (PET) dataset, or an Ultrasound dataset.
Thomas explicitly teaches wherein the three-dimensional dataset includes one or more of a CT dataset, a Magnetic Resonance Imaging dataset, a Positron Emission Tomography (PET) dataset, or an Ultrasound dataset (in the above three example MRI techniques (diffusion MRI, fMRI, perfusion MRI), what is generated is a 4d dataset (i.e. 3d volumes evolving over time) which includes data relating to either water diffusion (diffusion MRI), blood oxygenation (fMRI), or a contrast agent moving through tissue (perfusion MRI), in addition to the static imaging data - see [p][0051]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Swayze as modified by Bouget a method for rendering metal objects translucent in medical images with the teachings of Thomas wherein the three-dimensional dataset includes one or more of a CT dataset, a Magnetic Resonance Imaging dataset, a Positron Emission Tomography (PET) dataset, or an Ultrasound dataset
Wherein having Swayze wherein the three-dimensional dataset includes one or more of a CT dataset, a Magnetic Resonance Imaging dataset, a Positron Emission Tomography (PET) dataset, or an Ultrasound dataset.
The motivation behind the modification would have been for displaying a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while producing a 3D image containing entry points to the tissue, computing and storing one or more surgical trajectory paths based on a surgical outcome criteria, and displaying a selected trajectory path at the user interface since both Swayze and Thomas relate to processing images for surgery, wherein Swayze displays a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while Thomas produces a 3D image containing entry points to the tissue, computing and storing one or more surgical trajectory paths based on a surgical outcome criteria, and displaying a selected trajectory path at the user interface (Swayze et al (Pub No.: 20180168741), see [p][0006], Thomas et al (Pub No.: US20170309069), [p][0022]).
Regarding claim 14, Swayze in view of Bouget does not explicitly teach the method of claim 1, wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises registering a first coordinate system of the three-dimensional dataset to a second coordinate system of the imaging space.
Thomas explicitly teaches wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises registering a first coordinate system of the three-dimensional dataset to a second coordinate system of the imaging space (“registration” or “co-registration” refers to the process of transforming different sets of data into one coordinate system, and “image registration” refers to the process of transforming different sets of imaging data into one coordinate system – see [p][0054]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Swayze as modified by Bouget a method for rendering metal objects translucent in medical images with the teachings of Thomas wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises registering a first coordinate system of the three-dimensional dataset to a second coordinate system of the imaging space
Wherein having Swayze wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises registering a first coordinate system of the three-dimensional dataset to a second coordinate system of the imaging space.
The motivation behind the modification would have been for displaying a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while producing a 3D image containing entry points to the tissue, computing and storing one or more surgical trajectory paths based on a surgical outcome criteria, and displaying a selected trajectory path at the user interface since both Swayze and Thomas relate to processing images for surgery, wherein Swayze displays a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while Thomas produces a 3D image containing entry points to the tissue, computing and storing one or more surgical trajectory paths based on a surgical outcome criteria, and displaying a selected trajectory path at the user interface (Swayze et al (Pub No.: 20180168741), see [p][0006], Thomas et al (Pub No.: US20170309069), [p][0022]).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over
Swayze et al (Pub No.: 20180168741) in view of Bouget et al (NPL titled: Detecting Surgical Tools by Modelling Local Appearance and Global Shape) as applied to claim 1 further in view of Gong et al (NPL titled: Near real-time reliable stereo matching using programmable graphics hardware)
Regarding claim 13, Swayze in view of Bouget teaches the method of claim 14, wherein real-time is less than or equal to 0.1 seconds.
Gong explicitly teaches wherein real-time is less than or equal to 0.1 seconds (matches in near real-time (0.05~0.1sec) – see abstract).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Swayze as modified by Bouget a method for rendering metal objects translucent in medical images with the teachings of Gong wherein real-time is less than or equal to 0.1 seconds
Wherein having Swayze wherein real-time is less than or equal to 0.1 seconds.
The motivation behind the modification would have been for displaying a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while generating semi-dense disparity maps using dynamic programming by implementing an algorithm on programmable graphics hardware, which improves the processing speed since both Swayze and Gong relate to processing images, wherein Swayze displays a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while Gong generates a semi-dense disparity maps using dynamic programming by implementing an algorithm on programmable graphics hardware, which improves the processing speed (Swayze et al (Pub No.: 20180168741), see [p][0006], Gong et al (NPL titled: Near real-time reliable stereo matching using programmable graphics hardware), see Abstract).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over
Swayze et al (Pub No.: 20180168741) in view of Bouget et al (NPL titled: Detecting Surgical Tools by Modelling Local Appearance and Global Shape) as applied to claim 10 further in view of Uneri et al (NPL titled: Known-component 3D-2D registration for quality assurance of spine surgery pedicle screw placement)
Regarding claim 16, Swayze in view of Bouget teaches the method of claim 1, Swayze teaches wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises tracking the metal items using infrared light in the imaging space (a light source (not shown) can generate light which is reflected by the surgical environment. The light can be visible light (e.g., having a wavelength of about 400 nm to 800 nm) or light of a wavelength that is outside of the visible spectrum (e.g., infrared and ultraviolet light). A portion of the reflected light is captured by the camera module 210, which comprises a lens 206 configured to focus visible-light onto a detector 212 – see [p][0043]) and however, Swayze in view of Bouget teaches does not teach registering the metal pins in the three-dimensional dataset to the metal pins in the imaging space.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Swayze as modified by Bouget a method for rendering metal objects translucent in medical images with the teachings of Uneri r registering the metal pins in the three-dimensional dataset to the metal pins in the imaging space.
Wherein having Swayze registering the metal pins in the three-dimensional dataset to the metal pins in the imaging space.
The motivation behind the modification would have been for displaying a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while performing visualization of registered devices relative to surgical planning since both Swayze and Uneri relate to processing images, wherein Swayze displays a modified image of a surgical environment in which the modified image displays at least a portion of the surgical device while replacing another portion of the surgical device with an image of tissue at the surgical environment underlying the replaced portion of the surgical device while Uneri performs visualization of registered devices relative to surgical planning (Swayze et al (Pub No.: 20180168741), see [p][0006], Uneri et al (NPL titled: Known-component 3D-2D registration for quality assurance of spine surgery pedicle screw placement), see Abstract).
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-4, 6-12 and 14-16 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-17 of U.S. Patent No. 12118650 (Herein referred to as Patent’650). Although the claims at issue are not identical, they are not patentably distinct from each other because Patent’650 claims 1-3 requires the additional step of “receiving, by a computer, a three-dimensional dataset of a subject, not required by claim instant claim 1.
Claims instant 1 and Patent’650 claims 1-3 recite common subject matter;
Whereby claim 1, which recites the open ended transitional phrase “comprising”, does not preclude the additional elements recited by Patent’650 claims 1-3, and
Whereby the elements of claim 1 are fully anticipated by patent claims 1-3 and anticipation is “the ultimate or epitome of obviousness” (In re Kalm, 154 USPQ 10 (CCPA 1967), also In re Dailey, 178 USPQ 293 (CCPA 1973) and In re Pearson, 181 USPQ 641 (CCPA 1974)).
Instant claims
US Patent No.: 12118650
1 A method for rendering metal objects translucent in medical images, the method comprising:
1. A method for rendering metal objects translucent in medical images, the method comprising:
determining an imaging space of an image capturing device, wherein the image capturing device is configured to generate a two-dimensional image of a subject from the imaging space;
receiving, by a computer, a three-dimensional dataset of a subject;
determining, by the computer, an imaging space of an image capturing device, wherein the image capturing device is configured to generate a two-dimensional image of the subject from the imaging space;
acquiring, by the image capturing device, a two-dimensional image of the subject from the imaging space;
registering a three-dimensional dataset of the subject to the imaging space of the image capturing device, thereby generating a registered three-dimensional dataset;
registering, by the computer, the three-dimensional dataset of the subject to the imaging space of the image capturing device, thereby generating a registered three-dimensional dataset; and
predicting at least part of the two-dimensional image of the subject using the registered three-dimensional dataset, wherein the at least part of the two-dimensional image of the subject includes one or more metal objects, and wherein the prediction of the at least part of the two-dimensional image of the subject includes a first set of anatomical information of the subject that is blocked by the one or more metal objects;
predicting, by the computer, at least part of the two-dimensional image of the subject using the registered three-dimensional dataset, wherein the at least part of the two-dimensional image of the subject includes one or more metal objects, and wherein the prediction of the at least part of the two-dimensional image of the subject includes a first set of anatomical information of the subject that is blocked by the one or more metal objects.
and generating an improved two-dimensional image of the subject based on the two-dimensional image and the prediction of the at least part of the two-dimensional image of the subject, wherein the improved two-dimensional image is unobscured by the one or more metal objects;
3. The method of claim 2, further comprising, subsequent to the step of predicting at least a part of the two-dimensional image of the subject, generating, by the computer, an improved two-dimensional image of the subject based on the two-dimensional image and the prediction of the at least part of the two-dimensional image of the subject.
4. The method of claim 3, wherein the improved two-dimensional image is unobscured by the one or more metal objects, and wherein the improved two-dimensional image includes the first set of anatomical information of the subject.
and presenting the improved two-dimensional image to a user.
6. The method of claim 3 further comprising, presenting, by a digital display, the improved two-dimensional image to a user during a medical procedure.
2. The method of claim 1, wherein generating an improved two-dimensional image of the subject comprises identifying a plurality of pixels that at least partly contains the one or more metal objects.
3. The method of claim 2, wherein generating an improved two-dimensional image of the subject comprises modifying data of the plurality of pixels.
4. The method of claim 3, wherein generating an improved two-dimensional image of the subject comprises combining the prediction of the at least part of the two-dimensional image of the subject and the data of the plurality of pixels pixel by pixel.
4. The method of claim 3, wherein the improved two-dimensional image is unobscured by the one or more metal objects, and wherein the improved two-dimensional image includes the first set of anatomical information of the subject.
5. The method of claim 3, wherein generating the improved two-dimensional image of the subject comprises:
identifying, by the computer, a plurality of pixels that at least partly contains the one or more metal objects;
optionally modifying, by the computer, data of the plurality of pixels; and
combining, by the computer, the prediction of the at least part of the two-dimensional image of the subject and the data of the plurality of pixels pixel by pixel.
6. The method of claim 2, wherein the two-dimensional image is an X-ray image, and wherein the one or more metal objects block the first set of anatomical information of the subject therewithin (see Fig 9A).
8. The method of claim 2, wherein the two-dimensional image is an X-ray image, and wherein the one or more metal objects block the first set of anatomical information of the subject therewithin
7. The method of claim 1, comprising adjusting registration of the three-dimensional dataset of the subject to the imaging space of the image capturing device until prediction of a second set of unblocked anatomical information of the two-dimensional image satisfies a pre-determined criterion.
. 9. The method of claim 8 further comprising adjusting registration of the three-dimensional dataset of the subject to the imaging space of the image capturing device until prediction of a second set of unblocked anatomical information of the two-dimensional image satisfies a pre-determined criterion.
8. The method of claim 7, wherein adjusting registration of the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises calculating a similarity of the prediction of the second set of unblocked anatomical information to the second set of unblocked anatomical information in the two-dimensional image
10. The method of claim 9, wherein adjusting registration of the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises calculating a similarity of the prediction of the second set of unblocked anatomical information to the second set of unblocked anatomical information in the two-dimensional image.
9. The method of claim 1, wherein the two-dimensional image is generated using projection imaging, and wherein the one or more metal objects block the first set of anatomical information of the subject within the two-dimensional image.
11. The method of claim 2, wherein the two-dimensional image is generated using projection imaging, and wherein the one or more metal objects block the first set of anatomical information of the subject within the two-dimensional image.
10. The method of claim 1, wherein the three-dimensional dataset includes one or more of a CT dataset, a Magnetic Resonance Imaging dataset, a Positron Emission Tomography (PET) dataset, or an Ultrasound dataset.
12. The method of claim 1, wherein the three-dimensional dataset includes a CT dataset, a Magnetic Resonance Imaging dataset, a Positron Emission Tomography (PET) dataset, or an Ultrasound dataset of the subject.
11. The method of claim 1, wherein presenting the improved two-dimensional image comprises presenting the prediction of the first set of blocked anatomical information as an overlay superimposed on the two-dimensional image thereby facilitating the medical procedure.
7. The method of claim 6, wherein presenting the improved two-dimensional image comprises presenting the prediction of the first set of blocked anatomical information as an overlay superimposed on the two-dimensional image thereby facilitating the medical procedure.
12. The method of claim 1, wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device; the prediction of the at least part of the two-dimensional image of the subject using the registered three-dimensional dataset; and generating an improved two-dimensional image are all completed in real-time.
13. The method of claim 14, wherein real-time is less than or equal to 0.1 seconds.
14. The method of claim 1, wherein one or more of: the registering of the three-dimensional dataset of the subject to the imaging space of the image capturing device; the prediction of the at least part of the two-dimensional image of the subject using the registered three-dimensional dataset; and the presentation of the improved two-dimensional image to the user is in real-time.
15. The method of claim 14, wherein the real-time includes a time duration of less than 1 second, 0.8 seconds, 0.6 seconds, 0.5 seconds, 0.4 seconds, 0.3 seconds, 0.2 seconds, 0.1 seconds, 0.08 seconds, 0.06 seconds, 0.05 seconds, 0.02 seconds, or 0.01 seconds.
14. The method of claim 1, wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises registering a first coordinate system of the three-dimensional dataset to a second coordinate system of the imaging space.
16. The method of claim 1, wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises registering a first coordinate system of the three-dimensional dataset to a second coordinate system of the imaging space.
15. The method of claim 1, wherein the three-dimensional dataset is acquired with metal items within a field of view thereof.
16. The method of claim 1, wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises tracking the metal items using infrared light in the imaging space and registering the metal pins in the three-dimensional dataset to the metal pins in the imaging space.
17. The method of claim 1, wherein the three-dimensional dataset is acquired with metal items within a field of view thereof, and wherein registering the three-dimensional dataset of the subject to the imaging space of the image capturing device comprises tracking the metal items using infrared light in the imaging space and registering the metal pins in the three-dimensional dataset to the metal pins in the imaging space.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Fouts et al (Pub No.: 20230149092) discloses method (200) involves receiving (202) a first image data i.e. X-ray image, for capturing an anatomy of interest of a patient and an obstruction i.e. surgical instrument, for obscuring a portion of the anatomy of the interest of the patient. The second image data is generated (204) using a machine learning model in which a portion of the obstruction is replaced. An attribute associated with the anatomy of interest is determined (208) based on the second image data. A visual guidance associated with the anatomy of interest is generated based on a determined attribute, and the guidance is added to the second image data. The second image data is displayed (206) intra-operatively for guiding a surgical procedure.
Finley et al (Pub No.: 20230036038 ) discloses a method comprising segmenting at least one vertebral body from at least one image of a first three-dimensional image data set. The method comprises receiving at least one image of a second three-dimensional image data set. The method comprises registering the segmented at least one vertebral body from the at least one image of the first three-dimensional image data set with the at least one image of the second three-dimensional image data set. The method comprises determining a position of the at least one surgical implant based on the at least one image of the second three-dimensional image data set and a three-dimensional geometric model of the at least one surgical implant. The method comprises overlaying a virtual representation of the at least one surgical implant on the registered and segmented at least one vertebral body from the at least one image of the first three-dimensional image data set.
Govari et al (US Patent No.: 11832883) discloses a system includes a processor and a display. The processor is configured to: (a) receive, from a camera inserted into an eye of a patient, at least an optical image of at least a region-of-interest (ROI) of the eye, (b) receive, from a position tracking system (PTS), a position signal indicative of a position of a medical instrument treating the eye, (c) register the optical image and the PTS in a common coordinate system, and (d) estimate the position of the medical instrument in the optical image. The display is configured to visualize at least the ROI and the medical instrument.
Fronk et al (Pub No.: 20130267838) discloses a system (100) has a computer (150) configured to generate a three-dimensional (3D) model of a structure of a patient using an image. The computer is configured to generate a virtual image of the patient structure from based on the 3D model, and to generate a virtual image of a imaging tool from a 3D model indicative of the shape of a tool. The computer is configured to superimpose the virtual image of the structure of the patient and the virtual image of the tool on the display in correct positions and orientations relative to the patient.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDRAE S ALLISON whose telephone number is (571)270-1052. The examiner can normally be reached on Monday-Friday 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns, can be reached on (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDRAE S ALLISON/Primary Examiner, Art Unit 2673
August 14, 2026
Prosecution Insights