Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 7 and 17 are cancelled, and claims 1-6, 8-16, and 18-20 remain pending in the application in response to the applicant’s amendments to the rejections previously set forth in the Non-Final Office Action mailed 04/08/2026.
Response to Arguments
Applicant’s arguments filed 06/17/2026 with respect to claim(s) 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Given the amendments to claim 1, reference to Heeren is being relied upon to teach dependent claims 4-6, 8-11, and 14-19 more-consistently with the instant claim language, as shown below.
Given the amendments to claim 1, reference to Adebar is being relied upon to teach dependent claim 2 more-consistently with the instant claim language, as shown below.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 3-6, 8-16, and 18-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated Heeren (US 20170280989 A1, published October 5, 2017), hereinafter referred to as Heeren.
Regarding claim 1, and similarly for claims 12 and 20, Heeren teaches a medical system (Fig. 1) comprising:
a first imaging system (Fig. 1; see para. 0038 – “Surgical microscope 100 includes integrated OCT and display systems [first imaging system]. Surgical microscope 100 may facilitate magnified viewing of a patient's eye 102 during a surgical procedure…”); and
control circuitry (see para. 0062 – “Processor 154 [control circuitry] may work, either alone or with other components depicted in FIG. 1, to provide the functionality described herein.”) configured to:
generate a graphical interface depicting a first spatial relationship between an instrument and a target within an anatomy (Fig. 4A as first spatial relationship between instrument and target; see para. 0089 – “Certain embodiments of tracking unit 144 include a user interface to receive user input regarding customized settings defining when, where, and how the visual indicator will be displayed and modified.”; see para. 0093 – “In FIG. 4A, where distal tip 149 of surgical instrument 146 is a relatively large distance from retina 103 [target within anatomy (eye)], indicator 300 appears as nearly transparent.”);
receive first image data depicting the anatomy having the instrument disposed therein, the first image data captured by the first imaging system while positioned external to the anatomy (Fig. 1, imaging system positioned external to anatomy; see para. 0104 – “In certain embodiments, tracking unit 144 receives image data from OCT system 114.”; see para. 0090 – “FIGS. 3 and 4 further illustrate, on the right, corresponding microscope images [first image data] displayed to a system operator via eyepieces 104 (with input from real-time data projection unit 116) and/or display 132. The microscope images show fundus 105, surgical instrument 146 (which may include marker 147, not shown), and indicator 300 generated by tracking unit 144.”);
display an instrument indicator overlaid on one or more images depicting the anatomy with the instrument disposed therein based on the first image data (see para. 0093 – “In FIG. 4A, where distal tip 149 of surgical instrument 146 is a relatively large distance from retina 103 [target within anatomy (eye)], indicator 300 [displayed instrument indicator] appears as nearly transparent.”);
receive user input for repositioning the instrument indicator over a distal end of the instrument in the one or more images (see para. 0015 – “The visual indicator [instrument indicator] may also be configurable [repositioning] by a user [user input].”; see para. 0089 – “Various embodiments of tracking unit 144 may allow a user to configure the appearance, characteristics, and behavior of the indicator. For example, a user may configure a particular size and shape for an indicator, and may configure how the indicator is modified to indicate distance. Certain embodiments of tracking unit 144 include a user interface to receive user input regarding customized settings defining when, where, and how the visual indicator will be displayed and modified.”);
determine a second spatial relationship between the instrument and the target within the anatomy based at least in part on the repositioned instrument indicator (Fig. 4B as second spatial relationship between instrument and target; see para. 0092 – “As with FIG. 3, FIG. 4 shows that indicator 300 (here, a circle) becomes relatively smaller as distal tip 149 approaches retina 103 in FIGS. 4B and 4C, and indicator 300 is maintained as an overlay even as distal tip 149 moves within the microscope images on the right side of the figure.”); and
update the graphical interface to depict the second spatial relationship between the instrument and the target (see para .0081 – “Additionally, the indicator may be modified or adjusted [updated graphical interface] as the distance between distal tip 149 and the retina of eye 102 changes.”).
Furthermore, regarding claims 3 and 13, Herren further teaches wherein the determining of the second spatial relationship comprises: displaying one or more images of the anatomy based on the first image data; and receiving user input indicating at least one of a position of the target or a position of the instrument in the one or more images (Fig. 4B; see para. 0089 – “Various embodiments of tracking unit 144 may allow a user to configure the appearance, characteristics, and behavior of the indicator. For example, a user may configure a particular size and shape for an indicator, and may configure how the indicator is modified to indicate distance. Certain embodiments of tracking unit 144 include a user interface to receive user input regarding customized settings defining when, where, and how the visual indicator will be displayed and modified.”).
Furthermore, regarding claims 4 and 14, Heeren further teaches wherein the determining of the second spatial relationship comprises: estimating at least one of a position of the target or a position of the instrument based on the first image data (see para. 0103 – “At step 512, tracking unit 144 may determine the position and orientation of distal tip 149 of surgical instrument 146 within an image frame.”).
Furthermore, regarding claims 5 and 15, Heeren further teaches wherein the generating of the graphical interface comprises:
receiving second image data depicting a position of the target within the anatomy (see para. 0069 – “Tracking unit 144 may use various techniques to determine and track the location of surgical instrument 146 within a microscope image (e.g., the X-Y position of distal tip 149 within a microscope image).”);
receiving sensor data from a sensor associated with the instrument, the sensor data indicating a position of the instrument within the anatomy (see para. 0069 – “In certain embodiments, surgical instrument 146 may have attached or embedded sensing devices. For example, surgical instrument 146 may have one or more gyroscopes, accelerometers, gravity sensors, linear acceleration sensors, rotation vector sensors, geomagnetic field sensors, or other types of sensors, to sense changes in position, location, or movement.”); and
determining the first spatial relationship based on the second image data and the sensor data (see para. 0069 – “Data generated from such sensors may be provided to tracking unit 144, which may analyze the data to determine the location, position, and/or movement of surgical instrument 146.”).
Furthermore, regarding claims 6 and 16, Heeren further teaches wherein the second image data is captured by a second imaging system while the instrument is not disposed within the anatomy (see para. 0050 – “The present disclosure contemplates that, although not depicted, certain embodiments may include one or more additional or alternative depth-imaging systems, such as an ultrasound imaging system, a multispectral imaging system, a computerized axial tomography (CAT) scan system, a magnetic resonance imaging (MRI) system, or a positron emission tomography (PET) imaging system.” Inherent and known in the art to image the human body with an external imaging device both with and without an instrument disposed within the anatomy).
Furthermore, regarding claims 8 and 18, Heeren further teaches wherein the control circuitry is further configured to: estimate a pose of the instrument based on the first image data, the instrument indicator overlaid on the one or more images based on the estimated pose of the instrument (see para. 0079 – “Processor 154 may track the location, orientation, and depth/proximity of distal tip 149 in order to provide a dynamic indicated updated in real time. Accordingly, the indicator may assist a surgeon by providing an accurate, real-time indication of the distance between distal tip 149 and the retina, which may be difficult to precisely discern from a stereo microscope image.”).
Furthermore, regarding claims 9 and 19, Heeren further teaches wherein the user input includes an adjustment to a position or orientation of the instrument indicator relative to the one or more images (see para. 0015 – “The visual indicator may also be configurable by a user.”; see para. 0081 – “Additionally, the indicator may be modified or adjusted as the distance between distal tip 149 and the retina of eye 102 changes.”).
Furthermore, regarding claim 10, Heeren further teaches wherein the user input includes an indication the instrument indicator is aligned with the distal end of the instrument (see para. 0015 – “The visual indicator may also be configurable by a user.”; see para. 0081 – “Additionally, the indicator may be modified or adjusted as the distance between distal tip 149 and the retina of eye 102 changes.”).
Furthermore, regarding claim 11, Heeren further teaches wherein the first spatial relationship is represented by a depiction of the instrument in a first pose relative to a position of the target (Fig. 4A, instrument 146 in first pose relative to retina 103 (position of target)) and the second spatial relationship is represented by a depiction of the instrument in a second pose relative to the position of the target (Fig. 4B, instrument 146 in second pose relative to retina 103 (position of target)).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Heeren in view of Adebar et al. (US 20240050160 A1, published February 15, 2024), hereinafter referred to as Adebar.
Regarding claim 2, Heeren teaches all of the elements disclosed in claim 1 above.
Heeren teaches an imaging system, but does not explicitly teach where the imaging system is a cone beam computed tomography (CBCT) imaging system.
Whereas, Adebar, in an analogous field of endeavor, teaches wherein the first imaging system is a cone beam computed tomography (CBCT) imaging system (see para. 0072 – “In some embodiments, the imaging system 718 includes a mobile C-arm cone-beam CT imaging system…”).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified an imaging system, as disclosed in Heeren, by having the imaging system as a cone beam computed tomography (CBCT) imaging system, as disclosed in Adebar. One of ordinary skill in the art would have been motivated to make this modification in order to generate 3D images, as taught in Adebar (see para. 0072).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Charron et al. (US 20180228555 A1, published August 16, 2018) discloses analyze the image frame to determine the location in the image frame of the shaft of the first surgical instrument, process the image frame to overlay received overlay information on the shaft of the first surgical instrument as it appears in the image frame, and display the processed image frame on the display device.
Weir (US 20170189131 A1, published July 6, 2017) discloses the movement of the first and second graspers is shown in real time on the display, and the first, second, and central markers, the connecting line, and the cursor can move on the display in real time with the movement of the selected instruments, which may help the user move the selected instruments in a desired way and make any needed movement corrections in real time with the selected instruments' movement.
Ren et al. (US 20150173644 A1, published June 25, 2015) discloses generating indicators indicating surgical data, such as a position of a surgical tool, an orientation of a surgical tool, an image, a surgical setting parameter; overlaying the indicators on the captured images or a processed image; and displaying the captured image or a processed image with the overlaid indicators on a display.
Gliner et al. (US 20180146884 A1, published May 31, 2018) discloses the processor further overlays, on modified image slice, an icon that represents intrabody tool (in particular, the distal end thereof) on a portion of modified image slice that corresponds to the location of the intrabody tool within the anatomical cavity.
Zhang et al. (US 20230360212 A1, published November 9, 2023) discloses the control system may populate the view windows of the graphical user interface with the respective instrument icons and the user may be asked only to confirm that the identified location of the distal tip of the instrument is accurate.
Weeks et al. (US 20220331014 A1, published October 20, 2022) discloses with several markings identified and/or overlaid upon the interface, the system may continuously track and update such markers as additional images are captured by the endoscope, and may display subsequent images and updated markings as the positions of objects within the image change.
Hufford et al. (US 20210256719 A1, published August 19, 2018) discloses graphical markings may be generated and overlayed on the images of the instrument tips to give an enhanced visual of the points between which the measurements are being taken.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nyrobi Celestine whose telephone number is 571-272-0129. The examiner can normally be reached on Monday - Thursday, 7:00AM - 5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached on 571-272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.C./Examiner, Art Unit 3798
/PASCAL M BUI PHO/Supervisory Patent Examiner, Art Unit 3798
Prosecution Insights