Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendments filed 04/20/2026 have been entered. Claims 1-20 remain pending in the application. Applicant’s amendment to the claims have overcome the 35 USC 112(b) rejections previously set forth in the Non-final Office Action mailed 01/20/2026. However, new 35 USC 112(b) rejections have been entered as necessitated by amendment. Applicant’s amendments to the claims have not overcome the previously set forth 35 USC 103 rejections for the reasons described below.
Response to Arguments
Applicant's arguments filed 04/20/2026 have been fully considered but they are not persuasive.
Applicant states on pages 13“Applicant respectfully submits that Liu does not disclose the registration mechanism (transform determination) in amended claims. Liu describes calculating transformation matrices across multiple spaces (e.g. pre-operative image space, patient space, wearable imaging space, handheld probe space) and performing co-registration.”
Applicant further states “While Liu describes registration via patient space and related transformation steps, Applicant respectfully submits that Liu does not disclose the above transmitter attached marker premise and the transform determination/registration mechanism between the first tracking system frame and the HMD frame as defined in the claims”. Examiner respectfully disagrees.
At worst, Liu suggests the transmitter attached marker premise, however the teachings outlined in the updated 103 rejection clearly show Liu describes the fiducial markers as attached to the system, marked as 100, which is capable of transmitting and receiving data, which is definitively a transmitter. Additionally, if Liu were to only suggest the transmitter attached marker premise this would still mean Liu teaches the premise, and the claim would remain rejected under 35 U.S.C. 103. Applicant’s argument regarding the transform determination/registration mechanism describes the process “including:
using a marker attached to a transmitter included in the first (magnetic) tracking system;
the marker having a pose that is known or determinable relative to the first tracking system frame;
determining the marker pose relative to the HMD frame based on detection by the second tracking system; and
determining a transform between the first tracking system frame and the HMD frame based on the two marker poses, and registering the frames according to the transform.”
Examiner does not find these arguments persuasive for the reasons outlined in the current and previous rejections. Additionally, applicant argues “Liu describes calculating transformation matrices across multiple space (e.g. pre-operative image space, patient image space, wearable imaging space, handheld probe space) and performing co-registration”, but applicant fails to describe how these differ from that of the claimed invention. Examiner agrees that Liu describes calculating transformation matrices across multiple spaces and performs co-registration, but it is unclear how this differs from the claimed mechanism.
Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references. Specifically, applicant describes Liu, Ben-Tzvi and Foruntanpur as failing to teach the transform determination and registration mechanism without providing arguments against the citations of the references. At the same time, applicant admits “Liu describes registration via patient space and related transformation steps” (page 14), and “Liu describes calculating transformation matrices across multiple spaces ([…]) and performing co-registration” which seemingly contradicts the argument.
Applicant further argues “there is no teaching or suggestion in the cited references to arrive at the claimed series of features requiring:
a marker attached to the transmitter of the magnetic system;
a known/determinable marker pose relative to the first tracking system frame;
determination of the marker pose in the HMD frame based on detection by the optical tracking system; and
determination of a transform and registration of the first tracking system frame and the HMD frame based on the two marker poses”
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., two marker poses) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Claim Objections
Claim 1 objected to because of the following informalities: there are duplicate limitations. Lines 31-32 and 33-34 . Appropriate correction is required.
Claim 1 objected to because of the following informalities: “determine, […], of a position and orientation”. Examiner is interpreting the claims as “determine, […], a position and orientation” because the “at least one” limitation is removed and examiner is assuming Applicant intended to also remove the following “of”. Appropriate correction is required.
Claim 3 objected to because of the following informalities: “visible light [[or]] infrared light or other non-visible light” is missing a comma to separate the visible light and infrared light limitations. As currently recited the claim describes “visible light infrared light” rather than “visible light, infrared light”. Appropriate correction is required.
Applicant is advised that should claim 1 be found allowable, claim 3 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. While claim 3 recites an additional limitation “infrared light” this is included in the previously recited “non-visible light” of claim 1. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Applicant is advised that should claim 1 be found allowable, claim 11 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 5 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. A broad range or limitation together with a narrow range or limitation that falls within the broad range or limitation (in the same claim) may be considered indefinite if the resulting claim does not clearly set forth the metes and bounds of the patent protection desired. See MPEP § 2173.05(c). In the present instance, claim 5 recites “and/or” which defines a broad recitation “the determined position or orientation of the patient”, and the claim also recites "the determined position and orientation of the patient" which is the narrower statement of the range/limitation. The claim(s) are considered indefinite because there is a question or doubt as to whether the feature introduced by such narrower language is (a) merely exemplary of the remainder of the claim, and therefore not required, or (b) a required feature of the claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-13, 15-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu (US 2017/0202633 A1) in view of Ben-Tzvi (US 2018/0081027 A1) and Foruntapour (US 2015/0145887 A1).
Regarding claim 1 Liu teaches A medical imaging system comprising:
a head mounted display device configured to be worn by a user, the head mounted display device comprising a display device configured to display to the user (paragraph [0057], figure 1) either:
(i) a view through the head mounted display device in which the user is able to visually recognize a patient, or
(ii) a captured image of the patient captured by a camera included in the head mounted display device (paragraphs [0057], [0059] fig. 1);
a first tracking system comprising a magnetic tracking system configured to use a magnetic field to determine a position and orientation of a medical device operated by the user (paragraphs [0017], [0018], [0036], [0080], [0082], [0086]);
a second tracking system comprising an optical tracking system configured to use visible light or non-visible light to determine a position and orientation of the head mount display device (paragraphs [0017], [0018], [0036], [0080], [0082], [0087], fig. 5);
at least one marker attached to a transmitter included in the first tracking system (paragraphs [0073] – “In one aspect, the communication interface 250 is embodied as a transceiver that is enabled to both transmit and receive data via the network 260.”, [0080] – “In some embodiments, the system 100 includes a tracking module, which can be considered another peripheral 350, and includes software suitable for tracking the spatial location of the patient, location of the detector 120 ( or 120A, 120B) and the location of peripherals 350, such as imaging cameras and probes, and registering these locations relative to the image(s) of the detector 120 or detectors 120A, 120B (in stereoscopic modalities)” [0082] – “If optical tracking is used, active markers such as LEDs may be attached to detector 120, the imaging or sensing probe employed as another peripheral 350 and the patients, to locate their locations, respectively.”, paragraph [0100] – “Thus, in one embodiment, information acquired by each system 100A-X can also be transmitted or received wirelessly,” The markers may be attached to detector 120, this detector 120 included in the system 100 which can be transmitted. In other words the markers are attached to 120, a part of 100, which is a transmitter), a position and orientation of the at least one marker being detectable by the second tracking system (paragraph [0087] – “With the position obtained using the tracking techniques described, enabled by tracking systems as a peripheral 350, registration, or alignment, of the different images obtained by the imaging and display device 100 and the handheld imaging peripheral 350 is performed by using transformation matrices between preoperative imaging space, intraoperative object space (i.e patient space), intraoperative imaging space ( device 100 imaging space), and the handheld peripheral imaging/sensing probe 350 space (i.e. peripheral imaging probe space) can be calculated”, paragraph [0088] – “Through the use of fiducial markers, images of the same subject produced with multiple distinct imaging systems-for example, the detector 120 as a first imaging system, and any desired peripheral 350 that generates a second image as the second imaging system may be correlated by placing fiducial markers in the area imaged by both systems”), wherein the at least one marker has a known or determinable position and orientation relative to a frame of reference of the first tracking system (paragraphs [0080] – “the corresponding imaging and sensing information obtained from the peripheral can be correlated with the field of view imaged by the detector”, [0082] – “Alternatively, one may use a videometric system to estimate patient pose and instrument orientation by identification of passive markers in video-image sequences”, [0087]-[0089])
processing circuitry configured to:
determine a transform of coordinates between the frame of reference of the first tracking system and the frame of reference of the head mounted display device (paragraphs [0080], [0087] – “registration, or alignment, of the different images obtained by the imaging and display device 100 and the handheld imaging peripheral 350 is performed by using transformation matrices between preoperative imaging space, intraoperative object space (i.e patient space), intraoperative imaging space ( device 100 imaging space), and the handheld peripheral imaging/sensing probe 350 space (i.e. peripheral imaging probe space) can be calculated”, [0089], [0090]), based on the position and orientation of the at least one marker relative to the frame of reference of the first tracking system (paragraphs [0080] – “In some embodiments, the system 100 includes a tracking module, which can be considered another peripheral 350, and includes software suitable for tracking the spatial location of the patient, location of the detector 120 ( or 120A, 120B) and the location of peripherals 350, such as imaging cameras and probes, and registering these locations relative to the image(s) of the detector 120 or detectors 120A, 120B (in stereoscopic modalities).”, paragraph [0082] – “Furthermore, the tracking functions may be carried out using optical tracking or magnetic tracking devices that are employed as a peripheral 350. If optical tracking is used, active markers such as LEDs may be attached to detector 120, the imaging or sensing probe employed as another peripheral 350 and the patients, to locate their locations, respectively […] Alternatively, one may use a videometric system to estimate patient pose and instrument orientation by identification of passive markers in video-image sequences” [0089] – “At step 624 a transformation matrix is calculated between the hand-held imaging/sensing probe 350 image space and the intra-operative object space (i.e. patient space).”, the registration process described in Liu includes the calculation of a transformation matrix based on the relationship between imaging spaces, these imaging spaces include the described tracking peripherals which in some embodiments are markers. Additionally, the markers are utilized to track a patient to be treated. In this manner it would have been obvious to one of ordinary skill in the art to further specify the application of registration to be between the markers and the tracking system, in the same way as is done with the registration of the peripherals relative to the detectors relative to peripherals, both of which may include markers and include tracking systems) and the position and orientation of the at least one marker relative to the frame of reference of the head mounted display device (paragraphs [0082], [0087] – “With the position obtained using the tracking techniques described, enabled by tracking systems as a peripheral 350, registration, or alignment, of the different images obtained by the imaging and display device 100 and the handheld imaging peripheral 350 is performed by using transformation matrices”, [0090]-[0091] In some embodiments, tracking systems determine position based on markers. Additionally, Liu suggests the estimation of patient pose and instrument orientation. Because the markers can be attached to both instruments and patients, and markers are specifically described as being used for this estimation, this is analogous to determining marker orientation. Furthermore, paragraphs [0089]-[0091] describe various embodiments which register patient or peripheral data relative to the imaging space (analogous to HMD), and is further used to calculate a transformation matric);
register the frame of reference of the first tracking system and the frame of reference of the head mount display device in accordance with the transform (paragraph [0080], [0087]-[0091], [0099]); and
display, using the head mounted display device, at least one of:
(a) a representation of the medical device aligned relative to [[a]] the view or the captured image; and
(b) a medical image and a representation of the medical device, wherein the medical image and the representation of the medical device are aligned with each other (paragraph [0080], [0087]).
Liu fails to explicitly teach determine, based on the position and orientation of the at least one marker detected by the second tracking system, a position and orientation of the least at one marker relative to a frame of reference of the head mounted display device;
However, Ben-Tzvi teaches determine at least one of a position and orientation of the at least one marker relative to the frame of reference of first tracking system (paragraph [0025], claim 6). Ben-Tzvi describes a system for tracking motion; this system specifically utilizes a marker to track orientation of the marker relative to a frame of reference of a tracker module. Ben-Tzvi is considered analogous to the claimed invention as it is in the same field of tracking systems and computer graphics. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Ben-Tzvi with the teachings of Liu in order to improve tracking the medical devices of Liu in relation to other system components.
Liu in view of Ben-Tzvi fails to explicitly teach
and determine the position and orientation of the at one marker relative to a frame of reference of the head mount display device.
However, Forutanpour teaches and determine the position and orientation of the at one marker relative to a frame of reference of the head mount display device (paragraph [0003]). Forutanpour describes determining the marker’s location relative to the location and orientation of the head mounted display. Forutanpour is considered analogous to the claimed invention as it is in the same field of head mounted displays. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Forutanpour with Liu in view of Ben-Tzvi to implement the determination of relative marker location or orientation with the head-mounted display and improve the presentation of images on the head mounted display.
Additionally, it would have been obvious to one of ordinary skill in the art to incorporate the teachings of Forutanpour and Ben-Tzvi with Liu with respect to registering the frame of reference using the at least one marker, and to do so with the position or orientation of the marker relative to the frame of reference of the tracking system and head mounted display for the same motivations previously described.
Liu in view of Ben-Tzvi and Foruntanpour fails to explicitly teach at least one marker attached to a transmitter included in the first tracking system
Method claim 20 is drawn to the method of using the corresponding apparatus claimed in claim 1. Therefore, the method claim 20 corresponds to apparatus claim 1 and is rejected for the same reasons of obviousness as used above.
Regarding claim 2, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches wherein the medical image and the representation of the medical device are overlaid on a view or the captured image (paragraphs [0058], [0080], Figure 4).
Regarding claim 3, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches wherein the second tracking system comprises an optical tracking system using visible light, infrared light or other non-visible light (paragraph [0082]).
Regarding claim 4, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches wherein the head mounted display device comprises at least one camera configured to determine a position and/or orientation of the patient (paragraphs [0061], [0080], Figure 1 #120A-B, Figure 5 #606, 614).
Regarding claim 5, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 4. Liu further teaches wherein the determined position and/or orientation of the patient are stored with image data obtained by the system (paragraphs [0064],[0074], [0080]-[0082]).
Regarding claim 6, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 4. Liu further teaches wherein the processing circuitry is further configured to use the head mounted display device to display an image from a prior scan aligned with the view or the captured image (paragraph [0087]). The pre-operative tomography images described in [0087] are a type of prior scan aligned with the image of the patient.
Regarding claim 7, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 4. Liu further teaches wherein the processing circuitry is further configured to use the head mounted display device to display guides (paragraph [0087]) to guide positioning of a measurement apparatus relative to the patient to align the measurement apparatus with a position of the measurement apparatus used to obtain a prior scan (paragraph [0087], [0088], figure 5 #622).
Regarding claim 8, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches wherein the first tracking system is configured to determine a position and orientation of a measurement apparatus of the system (paragraph [0080], [0082]) and the processing circuitry is configured to display a representation of the measurement apparatus using the head mounted display device (paragraph [0057]), wherein at least one of:
(a) the medical image, the representation of the medical device and the representation of the measurement apparatus are aligned based on the registration; or
(b) the representation of the medical device and the representation of the measurement apparatus are aligned relative to the view or the captured image based on the registration (paragraphs [0080], [0087], [0099]).
Regarding claim 9, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 7. Liu further teaches wherein the system comprises an ultrasound imaging system and the measurement apparatus comprises a measurement probe (paragraph [0089]).
Regarding claim 10, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches wherein the head mounted display device comprises an augmented reality (AR) or virtual reality (VR) headset or other wearable head set (paragraph [0057]).
Regarding claim 11, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches wherein the first tracking system comprises a magnetic tracking system configured to generate the magnetic field and to determine the position and orientation of the medical device using the magnetic field (paragraph [0080], [0082], [0086]).
Regarding claim 12, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches wherein the first tracking system comprises acoustic tracking; (paragraph [0092], Figure 2)
Regarding claim 13, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches, wherein the medical imaging system further comprises a screen configured to display the medical image (paragraph [0139]) and the processing circuitry is configured to display a representation of the screen using the head mounted display device (paragraphs [0008], [0057], Figure 1).
Regarding claim 15, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches wherein the first tracking system comprises a first position sensor, the first position sensor being part of or comprised in the medical device (Paragraph [0087], [0089], Figure 5 #622). Liu describes markers attached to the imaging peripheral, which in multiple embodiments is a medical device, and has its position tracked via these markers. These markers can therefore be considered position sensors and they are attached to the device, therefore part of the device.
Regarding claim 16, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 9. Liu further teaches wherein the first tracking system comprises a second position sensor, the second position sensor being part of or comprised in the measurement apparatus (Paragraph [0087], [0089], Figure 2 #350, Figure 5 #616). Liu describes markers attached to the imaging peripheral, which in multiple embodiments is a measurement apparatus such as a vital sign module, GPS module or tracking module. The peripherals have their position tracked via these markers, which are attached, or a part of, the apparatus.
Regarding claim 17, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches wherein the system comprises a position detector configured to detect movement of a part of first tracking system relative to the at least one marker (Paragraphs [0018], [0089], Figure 5 #606).
Regarding claim 18, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 17. Liu further teaches wherein when movement of the part of the first tracking system has been detected, the processing circuitry is configured to perform: updating the registration based on a new position of the part of the first tracking system (Figure 5 #606, paragraph [0018]).
Regarding claim 19 Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches wherein the medical device comprises at least one of: a catheter, and an endoscopic device (paragraph [0145]).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Kozak (DE 202012100230 U1).
Regarding claim 14, Liu in view of Ben-Tzvi and Forutanpour teach the system according to claim 1. Liu further teaches wherein the processing circuitry is configured to display at least one of the medical image and the representation of the medical device using the head mounted display device (paragraph [0057]).
Liu in view of Ben-Tzvi and Forutanpour fails to teach said displaying when the user faces the medical device.
However, Kozak teaches to display at least one of the medical image and the representation of the medical device using the display device when the user faces the medical device (Description, Paragraph [0009], “For example, a user-oriented representation of the ultrasound image can thereby be achieved, which is stationary relative to the operator when the orientation of the display device changes in space.”). Kozak is considered analogous to the claimed invention as it is in the same field of medical device display methods.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing data, to incorporate the user-oriented representation of a medical imaging device with the head mounted system of Liu.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
(US 8353818 B1) describes a system involving a “endoscope tracking transmitter” and “a plurality of anatomy mapping transmitters” which are used to map an endoscopic image to a patients anatomy.
C. M. Andrews, et al., "Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices," in IEEE Journal of Translational Engineering in Health and Medicine, vol. 9, pp. 1-14, 2021, Art no. 4900214, doi: 10.1109/JTEHM.2020.3045642
C. Leuze, et al., "Landmark-based mixed-reality perceptual alignment of medical imaging data and accuracy validation in living subjects," 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Porto de Galinhas, Brazil, pp. 659-664, doi:10.1109/ISMAR50242.2020.00095
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Aidan W McCoy whose telephone number is (571)272-5935. The examiner can normally be reached 8:00 AM-5:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571)272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AIDAN W MCCOY/Examiner, Art Unit 2611
/TAMMY GODDARD/Supervisory Patent Examiner, Art Unit 2611
