DETAILED ACTION
Claims 1-3, 5-6, 8-9, 12-14, 16 and 18-26 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5-6, 8, 12, 16 and 20-24 are rejected under 35 U.S.C. 103 as being unpatentable over Mueller (US 20150282714) in view of Hallen (US 20190099226).
As per claim 1, Mueller discloses a method to assist in positioning one or more optical detectors of an optical tracking system (Abstract; [0005]), the method comprising:
displaying at least a portion of a view from a visible light detector (Figs. 4-5, #54a) on a display (#40), the visible light detector (#54a) being in a fixed position relative to the one or more optical detectors (#54b-c; [0083]; [0085]); and
displaying a graphic indication (Fig. 5, #58 and Fig. 25, #148) on the displayed view, wherein the graphical indication represents at least a portion of a first field-of-view of the one or more optical detectors ([0083]-[0084]; [0127]).
However, Mueller does not teach the graphic indication is generated by a computer processor.
Hallen discloses the graphic indication (Fig. 4A, #408, 410, 412, 414, 416) is generated by a computer processor ([0056]; where a computer processor is inherently present).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have the graphic indication of Mueller generated according to Hallen so that a variety of graphical overlays can be displayed along with the image.
As per claim 2, Mueller in view of Hallen discloses the method of claim 1, wherein the graphic indication is an outline (Mueller: [0084]; [0127]).
As per claim 5, Mueller in view of Hallen discloses the method of claim 2, wherein the outline on the view comprises a bounded geometrical shape, a semi-translucent shaded region, or a bounded region filled with a gradient pattern (Mueller: [0084]; [0127]).
As per claim 6, Mueller in view of Hallen discloses the method of claim 1, further comprising displaying a marking indicating a center of the portion of the first field-of-view of the one or more optical detectors on the view (Mueller: [0084]; [0127]).
As per claim 8, Mueller in view of Hallen discloses the method of claim 21, further comprising displaying two or more graphic indications representing two or more fields-of-view of the two or more optical detectors on the displayed view (Mueller: [0083]-[0084]),
wherein a first graphic indication represents a field-of-view of a first optical detector and a second graphic indication represents a field-of-view of a second optical detector (Mueller: [0083]-[0084]).
As per claim 12, Mueller in view of Hallen discloses the method of claim 8, wherein each of the two or more graphic indications are differentiated from one another by at least one of: an indicia, a color, or a label (Mueller: [0127]; [0133]).
As per claim 16, Mueller discloses a system (Abstract), comprising:
an optical tracking system comprising one or more optical detectors (Figs. 4-5, #54a-b) having a first field-of-view ([0083]);
a visible light detector (#54c) being in a fixed position with respect to the one or more optical detectors (#54a-b), the visible light detector (#54c) having a second field-of-view ([0083]);
a display (#40) for displaying at least a portion of a view of the visible light detector ([0083]); and
one or more processors (#1006) executing software, wherein the software, when executed by the one or more processors ([0143]; [0173]), is configured to display a graphic indication (Fig. 5, #58 and Fig. 25, #148) on the displayed view, wherein the graphic indication represents at least a portion of the first field-of-view of the one or more optical detectors ([0083]-[0084]; [0127]).
However, Mueller does not teach the graphic indication is generated by the one or more processors.
Hallen discloses the graphic indication (Fig. 4A, #408, 410, 412, 414, 416) is generated by the one or more processors ([0056]; where one or more processors are inherently present).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have the graphic indication of Mueller generated according to Hallen so that a variety of graphical overlays can be displayed along with the image.
As per claim 20, Mueller in view of Hallen discloses the system of claim 16, wherein the software when executed by the one or more processors displays a marking on the displayed visible detector view, wherein the marking indicates a center of the first field-of-view of the one or more optical detectors (Mueller: [0084]; [0172]).
As per claim 21, Mueller in view of Hallen discloses the method of claim 1, wherein the optical tracking system comprises two or more optical detectors (Mueller: [0083]).
As per claim 22, Mueller in view of Hallen discloses the system of claim 16, wherein the graphic indication is displayed as an outline representing the portion of the first field-of-view (Mueller: [0084]; [0127]).
As per claim 23, Mueller in view of Hallen discloses the system of claim 16, wherein the optical tracking system comprises two or more optical detectors (Mueller: [0083]).
As per claim 24, Mueller in view of Hallen discloses the system of claim 16, wherein the optical tracking system comprises the one or more processors and the software (Mueller: [0143]; [0173]).
Claims 3, 13-14, 18-19 and 25-26 are rejected under 35 U.S.C. 103 as being unpatentable over Mueller in view of Hallen in view of Berend (US 20160278868).
As per claims 3 and 18, Mueller in view of Hallen discloses the method (system) of claim 1 (claim 16).
However, the prior art of Mueller and Hallen do not teach the one or more optical detectors are one or more tracking cameras configured to detect infrared light.
Berend teaches the one or more optical detectors are one or more tracking cameras configured to detect infrared light ([0032]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to the have the one or more optical detectors of Mueller in view of Hallen configured according to Berend so as to provide determining three-dimensional location of devices carrying or incorporating markers that serve as tracking indicia (Berend: [0032]).
As per claim 13, Mueller in view of Hallen discloses the method of claim 1.
However, the prior art of Mueller and Hallen do not teach displaying positions of fiducial markers on the displayed view.
Berend teaches displaying positions of fiducial markers on the displayed view ([0021]-[0023]; [0034]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to the have the one or more optical detectors of Mueller in view of Hallen configured according to Berend so as to provide arrays that can assist in navigating surgical instruments into particular locations and orientations within an operating room using kinematic motion data collected from a patient pre-operatively using the visual motion tracking system (Berend: [0021]).
As per claim 14, Mueller in view of Hallen in view of Berend discloses the method of claim 3, wherein the one or more optical detectors is a camera (Mueller: [0083]).
As per claim 19, Mueller in view of Hallen discloses the system of claim 16.
However, the prior art of Mueller and Hallen do not teach a hand-held surgical device or a surgical robot.
Berend teaches a hand-held surgical device or a surgical robot ([0021]; [0026]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to the have the one or more optical detectors of Mueller in view of Hallen configured according to Berend so as to provide arrays that can assist in navigating surgical instruments into particular locations and orientations within an operating room using kinematic motion data collected from a patient pre-operatively using the visual motion tracking system (Berend: [0021]).
As per claim 25, Mueller in view of Hallen discloses the system of claim 16, wherein the visible light detector is a visible light camera configured to detect visible light (Mueller: [0083]).
However, the prior art of Mueller and Hallen do not teach the one or more optical detectors are one or more tracking cameras configured to detect infrared light.
Berend teaches the one or more optical detectors are one or more tracking cameras configured to detect infrared light ([0032]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to the have the one or more optical detectors of Mueller in view of Hallen configured according to Berend so as to provide determining three-dimensional location of devices carrying or incorporating markers that serve as tracking indicia (Berend: [0032]).
As per claim 26, Mueller in view of Hallen discloses the system of claim 16, wherein the visible light detector is a visible light camera configured to detect visible light, and wherein the graphic indication represents a boundary of the at least a portion of the first field-of-view of the two or more tracking cameras (Mueller: [0083]-[0084]; [0127]).
However, the prior art of Mueller and Hallen do not teach the one or more optical detectors are two or more tracking cameras configured to detect infrared light
Berend teaches the one or more optical detectors are two or more tracking cameras configured to detect infrared light ([0032]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to the have the one or more optical detectors of Mueller in view of Hallen configured according to Berend so as to provide determining three-dimensional location of devices carrying or incorporating markers that serve as tracking indicia (Berend: [0032]).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Mueller in view of Hallen in view of NG (US 20170104925).
As per claim 9, Mueller in view of Hallen discloses the method of claim 8.
However, the prior art of Mueller and Hallen do not teach automatically updating a size of each of the two or more graphic indications based on a distance of an object relative to the two or more optical detectors.
NG teaches automatically updating a size of each of the two or more graphic indications based on a distance of an object relative to the two or more optical detectors (Fig. 8; [0046]; [0051]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have included the size decider disclosed by NG to the method of Mueller in view of Hallen so that a two-dimensional (2D) display size of a crosshair can be expanded or contracted with respect to a depth (or Z distance) ([0038]).
Response to Arguments
Applicant's arguments filed 04/08/2026 have been fully considered but they are not persuasive.
The Applicant states on pages 3-4 in the Remarks, “It is respectfully submitted that the secondary and tertiary visible light cameras of Mueller are improperly conflated with the optical tracking detectors as claimed. Support for this position is found in the pending application at paragraphs [0032] and [0005] that explicitly state in relevant
part: [0032]...The optical tracking detectors (18 a, 18 b, 18 c, 18 d) are configured to detect infrared light emitted or reflected from fiducial markers attached to a tracked object. The optical tracking detectors (18 a, 18 b, 18 c, 18 d) may be CCD cameras, CMOS cameras, optical scanners, or other light-sensing devices tuned to detect infrared light by way of a filter, embedded software, or other techniques known in the art. The one or more tracking computers 34 include hardware (e.g., processor(s), non- volatile memory, and/or controllers) and software to detect the POSE of fiducial markers, tracking arrays, and/or objects in 3-D space. Methods of tracking an object with two or more optical detectors and a processor are known in the art, such as the
tracking system described U.S. Pat. No. 6,601,644. [0005] ... The fiducial markers likewise reflect or emit infrared light. Mueller is wholly silent as to cameras, such as those detailed as to reference numbers (54a, 54b, 54c) or detailed in paragraphs [0083] and [0085] being "configured to detect infrared light emitted or reflected from fiducial markers attached to a tracked object." Mueller nowhere discloses camera (54a, 54b, 54c) that detect infrared. In the only contemplation of markers, Mueller states: "In another embodiment of the invention, the imaging system 10 can be used in conjunction with placing the 3D markers on the skin or adjacent thereto. 3D markers for motion capture and the like are known. Accordingly, a description thereof will be omitted." This limited teaching of Mueller still cannot be equated with the IR emission or reflect function of the fiducial markers in the instant application”.
However, it is noted that the features upon which applicant relies (i.e., cameras being configured to detect infrared light emitted or reflected from fiducial markers attached to a tracked object, camera that detect infrared, the IR emission or reflect function of the fiducial markers) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
The Applicant states on page 4, in the Remarks, “Separate from the lack of teachings in Mueller as to the light spectrum detection alone or from fiducial markers attached to the subject, any movement of the cameras (54a, 54b, 54c) per Mueller is rotationally controlled around axes of the subject's head and as such there is no teaching or ability to "assist in positioning one or more optical detectors of an optical tracking system" per claim 1”.
The Examiner does not agree. Applicant’s arguments rely on language solely recited in preamble recitations in claim 1. When reading the preamble in the context of the entire claim, the recitation “A method to assist in positioning one or more optical detectors of an optical tracking system” is not limiting because the body of the claim describes a complete invention and the language recited solely in the preamble does not provide any distinct definition of any of the claimed invention’s limitations. Thus, the preamble of the claim(s) is not considered a limitation and is of no significance to claim construction. See Pitney Bowes, Inc. v. Hewlett-Packard Co., 182 F.3d 1298, 1305, 51 USPQ2d 1161, 1165 (Fed. Cir. 1999). See MPEP § 2111.02.
The Applicant states on page 5 in the Remarks, “It is respectfully submitted that Mueller does not teach or motivate the claim recited display of the graphical indication representative of at least a portion of a first field-of-view of the one or more optical detectors. Instead, the markers 58 of Mueller are provided on a set of monitors 56a- 56c to represent the optimal centered position so that "the surgeon can tell the patient to move their head, left, right, back, forth, etc. until their head is positioned as desired by the surgeon." (Mueller [0083]). Thus, there is no suggestion that the markers 58 of Mueller are in any way related to the field-of-view of the "three cameras or image capturing devices" ([0083]) 54a, 54b, 54c, let alone a marker displayed on a view from one camera that represents a FOV of a different camera”.
The Examiner, respectfully, does not agree. Mueller discloses on paragraph 0083, “FIGS. 4-5 shows the imaging system 10 together with a centering system 52. In a preferred embodiment, centering system 52 includes three cameras or image capturing devices 54a, 54b and 54c positioned such that they are directed toward the point where the object to be imaged is optimally centered. These cameras are positioned to capture the front view (x-axis camera 54a), side or lateral view (y-axis camera 54b) and top view (z-axis camera 54c). In a preferred embodiment, the images from these cameras 54a-54c are communicated to one or more monitors 56a, 56b and 56c where the user of the system 52 can position the object to be filmed as desired. It will be understood that the images can be positioned on a single monitor or on separate monitors. In another embodiment, the images can be shown on monitor 40”.
Mueller discloses on paragraph 0084, “In a preferred embodiment, the three monitors 56a-56c are positioned on the wall and each include circles or markers 58 thereon that represent the optimal centered position. In use, using an example where the patient's head is being imaged, after the patient is seated, the surgeon can tell the patient to move their head, left, right, back, forth, etc. until their head is positioned as desired by the surgeon. This arrangement helps with repeatability between the before and after images”.
Mueller discloses on paragraph 0127, “In a preferred embodiment, the openings 142a, 142b and 142c and monitors 144a, 144b and 144c are positioned such that a patient can look straight ahead (the central or neutral monitor 144b—see FIG. 24A) with their head tilted up (the tilt up or upper monitor 144a—see FIG. 24B) and with their head tilted down (the tilt down or lower monitor 144c—see FIG. 24C). The monitors are used in conjunction with an alignment camera 146 that is positioned above the patient and preferably is co-axial with the pivot axis of the camera 26 and backdrop 20 (rotatable pivot/pivot mechanism 77). The image captured by the alignment camera 146 can be viewed on the upper, center and lower monitors 144a, 144b and 144c. In a preferred embodiment, as shown in FIG. 25, the upper, center and lower monitors 144a, 144b and 144c each include alignment markings 148 thereon that aid with the patient's alignment and centering. For example, a circle 149 for positioning of the patients head and lines 150 for positioning of the patient's shoulders and to aid in centering can be included on the monitors. In use, the circle 149 is used to center the head and the patient can rotate or move their upper body so that their shoulders are aligned with the one or two horizontal lines 150. The markings 148 can be stickers or the like that are placed on the monitor or can be images projected thereon. The positioning of the monitors in conjunction with the alignment camera 146 allow a patient to keep the angle of chin flexing and extension approximately the same in both before and after images”.
Therefore, Mueller discloses the limitation of displaying a graphic indication on the displayed view, wherein the graphical indication represents at least a portion of a first field-of-view of the one or more optical detectors as recited in claim 1 and similarly in claim 16.
The Applicant states on pages 5-6 in the Remarks, “In contrast, the claimed invention of claim 1 advantageously provides a graphical indication of an optical detectors FOV on the displayed view of a single display as captured by the visible light detector to assist the user in positioning either the optical detectors or the patient's anatomy. For example, the visible light detector provides a view of the surgical site that is displayed on the display. A graphic indication representing the FOV of the optical detectors (which is a different detector than the visible light detector) is shown on that same view from the visible light detector. This allows the user to visualize the surgical site on the display (as the viewed from the visible light detector) and then adjust the position of the anatomy or the optical detectors such that the anatomy is within the FOV of the optical detectors (again, which is a different detector than the visible light detector). The user is using the view from one detector to position a different detector with the assistance of the graphic indication. This is unlike Mueller's markers, which are stickers placed on the monitors or images projected thereon (Mueller [0127]) are not an indication of any camera's FOV, and especially not a FOV of a different camera or image capturing devices. Instead, these are markings for aligning a user's head and shoulder relative to a monitor. Thus, a PHOSITA cannot find a teaching or motivation in Mueller for the claimed invention”.
However, it is noted that the features upon which applicant relies (i.e., advantageously provides a graphical indication of an optical detectors FOV on the displayed view of a single display as captured by the visible light detector to assist the user in positioning either the optical detectors or the patient's anatomy, the visible light detector provides a view of the surgical site that is displayed on the display, allows the user to visualize the surgical site on the display (as the viewed from the visible light detector) and then adjust the position of the anatomy or the optical detectors such that the anatomy is within the FOV of the optical detectors, the user is using the view from one detector to position a different detector with the assistance of the graphic indication) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
The Applicant states on page 6 in the Remarks, “It is respectfully submitted that this is a misplaced motivation as the markers of Mueller functional of align the orthogonal views on the three separate monitors and thus, the markers 58 are of value only as to assuring the triaxial alignment of the subject in the views of the three visible light cameras. As a result, even if such markers 58 are rendered as digital instead of ink or other markings on the displays, these would still fail to contemplate or address the problem solved by the claimed invention. It is further submitted that any combination of Mueller and Hallen is based on improper hindsight with the only motivation coming from the claimed invention.
The Examiner does not agree and has not applied the prior art of Mueller and Hallen in the manner as argued by the Applicant.
As applied in the Non-Final and Instant action, Mueller discloses the limitation of
displaying a graphic indication (see Mueller, Fig. 5, #58 and Fig. 25, #148) on the displayed view, wherein the graphical indication represents at least a portion of a first field-of-view of the one or more optical detectors (see Mueller paragraphs 0083-0084 and 0127).
The prior art of Hallen discloses the graphic indication (see Hallen, Figure. 4A, #408, 410, 412, 414, 416) is generated by a computer processor (see Hallen, paragraph 0056; where a computer processor is inherently present).
Hallen explicitly discloses on paragraph 0056, “Also, a variety of graphical overlays 408, 410, 412, 414, 416 can be displayed along with the image of the eye”.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have the graphic indication of Mueller generated according to Hallen so that a variety of graphical overlays can be displayed along with the image.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nelson Lam whose telephone number is (571)272-8044. The examiner can normally be reached 1pm-9pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ke Xiao can be reached on 571 272-7776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Nelson Lam/Examiner, Art Unit 2627
/KE XIAO/Supervisory Patent Examiner, Art Unit 2627
Prosecution Insights