Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claims 1-52 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-35 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Ryan et al. (US 20220168051 A1).
Regarding claim 1, Ryan et al. discloses a method for referencing anatomical structure using an AR headset (e.g., headset 3600, figure 36A worn by surgeon 4208), comprising (self-contained surgical navigation systems which include a head-worn display device to be worn by a user during surgery, Abstract, novel visualization and sensory augmentation devices - for positioning, localization, and situational awareness during medical procedures including, but not limited to, surgical, diagnostic, therapeutic, and anesthetic procedures, para 0002, para 0004-0014, para 0017- 0020, head-worn surgical navigation system for determining a joint center - one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104. For the purpose of this specification, "pose" is defined as the combination of position and orientation of an object, paras. 0243-0246, para 0267-0270, para 0307-0311, augmented reality headset, para 0368-0375):
registering a marker located on a movable anatomical structure, wherein the marker has a first pose that includes a position and orientation in a 3D coordinate system (regions indicate captured and valid depth reference points, para. 0077) of the AR headset (one reference marker 4210 affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; reference marker 4212 affixed to a tibia with pins create a femoral coordinate frame based on the registered points, paras. 0050-0060, registering a condylar surface before setting a resection angle, such that the method is performed by any of the head-worn surgical navigation systems described herein. Performed by a processor unit and displays, on a display of the head-worn surgical navigation system, a target of one or more regions; providing, on the display, a movable icon that represents one or more angles received from a condylar guide in real-time, para. 0058),
identifying the marker at a second pose of the movable anatomical structure having a second position and second orientation (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para 0050-0060, algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]); and
determining a joint pivot axis (distraction paddles 4706 and 4707 are pushed by the spring 4902 and pivot about an anteroposterior axis to provide a nearly equal and constant distraction force between each femoral condyle (4708, 4712) and the tibia 4704. The base element 4702 and distraction paddles (4706, 4704) include optical markers (4714, 4716) which allow the software to measure the degree of distraction of each femoral condyle (4708, 4712), para. 0318)
and an angle by comparing the first pose and the second pose (para 0050-0060, The system 10 can then establish a reference frame 5016 for the femur 5002 relative to the sensor suite 210 (5906). Then, repeatedly scanning the exposed lesser trochanter 5010, the system 10 generates a displaced 3-dimensional surface map 5018 for each scan (5908). With each successive scan, the system can compare the displaced surface map 5018 to the reference surface map 5014 for the same region on the lesser trochanter 5010. Based on this comparison, the system 10 can track the pose of the femur 5002 relative to sensor suite 210 by determining the translation and rotation required to best fit the displaced surface map 5018 with the reference surface map 5014 (5910), para [0267]-[0270], para [0307]-[0311], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]).
Regarding claim 2, Ryan et al. discloses the method as in claim 1, further comprising displaying the joint pivot axis, using the AR headset (displaying a mixed reality user interface comprising stereoscopic virtual images of the features of the surgical tool and the features of the anatomical object in the user's field of view, para. 0004-0014; see also para. 0217).
Regarding claim 3, Ryan et al. discloses the method as in claim 1, further comprising: registering a second marker having a third position and third orientation on a proximal anatomical structure at a second pose; and computing the first pose and second pose with respect to the proximal anatomical structure using the second marker to enable free movement of the movable anatomical structure and the proximal anatomical structure (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060], algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, paras. 0243-0246, augmented reality headset, paras. [0368]-[0375]).
.
Regarding claim 4, Ryan et al. discloses the method as in claim 1, further comprising displaying a plurality of points surrounding the joint pivot axis that are isometric to the joint pivot axis (markers (e.g., 100, 108, 110, etc.) for anatomic landmarks and tools are used for data collection (1000), which may be combined with pre-operative CT scan or MRI data for the determination of position and orientation (1002) of isometric points for ligament reconstruction and surgical tools, paras. 0355-0358).
Regarding claim 5, Ryan et al. discloses the method as in claim 4, wherein the plurality of points forms at least one of: an arc, a circle (virtual circle 1908, circle 4306), a cylinder, a curved surface (When each point is registered, a virtual marker, such as a small sphere, may be positioned and remain at the location of the tip at the time of registration and beyond to provide the user 106 a visual confirmation to the user 106 and check on the quality of the registration, para. 0247), or an irregular shape.
Regarding claim 6, Ryan et al. discloses the method as in claim 1, further comprising displaying a plurality of isometric points surrounding the joint pivot axis based on a defined ligament length to identify locations on a bone where a ligament is affixable at isometric distances to the joint pivot axis, using the AR headset (markers (e.g., 100, 108, 110, etc.) for anatomic landmarks and tools are used for data collection (1000), which may be combined with pre-operative CT scan or MRI data for the determination of position and orientation (1002) of isometric points for ligament reconstruction and surgical tools, paras. 0355-0358, augmented reality headset, paras. 0368-0375).
Regarding claim 7, Ryan et al. discloses the method as in claim 1, measuring a length of a bone from the joint pivot axis to a point defined on the bone (measurements of physical parameters from pre- to post-operative states can be presented, including, but not limited to, change in overall leg length - The change in leg length can also be calculated at this point in the procedure using the marker position and orientation of the replaced femur (918), para [0237]-[0247], para [0290]-[0293]).
.
Regarding claim 8, Ryan et al. discloses the method as in claim 1, further comprising:
calculating a change in the angle between the first pose and the second pose; and displaying a numerical output for the angle defining an angular change based in part on movement of the marker around the joint pivot axis (comparing the displaced 3-dimensional surface map 5018 to the reference 3-dimensional surface map 5014 created for the same surface, the system 10 determines the geometric rotation and translation required to align the displaced surface map 5018 and reference surface map 5014 for best fit. The system 10 then applies the same rotation and translation to all stored reference points and structures on the rigid body of the femur 5002, calculating the current pose of all such points and structures relative to the reference frame of sensor suite 210, para [0265], The angle and depth of cutting guide 6616 relative to femur 6602 can still be measured and reported by system 10 by tracking marker 6620, which is still rigidly mounted on cutting guide 6616, para [0344]-[0345]).
Regarding claim 9, Ryan et al. discloses the method as in claim 1, further comprising aligning an image data set to a person using the marker attached to the person (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip, para [0050]-[0060], "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], Re-positioning the femur to the baseline orientation is achieved by manipulating the femur to align the virtual femur frame 2602 with the virtual femur target 2600 in abduction, flexion, and rotation, para [0254]-[0260], augmented reality headset, para [0368]-[0375]).
Regarding claim 10, Ryan et al. discloses the method as in claim 1, further comprising identifying the joint pivot axis as an intersection of a first line that extends down a surface of a bone of a joint substantially parallel to the first pose of the marker and a second line that extends along a surface of the bone of the anatomic structure at the second pose (algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para 0243-0246, augmented reality headset, para. 0368-0375).
Regarding claim 11, Ryan et al. discloses the method as in claim 1, wherein the movable anatomical structure and a proximal anatomical structure include bones forming a joint (at least one reference marker affixed to a bone for tracking the bone, wherein the bone is positioned such that the bone pivots at a joint or relative to the joint; at least one stationary reference marker positioned such that is it substantially fixed with respect to the joint; and a processor unit, para [0050]-[0060]).
Regarding claim 12, Ryan et al. discloses the method as in claim 1, wherein a proximal anatomical structure includes a bone (A tibial reference frame is defined with its origin at the center of the proximal tibia, with a first axis extending toward the center of the ankle, a second axis defined by the flexion axis of the knee and a third axis defined as the normal to the first and second axes, para [0306]-[0311]).
Regarding claim 13, Ryan et al. discloses the method as in claim 1, further comprising providing a graphical guide to guide surgical access to the joint pivot axis or a plurality of isometric points on a bone that is related to a joint (The markers (e.g., 100, 108, 110, etc.) for anatomic landmarks and tools are used for data collection (1000), which may be combined with pre-operative CT scan or MRI data for the determination of position and orientation (1002) of isometric points for ligament reconstruction and surgical tools, para [0355]-[0358], The system 10 continues to track a position and an orientation of a probe (e.g., needle, injection, pin, screw, etc.) and displays an axis (e.g., along an axial length of the probe) and/or location of the tip of the probe relative to the 3D image of the internal anatomy of the patient. The axis may be a virtual axis of the probe or a graphical representation of the probe. The tip of probe is then advanced to a desired position based on the location relative to the internal anatomy of the patient - augmented reality
headset, para [0368]-[0375]).
.
Regarding claim 14, Ryan et al. discloses the method as in claim 1, wherein the marker is at least one of: an optical code, a 2D bar code, an infrared marker, or a radiopaque marker (the anatomy marker 1300 would incorporate radio-opaque features of known geometry in a known pattern. The C-arm image is captured and scaled based on known marker features and displayed in the AR headset 3600, para [0292]).
Regarding claim 15, Ryan et al. discloses the method as in claim 1, further comprising identifying a proximal anatomical structure, which is fixed in the 3D coordinate system, and the movable anatomical structure is connected to the proximal anatomical structure (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060]).
Regarding claim 16, Ryan et al. discloses the method as in claim 15, further comprising determining the joint pivot axis and angle with respect to the proximal anatomical structure by comparing the first pose and the second pose (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060]).
Regarding claim 17, Ryan et al. discloses a method for referencing anatomical structure of a person using an AR headset, comprising:
registering a marker on a distal anatomical structure, which is connected with a proximal anatomical structure that does not move during a medical procedure (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points, para [0050]-[0060], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]); identifying a first pose for the marker in a 3D (three dimensional) coordinate system and a second pose for the marker; calculating a rotational angle of a displacement of the distal anatomical structure using the first pose and second pose (para [0050]-[0060], The system 10 can then establish a reference frame 5016 for the femur 5002 relative to the sensor suite 210 (5906). Then, repeatedly scanning the exposed lesser trochanter 5010, the system 10 generates a displaced 3-dimensional surface map 5018 for each scan (5908). With each successive scan, the system can compare the displaced surface map 5018 to the reference surface map 5014 for the same region on the lesser trochanter 5010. Based on this comparison, the system 10 can track the pose of the femur 5002 relative to sensor suite 210 by determining the translation and rotation required to best fit the displaced surface map 5018 with the reference surface map 5014 (5910), para [0267]-[0270], para [0307]-[0311], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]); and displaying a graphical reference for the rotational angle of displacement, using the AR headset (The system 10 continues to track a position and an orientation of a probe (e.g., needle, injection, pin, screw, etc.) and displays an axis (e.g., along an axial length of the probe) and/or location of the tip of the probe relative to the 3D image of the internal anatomy of the patient. The axis may be, for example, a virtual axis of the probe or a graphical representation of the probe. The tip of probe is then advanced to a desired position based on the location relative to the internal anatomy of the patient - augmented reality headset, para [0368]-[0375]).
Regarding claim 18, Ryan et al. discloses the method as in claim 17, further comprising:
setting a target position and rotation for the marker in the 3D coordinate system;
tracking a positional distance and the rotational angle of a displacement of the distal anatomical structure from a start point using the marker; and
displaying the graphical reference for positional distance and rotational angle of displacement as the distal anatomical structure is moved and rotated from the start point (para [0050]-[0060], The markers (e.g., 100, 108, 110, etc.) for anatomic landmarks and tools are used for data collection (1000), which may be combined with pre-operative CT scan or MRI data for the determination of position and orientation (1002) of isometric points for ligament reconstruction and surgical tools, para [0355]-[0358]; The system 10 continues to track a position and an orientation of a probe (e.g., needle, injection, pin, screw, etc.) and
displays an axis (e.g., along an axial length of the probe) and/or location of the tip of the probe relative to the 3D image of the internal anatomy of the patient. The axis may be, for example, a virtual axis of the probe or a graphical representation of the probe. The tip of probe is then advanced to a desired position based on the location relative to the internal anatomy of the patient - augmented reality headset, para [0368]-[0375])
Regarding claim 19, Ryan et al. discloses the method as in claim 18, further comprising providing at least one of a visual, audible or tactile indicator when the target position and rotation for the marker has been reached (when a valid reference depth point was recorded in the database or was interpolated, outputting an indicator that the valid reference point is available. The indicator of blocks S7820 and S7830 may be a visual indicator (e.g., displayed on the display of the head-worn display, flashing signal, lighted indicator, text indicator, pop-up, etc.), an audible indicator (e.g., beep, specific tone, specific sound, etc.), or a haptic indicator (e.g., haptics or feedback in head worn display, support module, helmet, etc.), para [0337]).
Regarding claim 20, Ryan et al. discloses the method as in claim 17, wherein the distal anatomical structure and the proximal anatomical structure are parts of at least one of: a broken bone, a malrotated structure, a dislocated joint, or dislocated anatomical structure (para [0265], used for corrective osteotomy for malunion of arm bones including the humerus, distal humerus, radius, and ulna with fractures that can be complicated and involve angular and rotational corrections. The markers (e.g., 100, 108, 110, etc.) for anatomic landmarks and tools are used for data collection (1000), which may be combined with pre-operative CT scan data for the determination of position and orientation (1002) of malunion and surgical tools. Algorithms (1006) are used to determine solutions including, but not limited to, location of osteotomy site, angle of cut, degree of correction, and assessment of results, para [0350]-[0351]).
Regarding claim 21, Ryan et al. discloses a method for referencing anatomical structures for a joint of a person using an AR headset, comprising (self-contained surgical navigation systems which include a head-worn display device to be worn by a user during surgery, Abstract, novel visualization and sensory augmentation devices - for positioning, localization, and situational awareness during medical procedures including, but not limited to, surgical, diagnostic, therapeutic, and anesthetic procedures, para [0002], para [0004]-[0014], para [0017]-[0020], head-worn surgical navigation system for determining a joint center - one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame;
create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060], algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104. For the purpose of this specification, "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], para [0267]-[0270], para [0307]-[0311], augmented reality headset, para [0368]-[0375]): registering a first marker on a proximal anatomical structure of the joint and a second marker on a distal anatomical structure of the joint (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points, para [0050]-[0060], algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]);
identifying a first pose position for the second marker in a 3D (three dimensional) space (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]);
setting a target position and rotation for the second marker in the 3D space register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]);
identifying a second pose position for the second marker in a 3D space (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]);
computing a distance and an angle of a displacement of the distal anatomical structure using the first pose position and second pose position (para [0050]-[0060], The system 10 can then establish a reference frame 5016 for the femur 5002 relative to the sensor suite 210 (5906). Then, repeatedly scanning the exposed lesser trochanter 5010, the system 10 generates a displaced 3-dimensional surface map 5018 for each scan (5908). With each successive scan, the system can compare the displaced surface map 5018 to the reference surface map 5014 for the same region on the lesser trochanter 5010. Based on this comparison, the system 10 can track the pose of the femur 5002 relative to sensor suite 210 by determining the translation and rotation required to best fit the displaced surface map 5018 with the reference surface map 5014 (5910), para [0267]-[0270], para [0307]-[0311], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]);and
displaying the positional distance and rotational angle as the distal anatomical structure is moved between the first pose position and the second pose position, using the AR headset (The system 10 continues to track a position and an orientation of a probe (e.g., needle, injection, pin, screw, etc.) and displays an axis (e.g., along an axial length of the probe) and/or location of the tip of the probe relative to the 3D image of the internal anatomy of the patient. The axis may be, for example, a virtual axis of the probe or a graphical representation of the probe. The tip of probe is then advanced to a desired position based on the location relative to the internal anatomy of the patient - augmented reality headset, para [0368]-[0375]).
Regarding claim 22, Ryan et al. discloses the method as in claim 21, further comprising aligning an image data set to the person using the first marker or the second marker attached to the person (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip, para [0050]-[0060], "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], Re-positioning the femur to the baseline orientation is achieved by manipulating the femur to align the virtual femur frame 2602 with the virtual femur target 2600 in abduction, flexion, and rotation, para [0254]-[0260], augmented reality headset, para [0368]-[0375]).
Regarding claim 23, Ryan et al. discloses the method as in claim 21, providing visual, audible, tactile, or graphical feedback when the target position and location have been reached (when a valid reference depth point was recorded in the database or was interpolated, outputting an indicator that the valid reference point is available. The indicator of blocks S7820 and S7830 may be a visual indicator (e.g., displayed on the display of the head-worn display, flashing signal, lighted indicator, text indicator, pop-up, etc.), an audible indicator (e.g., beep, specific tone, specific sound, etc.), or a haptic indicator (e.g., haptics or feedback in head worn display, support module, helmet, etc.), para [0337]).
Regarding claim 24, Ryan et al. discloses the method as in claim 23, further comprising displaying a graphical icon when the target position or rotation is reached (when a valid reference depth point was recorded in the database or was interpolated, outputting an indicator that the valid reference point is available. The indicator of blocks S7820 and S7830 may be a visual indicator (e.g., displayed on the display of the head-worn display, flashing signal, lighted indicator, text indicator, pop-up, etc.), an audible indicator (e.g., beep, specific tone, specific sound, etc.), or a haptic indicator (e.g., haptics or feedback in head worn display, support module, helmet, etc.), para [0337]).
Regarding claim 25, Ryan et al. discloses the method as in claim 21, further comprising:
segmenting structures of anatomy related to the joint to form segments; and
moving the segments, as viewed through the AR headset, using changes in position and orientation of the first marker or the second marker (First, a scan of the region of interest of the patient such as CT or MRI is obtained. If possible, the patient should be positioned in a way that approximates positioning during surgery. Second, segmentation of the scan data is performed in order to convert it into three-dimensional models of items of interest including but not limited to: teeth and bony structures, veins and arteries of interest, nerves, glands, tumors or masses, implants and skin surfaces. Models are segregated so that they can later be displayed, labeled or manipulated independently, para [0231]-[0239]).
Regarding claim 26, Ryan et al. discloses the method as in claim 21, wherein the proximal anatomical structure and the distal anatomical structure include a broken portion of a bone (para [0265], used for corrective osteotomy for malunion of arm bones including the humerus, distal humerus, radius, and ulna with fractures that can be complicated and involve angular and rotational corrections. The markers (e.g., 100, 108, 110, etc.) for anatomic landmarks and tools are used for data collection (1000), which may be combined with pre-operative CT scan data for the determination of position and orientation (1002) of malunion and surgical tools. Algorithms (1006) are used to determine solutions including, but not limited to, location of osteotomy site, angle of cut, degree of correction, and assessment of results, para [0350]-[0351]).
Regarding claim 27, Ryan et al. discloses the method as in claim 21, further comprising defining a rotation axis that is a longitudinal axis of a bone in the proximal anatomical structure or the distal anatomical structure (para. 0050-0060, used for corrective osteotomy for malunion of arm bones including the humerus, distal humerus, radius, and ulna with fractures that can be complicated and involve angular and rotational corrections. The markers (e.g., 100, 108, 110, etc.) for anatomic landmarks and tools are used for data collection (1000), which may be combined with pre-operative CT scan data for the determination of position and orientation (1002) of malunion and surgical tools. Algorithms (1006) are used to determine solutions including, but not limited to, location of osteotomy site, angle of cut, degree of correction, and assessment of results, para [0350]-[0358], The system 10 continues to track a position and an orientation of a probe (e.g., needle, injection, pin, screw, etc.) and displays an axis (e.g., along an axial length of the probe) and/or location of the tip of the probe relative to the 3D image of the internal anatomy of the patient. The axis may be, for example, a virtual axis of the probe or a graphical representation of the probe. The tip of probe is then advanced to a desired position based on the location relative to the internal anatomy of the patient - augmented reality headset, para 0368-0375).
Regarding claim 28, Ryan et al. discloses the method as in claim 21, wherein the first marker and the second marker are at least one of: an optical code, 2D optical code, an infrared marker, or a radiopaque marker (the anatomy marker 1300 would incorporate radio-opaque features of known geometry in a known pattern. The C-arm image is captured and scaled based on known marker features and displayed in the AR headset 3600, para [0292]).
Regarding claim 29, Ryan et al. discloses the method as in claim 21, further comprising registering a plurality of points on an optical code (combining inertial data and optical tracking includes: determining the camera to eyepiece rotation matrix from mechanical design (shown as RCameraEye) at block S8300; determining the eyepiece to IMU rotation matrix from the mechanical design (shown as REyeIMU) at block S8310; and calibrating the marker to camera rotation matrix (shown as RMaarkerCamera) at block S8320, para [0270]-[0273]).
Regarding claim 30, Ryan et al. discloses the method as in claim 21, wherein the proximal anatomical structure is fixed in place within the 3D space (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060]).
Regarding claim 31, Ryan et al. discloses a method for identifying anatomical structures for a ball joint of a person using an AR headset, comprising: (self-contained surgical navigation systems which include a head-worn display device to be worn by a user during surgery, Abstract, novel visualization and sensory augmentation devices - for positioning, localization, and situational awareness during medical procedures including, but not limited to, surgical, diagnostic, therapeutic, and anesthetic procedures, para [0002], para [0004]-[0014], para [0017]-[0020], head-worn surgical navigation system for determining a joint center - one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104. For the purpose of this specification, "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], para [0267]-[0270], the bearing 4916 may be a ball-type allowing medial/lateral and flexion/extension tilt of the condylar plate 4906, para [0307]-[0317], augmented reality headset, para [0368]-[0375]): registering a marker in a first pose on a distal anatomical structure which is connected with a proximal anatomical structure (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points, para [0050]-[0060], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]);
registering a second pose of the marker after the distal anatomical structure has been moved with respect to the proximal anatomical structure (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]); determining an angle with respect to the proximal anatomical structure by comparing the first pose and the second pose (para [0050]-[0060], The system 10 can then establish a reference frame 5016 for the femur 5002 relative to the sensor suite 210 (5906). Then, repeatedly scanning the exposed lesser trochanter 5010, the system 10 generates a displaced 3-dimensional surface map 5018 for each scan (5908). With each successive scan, the system can compare the displaced surface map 5018 to the reference surface map 5014 for the same region on the lesser trochanter 5010. Based on this comparison, the system 10 can track the pose of the femur 5002 relative to sensor suite 210 by determining the translation and rotation required to best fit the displaced surface map 5018 with the reference surface map 5014 (5910), para [0267]-[0270], para [0307]-[0311], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]); registering a third pose and fourth pose with the marker (one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060], Algorithms in the AR headset 3600 are used to process the images from the stereoscopic cameras (3904) to calculate the point of intersection of each fiducial (1108, 1110, 1112) and thereby determine the six-degrees of freedom pose of the marker 1104 - "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], augmented reality headset, para [0368]-[0375]); and determining an intersection of a first joint pivot axis and the second joint pivot axis that represents a joint pivot point of the ball joint (head-worn surgical navigation system for determining a joint center – one reference marker affixed to a femur for tracking the femur, wherein the femur is positioned such that the femur pivots at a hip or relative to the hip; at least one stationary reference marker positioned such that is it substantially fixed with respect to the hip - register points on the femur in a reference coordinate frame; create a femoral coordinate frame based on the registered points; transform from the reference coordinate frame to the femoral coordinate frame; acquire, using the at least one tracking camera, points of the at least one stationary marker in the reference frame, wherein, during acquisition, a position of at least a portion of the visual display moves synchronously with movement of the head-worn surgical navigation system; and determine a hip center in the femoral coordinate frame, para [0050]-[0060], "pose" is defined as the combination of position and orientation of an object, para [0243]-[0246], para [0267]-[0270], the bearing 4916 may be a ball-type allowing medial/lateral and flexion/extension tilt of the condylar plate 4906, para [0307]-[0317], augmented reality headset, para [0368]-[0375]).
Regarding claim 32, Ryan et al. discloses the method as in claim 31, further comprising displaying the joint pivot point using the AR headset (displaying a mixed reality user interface comprising stereoscopic virtual images of the features of the surgical tool and the features of the anatomical object in the user's field of view, para [0004]-[0014], augmented reality headset, para [0368]-[0375]).
Regarding claim 33, Ryan et al. discloses the method as in claim 31, further comprising displaying a plurality of points surrounding the joint pivot point based on a defined ligament length to identify locations on a bone where a ligament is affixable at isometric distances to the joint pivot point, using the AR headset (The markers (e.g., 100, 108, 110, etc.) for anatomic landmarks and tools are used for data collection (1000), which may be combined with pre-operative CT scan or MRI data for the determination of position and orientation (1002) of isometric points for ligament reconstruction and surgical tools, para [0355]-[0358], augmented reality headset, para [0368]-[0375]).
Regarding claim 34, Ryan et al. discloses the method as in claim 33, wherein the plurality of points is at least one of: an arc, a circle (virtual circle 1908, circle 4306), a cylinder, a curved surface (When each point is registered, a virtual marker, such as a small sphere, may be positioned and remain at the location of the tip at the time of registration and beyond to provide the user 106 a visual confirmation to the user 106 and check on the quality of the registration, para. 0247), or an irregular shape.
Regarding claim 35, Ryan et al. discloses the method as in claim 31, wherein the joint pivot point is a ball joint in the person (para [0267]-[0270], the bearing 4916 may be a ball-type allowing medial/lateral and flexion/extension tilt of the condylar plate 4906, para [0307]-[0317], augmented reality headset, para [0368]-[0375]).
Allowable Subject Matter
Claims 36-52 are allowed.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS J LETT whose telephone number is (571)272-7464. The examiner can normally be reached Mon-Fri 9-6 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571) 272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS J LETT/ Primary Examiner, Art Unit 2611
Prosecution Insights