DETAILED ACTION
This Office action is responsive to communications filed on 03/02/2026. Claims 60-62. 64. 69-71, 74. 77-79 have been amended. Presently, Claims 60-79 remain pending and are hereinafter examined on the merits.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Previous objections to the Abstract is NOT withdrawn in view of the response filed on 03/02/2026. No amendments were received with the response submitted on 03/02/2026.
Previous objections to the Drawings are withdrawn in view of the amendments filed on 03/02/2026.
Previous rejections under 35 USC § 112(b) are withdrawn in view of the amendments filed on 03/02/2026.
Previous claim objections are withdrawn in view of the amendments filed on 03/02/2026.
The Applicant’s arguments with respect to rejections under 35 USC § 101 have been fully, considered, but are not persuasive.
Firstly, the Examiner agrees that the eligibility must be evaluated on a claim-by-claim basis and that the rejection should identify the limitations believed to recite the abstract idea. The Office Action did so by identifying the specific limitations in claims 60, 70, and 79 that recite determining the coordinates, determining distances between coordinates, and determining whether a condition is satisfied based on those distances. As explained in the Office Action filed on 12/02/2025 in the Interview summary 01/14/2026, these steps correspond to identifying positional values, performing mathematical calculations on those values, and evaluating results. These operations fall squarely within the abstract idea groupings of mathematical concepts and mental processes. The Applicant’s assertion that the rejection generalizes the claims does not overcome the fact that the recite steps themselves are framed in terms of determining coordinate values, computing distances, and evaluating a condition based on those values.
The Applicant argues that the limitations cannot be performed mentally because the coordinates are determined “based on one or more images received by the tracking system”. This argument is tantamount to stating that the limitations preclude a human from determining coordinates by looking at already received images by the tracking system. However, the claims do not recite any specific imaging processing technique, tracking algorithm, or any other technological mechanism for extracting the coordinates from the images. The claims merely state the coordinates are determined based on images already received from a tracking system. As written, the claim language still reflects the abstract operations of determining positions, computing distances, and evaluating whether a threshold or condition satisfied. The presence of image input does not change the character of the recited operations, which remain mathematical evaluations performed on positional data. Accordingly, the limitations continue to fall with the abstract idea groupings identified in the Office Action.
The Applicant further contends that the claims do not recite a “mathematical concept” because they do not recite a mathematical formula. This argument is not persuasive. Mathematical concepts under the eligibility guidance are not limited to explicit equations. The claim expressly requires determining distances between coordinate values and evaluating the distances to determine whether registration remains valid. Determining distances between coordinates involves “mathematical relationships”, which the Applicant has identified, pg 13, that mathematical relationships are a mathematical concept. That fact that the specific formula for distance is not explicitly written in the claim does not remove the mathematical nature of the operation that the claim requires.
With respect to Step 2A, Prong Two, the Applicant argues that the additional limitations integrate the alleged abstract idea into practical application because the claims operate within a registration framework involving transformation, using tracking system images, and near-eye display. However, these elements are recited at high-level of generality as explain previously and in the interview on 01/14/2026 and merely provide the technological environment in which the abstract calculations are performed. The claims do not specify any particular structure or technological improvement the tracking system “itself” or tracking systems, the display device, or the image guided surgical technology. In fact there is no step that places the judicial exception into a practical application. The additional elements therefore function as generic data sources, processors, and output components used to implement the abstract calculations.
The Applicant also asserts that the claims improve the technological field of image-guided medical procedures by detecting loss of registration during a procedure. While the specification may describe advantages relating to surgical navigation or augmented reality, those asserted advantages are not reflected in the claims. In addition, to merely or generically assert a desired outcome or advantage does not define “a specific technological mechanism” that achieves that outcome. The claims themselves merely determine coordinate values, compute distance between those values, and determine whether the registration is valid based on those distance, without any further implementation of those steps into a practical application. The claims do not recite any specific improvement to how the tracking system operates, how the images are produced, or how display renders the registered anatomy. Thus, the alleged technological improvement is not supported by the limitations in the claims themselves.
Turning, to Step 2B, the Applicant argues the claims recite significantly more because they improve the technological field of image guided procedures. As discussed above, the claims do not recite a technological improvement.
The Applicant’s arguments with respect to rejections under 35 USC § 103 have been fully, considered, but are not persuasive.
With respect to the argument that both references Ben-Yishai (US 2019/0108645 A1, herein Ben-Yishai 2019) in view of Ben-Yishai et al (US 2021/0186355 A1, herein Ben-Yishai 2021), fail to teach: “after obtaining the transformation, determine initial coordinates of the plurality of landmarks relative to the representation”, the Applicant’s position is not persuasive. Ben-Yishai 2019 teaches that reference points (i.e., landmarks) have known positions in the reference coordinate system once the model of the internal anatomy is registered with the internal body part. The processor determines the expected location of these reference points based on their known positions and the determined pose of the imaging system, and compares them with the detected locations in the acquired images to verify registration. Specifically,
After the model is registered with the reference coordinate system (i.e., obtaining the transformation), the initial coordinates (i.e., predetermined position) of the landmarks are determined, ¶0014, ¶0034, ¶0043, ¶0048, ¶0049. Because the model and the landmarks are already registered, those landmarks coordinates correspond to coordinates relative the representation under the obtained transformation. Therefore, contrary to the Applicant’s assertion, Ben-Yishai does disclose determining initial landmark coordinates following registration of the model.
The Applicant arguments directed to characterizing the “expected locations” described in Ben-Yishai 2019 as merely 2D image plane values and asserting that the claim requires a different type of coordinate, is not persuasive. The claim does not require that the coordinates are stored or represented in any particular dimensionality or coordinate frame beyond being relative the representation after the transformation is obtained. The expected locations calculated in Ben-Yishai 2019 are derived from the known positions of the reference points in the coordinate system in which the model is registered, which reflects their coordinates relative to the representation used by the system, ¶Abstract, ¶0005, ¶0014, ¶0018, ¶0023, ¶0034, ¶0037, ¶0048, ¶0053. Accordingly, the Applicant’s distinction between the image-plane locations and the representation of relative coordinates introduces limitations not reflected in the claim.
The Applicant also argues that the cited passages of Ben-Yishai 2019 only relate to varying the registration using the image congruence and therefore cannot correspond to determining “initial coordinates”. This argument is not persuasive because the congruence determination relies on the comparing the detected landmark position with an expected position derived from the landmark’s known coordinate in the registration coordinate system. Specifically, Ben-Yishai 2019 explicitly confirms that the system determines the position (i.e., coordinates) of the reference points (i.e., landmarks) in the reference coordinate system before checking for congruence, ¶0018, ¶0034, ¶0043, Claim 27-28. The congruence check cannot occur without a balance. The expected location of the reference points in the acquired image is directly calculated using these initial determined coordinates, ¶0044-0045, ¶0056-0057, Claim 26. Hence, Ben-Yishai 2019 establishes those coordinates prior to the comparison step.
Regarding the limitation of “determine one or more distances between the subsequent coordinates and the initial coordinates”, the rejection explicitly acknowledged that Ben-Yishai 2019 does not expressly disclose computing the difference in terms of a distance measurement. The rejection relied on Ben-Yishai 2021 for teaching the use of a distance error estimation between marker positions in the registered model coordinate system and the tracker coordinate system to evaluate registration quality. Specifically, Ben-Yishai 2021 calculates the difference between the expected position and the measured position of a landmark defined as the “type two error estimation”, ¶0097. The position of the markers in the registered model coordinate system (i.e., the initial expected location, transformed), ¶0097, & the position of the marker in the reference coordinate system (i.e., the subsequent measured location), ¶0097. The comparison is motived by the possibility that a specific marker may have been displaced between the time initial imaging was performed and the time registration is performed, ¶0097. Furthermore, Ben-Yishai 2021 confirms that the distance measured between these position represents an error value used to evaluate or improve registration and determine whether a marker should be discarded, ¶0096-0097, ¶0064-0065, Claim 12.
Arguments directed to this distance computation occurs during the registration process rather than after the registration is not persuasive because the claim does not require that the distance computation occur only after a completed registration process or exclude where the same metric is used during or after registration. The combination merely applies the known distance error estimation in Ben-Yishai 2021 to the landmark comparison already performed in Ben-Yishai 2019. Doing so would have been obvious modification because Ben-Yishai 2021 teaches that the distance between marker positions provide a quantitative measure of registration quality. A person of ordinary skill in the art would reasonably apply that known metric to the landmark comparison to Ben-Yishai in order to improve the reliability and evaluation of registration accuracy.
For these reasons, the 35 USC § 103 rejection is maintained.
Abstract
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details. In the current Abstract filed on 02/19/2024, the word count is 40.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 60-79 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 of the subject matter eligibility test (see MPEP 2106.03).
Claims 60-69 are drawn to a “system” which describes one of the four statutory categories, i.e., a machine.
Claims 70-78 are drawn to a “system” which describes one of the four statutory categories, i.e., a machine.
Claim 79 is directed to a “method” which describes one of the four statutory categories of patentable subject matter, i.e., a process.
Step 2A of the subject matter eligibility test (see MPEP 2106.04).
Prong One:
Claims 60 recite (“sets forth” or “describes”) the abstract idea of “a mental process” (MPEP 2106.04(a)(2).III.) & the abstract idea of “mathematical concepts” (MPEP 2106.04(a)(2).I.), substantially as follows:
“ after obtaining the transformation, determine initial coordinates of the plurality of landmarks relative to the representation based on one or more images received by the tracking system of the plurality of landmarks and a site of the medical procedure;
[...]
determine, at a time subsequent to determining the initial coordinates and during the medical procedure, subsequent coordinates of the plurality of landmarks relative to the representation;
determine one or more distances between the subsequent coordinates and the initial coordinates; and
determine whether the registration is valid during the medical procedure based at least in part on the one or more distances. ”
Claims 70 recite (“sets forth” or “describes”) the abstract idea of “a mental process” (MPEP 2106.04(a)(2).III.) & the abstract idea of “mathematical concepts” (MPEP 2106.04(a)(2).I.), substantially as follows:
“ after obtaining the transformation, determine initial coordinates of the plurality of landmarks relative to the representation based on one or more images received by the tracking system of the plurality of landmarks and a site of the medical procedure;
[...] determine, at a time subsequent to determining the initial coordinates and during the medical procedure, subsequent coordinates of the plurality of landmarks relative to the representation;
determine one or more distances between the subsequent coordinates and the initial coordinates; and
determine whether the registration is valid during the medical procedure based at least in part on the one or more distances. ”
Claims 79 recite (“sets forth” or “describes”) the abstract idea of “a mental process” (MPEP 2106.04(a)(2).III.) & the abstract idea of “mathematical concepts” (MPEP 2106.04(a)(2).I.), substantially as follows:
“ computing and recording initial locations of a plurality of landmarks [...]
at a time subsequent to computing and recording the initial locations and during the image-guided medical procedure, computing a subsequent location of one or more landmarks of the plurality of landmarks in the target coordinate system of the registration;
for each respective landmark of the one or more landmarks, computing a distance between an initial location, among the initial locations, associated with the respective landmark and the subsequent location of the respective landmark; and
wherein the computed distance for at least one landmark of the one or more landmarks exceeds a threshold, [...].”
In claims (60, 70, 79), recite mathematical concepts and mental processes because they describe operations that involve determine spatial coordinates, calculating distances (i.e., a mathematical relationship / mathematical calculation) between coordinate values, and evaluating those distances to assess whether a condition is satisfied. Determining initial and subsequent coordinates of landmarks relative to a representation involves identifying positional values within a coordinate system. The subsequent step of determining between the coordinates applies mathematical relationships to those positional values. Evaluating whether the registration remains valid based on those distances represents a comparison of calculated values against a condition. These operations of identifying coordinate positions, computing distances between positions, and assessing the result are forms of mathematical calculations, and mathematical relationships that can be performing using mathematical formulas or reasoning and can also be carried out mentally or with a pencil and paper. Accordingly, the limitations are indeed described as mathematical relationships and evaluations that constitute mathematical concepts and mental processes within the meaning of MPEP 2106.04(a)(2).
Similarly, the indicated limitations of 79 recite computing and recording initial locations of landmarks, computing subsequent locations, computing distances between those locations, and determining whether a computed distance exceeds a threshold to indicate invalid registration. These steps as generally recited involve identing positional coordinates within a coordinate system, performing mathematical calculations to determine distances between coordinate values, and comparing the calculated distances to a threshold decision/condition. Such operations rely on mathematical relationships applied to positional data and involve evaluating the results of those calculations to determine whether a specified condition is met. The identification of positions, calculation of distances, and comparison of a threshold reflect a mathematical operation and evaluative reasoning that can be performed mentally or with pen and paper. Therefore these limitations too set forth mathematical concepts and mental processes under MPEP 2106.04(a)(2). There is nothing recited in the claim to suggest an undue level of complexity in how these noted steps are conducted.
Prong Two: Claims 60, 70 & 79 do not include additional elements that integrate the mental process into a practical application.
This judicial exception is not integrated into a practical application. In particular, the claims recites:
[1] additional steps of
“tracking a transformation during a medical procedure, the transformation registering a representation of internal anatomy of a subject with a coordinate system with respect to one or more fiducial objects coupled to the subject, the system comprising: a head-mounted display device comprising a near-eye display and a tracking system; a plurality of landmarks configured to provide the one or more fiducial objects, and to be disposed on skin of the subject in proximity to a site of the medical procedure; and one or more processors, that upon execution of program instructions stored on a non-transitory computer-readable medium, are configured to cause the system to:”-(claim 60),
“tracking a transformation during a medical procedure, the transformation registering a representation of internal anatomy of a subject with a coordinate system with respect to one or more fiducial objects coupled to the subject, the system comprising: a display device comprising a near-eye display and a tracking system; a plurality of landmarks configured to provide the one or more fiducial objects; and one or more processors, that upon execution of program instructions stored on a non- transitory computer-readable medium, are configured to cause the system to:”-(claim 70)
[2] further an addition step of
“display the representation in registration with the coordinate system on the near- eye display of the head-mounted display device using the transformation;”-(claim 60),
“display the representation in registration with the coordinate system on the near-eye display of the display device using the transformation;”-(claim 70),
“validating a registration between a tracking system and image data of an anatomy of a patient during an image-guided medical procedure, the method comprising:”-(claim 79).
The steps in (1) represent merely data gathering or pre-solution activities that are necessary for use of the recited judicial exception and are recited at a high level of generality with conventionally used tools (see below Step IIB for further details). Data gathering and mere instructions to implement an abstract idea on a computer do not integrate a judicial exception into a practical application (MPEP 2106.05 (f and g)). Regarding the processor language written at such a high level of generality of structural limitations, the processor language amounts to a generic computer component with mere instructions to implement the abstract idea on a computer.
The step in (2) represents merely outputting the insignificant extra solution activity on a display amounting to nothing more than insignificant extra solution activity and are recited at a high level of generality with conventionally used tools (see below Step IIB for further details). Data gathering and mere instructions to implement an abstract idea on a computer do not integrate a judicial exception into a practical application (MPEP 2106.05 (f and g)). Regarding the processor language written at such a high level of generality of structural limitations, the processor language amounts to a generic computer component with mere instructions to implement the abstract idea on a computer.
Regarding the limitations of claim 79 directed to the “tracked by the tracking system in a target coordinate system of the registration, wherein the landmarks are locations on the skin of the patient indicated by a user;” [emphasis added], (i.e., “computing and recording initial locations of a plurality of landmarks tracked by the tracking system in a target coordinate system of the registration, wherein the landmarks are locations on the skin of the patient indicated by a user;” are treated as a generic computer implementation, which falls under mere instructions to apply the abstract idea on a computer and therefore does not place the abstract idea into a practical application that solves a technological solution in a meaningful way or improve the functionality of the technology or generic computer “itself”. Simply, it’s a generic computer implementation of a mental process rather than a meaningful limitation. Regarding the processor language written at such a high level of generality of structural limitations, the processor language amounts to a generic computer component with mere instructions to implement the abstract idea on a computer.
As a whole, the additional elements merely serve to gather and feed information to the abstract idea and to output a notification based on the abstract idea, while generically implementing it on conventionally used tools. There is no practical application because the abstract idea is not applied, relied on, or used in a meaningful way. No improvement to the technology is evident, and the estimated tracking location information is not outputted in any way such that a practical benefit is realized. Therefore, the additional elements, alone or in combination, do not integrate the abstract idea into a practical application.
Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Further, there is no evidence of record that would support the assertion that this step is an improvement to a computer or technological solution to a technological problem. Ultimately, the Applicant’s describe improvement in the process of using known tracking techniques, but this is not an improvement in the function of a computer or other technology (See MPEP 2106.05(a)(ii); “the court determined that the claimed user interface simply provided a trader with more information to facilitate market trades, which improved the business process of market trading but did not improve computers or technology”; See MPEP 2106.04(d)(1); 2106.05(a); and 2106.05(f)). The claims are directed to the abstract idea. Also, there does not appear to be any particular structure or machine, treatment or prophylaxis, transformation, or any other meaningful application that would render the claim eligible at step 2A, prong 2.
Step 2B of the subject matter eligibility test (see MPEP 2106.05).
Claims 60, 70 & 79 do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, the claims recite additional steps of head-mounted displays/generic display and landmarks forming fiducial objects and one or more processors. These steps represents mere data gathering, data outputting or pre/post/extra-solution activities that are necessary for use of the recited judicial exception and are recited at a high level of generality. Furthermore, as discussed above, limitations with respect to the processor languages/terms, respectively, amount to mere instructions to implement the abstract idea on a computer. As discussed with respect to Step 2A Prong Two, the additional elements in the claims amount to no more than insignificant extra solution activity and mere instructions to apply the exception using a generic computer component. The same analysis applies here in 2B and does not provide an inventive concept. The data gathering steps that were considered insignificant extra-solution activity in Step 2A Prong Two, have been re-evaluated in Step 2B and determined to be well-understood, routine, conventional activity in the field.
As an evidence, Chiou (US 12,053,247 B1) discloses:
[Col 104 l.4-11], ‘Any other techniques known in the art, including as described in this specification, that can be used to determine the location, position, orientation, alignment, surfaces or shape of the fiducial marker, sub-array or array in relationship to the patient, the operator and/or the HMD, can be used, including, but not limited to surgical navigation including optical or RF tracking, laser based distance measurements and the like.’
For these reasons, there is no inventive concept. The claim is not patent eligible. Even when viewed as a whole, nothing in the claim adds significantly more to the abstract idea.
Dependent Claims
The following dependent claims merely further describe the extra-solution activities and therefore, do not amount to significantly more than the judicial exception or integrate the abstract idea into a practical application for similar reasons:
describing wherein the head-mounted display device comprises a wearable display device adapted to be worn on a head-(claim 61), wherein the display head-mounted device display device comprises a pair of glasses-(claim 62), wherein the tracking system comprises an infrared camera-(claim 63), wherein the tracking system comprises a projector configured to project infrared light toward the site of the medical procedure.-(claim 64), wherein the plurality of landmarks comprise registration markers.-(claim 65), wherein the plurality of landmarks comprise stickers.-(claim 66), wherein the plurality of landmarks comprise one or more uniquely identifying retroreflective elements.-(claim 67), wherein the plurality of landmarks comprise one or more radiopaque elements.-(claim 68). The above recitations amount to the data gathering steps and pre-solution activity are conventional and recited at high level of generality. As such, the abstract idea is not applied, relied on, or used in a meaningful way. No improved to the technology is evident, and the determined visualization of context is not outputted in any way such that the practical benefit is realized.
describing (claim 69) wherein to cause the system to determine whether the registration is valid during the medical procedure, the one or more processors are configured to cause the system to determine, during the medical procedure, whether the one or more distances exceed a threshold, amounts the mental processes and mathematical concepts as outlined above, whereas the “wherein the one or more processors are configured to” to preform the abstract idea falls under mere instructions to apply the abstract idea on a computer and therefore does not place the abstract idea into a practical application that solves a technological solution in a meaningful way or improve the functionality of the technology or generic computer “itself”. Simply, it’s a generic computer implementation of a mental process rather than a meaningful limitation. Regarding the processor language written at such a high level of generality of structural limitations, the processor language amounts to a generic computer component with mere instructions to implement the abstract idea on a computer.
describing wherein the plurality of landmarks are configured to be disposed on skin of the subject in proximity to the site of the medical procedure.-(claim 71), wherein the display device comprises a wearable display device adapted to be worn on a head.-(claim 72), wherein the display device comprises a pair of glasses.-(claim 73), wherein the tracking system comprises an infrared camera and a projector configured to project infrared light toward the site of the medical procedure.-(claim 74), wherein the plurality of landmarks comprise registration markers.-(claim 75), wherein the plurality of landmarks comprise stickers.-(claim 76). The above recitations amount to the data gathering steps and pre-solution activity are conventional and recited at high level of generality. As such, the abstract idea is not applied, relied on, or used in a meaningful way. No improved to the technology is evident, and the determined visualization of context is not outputted in any way such that the practical benefit is realized.
describing “wherein to cause the system to determine whether the registration is valid during the medical procedure, the one or more processors are configured to cause the system to determine, during the medical procedure, whether the one or more distances exceed a threshold.”-(claim 77), and “wherein: to cause the system to determine whether the one or more distances exceed the threshold, the one or more processors are configured to cause the system to determine, during the medical procedure, that at least one distance of the one or more distances exceeds the threshold; and the one or more processors are configured to notify a user that the registration is not valid based on the at least one distance exceeding the threshold.”-(claim 78) amounts the mental processes and mathematical concepts as outlined above, whereas the “wherein the one or more processors are configured to” to preform the abstract idea falls under mere instructions to apply the abstract idea on a computer and therefore does not place the abstract idea into a practical application that solves a technological solution in a meaningful way or improve the functionality of the technology or generic computer “itself”. Simply, it’s a generic computer implementation of a mental process rather than a meaningful limitation. Regarding the processor language written at such a high level of generality of structural limitations, the processor language amounts to a generic computer component with mere instructions to implement the abstract idea on a computer.
Taken alone and in combination, the additional elements do not integrate the judicial exception into a practical application at least because the abstract idea is not applied, relied on, or used in a meaningful way. They also do not add anything significantly more than the abstract idea. Their collective functions merely provide computer/electronic implementation and processing, and no additional elements beyond those of the abstract idea. Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements individually. There is no indication that the combination of elements improves the functioning of a computer, output device, improves technology other than the technical field of the claimed invention, etc. Therefore, the claims are rejected as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 60-62, 66, 68-73, 76-79 are rejected under 35 U.S.C. 103 as being unpatentable over Ben-Yishai (US 2019/0108645 A1, herein Ben-Yishai 2019) in view of Ben-Yishai et al (US 2021/0186355 A1, herein Ben-Yishai 2021).
Claim 60: Ben-Yishai 2019 discloses, A system (system 200, Fig. 2) for tracking a transformation during a medical procedure, the transformation registering a representation of internal anatomy of a subject with a coordinate system with respect to one or more fiducial objects coupled to the subject, the system comprising:
-Ben-Yishai 2019 discloses, a system for verifying registration for a model of a body part with the internal body part in a reference coordinate system, ¶Abstract, ¶0006, Verifying registration (i.e., the tracking accuracy of the transformation) of the internal body during a medical procedure of a subject in a coordinate is described with respect to one or more fiducial objects, ¶Abstract, ¶0006, ¶0048. Ben-Yishai 2019 utilizes fiducial objects, referred to as reference points, ¶Abstract, ¶0005, ¶0007, ¶0034, ¶0037. The principle of Ben-Yishai 2019 is to ensure that the visualization of the internal anatomy remains accurately registered with the real anatomy, ¶Abstract, ¶0013.
a head-mounted display device comprising a near-eye display and a tracking system;
-The system of Ben-Yishai 2019 includes a head-mounted display (e.g., HMD 206) to be donned by the user, ¶0006, ¶0023, Claim 10. The HMD includes a see through display (visor 212, Fig. 2), ¶0023, Claim 10. The visor relates to optically see through displays such as smart glass, augmented reality, or near to eye displays, ¶0013, ‘[...] visor’ used herein relates in general to optically see-through displays such as smart glasses, augmented reality glasses, near-to-eye displays, head mounted displays and head wearable displays’. The system also includes an HMD tracking module (HMD tracking unit 202 and a reference tracking unit 204) coupled to the processor (processor 208) that continuously determines position and orientation (‘P&O’) of the HMD in the reference coordinate system, ¶0006, ¶0025, Claim 10.
a plurality of landmarks configured to provide the one or more fiducial objects, and to be disposed on skin of the subject in proximity to a site of the medical procedure; and
-Ben-Yishai 2019 utilizes fiducial objects, referred to as reference points, ¶Abstract, ¶0005, ¶0007, ¶0034. The plurality of landmarks forming one or more fiducial objects, is described by the technique of using at least one reference point. Examples also include three reference points, ¶Abstract, ¶0005, Claim 1. The reference points can be a marking on the skin, ¶0017, Claim 7, Claim 15, an adhesive marker attached to the skin of the patient (i.e., sticker), ¶0034, Claim 7, Claim 15, or anatomical feature which is directly visible to the user, Claim 7, Claim 15. These references points are determined to be directly visible to the user, and the relative position between them and the internal body part is substantially constant. Their position in the reference coordinate system is predetermined, ¶Abstract, ¶0005-0007, ¶0013, ¶0034, Claim 1.
one or more processors, that upon execution of program instructions stored on a non-transitory computer-readable medium, are configured to cause the system to:
-Ben-Yishai 2019 describes the system 200 including a processor 208 that performs procedures to verify registration by comparing predicted and determined locations of reference points (i.e., landmarks), ¶0044-0045, ¶0056-0057. The processor 208 is coupled to the HMD tracking unit 202 and a reference tracking unit 204. This processor executes the steps required verification, ¶0006, ¶0026.
after obtaining the transformation, determine initial coordinates of the plurality of landmarks relative to the representation based on one or more images received by the tracking system of the plurality of landmarks and the site of the medical procedure;
-Ben-Yishai 2019 processor determines the position of the reference points (e.g., landmarks 2301-2303) in the reference coordinate system (e.g., 222) prior to the onset of the procedure, the claims do not require the extent of “during a medical procedure”, ¶0034, ¶0049. The model (i.e., representation of the internal anatomy) is registered with the internal body part in the same reference coordinate system, ¶0029, ¶0048. There a multitude of methods described by Ben-Yishai 2019 to determine initial positions which include: Using a tracked pointer (tool 220) to touch teach reference point, allowing the processor to determine its position, ¶0035, ¶0049. The reference points (i.e. landmarks) may be automatically identified in pre-acquired or inter-operative images (i.e., during the medical procedure), and since the model is registered, the processor determines the position of these reference points in the reference coordinate system, ¶0037. The reference points are limit emitters, the processor can determine their location in the reference coordinate system using images acquired by the optical detector of the HMD tracking unit, ¶0043.
display the representation in registration with the coordinate system on the near- eye display of the head-mounted display device using the transformation;
-Ben-Yishai 2019 discloses the processor 208 renders the model of the internal body part (i.e., representation) and provides it to the HMD 206 to be displayed on its visor 212. This model is displayed as a perspective corresponding to P&O of the user relative to the internal body part, such that the user perceives the model to be located at the correct position and orientation (i.e., in registration), ¶0023, ¶0027, ¶0029, ¶0052.
determine, at a time subsequent to determining the initial coordinates and during the medical procedure, subsequent coordinates of the plurality of landmarks relative to the representation; determine one or more congruences between the subsequent coordinates and the initial coordinates; and determine whether the registration is valid during the medical procedure based at least in part on the one or more congruences.
-Ben-Yishai 2019 describes automatic detection of effective miss-registration that involves comparing the determined real-time location of the landmarks with their predicted locations, these landmarks being indicative of coordinates, which constitutes determination of subsequent coordinates, and determining validity. Specifically, Ben-Yishai 2019 teaches during the procedure, and image of at least one reference point is acquired from a known P&O in the reference coordinate system, ¶0007, ¶0055. This image may come from an camera 111 fixed to the HMD, or from the optical detector in HMD tracking unit 202 if the landmarks are light emitters, ¶0007, ¶0042, ¶0044-0055. The processor determines the actual location (i.e., subsequent coordinates/image location) of the reference point(s) in the acquired image, ¶0056-0057. The processor also determines the expected image location of the reference point(s) based on its known, predetermined position (i.e., initial coordinates) in the reference coordinate system and the determined P&O from which the image was acquired, ¶0044-0045, ¶0056-0057, Claim 26. The processor then determines the congruence (i.e., similarity or difference) between the expected location (i.e., indicative of the initial coordinates) and the determined actual location of the reference point in the image, ¶0044-0045, ¶0058. When the model and the internal body part are effectively registered, the expected (initial) and determined (subsequent) image locations are congruent (i.e., substantially equal) with each other, meaning the registration is valid, ¶0044-0045, ¶0058. When the locations are not congruent, the processor determines that the internal body part and the model thereof are effectively not registered (i.e., invalid registration), ¶0013, ¶0044-0045, ¶0058. The congruence check functions as the comparison of coordinates and the determined upon which the system determines the registration validity, ¶0044-0055, ¶0058.
Ben-Yishai 2019 fails to explicitly teach (distances) (i.e., determine one or more distances between the subsequent coordinates and the initial coordinates,) such that the validity of the registration during the medical procedure is based at least in part on the one or more distance, as required by the claim.
However, Ben-Yishai 2021 in the context of registration systems discloses, determine one or more distances between the subsequent coordinates and the initial coordinates; and determine whether the registration is valid during the medical procedure based at least in part on the one or more distances.
-Ben-Yishai 2021 explicitly defines error estimations used during registration. The “type two error estimation” relating to the distance between the position of the markers in the registered model coordinate system (i.e., initial reference point from the model) and the position of the marker in the reference coordinate system (the subsequent coordinate), ¶0097. In this two type estimation if the error (i.e., distance) is large, the system may discard that marker automatically, or recommend the user discard it manually, this improving the registration (i.e., validity), ¶0097. A third type of error estimation is also included which is the average of the type two errors of all the markers. Registration related information is then displayed to the user to include registration scores. These scores are another exampled of indicating the quality or validity of the registration, ¶0065, ¶0068, ¶0097.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the determination between the subsequent and initial coordinates of Ben-Yishai 2019 in view of the teachings of Ben-Yishai 2021 utilization of a determined distance. The motivation to do this yields predictable results such as improving the registration as explicitly suggested by Ben-Yishai 2021, ¶0065, ¶0097.
Claim 61: Ben-Yishai 2019 as modified discloses all the elements above in claim 60, Ben-Yishai 2019 discloses, wherein the display head-mounted device comprises a wearable display device adapted to be worn on a head. (¶0006, ¶0013, ¶0033, ¶0040, Claim 10, FIG. 2)
Claim 62: Ben-Yishai 2019 as modified discloses all the elements above in claim 60, Ben-Yishai 2019 discloses, wherein the display head-mounted device comprises a pair of glasses. (¶0013)
Claim 66: Ben-Yishai 2019 as modified discloses all the elements above in claim 60, Ben-Yishai 2019 discloses, wherein the plurality of landmarks comprise stickers.
-The reference points can be a marking on the skin, ¶0017, Claim 7, Claim 15, an adhesive marker attached to the skin of the patient (i.e., sticker), ¶0034.
Claim 68: Ben-Yishai 2019 as modified discloses all the elements above in claim 60, Ben-Yishai 2019 discloses, wherein the plurality of landmarks comprise one or more radiopaque elements (¶0037, ‘For example, when the reference points are adhesive radio-opaque fiducial markers’).
Claim 69: Ben-Yishai 2019 as modified discloses all the elements above in claim 60, Ben-Yishai 2019 fails to disclose, wherein to cause the system to determine whether the registration is valid during the medical procedure, the one or more processors are configured to cause the system to determine, during the medical procedure, whether the one or more distances exceed a threshold.
However, Ben-Yishai 2021 is relied upon above discloses: wherein to cause the system to determine whether the registration is valid during the medical procedure, the one or more processors are configured to cause the system to determine, during the medical procedure, whether the one or more distances exceed a threshold.
-Ben-Yishai 2021 explicitly defines error estimations used during registration. The “type two error estimation” relating to the distance between the position of the markers in the registered model coordinate system (i.e., initial reference point from the model) and the position of the marker in the reference coordinate system (the subsequent coordinate), ¶0097. In this two type estimation if the error (i.e., distance) is large, the system may discard that marker automatically, or recommend the user discard it manually, this improving the registration (i.e., validity), ¶0097. A third type of error estimation is also included which is the average of the type two errors of all the markers. Registration related information is then displayed to the user to include registration scores. These scores are another exampled of indicating the quality or validity of the registration, ¶0065, ¶0068, ¶0097.
-Ben-Yishai 2021 explicitly discloses comparing the registration error to a threshold to determine if the registration process should continue, ¶0096. Specifically, if the registration error is above (i.e., exceeds) a threshold.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the processor’s validity determination of modified Ben-Yishai 2019 in view of the teachings of Ben-Yishai 2021 utilization of the determined distance by determining whether the one or more distances exceed a predefined threshold. The motivation to do this yields predictable results such as improving the registration as explicitly suggested by Ben-Yishai 2021, ¶0065, ¶0097
Claim 70: Ben-Yishai 2019 discloses, A system for tracking a transformation during a medical procedure, the transformation registering a representation of internal anatomy of a subject with a coordinate system with respect to one or more fiducial objects coupled to the subject, the system comprising:
-Ben-Yishai 2019 discloses, a system for verifying registration for a model of a body part with the internal body part in a reference coordinate system, ¶Abstract, ¶0006, Verifying registration (i.e., the tracking accuracy of the transformation) of the internal body during a medical procedure of a subject in a coordinate is described with respect to one or more fiducial objects, ¶Abstract, ¶0006, ¶0048. Ben-Yishai 2019 utilizes fiducial objects, referred to as reference points, ¶Abstract, ¶0005, ¶0007, ¶0034, ¶0037. The principle of Ben-Yishai 2019 is to ensure that the visualization of the internal anatomy remains accurately registered with the real anatomy, ¶Abstract, ¶0013.
a display device comprising a near-eye display and a tracking system;
-The system of Ben-Yishai 2019 includes a head-mounted display (e.g., HMD 206) to be donned by the user, ¶0006, ¶0023, Claim 10. The HMD includes a see through display (visor 212, Fig. 2), ¶0023, Claim 10. The visor relates to optically see through displays such as smart glass, augmented reality, or near to eye displays, ¶0013, ‘[...] visor’ used herein relates in general to optically see-through displays such as smart glasses, augmented reality glasses, near-to-eye displays, head mounted displays and head wearable displays’. The system also includes an HMD tracking module (HMD tracking unit 202 and a reference tracking unit 204) coupled to the processor (processor 208) that continuously determines position and orientation (‘P&O’) of the HMD in the reference coordinate system, ¶0006, ¶0025, Claim 10.
a plurality of landmarks configured to provide the one or more fiducial objects; and
-Ben-Yishai 2019 utilizes fiducial objects, referred to as reference points, ¶Abstract, ¶0005, ¶0007, ¶0034. The plurality of landmarks forming one or more fiducial objects, is described by the technique of using at least one reference point. Examples also include three reference points, ¶Abstract, ¶0005, Claim 1. The reference points can be a marking on the skin, ¶0017, Claim 7, Claim 15, an adhesive marker attached to the skin of the patient (i.e., sticker), ¶0034, Claim 7, Claim 15, or anatomical feature which is directly visible to the user, Claim 7, Claim 15. These references points are determined to be directly visible to the user, and the relative position between them and the internal body part is substantially constant. Their position in the reference coordinate system is predetermined, ¶Abstract, ¶0005-0007, ¶0013, ¶0034, Claim 1.
one or more processors, that upon execution of program instructions stored on a non- transitory computer-readable medium, are configured to cause the system to:
-Ben-Yishai 2019 describes the system 200 including a processor 208 that performs procedures to verify registration by comparing predicted and determined locations of reference points (i.e., landmarks), ¶0044-0045, ¶0056-0057. The processor 208 is coupled to the HMD tracking unit 202 and a reference tracking unit 204. This processor executes the steps required verification, ¶0006, ¶0026.
after obtaining the transformation, determine initial coordinates of the plurality of landmarks relative to the representation based on one or more images received by the tracking system of the plurality of landmarks and a site of the medical procedure;
-Ben-Yishai 2019 processor determines the position of the reference points (e.g., landmarks 2301-2303) in the reference coordinate system (e.g., 222) prior to the onset of the procedure, the claims do not require the extent of “during a medical procedure”, ¶0034, ¶0049. The model (i.e., representation of the internal anatomy) is registered with the internal body part in the same reference coordinate system, ¶0029, ¶0048. There a multitude of methods described by Ben-Yishai 2019 to determine initial positions which include: Using a tracked pointer (tool 220) to touch teach reference point, allowing the processor to determine its position, ¶0035, ¶0049. The reference points (i.e. landmarks) may be automatically identified in pre-acquired or inter-operative images (i.e., during the medical procedure), and since the model is registered, the processor determines the position of these reference points in the reference coordinate system, ¶0037. The reference points are limit emitters, the processor can determine their location in the reference coordinate system using images acquired by the optical detector of the HMD tracking unit, ¶0043.
display the representation in registration with the coordinate system on the near-eye display of the display device using the transformation;
-Ben-Yishai 2019 discloses the processor 208 renders the model of the internal body part (i.e., representation) and provides it to the HMD 206 to be displayed on its visor 212. This model is displayed as a perspective corresponding to P&O of the user relative to the internal body part, such that the user perceives the model to be located at the correct position and orientation (i.e., in registration), ¶0023, ¶0027, ¶0029, ¶0052.
determine, at a time subsequent to determining the initial coordinates and during the medical procedure, subsequent coordinates of the plurality of landmarks relative to the representation; determine one or more congruences between the subsequent coordinates and the initial coordinates; and determine whether the registration is valid during the medical procedure based at least in part on the one or more congruences.
-Ben-Yishai 2019 describes automatic detection of effective miss-registration that involves comparing the determined real-time location of the landmarks with their predicted locations, these landmarks being indicative of coordinates, which constitutes determination of subsequent coordinates, and determining validity.
-Specifically, Ben-Yishai 2019 teaches during the procedure, and image of at least one reference point is acquired from a known P&O in the reference coordinate system, ¶0007, ¶0055. This image may come from an camera 111 fixed to the HMD, or from the optical detector in HMD tracking unit 202 if the landmarks are light emitters, ¶0007, ¶0042, ¶0044-0055. The processor determines the actual location (i.e., subsequent coordinates/image location) of the reference point(s) in the acquired image, ¶0056-0057. The processor also determines the expected image location of the reference point(s) based on its known, predetermined position (i.e., initial coordinates) in the reference coordinate system and the determined P&O from which the image was acquired, ¶0044-0045, ¶0056-0057, Claim 26. The processor then determines the congruence (i.e., similarity or difference) between the expected location (i.e., indicative of the initial coordinates) and the determined actual location of the reference point in the image, ¶0044-0045, ¶0058. When the model and the internal body part are effectively registered, the expected (initial) and determined (subsequent) image locations are congruent (i.e., substantially equal) with each other, meaning the registration is valid, ¶0044-0045, ¶0058. When the locations are not congruent, the processor determines that the internal body part and the model thereof are effectively not registered (i.e., invalid registration), ¶0013, ¶0044-0045, ¶0058. The congruence check functions as the comparison of coordinates and the determined upon which the system determines the registration validity, ¶0044-0055, ¶0058.
Ben-Yishai 2019 fails to explicitly teach (distances) (i.e., determine one or more distances between the subsequent coordinates and the initial coordinates; and determine whether the registration is valid during the medical procedure based at least in part on the one or more distances.), as required by the claim.
However, Ben-Yishai 2021 in the context of registration systems discloses, determine one or more distances between the subsequent coordinates and the initial coordinates; and determine whether the registration is valid during the medical procedure based at least in part on the one or more distances.
-Ben-Yishai 2021 explicitly defines error estimations used during registration. The “type two error estimation” relating to the distance between the position of the markers in the registered model coordinate system (i.e., initial reference point from the model) and the position of the marker in the reference coordinate system (the subsequent coordinate), ¶0097. In this two type estimation if the error (i.e., distance) is large, the system may discard that marker automatically, or recommend the user discard it manually, this improving the registration (i.e., validity), ¶0097. A third type of error estimation is also included which is the average of the type two errors of all the markers. Registration related information is then displayed to the user to include registration scores. These scores are another exampled of indicating the quality or validity of the registration, ¶0065, ¶0068, ¶0097.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the determination between the subsequent and initial coordinates of Ben-Yishai 2019 in view of the teachings of Ben-Yishai 2021 utilization of a determined distance. The motivation to do this yields predictable results such as improving the registration as explicitly suggested by Ben-Yishai 2021, ¶0065, ¶0097
Claim 71: Ben-Yishai 2019 as modified discloses all the elements above in claim 70, Ben-Yishai 2019 discloses, wherein the plurality of landmarks are configured to be disposed on skin of the subject in proximity to the site of the medical procedure.
-Ben-Yishai 2019 utilizes fiducial objects, referred to as reference points, ¶Abstract, ¶0005, ¶0007, ¶0034. The plurality of landmarks forming one or more fiducial objects, is described by the technique of using at least one reference point. Examples also include three reference points, ¶Abstract, ¶0005, Claim 1. The reference points can be a marking on the skin, ¶0017, Claim 7, Claim 15, an adhesive marker attached to the skin of the patient (i.e., sticker), ¶0034, Claim 7, Claim 15, or anatomical feature which is directly visible to the user, Claim 7, Claim 15. These references points are determined to be directly visible to the user, and the relative position between them and the internal body part is substantially constant. Their position in the reference coordinate system is predetermined, ¶Abstract, ¶0005-0007, ¶0013, ¶0034, Claim 1.
Claim 72: Ben-Yishai 2019 as modified discloses all the elements above in claim 70, Ben-Yishai 2019 discloses, wherein the display device comprises a wearable display device adapted to be worn on a head. (¶0006, ¶0013, ¶0033, ¶0040, Claim 10, FIG. 2)
Claim 73: Ben-Yishai 2019 as modified discloses all the elements above in claim 70, Ben-Yishai 2019 discloses, wherein the display device comprises a pair of glasses. (¶0013)
Claim 76: Ben-Yishai 2019 as modified discloses all the elements above in claim 70, Ben-Yishai 2019 discloses, wherein the plurality of landmarks comprise stickers.
-The reference points can be a marking on the skin, ¶0017, Claim 7, Claim 15, an adhesive marker attached to the skin of the patient (i.e., sticker), ¶0034.
Claim 77: Ben-Yishai 2019 as modified discloses all the elements above in claim 70, Ben-Yishai 2019 fails to disclose: wherein to cause the system to determine whether the registration is valid during the medical procedure, the one or more processors are configured to cause the system to determine, during the medical procedure, whether the one or more distances exceed a threshold.
However, Ben-Yishai 2021 is relied upon above discloses: wherein to cause the system to determine whether the registration is valid during the medical procedure, the one or more processors are configured to cause the system to determine, during the medical procedure, whether the one or more distances exceed a threshold.
-Ben-Yishai 2021 explicitly defines error estimations used during registration. The “type two error estimation” relating to the distance between the position of the markers in the registered model coordinate system (i.e., initial reference point from the model) and the position of the marker in the reference coordinate system (the subsequent coordinate), ¶0097. In this two type estimation if the error (i.e., distance) is large, the system may discard that marker automatically, or recommend the user discard it manually, this improving the registration (i.e., validity), ¶0097. A third type of error estimation is also included which is the average of the type two errors of all the markers. Registration related information is then displayed to the user to include registration scores. These scores are another exampled of indicating the quality or validity of the registration, ¶0065, ¶0068, ¶0097.
-Ben-Yishai 2021 explicitly discloses comparing the registration error to a threshold to determine if the registration process should continue, ¶0096. Specifically, if the registration error is above (i.e., exceeds) a threshold.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the processor’s validity determination of modified Ben-Yishai 2019 in view of the teachings of Ben-Yishai 2021 utilization of the determined distance by determining whether the one or more distances exceed a predefined threshold. The motivation to do this yields predictable results such as improving the registration as explicitly suggested by Ben-Yishai 2021, ¶0065, ¶0097
Claim 78: Ben-Yishai 2019 as modified discloses all the elements above in claim 70, Ben-Yishai 2019 fails to disclose: wherein: to cause the system to determine whether the one or more distances exceed the threshold, the one or more processors are configured to cause the system to determine, during the medical procedure, that at least one distance of the one or more distances exceeds the threshold; and the one or more processors are configured to notify a user that the registration is not valid based on the at least one distance exceeding the threshold.
However, Ben-Yishai 2021 is relied upon above discloses: wherein: to cause the system to determine whether the one or more distances exceed the threshold, the one or more processors are configured to cause the system to determine, during the medical procedure, that at least one distance of the one or more distances exceeds the threshold; and the one or more processors are configured to notify a user that the registration is not valid based on the at least one distance exceeding the threshold.
-Ben-Yishai 2021 explicitly defines error estimations used during registration. The “type two error estimation” relating to the distance between the position of the markers in the registered model coordinate system (i.e., initial reference point from the model) and the position of the marker in the reference coordinate system (the subsequent coordinate), ¶0097. In this two type estimation if the error (i.e., distance) is large, the system may discard that marker automatically, or recommend the user discard it manually, this improving the registration (i.e., validity), ¶0097. A third type of error estimation is also included which is the average of the type two errors of all the markers. Registration related information is then displayed to the user to include registration scores. These scores are another exampled of indicating the quality or validity of the registration, ¶0065, ¶0068, ¶0097.
-Ben-Yishai 2021 explicitly discloses comparing the registration error to a threshold to determine if the registration process should continue, ¶0096. Specifically, if the registration error is above (i.e., exceeds) a threshold.
-Ben-Yishai teaches that the markers distance error exceeds an acceptable level should be removed from the registration calculation, which constitutes to not using them for subsequent validation, ¶0065, ¶0097. The user is indeed notified of such an exceed threshold, ¶0065, ¶0096, ¶0097,
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the processor’s validity determination of modified Ben-Yishai 2019 in view of the teachings of Ben-Yishai 2021 utilization of the determined distance by determining whether the one or more distances exceed a predefined threshold should not be used for subsequent validation. The motivation to do this yields predictable results such as improving the registration as explicitly suggested by Ben-Yishai 2021, ¶0065, ¶0097
Claim 79: Ben-Yishai 2019 discloses, A method for validating a registration between a tracking system and image data of an anatomy of a patient during an image-guided medical procedure, the method comprising:
-Ben-Yishai 2019 discloses, a system for verifying registration for a model of a body part with the internal body part in a reference coordinate system, ¶Abstract, ¶0006, Verifying registration (i.e., the tracking accuracy of the transformation) of the internal body during a medical procedure of a subject in a coordinate is described with respect to one or more fiducial objects, ¶Abstract, ¶0006, ¶0048. Ben-Yishai 2019 utilizes fiducial objects, referred to as reference points, ¶Abstract, ¶0005, ¶0007, ¶0034, ¶0037. The principle of Ben-Yishai 2019 is to ensure that the visualization of the internal anatomy remains accurately registered with the real anatomy, ¶Abstract, ¶0013.
computing and recording initial locations of a plurality of landmarks tracked by the tracking system in a target coordinate system of the registration, wherein the plurality of landmarks are locations on a skin of the patient indicated by a user;
-Ben-Yishai 2019 discloses, a system for verifying registration for a model of a body part with the internal body part in a reference coordinate system, ¶Abstract, ¶0006, Verifying registration (i.e., the tracking accuracy of the transformation) of the internal body during a medical procedure of a subject in a coordinate is described with respect to one or more fiducial objects, ¶Abstract, ¶0006, ¶0048. Ben-Yishai 2019 utilizes fiducial objects, referred to as reference points, ¶Abstract, ¶0005, ¶0007, ¶0034, ¶0037. The principle of Ben-Yishai 2019 is to ensure that the visualization of the internal anatomy remains accurately registered with the real anatomy, ¶Abstract, ¶0013.
-Ben-Yishai 2019 processor determines the position of the reference points (e.g., landmarks 2301-2303) in the reference coordinate system (e.g., 222) prior to the onset of the procedure, the claims do not require the extent of “during a medical procedure”, ¶0034, ¶0049. The model (i.e., representation of the internal anatomy) is registered with the internal body part in the same reference coordinate system, ¶0029, ¶0048. There a multitude of methods described by Ben-Yishai 2019 to determine initial positions which include: Using a tracked pointer (tool 220) to touch teach reference point, allowing the processor to determine its position, ¶0035, ¶0049. The reference points (i.e. landmarks) may be automatically identified in pre-acquired or inter-operative images (i.e., during the medical procedure), and since the model is registered, the processor determines the position of these reference points in the reference coordinate system, ¶0037. The reference points are limit emitters, the processor can determine their location in the reference coordinate system using images acquired by the optical detector of the HMD tracking unit, ¶0043.
at a time subsequent to computing and recording the initial locations and during the image-guided medical procedure, computing a subsequent location of one or more landmarks of the plurality of landmarks in the target coordinate system of the registration;
-Ben-Yishai 2019 describes automatic detection of effective miss-registration that involves comparing the determined real-time location of the landmarks with their predicted locations, these landmarks being indicative of coordinates, which constitutes determination of subsequent coordinates, and determining validity.
-Specifically, Ben-Yishai 2019 teaches during the procedure, and image of at least one reference point is acquired from a known P&O in the reference coordinate system, ¶0007, ¶0055. This image may come from an camera 111 fixed to the HMD, or from the optical detector in HMD tracking unit 202 if the landmarks are light emitters, ¶0007, ¶0042, ¶0044-0055. The processor determines the actual location (i.e., subsequent coordinates/image location) of the reference point(s) in the acquired image, ¶0056-0057. The processor also determines the expected image location of the reference point(s) based on its known, predetermined position (i.e., initial coordinates) in the reference coordinate system and the determined P&O from which the image was acquired, ¶0044-0045, ¶0056-0057, Claim 26. The processor then determines the congruence (i.e., similarity or difference) between the expected location (i.e., indicative of the initial coordinates) and the determined actual location of the reference point in the image, ¶0044-0045, ¶0058. When the model and the internal body part are effectively registered, the expected (initial) and determined (subsequent) image locations are congruent (i.e., substantially equal) with each other, meaning the registration is valid, ¶0044-0045, ¶0058. When the locations are not congruent, the processor determines that the internal body part and the model thereof are effectively not registered (i.e., invalid registration), ¶0013, ¶0044-0045, ¶0058. The congruence check functions as the comparison of coordinates and the determined upon which the system determines the registration validity, ¶0044-0055, ¶0058.
Ben-Yishai 2019 fails to disclose:
for each respective landmark of the one or more landmarks, computing a distance between an initial location, among the initial locations, associated with the respective landmark and the subsequent location of the respective landmark; and
wherein the computed distance for at least one landmark of the one or more landmarks exceeds a threshold, notifying the user that the registration is not valid.”
However, Ben-Yishai 2021 in the context of registration systems discloses,
for each respective landmark of the one or more landmarks, computing a distance between an initial location, among the initial locations, associated with the respective landmark and the subsequent location of the respective landmark; and
wherein the computed distance for at least one landmark of the one or more landmarks exceeds a threshold, notifying the user that the registration is not valid.”
-Ben-Yishai 2021 explicitly defines error estimations used during registration. The “type two error estimation” relating to the distance between the position of the markers in the registered model coordinate system (i.e., initial reference point from the model) and the position of the marker in the reference coordinate system (the subsequent coordinate), ¶0097. In this two type estimation if the error (i.e., distance) is large, the system may discard that marker automatically, or recommend the user discard it manually, this improving the registration (i.e., validity), ¶0097. A third type of error estimation is also included which is the average of the type two errors of all the markers. Registration related information is then displayed to the user to include registration scores. These scores are another exampled of indicating the quality or validity of the registration, ¶0065, ¶0068, ¶0097.
-Ben-Yishai 2021 explicitly discloses comparing the registration error to a threshold to determine if the registration process should continue, ¶0096. Specifically, if the registration error is above (i.e., exceeds) a threshold.
-Ben-Yishai teaches that the markers distance error exceeds an acceptable level should be removed from the registration calculation, which constitutes to not using them for subsequent validation, ¶0065, ¶0097. The user is indeed notified of such an exceed threshold, ¶0065, ¶0096, ¶0097,
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the registration of Ben-Yishai 2019 in view of the teachings of Ben-Yishai 2021. The motivation to do this yields predictable results such as improving the registration as explicitly suggested by Ben-Yishai 2021, ¶0065, ¶0097
Claims 63-64 & 74 are rejected under 35 U.S.C. 103 as being unpatentable over Ben-Yishai (US 2019/0108645 A1, herein Ben-Yishai 2019) in view of Ben-Yishai et al (US 2021/0186355 A1, herein Ben-Yishai 2021), as applied to claim 60 & 70 respectively, in further view of Healy et al (US 2021/0352267 A1).
Claim 63: Ben-Yishai 2019 as modified discloses all the elements above in claim 60, Ben-Yishai 2019 fails to disclose: wherein the tracking system comprises an infrared camera.
However, Healy in the context of head-mounted displays for computed assisted navigation in surgery discloses: wherein the tracking system (tracking system 6) comprises an infrared camera. (FIG. 9; ¶Abstract, ‘A head-mounted extended reality (XR) display device includes a rigid mounting element coupled to a frame. The XR display device further includes right-side and left-side visible light cameras coupled to the rigid mounting element, right-side and left-side near-infrared (NIR) cameras coupled to the rigid mounting element, and an NIR light-emitting diode (LED) configured to illuminate a region within a field of view of the NIR cameras. The visible light cameras are configured to capture stereoscopic visible light images within a field of view of the user when the user is wearing the frame, and the NIR cameras are configured to capture stereoscopic NIR images within the field of view of the user when the user is wearing the frame.’, see also Claim 1.)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the tacking system of modified Ben-Yishai 2019 to comprise an infrared camera as taught by Healy. The motivation to do this yield predictable results such as providing improvements to computer assisted navigation during surgery, as suggested by Healy, ¶0003, ¶0032.
Claim 64: Ben-Yishai 2019 as modified discloses all the elements above in claim 63, Ben-Yishai 2019 fails to disclose: wherein the tracking system further comprises a projector configured to project infrared light toward the site of the medical procedure.
However, Healy is relied upon above discloses: wherein the tracking system (tracking system 6) further comprises a projector configured to project infrared light toward the site of the medical procedure. (FIG. 9; ¶Abstract, ‘A head-mounted extended reality (XR) display device includes a rigid mounting element coupled to a frame. The XR display device further includes right-side and left-side visible light cameras coupled to the rigid mounting element, right-side and left-side near-infrared (NIR) cameras coupled to the rigid mounting element, and an NIR light-emitting diode (LED) configured to illuminate a region within a field of view of the NIR cameras. The visible light cameras are configured to capture stereoscopic visible light images within a field of view of the user when the user is wearing the frame, and the NIR cameras are configured to capture stereoscopic NIR images within the field of view of the user when the user is wearing the frame.’, see also Claim 1.)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the tacking system of modified Ben-Yishai 2019 to further comprise a projector configured to project infrared light toward the site of the medical procedure as taught by Healy. The motivation to do this yield predictable results such as providing improvements to computer assisted navigation during surgery, as suggested by Healy, ¶0003, ¶0032.
Claim 74: Ben-Yishai 2019 as modified discloses all the elements above in claim 70, Ben-Yishai 2019 fails to disclose: wherein the tracking system comprises an infrared camera and a projector configured to project infrared light toward the site of the medical procedure.
However, Healy in the context of head-mounted displays for computed assisted navigation in surgery discloses: wherein the tracking system comprises (tracking system 6) an infrared camera and a projector configured to project infrared light toward the site of the medical procedure. (FIG. 9; ¶Abstract, ‘A head-mounted extended reality (XR) display device includes a rigid mounting element coupled to a frame. The XR display device further includes right-side and left-side visible light cameras coupled to the rigid mounting element, right-side and left-side near-infrared (NIR) cameras coupled to the rigid mounting element, and an NIR light-emitting diode (LED) configured to illuminate a region within a field of view of the NIR cameras. The visible light cameras are configured to capture stereoscopic visible light images within a field of view of the user when the user is wearing the frame, and the NIR cameras are configured to capture stereoscopic NIR images within the field of view of the user when the user is wearing the frame.’, see also Claim 1.)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the tacking system of modified Ben-Yishai 2019 to an infrared camera and wherein the tracking system further comprises a projector configured to project infrared light toward the site of the medical procedure as taught by Healy. The motivation to do this yield predictable results such as providing improvements to computer assisted navigation during surgery, as suggested by Healy, ¶0003, ¶0032.
Claim 67 is rejected under 35 U.S.C. 103 as being unpatentable over Ben-Yishai (US 2019/0108645 A1, herein Ben-Yishai 2019) in view of Ben-Yishai et al (US 2021/0186355 A1, herein Ben-Yishai 2021), as applied to claim 60, in further view of Leis (US 6,061,644).
Claim 67: Ben-Yishai 2019 as modified discloses all the elements above in claim 60, Ben-Yishai 2019 fails to disclose: wherein the plurality of landmarks comprise one or more uniquely identifying retroreflective elements.
However, Leis in the context for system determine spatial position and orientation of a body disclsoes, wherein the plurality of landmarks comprise one or more uniquely identifying retroreflective elements. ([Col 3 l.63-66], ‘Here, each of the energy retro-reflecting markers 12a, 12c includes a sphere, affixable to body 11a, covered with a retro-reflective material as is generally available and well known in the art.’; [Col. 4 l.55-59], ‘both the reflected energy from the energy retro-reflecting markers 12a, 12c and the emitted energy from the energy emitting marker 12b, in order to create a focused energy image of the emitted or reflected energy from the markers 12a, 12b, 12c respectively’; [Col 5-6 l.66-67 to l.1-3], ‘That is, the markers 12a-12c, 12d-12f are placed on bodies 11a, 11b, respectively to provide each one of the bodies 11a, 11b with a unique signature, or finger-print, that can be recognized and tracked by the processor 28.’)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the plurality of landmarks of modified Ben-Yishai 2019 to be configured to comprise retro-reflective elements that are uniquely identified as taught by Leis. The motivation to do this yield predictable results such as providing unique signatures for each marker so each one can be automatically identified, as suggested by Leis, [Col 5-6], Claim 12.
Claims 65 & 75 is rejected under 35 U.S.C. 103 as being unpatentable over Ben-Yishai (US 2019/0108645 A1, herein Ben-Yishai 2019) in view of Ben-Yishai et al (US 2021/0186355 A1, herein Ben-Yishai 2021), as applied to claim 60 & 70 respectively, in further view of Weinstein et al (US 2018/0185100 A1).
Claim 65: Ben-Yishai 2019 as modified discloses all the elements above in claim 60, Ben-Yishai 2019 discloses, wherein the plurality of landmarks comprise reference points.
-Ben Yishai 2019 discloses that plurality of landmarks comprise reference points, ¶Abstract.
Ben-Yishai 2019 fails to explicitly disclose that the plurality of landmarks comprise registration markers.
However, Weinstein in the context of surgical navigation system requiring registration of anatomy and tools in a common coordinate system discloses, that the plurality of landmarks comprise registration markers. (¶0008, ‘The registration device comprises a plurality of registration markers for being analyzed in the images captured by the camera of the head-mounted display to determine a pose of the registration coordinate system relative to the HMD coordinate system.’)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the plurality of landmarks that comprise reference points of modified Ben-Yishai 2019, to comprise registration markers as taught by Weinstein. The motivation to do this yields predictable results such as improving visualization to users, as suggested by Weinstein, ¶Abstract.
Claim 75: Ben-Yishai 2019 as modified discloses all the elements above in claim 70, Ben-Yishai 2019 discloses, wherein the plurality of landmarks comprise reference points.
-Ben Yishai 2019 discloses that plurality of landmarks comprise reference points, ¶Abstract.
Ben-Yishai 2019 fails to explicitly disclose that the plurality of landmarks comprise registration markers.
However, Weinstein in the context of surgical navigation system requiring registration of anatomy and tools in a common coordinate system discloses, that the plurality of landmarks comprise registration markers. (¶0008, ‘The registration device comprises a plurality of registration markers for being analyzed in the images captured by the camera of the head-mounted display to determine a pose of the registration coordinate system relative to the HMD coordinate system.’)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the plurality of landmarks that comprise reference points of modified Ben-Yishai 2019, to comprise registration markers as taught by Weinstein. The motivation to do this yields predictable results such as improving visualization to users, as suggested by Weinstein, ¶Abstract.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nicholas Robinson whose telephone number is (571)272-9019. The examiner can normally be reached M-F 9:00AM-5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at (571) 272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.A.R./Examiner, Art Unit 3798
/PASCAL M BUI PHO/Supervisory Patent Examiner, Art Unit 3798
Prosecution Insights