DETAILED ACTION
This Office action is responsive to communications filed on 11/11/2025. Claims 1-2, 4-8, 10-12, 14-17, 19 have been amended. Claims 13, 18, 20 canceled. Presently, Claims 1-12, 14-17, & 19 remain pending and are hereinafter examined on the merits.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Previous objections to the Abstract are withdrawn in view of the amendments filed on 11/11/2025.
Previous interpretations under 35 USC § 112(f) are NOT withdrawn in view of the amendments filed on 11/11/2025.
Previous rejections under 35 USC § 112(b) are withdrawn in view of the amendments filed on 11/11/2025.
Previous claim objections are withdrawn in view of the amendments filed on 11/11/2025.
Previous rejections under 35 USC § 101 are withdrawn in view of the amendments filed on 11/11/2025.
Applicant’s arguments with respect to claim(s) rejected under 35 USC § 103 have been considered but are moot because the new ground of rejection does not rely on Marti et al (US 2021/0030479) in view of Falco (US 2007/0034731 A1) applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The new grounds of rejection now relied on, Falco (US 2007/0034731 A1) in view of Chen et al (US 20120154604 A1).
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action. This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
The nonce term “device” in the term “notification device” for outputting a notification is used in claim(s) 10-12, configured for outputting the re-registration notification invokes 35 USC 112(f)
The term, “device” is a non-structural generic placeholder that does not include any specific structure for performing the accompany functions. See MPEP 2181.I.A: The following is a list of non-structural generic placeholders that may invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, paragraph 6: "mechanism for," "module for," "device for," "unit for," "component for," "element for," "member for," "apparatus for," "machine for," or "system for." Welker Bearing Co., v. PHD, Inc., 550 F.3d 1090, 1096, 89 USPQ2d 1289, 1293-94 (Fed. Cir. 2008); Massachusetts Inst. of Tech. v. Abacus Software, 462 F.3d 1344, 1354, 80 USPQ2d 1225, 1228 (Fed. Cir. 2006); Personalized Media, 161 F.3d at 704, 48 USPQ2d at 1886–87; Mas-Hamilton Group v. LaGard, Inc., 156 F.3d 1206, 1214-1215, 48 USPQ2d 1010, 1017 (Fed. Cir. 1998).
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f).
Please note that for the purposes of this examination the phrase “notification device” is being interpreted to include generic optical device e.g., LED or multiple LEDs as described in ¶0071 in the specification as performing the claimed function, and equivalents thereof.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-12, 14-17, & 19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as failing to set forth the subject matter which the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the applicant regards as the invention.
Claim 1: “the impact on the camera”. There is insufficient antecedent basis for this limitation in the claim, as required by MPEP 2173.05(e). For examination purposes, the Examiner assumes, an impact on the camera system. Note; the Applicant’s remarks filed on 11/11/2025 pg. 13 reflect such the interpretation. Accordingly, proper antecedent basis is required.
The above rejections to claim 1 apply to claim 19 for substantially identical claim
limitations recited in the claim. Appropriate correction is required.
The dependent claims of the above rejected claims are rejected due to their dependency.
Claim Objections
The following claims are objected to because of the following informalities and should recite:
Claim 1: lines 5-6, “
Claim 5: line 10, “the at least one predetermined movement pattern.”
Claim 14: line 3, “generated at [[at]] least the tracker”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 8-12, 14, 17, & 19 are rejected under 35 U.S.C. 103 as being unpatentable over Falco (US 2007/0034731 A1) in view of Chen et al (US 20120154604 A1).
Claim 1: Falco discloses, A method for navigating a tracked surgical tool (¶0008-0009, ¶0013, ¶0032, ¶0037, ¶0041 – intended for tracking the physical location and orientation of objects ssuch as a “surgical instrument was they move with respect to a coordinate system.) based on a re-registration of a coordinate system (recalibrating or adjustment factor, ¶0012, Claim 1-4, ¶Abstract) associated with a tracker (tracked object 40 with included passive or active markers 50) attached to a patient or associated with a camera system (optical camera, optical tracker or infrared camera) configured to generate image data for tracking the tracker with a coordinate system associated with image data of the patient obtained by a medical imaging modality, the camera system comprises a first acceleration sensor configured to generate first inertial data indicative of an acceleration of the camera system,
-Falco refers to recalibrating or applying an adjustment factor to correct coordinate systems and compensation for changes in orientation over time, ¶Abstract, ¶0012, Claims 1-4. The tracking system utilized to monitor the surgical tools described is an optical camera, optical tracker, or infrared camera, ¶0004, ¶0013, ¶0041, Claim 14. If the camera system gets bumped or drifts over time, the relationship between the camera’s coordinate system and the fixed reference coordinate system becomes inaccurate, ¶0005, ¶0042. The system detects this alignment error and applies an adjustment factor, specifically, defined as a transformation, to adjust the location of the object from the second coordinate system (i.e., associated with the tracker) back into the first reference coordinate system, ¶0010-0012, Claims 1-3, (i.e., recalibrating the position and orientation of the position-tracking device) to the reference coordinate system to reduce the alignment errors, ¶0016, Claim 23-26.
-Falco discloses, using a hand-held ultrasound scanner or imaging probe as a primary example of a tracked medical device used to generate 3D images of the patient, (i.e., an association with image data of the patient obtained a medical imaging modality), ¶0009, ¶0032, ¶0037, ¶0041. When misalignment of tracking camera occurs, the system generates corrective information. This corrective is forwarded to the imaging system and used to compensate for the misalignment when “registering images to the reference coordinate system”, ¶0044.
-Falco discloses the integration of an acceleration sensor to detect movement of the camera tracking system. Specifically, the tracking system includes an independent motion detector device (90) mounted on or within the tracker to locally monitor its position and orientation, Claim 1, ¶0008, ¶0034, ¶0039. The accelerometer detects displacement and rotation to monitor whether the camera system has shifted from is calibrated position. The accelerometer functions by measuring motion date, such as “two out of three degrees of rotation” to capture misalignments of the tacker, Claim 1, ¶0039.
the method comprising the following steps performed by a processor (processing device, ¶0010-0011, Claim 1):
receiving the image data for tracking the tracker from the camera system; (¶0004-0005, ¶0008-0009, ¶0013, ¶0024, ¶0026, ¶0040)
analyzing the received image data for a positional change of the tracker indicative (-The primary focus of Falco is to analyze the tracking data to detect misalignments, ¶0010-0011, ¶0016) of at least one of
a drift of the tracker; and (temporal drifts, experimental drift of the system, slight drifts of the tracker housing, ¶0006, ¶0039, ¶0042.)
an impact on the tracker; (accidental collisions, physical collision with another object or person that knocks/bumps the tacker out of its calibrated alignment, ¶0006, ¶0042)
receiving, from the first acceleration sensor, the first inertial data (motion-detector device(s) ¶0008-0009, ¶0014, ¶0039, Claim 17 – motion-detector devices identifies an accelerometer mounted directly on or within the tracker. The accelerometer transmits motion/position data of the system, specifically, noting that an accelerometer is used to measure degrees of rotation to detect disturbances in the tracker’s orientation.);
determining if at least one first predetermined condition indicative of an impact on the camera system was fulfilled or not fulfilled based upon analyzing the received first inertial data;
-Falco discloses, the system evaluates the movement data against a set condition to determine if an action needs to be taken. Specifically, the processing device features an alert module that activates based on “the detected movement exceeding a predetermined threshold”, ¶0006, ¶0008-0009, ¶0015, Claim 18. This threshold acts as a predetermined condition if the accelerometer or optical data indicates a sudden displacement (i.e., an impact) that fulfills or exceeds this threshold, the system then recognizes that the camera is misaligned and requires recalibration, ¶0007, ¶0039, ¶0043. Further note, ¶0039, “multiple motion detectors 90, each with different degrees of freedom detection capabilities, can be combined to capture all possible types of tracking system 30 misalignment and fully correct for such misalignment.”
re-registering the coordinate system associated with the tracker attached to the patient or associated with the camera system configured to generate the image data for tracking the tracker with the coordinate system associated with the image data of the patient when the positional change of the tracker indicative of at least one of the drift of the tracker and the impact on the tracker is identified based on the received image data and the first predetermined condition is fulfilled; and
Note; upon review of the limitation, under the broadest reasonable interpretation the claim states that “the positional change of the tracker indicative of at least one of the drift of the tracker and the impact of the tracker is identified based on [emphasis added] the received image data and the at least one first predetermined condition is fulfilled”. The term “based on” is ambiguous and lacks precision. The term “based on” is a broad and ambiguous term that doesn’t clearly define the extent or nature of the relationship that claimed invention is intended to present. As such, the term “based on” implies that the claim invention is derived from or closely related to received image data and the at least one first predetermined condition is fulfilled. There are not explicit steps or additional limitations recited in the claim that preclude this interpretation.
-Accordingly, Falco discloses, that the spatial relationships between the tracking system’s coordinates and the room’s reference coordinates can be compromised by temporal drifts or accidental collisions, ¶0006. These misalignments can be caused by physical collisions with another object or person, slight drifts of the tracker housing, or experimental drift, ¶0033, ¶0042. To detect the positional changes image data is used or complementary with accelerometers where the tracking system (camera) monitors its own location relative to a set of fixed mounted markers, ¶0038, ¶0040, ¶0045. By receiving signals or image data from these emitters, the optical camera can detect its own movement, (i.e., movement of the calibrated tracking system relative to the wall-mounted markers would indicate misalignment of the tracking system), ¶0038, ¶0040, ¶0045.
-Once a positional change is identified, the system of Falco calculates an adjustment factor, (i.e. a transformation) based on the detected displacement, ¶Abstract, ¶0010-0012, Claim 1. Falco teaches that the system then adjust the location data form the tracking system’s coordinate system back into the fixed reference coordinate system, ¶0011-0012, Claim 1. Specifically, Flaco connects this re-registration process to the patient image data by describing the use of a tracked medical device, such as the ultrasound probe, ¶0032, ¶0041. When the tracking system is misaligned, the corrective information generated by the system is forwarded to the imaging system and is specifically “used to compensate for the misalignment when locating devices or registering images to the reference coordinate system 10.”, ¶0044. The system continuously checks if the tracker has moved stating that the system responds to an alert trigger based on the detected movement exceeding a predetermined threshold, ¶0009.
navigating a tracked surgical tool based on the re-registration (Because an un-calibrated tracker can no longer effectively determine the position of the medical instrument, the system must apply the corrective information to fix the coordinate alignment. This corrective information is used to accurately compensate for the misalignment when locating devices, effectively teaching that an ongoing navigation and tracking of the surgical tool is based directly on the re-registration, ¶0035-0037, ¶0042, ¶0044).
Falco fails to explicitly disclose the feature of, based on “the first predetermined condition is not [emphasis added] fulfilled.
However, Chen in the context of camera calibration systems discloses, a first predetermined condition is not fulfilled. (¶0027, ‘the control unit 140 is set in an auto-adjustment mode and the camera motion does not exceed the predetermined threshold; for example, the motion magnitude or motion direction is less than the predetermined threshold but more than zero, the control unit 140 can perform transformation of image coordinate so as to transform the coordinate system of the image captured by the first camera 110 to that of its original setting, and hence to reduce the labor maintenance cost for the camera 110.’)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the re-registration of Falco to include being based on the first determined condition not fulfilled as taught by Chen. The motivation to do this yield predictable results such as to reduce the labor maintenance cost for the camera, as suggested by Chen, ¶0027.
Claim 3: Falco as modified discloses all the elements above in claim 1, Falco discloses, wherein the at least one first predetermined condition comprises a threshold decision.
-Falco discloses, the system evaluates the movement data against a set condition to determine if an action needs to be taken. Specifically, the processing device features an alert module that activates based on “the detected movement exceeding a predetermined threshold”, ¶0006, ¶0008-0009, ¶0015, Claim 18. This threshold acts as a predetermined condition if the accelerometer or optical data indicates a sudden displacement (i.e., an impact) that fulfills or exceeds this threshold, the system then recognizes that the camera is misaligned and requires recalibration, ¶0007, ¶0039, ¶0043. Further note, ¶0039, “multiple motion detectors 90, each with different degrees of freedom detection capabilities, can be combined to capture all possible types of tracking system 30 misalignment and fully correct for such misalignment.”
Claim 8: Flaco as modified discloses all the elements above in claim 1, Flaco discloses: further comprising triggering a re- registration notification (¶0006, ‘The invention provides a method and apparatus to independently monitor a calibrated tracking system in order to detect temporal drifts or accidental collisions that can put the system out of alignment or calibration with respect to a fixed reference coordinate system. The invention may be configured to alert users to such misalignments, to disable related tracked objects such as treatment or imaging devices until such errors are corrected, and in some cases to automatically apply corrective actions by moving the tracking system or adjusting the treatment position or medical image accordingly.’; ¶0007, ‘a motion-system device is placed on or about the tracking device prior to, during, or soon after calibration. Once the tracker is calibrated to the fixed coordinate system, the motion-system device continuously or periodically determines if the tracker has moved relative to the fixed coordinate system, and provides feedback to the user and/or a control device. The feedback can be used merely to indicate that a new calibration event is necessary, or in some cases to recalibrate the tracker automatically.’, ¶0009, ‘The alert may be based, in some cases, on the detected movement exceeding a predetermined threshold. In some embodiments, the motion-detector provides a signal to the tracker (or to a control system for positioning the tracker) that may be used to automatically recalibrate the tracker, and/or to compensate for the detected movement.’; ¶0015, ‘an alert module for alerting a user of the apparatus of a movement of the tracking system. The alert may be based on the detected movement exceeding a predetermined threshold. The apparatus can also include a position-adjustment device. The processing device can provide a signal to the position-adjustment device such that the positioning device adjusts the position of the tracking system, thereby bringing the tracker into calibration with the first coordinate system. The apparatus can also include a control device for controlling the operation of one or more object of a medical device, treatment device and diagnostic device.’, ¶0039, ‘A signal may also be sent to a user of the portable device 40, either through a warning message being displayed on a display unit or by an audible warning signal being broadcast, if a misalignment has been detected. In some cases, the motion detector 90 can only detect a limited number of degrees of freedom of position and rotation.’).
Claim 9: Flaco as modified discloses all the elements above in claim 8, Flaco discloses: wherein the re-registration notification is at least one of an acoustic notification and an optical notification (¶0006, ‘The invention provides a method and apparatus to independently monitor a calibrated tracking system in order to detect temporal drifts or accidental collisions that can put the system out of alignment or calibration with respect to a fixed reference coordinate system. The invention may be configured to alert users to such misalignments, to disable related tracked objects such as treatment or imaging devices until such errors are corrected, and in some cases to automatically apply corrective actions by moving the tracking system or adjusting the treatment position or medical image accordingly.’; ¶0007, ‘a motion-system device is placed on or about the tracking device prior to, during, or soon after calibration. Once the tracker is calibrated to the fixed coordinate system, the motion-system device continuously or periodically determines if the tracker has moved relative to the fixed coordinate system, and provides feedback to the user and/or a control device. The feedback can be used merely to indicate that a new calibration event is necessary, or in some cases to recalibrate the tracker automatically.’, ¶0009, ‘The alert may be based, in some cases, on the detected movement exceeding a predetermined threshold. In some embodiments, the motion-detector provides a signal to the tracker (or to a control system for positioning the tracker) that may be used to automatically recalibrate the tracker, and/or to compensate for the detected movement.’; ¶0015, ‘an alert module for alerting a user of the apparatus of a movement of the tracking system. The alert may be based on the detected movement exceeding a predetermined threshold. The apparatus can also include a position-adjustment device. The processing device can provide a signal to the position-adjustment device such that the positioning device adjusts the position of the tracking system, thereby bringing the tracker into calibration with the first coordinate system. The apparatus can also include a control device for controlling the operation of one or more object of a medical device, treatment device and diagnostic device.’, ¶0039, ‘A signal may also be sent to a user of the portable device 40, either through a warning message being displayed on a display unit or by an audible warning signal being broadcast, if a misalignment has been detected. In some cases, the motion detector 90 can only detect a limited number of degrees of freedom of position and rotation.’).
Claim 10: Flaco as modified discloses all the elements above in claim 8, Flaco discloses: wherein a notification device is configured output the re-registration notification (¶0006, ‘The invention provides a method and apparatus to independently monitor a calibrated tracking system in order to detect temporal drifts or accidental collisions that can put the system out of alignment or calibration with respect to a fixed reference coordinate system. The invention may be configured to alert users to such misalignments, to disable related tracked objects such as treatment or imaging devices until such errors are corrected, and in some cases to automatically apply corrective actions by moving the tracking system or adjusting the treatment position or medical image accordingly.’; ¶0007, ‘a motion-system device is placed on or about the tracking device prior to, during, or soon after calibration. Once the tracker is calibrated to the fixed coordinate system, the motion-system device continuously or periodically determines if the tracker has moved relative to the fixed coordinate system, and provides feedback to the user and/or a control device. The feedback can be used merely to indicate that a new calibration event is necessary, or in some cases to recalibrate the tracker automatically.’, ¶0009, ‘The alert may be based, in some cases, on the detected movement exceeding a predetermined threshold. In some embodiments, the motion-detector provides a signal to the tracker (or to a control system for positioning the tracker) that may be used to automatically recalibrate the tracker, and/or to compensate for the detected movement.’; ¶0015, ‘an alert module for alerting a user of the apparatus of a movement of the tracking system. The alert may be based on the detected movement exceeding a predetermined threshold. The apparatus can also include a position-adjustment device. The processing device can provide a signal to the position-adjustment device such that the positioning device adjusts the position of the tracking system, thereby bringing the tracker into calibration with the first coordinate system. The apparatus can also include a control device for controlling the operation of one or more object of a medical device, treatment device and diagnostic device.’, ¶0039, ‘A signal may also be sent to a user of the portable device 40, either through a warning message being displayed on a display unit or by an audible warning signal being broadcast, if a misalignment has been detected. In some cases, the motion detector 90 can only detect a limited number of degrees of freedom of position and rotation.’).
Claim 11: Flaco as modified discloses all the elements above in claim 10, Flaco discloses: wherein the notification device is part of the camera system (¶0006, ‘The invention provides a method and apparatus to independently monitor a calibrated tracking system in order to detect temporal drifts or accidental collisions that can put the system out of alignment or calibration with respect to a fixed reference coordinate system. The invention may be configured to alert users to such misalignments, to disable related tracked objects such as treatment or imaging devices until such errors are corrected, and in some cases to automatically apply corrective actions by moving the tracking system or adjusting the treatment position or medical image accordingly.’; ¶0007, ‘a motion-system device is placed on or about the tracking device prior to, during, or soon after calibration. Once the tracker is calibrated to the fixed coordinate system, the motion-system device continuously or periodically determines if the tracker has moved relative to the fixed coordinate system, and provides feedback to the user and/or a control device. The feedback can be used merely to indicate that a new calibration event is necessary, or in some cases to recalibrate the tracker automatically.’, ¶0009, ‘The alert may be based, in some cases, on the detected movement exceeding a predetermined threshold. In some embodiments, the motion-detector provides a signal to the tracker (or to a control system for positioning the tracker) that may be used to automatically recalibrate the tracker, and/or to compensate for the detected movement.’; ¶0015, ‘an alert module for alerting a user of the apparatus of a movement of the tracking system. The alert may be based on the detected movement exceeding a predetermined threshold. The apparatus can also include a position-adjustment device. The processing device can provide a signal to the position-adjustment device such that the positioning device adjusts the position of the tracking system, thereby bringing the tracker into calibration with the first coordinate system. The apparatus can also include a control device for controlling the operation of one or more object of a medical device, treatment device and diagnostic device.’, ¶0039, ‘A signal may also be sent to a user of the portable device 40, either through a warning message being displayed on a display unit or by an audible warning signal being broadcast, if a misalignment has been detected. In some cases, the motion detector 90 can only detect a limited number of degrees of freedom of position and rotation.’).
Claim 12: Flaco as modified discloses all the elements above in claim 9, Flaco discloses: wherein a notification device is configured to output the re-registration notification(¶0006, ‘The invention provides a method and apparatus to independently monitor a calibrated tracking system in order to detect temporal drifts or accidental collisions that can put the system out of alignment or calibration with respect to a fixed reference coordinate system. The invention may be configured to alert users to such misalignments, to disable related tracked objects such as treatment or imaging devices until such errors are corrected, and in some cases to automatically apply corrective actions by moving the tracking system or adjusting the treatment position or medical image accordingly.’; ¶0007, ‘a motion-system device is placed on or about the tracking device prior to, during, or soon after calibration. Once the tracker is calibrated to the fixed coordinate system, the motion-system device continuously or periodically determines if the tracker has moved relative to the fixed coordinate system, and provides feedback to the user and/or a control device. The feedback can be used merely to indicate that a new calibration event is necessary, or in some cases to recalibrate the tracker automatically.’, ¶0009, ‘The alert may be based, in some cases, on the detected movement exceeding a predetermined threshold. In some embodiments, the motion-detector provides a signal to the tracker (or to a control system for positioning the tracker) that may be used to automatically recalibrate the tracker, and/or to compensate for the detected movement.’; ¶0015, ‘an alert module for alerting a user of the apparatus of a movement of the tracking system. The alert may be based on the detected movement exceeding a predetermined threshold. The apparatus can also include a position-adjustment device. The processing device can provide a signal to the position-adjustment device such that the positioning device adjusts the position of the tracking system, thereby bringing the tracker into calibration with the first coordinate system. The apparatus can also include a control device for controlling the operation of one or more object of a medical device, treatment device and diagnostic device.’, ¶0039, ‘A signal may also be sent to a user of the portable device 40, either through a warning message being displayed on a display unit or by an audible warning signal being broadcast, if a misalignment has been detected. In some cases, the motion detector 90 can only detect a limited number of degrees of freedom of position and rotation.’).
Claim 14: Falco as modified discloses all the elements above in claim 1, Falco discloses,
wherein the image data generated by the camera system for tracking the tracker is continuously generated at at least the tracker is imaged in the continuously generated image data and the method further comprising visualizing the image data generated by the camera system for tracking the tracker at least for a point in associated with the detected positional change of the tracker.
-Falco refers to recalibrating or applying an adjustment factor to correct coordinate systems and compensation for changes in orientation over time, ¶Abstract, ¶0012, Claims 1-4. The tracking system utilized to monitor the surgical tools described is an optical camera, optical tracker, or infrared camera, ¶0004, ¶0013, ¶0041, Claim 14. If the camera system gets bumped or drifts over time, the relationship between the camera’s coordinate system and the fixed reference coordinate system becomes inaccurate, ¶0005, ¶0042. The system detects this alignment error and applies an adjustment factor, specifically, defined as a transformation, to adjust the location of the object from the second coordinate system (i.e., associated with the tracker) back into the first reference coordinate system, ¶0010-0012, Claims 1-3, (i.e., recalibrating the position and orientation of the position-tracking device) to the reference coordinate system to reduce the alignment errors, ¶0016, Claim 23-26.
-Falco discloses, using a hand-held ultrasound scanner or imaging probe as a primary example of a tracked medical device used to generate 3D images of the patient, (i.e., an association with image data of the patient obtained a medical imaging modality), ¶0009, ¶0032, ¶0037, ¶0041. When misalignment of tracking camera occurs, the system generates corrective information. This corrective is forwarded to the imaging system and used to compensate for the misalignment when “registering images to the reference coordinate system”, ¶0044.
-Falco discloses the integration of an acceleration sensor to detect movement of the camera tracking system. Specifically, the tracking system includes an independent motion detector device (90) mounted on or within the tracker to locally monitor its position and orientation, Claim 1, ¶0008, ¶0034, ¶0039. The accelerometer detects displacement and rotation to monitor whether the camera system has shifted from is calibrated position. The accelerometer functions by measuring motion date, such as “two out of three degrees of rotation” to capture misalignments of the tacker, Claim 1, ¶0039.
-Accordingly, Falco discloses, that the spatial relationships between the tracking system’s coordinates and the room’s reference coordinates can be compromised by temporal drifts or accidental collisions, ¶0006. These misalignments can be caused by physical collisions with another object or person, slight drifts of the tracker housing, or experimental drift, ¶0033, ¶0042. To detect the positional changes image data is used or complementary with accelerometers where the tracking system (camera) monitors its own location relative to a set of fixed mounted markers, ¶0038, ¶0040, ¶0045. By receiving signals or image data from these emitters, the optical camera can detect its own movement, (i.e., movement of the calibrated tracking system relative to the wall-mounted markers would indicate misalignment of the tracking system), ¶0038, ¶0040, ¶0045.
-Once a positional change is identified, the system of Falco calculates an adjustment factor, (i.e. a transformation) based on the detected displacement, ¶Abstract, ¶0010-0012, Claim 1. Falco teaches that the system then adjust the location data form the tracking system’s coordinate system back into the fixed reference coordinate system, ¶0011-0012, Claim 1. Specifically, Flaco connects this re-registration process to the patient image data by describing the use of a tracked medical device, such as the ultrasound probe, ¶0032, ¶0041. When the tracking system is misaligned, the corrective information generated by the system is forwarded to the imaging system and is specifically “used to compensate for the misalignment when locating devices or registering images to the reference coordinate system 10.”, ¶0044. The system continuously checks if the tracker has moved stating that the system responds to an alert trigger based on the detected movement exceeding a predetermined threshold, ¶0009.
Claim 16: Falco as modified discloses all the elements above in claim 1, Falco discloses, wherein the first acceleration sensor is configured to measure a gravity vector.
-The detected movement is indicative of acceleration, as the accelerometer is explicitly listed as a suitable motion-detector device for detecting movement of the tracking system, ¶0008-0009, ¶0014, Claims 14-17. Flaco describes how a motion-detector is associated with the tracking system, FIG. 1, to detect its displacement, which includes changes in its location or orientation, ¶0010-0011, ¶0033-0034, Claim 1. The motion-detector may be an accelerometer, ¶0008-0009, ¶0014, Claims 15-17. Data this is relevant to the measurement of acceleration is inherently the effect of gravity, and Falco uses acceleration for determining position and orientation. Acceleration is due to gravity such that it indeed is a vector of gravity.
-Falco discloses the systema and method for detecting drifts in calibrated tracking systems include an alert module for altering a user of the apparatus of a movement of the tracking system, ¶0015, ¶0021-0023. This alert is based on the detected movement exceed a predetermined threshold, ¶0009, ¶0015. The motion detector associated with the tracker can alert a user if the tracker is no longer calibrated, and this alert is based on said detected movement exceeding the predetermined threshold, ¶0008-0009, ¶0015. Once the user is altered the system is triggered for automatic recalibration for the detected movement, ¶0008-0009, ¶0010-0011, ¶0017, ¶0023, ¶0031, ¶0035-0036, ¶0039, ¶0044. Therefore, by explicitly including accelerometers in this list of motion detection devices, the source teaches that the detected movement can indeed encompass acceleration.
-Since the accelerometer data, which is indicative of inertial data received, is the motion detection device use, then the predetermined threshold is indeed indicative of acceleration, ¶0008-0009, ¶0014-0015, ¶0024. Otherwise, how else would a predetermined threshold indicative of an accelerometer not include thresholding corresponding to acceleration to determine if the detected movement exceeds the predetermined threshold. Therefore, if the motion-detector device is an accelerometer, it measured acceleration as the “detected movement”. Consequently, the predetermined threshold would be set for these acceleration values, meaning that an alert is triggered when the measured acceleration surpasses that specific limit, ¶0008-0009, ¶0015, ¶0024.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the system and method of the camera system of Marti to include the teachings of Falco. The motivation to do this yields predictable results such as to address the problem of drift or accident misalignment of the tracking system itself, ¶0003-0004, ¶0005, ¶0033-0034 of Falco; thereby allowing tracking system to locate features accurately within a reference coordinate system, ¶Abstract of Falco.
Claim 17: Falco as modified discloses all the elements above in claim 16, Falco disclose:
further comprising: creating a coordinate system based on the measured gravity vector and a tracker position, wherein the step of determining the at least one first predetermined condition indicative on the impact on the camera system is fulfilled or not fulfilled based upon analyzing the received first inertial data comprises: verifying the positional change of the tracker based on the created coordinate system.
-Falco discloses, the system evaluates the movement data against a set condition to determine if an action needs to be taken. Specifically, the processing device features an alert module that activates based on “the detected movement exceeding a predetermined threshold”, ¶0006, ¶0008-0009, ¶0015, Claim 18. This threshold acts as a predetermined condition if the accelerometer or optical data indicates a sudden displacement (i.e., an impact) that fulfills or exceeds this threshold, the system then recognizes that the camera is misaligned and requires recalibration, ¶0007, ¶0039, ¶0043. Further note, ¶0039, “multiple motion detectors 90, each with different degrees of freedom detection capabilities, can be combined to capture all possible types of tracking system 30 misalignment and fully correct for such misalignment.”
-Accordingly, Falco discloses, that the spatial relationships between the tracking system’s coordinates and the room’s reference coordinates can be compromised by temporal drifts or accidental collisions, ¶0006. These misalignments can be caused by physical collisions with another object or person, slight drifts of the tracker housing, or experimental drift, ¶0033, ¶0042. To detect the positional changes image data is used or complementary with accelerometers where the tracking system (camera) monitors its own location relative to a set of fixed mounted markers, ¶0038, ¶0040, ¶0045. By receiving signals or image data from these emitters, the optical camera can detect its own movement, (i.e., movement of the calibrated tracking system relative to the wall-mounted markers would indicate misalignment of the tracking system), ¶0038, ¶0040, ¶0045.
-Once a positional change is identified, the system of Falco calculates an adjustment factor, (i.e. a transformation) based on the detected displacement, ¶Abstract, ¶0010-0012, Claim 1. Falco teaches that the system then adjust the location data form the tracking system’s coordinate system back into the fixed reference coordinate system, ¶0011-0012, Claim 1. Specifically, Flaco connects this re-registration process to the patient image data by describing the use of a tracked medical device, such as the ultrasound probe, ¶0032, ¶0041. When the tracking system is misaligned, the corrective information generated by the system is forwarded to the imaging system and is specifically “used to compensate for the misalignment when locating devices or registering images to the reference coordinate system 10.”, ¶0044. The system continuously checks if the tracker has moved stating that the system responds to an alert trigger based on the detected movement exceeding a predetermined threshold, ¶0009.
Claim 19: Falco discloses: A surgical system comprising: (¶0008-0009, ¶0013, ¶0032, ¶0037, ¶0041 – intended for tracking the physical location and orientation of objects ssuch as a “surgical instrument was they move with respect to a coordinate system.)
a tracker attached to a patient (tracked object 40 with included passive or active markers 50, see ¶0013);
a camera system (optical camera, optical tracker or infrared camera) configured to generate image data for tracking the tracker, the camera system comprising an acceleration sensor configured to generate first inertial data indicative of an acceleration of the camera system;
a data processing system for re-registration of a coordinate system associated with the tracker attached to the patient or associated with the camera system configured to generate the image data for tracking the tracker with a coordinate system associated with image data of the patient obtained by a medical imaging modality, the data processing system comprising a processor configured for:
-Falco refers to recalibrating or applying an adjustment factor to correct coordinate systems and compensation for changes in orientation over time, ¶Abstract, ¶0012, Claims 1-4. The tracking system utilized to monitor the surgical tools described is an optical camera, optical tracker, or infrared camera, ¶0004, ¶0013, ¶0041, Claim 14. If the camera system gets bumped or drifts over time, the relationship between the camera’s coordinate system and the fixed reference coordinate system becomes inaccurate, ¶0005, ¶0042. The system detects this alignment error and applies an adjustment factor, specifically, defined as a transformation, to adjust the location of the object from the second coordinate system (i.e., associated with the tracker) back into the first reference coordinate system, ¶0010-0012, Claims 1-3, (i.e., recalibrating the position and orientation of the position-tracking device) to the reference coordinate system to reduce the alignment errors, ¶0016, Claim 23-26.
-Falco discloses, using a hand-held ultrasound scanner or imaging probe as a primary example of a tracked medical device used to generate 3D images of the patient, (i.e., an association with image data of the patient obtained a medical imaging modality), ¶0009, ¶0032, ¶0037, ¶0041. When misalignment of tracking camera occurs, the system generates corrective information. This corrective is forwarded to the imaging system and used to compensate for the misalignment when “registering images to the reference coordinate system”, ¶0044.
-Falco discloses the integration of an acceleration sensor to detect movement of the camera tracking system. Specifically, the tracking system includes an independent motion detector device (90) mounted on or within the tracker to locally monitor its position and orientation, Claim 1, ¶0008, ¶0034, ¶0039. The accelerometer detects displacement and rotation to monitor whether the camera system has shifted from is calibrated position. The accelerometer functions by measuring motion date, such as “two out of three degrees of rotation” to capture misalignments of the tacker, Claim 1, ¶0039.
receiving the image data for tracking the tracker from the camera system; (¶0004-0005, ¶0008-0009, ¶0013, ¶0024, ¶0026, ¶0040)
analyzing the received image data for a positional change of the tracker (-The primary focus of Falco is to analyze the tracking data to detect misalignments, ¶0010-0011, ¶0016) indicative of at least one of
i) a drift of the tracker (temporal drifts, experimental drift of the system, slight drifts of the tracker housing, ¶0006, ¶0039, ¶0042.); and
ii) an impact on the tracker (accidental collisions, physical collision with another object or person that knocks/bumps the tacker out of its calibrated alignment, ¶0006, ¶0042.);
receiving, from the acceleration sensor, the first inertial data (motion-detector device(s) ¶0008-0009, ¶0014, ¶0039, Claim 17 – motion-detector devices identifies an accelerometer mounted directly on or within the tracker. The accelerometer transmits motion/position data of the system, specifically, noting that an accelerometer is used to measure degrees of rotation to detect disturbances in the tracker’s orientation.);
determining if at least one first predetermined condition indicative of the impact on the camera system is fulfilled or not fulfilled based upon analyzing the received first inertial data;
-Falco discloses, the system evaluates the movement data against a set condition to determine if an action needs to be taken. Specifically, the processing device features an alert module that activates based on “the detected movement exceeding a predetermined threshold”, ¶0006, ¶0008-0009, ¶0015, Claim 18. This threshold acts as a predetermined condition if the accelerometer or optical data indicates a sudden displacement (i.e., an impact) that fulfills or exceeds this threshold, the system then recognizes that the camera is misaligned and requires recalibration, ¶0007, ¶0039, ¶0043. Further note, ¶0039, “multiple motion detectors 90, each with different degrees of freedom detection capabilities, can be combined to capture all possible types of tracking system 30 misalignment and fully correct for such misalignment.”
re-registering the coordinate system associated with the tracker attached to the patient or associated with the camera system configured to generate the image data for tracking the tracker with the coordinate system associated with the image data of the patient when the positional change of the tracker indicative of at least one of the drift of the tracker and the impact on the tracker is identified based on the received image data and the at least one first predetermined condition is fulfilled; and
Note; upon review of the limitation, under the broadest reasonable interpretation the claim states that “the positional change of the tracker indicative of at least one of the drift of the tracker and the impact of the tracker is identified based on [emphasis added] the received image data and the at least one first predetermined condition is fulfilled”. The term “based on” is ambiguous and lacks precision. The term “based on” is a broad and ambiguous term that doesn’t clearly define the extent or nature of the relationship that claimed invention is intended to present. As such, the term “based on” implies that the claim invention is derived from or closely related to received image data and the at least one first predetermined condition is fulfilled. There are not explicit steps or additional limitations recited in the claim that preclude this interpretation.
-Accordingly, Falco discloses, that the spatial relationships between the tracking system’s coordinates and the room’s reference coordinates can be compromised by temporal drifts or accidental collisions, ¶0006. These misalignments can be caused by physical collisions with another object or person, slight drifts of the tracker housing, or experimental drift, ¶0033, ¶0042. To detect the positional changes image data is used or complementary with accelerometers where the tracking system (camera) monitors its own location relative to a set of fixed mounted markers, ¶0038, ¶0040, ¶0045. By receiving signals or image data from these emitters, the optical camera can detect its own movement, (i.e., movement of the calibrated tracking system relative to the wall-mounted markers would indicate misalignment of the tracking system), ¶0038, ¶0040, ¶0045.
-Once a positional change is identified, the system of Falco calculates an adjustment factor, (i.e. a transformation) based on the detected displacement, ¶Abstract, ¶0010-0012, Claim 1. Falco teaches that the system then adjust the location data form the tracking system’s coordinate system back into the fixed reference coordinate system, ¶0011-0012, Claim 1. Specifically, Flaco connects this re-registration process to the patient image data by describing the use of a tracked medical device, such as the ultrasound probe, ¶0032, ¶0041. When the tracking system is misaligned, the corrective information generated by the system is forwarded to the imaging system and is specifically “used to compensate for the misalignment when locating devices or registering images to the reference coordinate system 10.”, ¶0044. The system continuously checks if the tracker has moved stating that the system responds to an alert trigger based on the detected movement exceeding a predetermined threshold, ¶0009.
wherein the surgical system further comprises a tracked surgical tool configured to be navigated based on the re-registration (Because an un-calibrated tracker can no longer effectively determine the position of the medical instrument, the system must apply the corrective information to fix the coordinate alignment. This corrective information is used to accurately compensate for the misalignment when locating devices, effectively teaching that an ongoing navigation and tracking of the surgical tool is based directly on the re-registration, ¶0035-0037, ¶0042, ¶0044.).
Falco fails to explicitly disclose the feature of, based on “the first predetermined condition is not [emphasis added] fulfilled.
However, Chen in the context of camera calibration systems discloses, a first predetermined condition is not fulfilled. (¶0027, ‘the control unit 140 is set in an auto-adjustment mode and the camera motion does not exceed the predetermined threshold; for example, the motion magnitude or motion direction is less than the predetermined threshold but more than zero, the control unit 140 can perform transformation of image coordinate so as to transform the coordinate system of the image captured by the first camera 110 to that of its original setting, and hence to reduce the labor maintenance cost for the camera 110.’)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the re-registration of Falco to include being based on the first determined condition not fulfilled as taught by Chen. The motivation to do this yield predictable results such as to reduce the labor maintenance cost for the camera, as suggested by Chen, ¶0027.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Falco (US 2007/0034731 A1) in view of Chen et al (US 20120154604 A1), as applied to claim 3, in further view of Kwon et al (US 2006/0070439 A1).
Claim 4: Falco as modified discloses all the elements above in claim 3, Flaco discloses wherein the coordinate system associated with the tracker attached to the patient or associated with the camera system configured to generate the image data for tracking the tracker is re-registered with the coordinate system associated with the image data of the patient when the received first inertial data are indicative of an acceleration above a decision threshold of the threshold decision of at least 5 m/s2.
-The detected movement is indicative of acceleration, as the accelerometer is explicitly listed as a suitable motion-detector device for detecting movement of the tracking system, ¶0008-0009, ¶0014, Claims 14-17. Flaco describes how a motion-detector is associated with the tracking system, FIG. 1, to detect its displacement, which includes changes in its location or orientation, ¶0010-0011, ¶0033-0034, Claim 1. The motion-detector may be an accelerometer, ¶0008-0009, ¶0014, Claims 15-17.
-Falco discloses the systema and method for detecting drifts in calibrated tracking systems include an alert module for altering a user of the apparatus of a movement of the tracking system, ¶0015, ¶0021-0023. This alert is based on the detected movement exceed a predetermined threshold, ¶0009, ¶0015. The motion detector associated with the tracker can alert a user if the tracker is no longer calibrated, and this alert is based on said detected movement exceeding the predetermined threshold, ¶0008-0009, ¶0015. Once the user is altered the system is triggered for automatic recalibration for the detected movement, ¶0008-0009, ¶0010-0011, ¶0017, ¶0023, ¶0031, ¶0035-0036, ¶0039, ¶0044. Therefore, by explicitly including accelerometers in this list of motion detection devices, the source teaches that the detected movement can indeed encompass acceleration.
-Since the accelerometer data, which is indicative of inertial data received, is the motion detection device use, then the predetermined threshold is indeed indicative of acceleration, ¶0008-0009, ¶0014-0015, ¶0024. Otherwise, how else would a predetermined threshold indicative of an accelerometer not include thresholding corresponding to acceleration to determine if the detected movement exceeds the predetermined threshold. Therefore, if the motion-detector device is an accelerometer, it measured acceleration as the “detected movement”. Consequently, the predetermined threshold would be set for these acceleration values, meaning that an alert is triggered when the measured acceleration surpasses that specific limit, ¶0008-0009, ¶0015, ¶0024.
Modified Flaco fail to disclose: a decision threshold of the threshold decision of at least 5 m/s.
However, Kwon in the context of detecting free fall of devices discloses, decision threshold of an acceleration sensor to be greater than 9.8 m/s2. (FIG. 4; ¶0003, ¶0007, ¶0029, ¶0052, ‘it is assumed that a predetermined threshold is 0.55*9.8 m/s.sup.2,’, which is at least 5 m/s2)
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the decision threshold of modified Falco, such that includes the decision threshold of at least 5 m/s2 as taught by Kwon because it has been held before that "in the case where the claimed ranges overlap or lie inside ranges disclosed by the prior art a prima facie case of obviousness exists" (see MPEP 2144.05 subsection I), no criticality is given for the claimed value, one of ordinary skill in the art could have made the modification with known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results to one of ordinary skill in the art at the time of the invention. Accordingly, one of ordinary skill in the art would be able to obtain the claimed value through routine experimentation. The motivation to do this yields predictable results such as preventing unwanted damage to the device, ¶0003 of Kwon.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Falco (US 2007/0034731 A1) in view of Chen et al (US 20120154604 A1), as applied to claim 1, in further view of Marti et al (US 2021/0030479) in view of Seth (US 2020/0011669 A1).
Claim 5: Falco as modified discloses all the elements above in claim 1, Falco discloses:
wherein the step of analyzing the received image data for the positional change of the tracker comprises: wherein the coordinate system associated with the tracker attached to the patient or associated with the camera system configured to generate the image data for tracking the tracker is re-registered with the coordinate system associated with the image data of the patient is re-registered when the positional change of the tracker is indicative of the predetermined movement pattern.
-Falco refers to recalibrating or applying an adjustment factor to correct coordinate systems and compensation for changes in orientation over time, ¶Abstract, ¶0012, Claims 1-4. The tracking system utilized to monitor the surgical tools described is an optical camera, optical tracker, or infrared camera, ¶0004, ¶0013, ¶0041, Claim 14. If the camera system gets bumped or drifts over time, the relationship between the camera’s coordinate system and the fixed reference coordinate system becomes inaccurate, ¶0005, ¶0042. The system detects this alignment error and applies an adjustment factor, specifically, defined as a transformation, to adjust the location of the object from the second coordinate system (i.e., associated with the tracker) back into the first reference coordinate system, ¶0010-0012, Claims 1-3, (i.e., recalibrating the position and orientation of the position-tracking device) to the reference coordinate system to reduce the alignment errors, ¶0016, Claim 23-26.
-Falco discloses, using a hand-held ultrasound scanner or imaging probe as a primary example of a tracked medical device used to generate 3D images of the patient, (i.e., an association with image data of the patient obtained a medical imaging modality), ¶0009, ¶0032, ¶0037, ¶0041. When misalignment of tracking camera occurs, the system generates corrective information. This corrective is forwarded to the imaging system and used to compensate for the misalignment when “registering images to the reference coordinate system”, ¶0044.
-Falco discloses the integration of an acceleration sensor to detect movement of the camera tracking system. Specifically, the tracking system includes an independent motion detector device (90) mounted on or within the tracker to locally monitor its position and orientation, Claim 1, ¶0008, ¶0034, ¶0039. The accelerometer detects displacement and rotation to monitor whether the camera system has shifted from is calibrated position. The accelerometer functions by measuring motion date, such as “two out of three degrees of rotation” to capture misalignments of the tacker, Claim 1, ¶0039.
-Accordingly, Falco discloses, that the spatial relationships between the tracking system’s coordinates and the room’s reference coordinates can be compromised by temporal drifts or accidental collisions, ¶0006. These misalignments can be caused by physical collisions with another object or person, slight drifts of the tracker housing, or experimental drift, ¶0033, ¶0042. To detect the positional changes image data is used or complementary with accelerometers where the tracking system (camera) monitors its own location relative to a set of fixed mounted markers, ¶0038, ¶0040, ¶0045. By receiving signals or image data from these emitters, the optical camera can detect its own movement, (i.e., movement of the calibrated tracking system relative to the wall-mounted markers would indicate misalignment of the tracking system), ¶0038, ¶0040, ¶0045.
-Once a positional change is identified, the system of Falco calculates an adjustment factor, (i.e. a transformation) based on the detected displacement, ¶Abstract, ¶0010-0012, Claim 1. Falco teaches that the system then adjust the location data form the tracking system’s coordinate system back into the fixed reference coordinate system, ¶0011-0012, Claim 1. Specifically, Flaco connects this re-registration process to the patient image data by describing the use of a tracked medical device, such as the ultrasound probe, ¶0032, ¶0041. When the tracking system is misaligned, the corrective information generated by the system is forwarded to the imaging system and is specifically “used to compensate for the misalignment when locating devices or registering images to the reference coordinate system 10.”, ¶0044. The system continuously checks if the tracker has moved stating that the system responds to an alert trigger based on the detected movement exceeding the predetermined threshold, ¶0009.
Falco fails to disclose: deriving a movement pattern of the positional change from the received image data; and comparing the derived movement pattern to at least one predetermined movement pattern,
However, Marti in the context of registering surgical navigation system discloses, deriving a movement pattern of the positional change from the received image data; and
-Marti discloses, the system analyzes image data for a positional change of a marker indicative of a drift of the marker, ¶0155-0156. The optical tracking system (10) uses optical sensors (11) to sense fiducials (21) located on markers (20), ¶Abstract. From this raw data, the system processing means compute the pose (position and/or orientation) of Markers (20), ¶0004, ¶0104-0105. This involves identify fiducials within the data generated by the optical sensors, ¶0096, ¶0107-0108. This cycle is a continuous determination of the marker’s pose from optical data represents the analysis of the marker’s position and orientation, ¶0097-0098, ¶0171.
-Marti discloses, the system analyzes image data for a positional change of a marker indicative of a drift of the marker, ¶0155-0156. The optical tracking system (10) uses optical sensors (11) to sense fiducials (21) located on markers (20), ¶Abstract. From this raw data, the system processing means compute the pose (position and/or orientation) of Markers (20), ¶0004, ¶0104-0105. This involves identify fiducials within the data generated by the optical sensors, ¶0096, ¶0107-0108. This cycle is a continuous determination of the marker’s pose from optical data represents the analysis of the marker’s position and orientation, ¶0097-0098, ¶0171.
-The Markers (2) often include embedded marker sensors such as accelerometers, ¶0004, ¶0047-0049, ¶0102, ¶0155-0156. These inertial sensors are known to be prone to “drift”, ¶0155-0156. The drift of an inertial sensor can also be corrected when the Marker (20) is in view of the optical tracking system (10), ¶0156. Drift is a type of movement pattern specifical an error pattern in the positional change.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the deriving of the positional change of modified Falco such that a derived movement pattern is indicative of the positional change from the received image data as taught by Marti. The motivation to do this yield predictable results such as improving marker position and orientation results as suggested by Marti, ¶0057.
Modified Falco fails to disclose,
and comparing the derived movement pattern to at least one predetermined movement pattern,
However, Seth in the context of motion sensor drift correction discloses,
comparing the derived movement pattern to at least one predetermined movement pattern, (¶0065-0066)
wherein the re-registration signal is generated in case the positional change of the tracker is indicative of the predetermined movement pattern.
-The direct trigger for the generating the drift correction signal is the detection of deviations from the objects known predetermined shape, this process of precisely identifying the object’s current movement pattern (via pattern recognition is an integral step, ¶0065-0066.
-The motion decomposition module utilizes pattern recognition techniques to analyze sensor data. It collects translational and rotational movements (e.g., constant gravity vector for an accelerometer) and stores theses as known patterns in memory, ¶0065-0066, ¶0079-0080, Claims 3-5, Claims 14-16.
-The motion synthesis module uses this (patten of movement) from the motion decomposition model to select algorithms and synthesis the motion of the object, ¶0067, ¶0081-0082, Claim 1.
-The drift correction is based on the synthesized movement. The drift correction module 252 calculates the current tracking device rectified data output. This calculation is based on the synthesized movement, which incorporates the identified predetermined movement pattern of each IMU, ¶0005-0006, ¶0065-0066, ¶0081-0082, Claim 1. This synthesis module, in turns provides the context for the drift correction module to determine if and how much the IMUs have drifted from their known movement pattern, and then generate the appropriate correction, ¶0065-0066, ¶0068.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the movement pattern of modified Flaco to include the teachings of Seth. The motivation to do this yields predictable results such as improving drift correction, ¶0041 of Seth.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Falco (US 2007/0034731 A1) in view of Chen et al (US 20120154604 A1), as applied to claim 1, in further view of Seth (US 2020/0011669 A1).
Claim 6: Falco as modified discloses all the elements above in claim 1, Falco discloses:
wherein the at least one first predetermined condition is indicative of at least one predetermined movement pattern, and wherein the step of determining if the at least one predetermined condition indicative of the impact on the camera system is fulfilled or not fulfilled based upon analyzing the received first inertial data comprises: (motion-detector device(s) ¶0008-0009, ¶0014, ¶0039, Claim 17 – motion-detector devices identifies an accelerometer mounted directly on or within the tracker. The accelerometer transmits motion/position data of the system, specifically, noting that an accelerometer is used to measure degrees of rotation to detect disturbances in the tracker’s orientation.);
-Falco discloses, the system evaluates the movement data against a set condition to determine if an action needs to be taken. Specifically, the processing device features an alert module that activates based on “the detected movement exceeding a predetermined threshold”, ¶0006, ¶0008-0009, ¶0015, Claim 18. This threshold acts as a predetermined condition if the accelerometer or optical data indicates a sudden displacement (i.e., an impact) that fulfills or exceeds this threshold, the system then recognizes that the camera is misaligned and requires recalibration, ¶0007, ¶0039, ¶0043. Further note, ¶0039, “multiple motion detectors 90, each with different degrees of freedom detection capabilities, can be combined to capture all possible types of tracking system 30 misalignment and fully correct for such misalignment.”
Falco fails to disclose:
deriving a movement pattern from the received first inertial data; and
comparing the derived movement pattern to the at least one predetermined movement pattern.
However, Seth in the context of motion sensor drift correction discloses, deriving a movement pattern from the received first inertial data; and comparing the derived movement pattern to the at least one predetermined movement pattern.
-The direct trigger for the generating the drift correction signal is the detection of deviations from the objects known predetermined shape, this process of precisely identifying the object’s current movement pattern (via pattern recognition is an integral step, ¶0065-0066.
-The motion decomposition module utilizes pattern recognition techniques to analyze sensor data. It collects translational and rotational movements (e.g., constant gravity vector for an accelerometer) and stores theses as known patterns in memory, ¶0065-0066, ¶0079-0080, Claims 3-5, Claims 14-16.
-The motion synthesis module uses this (patten of movement) from the motion decomposition model to select algorithms and synthesis the motion of the object, ¶0067, ¶0081-0082, Claim 1.
-The drift correction is based on the synthesized movement. The drift correction module 252 calculates the current tracking device rectified data output. This calculation is based on the synthesized movement, which incorporates the identified predetermined movement pattern of each IMU, ¶0005-0006, ¶0065-0066, ¶0081-0082, Claim 1. This synthesis module, in turns provides the context for the drift correction module to determine if and how much the IMUs have drifted from their known movement pattern, and then generate the appropriate correction, ¶0065-0066, ¶0068.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify first predetermined condition of modified Falco to include the teachings of Seth. The motivation to do this yields predictable results such as improving drift correction, ¶0041 of Seth.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Falco (US 2007/0034731 A1) in view of Chen et al (US 20120154604 A1) in view of Seth (US 2020/0011669 A1), as applied to claim 6, in further view of Nazzaro et al (US 2021/0360100 A1).
Claim 7: Falco as modified discloses all the elements above in claim 6, Falco fails to disclose: wherein the at least one predetermined movement pattern is indicative of a damped oscillation.
However, Nazzaro in the context of electronic devices discloses, wherein the at least one predetermined movement pattern is indicative of a damped oscillation. (¶0079, ‘FIGS. 2A-2C, for induced oscillatory motion, the rate of decay of the peaks of the induced motion may be greater when the device is subjected to relatively more damping. Accordingly, the first condition may be that the rate of decay of the peaks is greater than a predetermined rate of decay. The predetermined rate of decay, in that case, may correspond to a rate of decay that is typically seen when the device is in contact with the user's body (though the predetermined rate may instead correspond to other conditions, such as when the device is on a table, or strapped securely to a user's wrist, and so on’; ¶0082, ‘if the captured motion data satisfies a second condition (e.g., the oscillations of the induced motion decay at a rate that is less than the predetermined rate), it is determined that the device is not in contact with the body of the user. In some embodiments, satisfaction of the second condition equates to a failure to satisfy the first condition. In other embodiments, the first and second condition are discontinuous, such that a failure to satisfy the first condition does not necessarily correspond to satisfaction of the second condition. In the latter case, the failure of the captured motion data to satisfy either the first or the second condition may correspond to an indeterminate result. In other words, the damping to which the device is subjected cannot be correlated with a particular physical condition with an appropriate degree of certainty.).
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the at least one predetermined movement pattern of modified Falco to be indicative of damped oscillation as taught by Nazzaro. The motivation to do this yields predictable results such as enabling accurate measurements by the sensor, ¶0031 of Nazzaro.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Falco (US 2007/0034731 A1) in view of Chen et al (US 20120154604 A1), as applied to claim 1, in further view of Marti et al (US 2021/0030479).
Claim 15: Falco as modified discloses all the elements above in claim 1, Falco fails to disclose: wherein the received first inertial data and the received image data are each associated with time stamps, and wherein the analyzed image data are associated with corresponding analyzed first inertial data based on the time stamps.
However, Marti in the context of registering surgical navigation system discloses, wherein the received first inertial data and the received image data are each associated with time stamps, and wherein the analyzed image data are associated with corresponding analyzed first inertial data based on the time stamps.
-Marti discloses, the system analyzes image data for a positional change of a marker indicative of a drift of the marker, ¶0155-0156. The optical tracking system (10) uses optical sensors (11) to sense fiducials (21) located on markers (20), ¶Abstract. From this raw data, the system processing means compute the pose (position and/or orientation) of Markers (20), ¶0004, ¶0104-0105. This involves identify fiducials within the data generated by the optical sensors, ¶0096, ¶0107-0108. This cycle is a continuous determination of the marker’s pose from optical data represents the analysis of the marker’s position and orientation, ¶0097-0098, ¶0171.
-The Markers (2) often include embedded marker sensors such as accelerometers, ¶0004, ¶0047-0049, ¶0102, ¶0155-0156. These inertial sensors are known to be prone to “drift”, ¶0155-0156. The drift of an inertial sensor can also be corrected when the Marker (20) is in view of the optical tracking system (10), ¶0156.
-To address the drift, the method of Marti involves sensor fusion. The optical pose data (derived from the image analysis by the optical tracking system (10) and the marker sensor data (from the inertial sensors) are fused to provide an improved marker position and/orientation, ¶0057-0059, ¶0097-0098, ¶0155-0156, ¶0117. Explicitly Marti discloses “drift of an inertial sensor can be also corrected when the Marker 20 is in view of the Optical Tracking System 10.”, ¶0156. This implies that the stable, absolute positional information obtained from the optical tracking, which analyzes image data) is used to detect and compensate for the accumulated errors (drift) from the relative measurements of the inertial sensors.
-To enable this precise sensor fusion and drift correction, synchronized timestamps in real time between the optical sensors data and the marker sensor data is determined, ¶0004, ¶0098-0100, ¶0156-0157, ¶0171.
It would have been obvious to one of ordinary skilled in the art before the effective filing date of the claimed invention to modify the received inertial and image data of modified Falco to be associated with time stamps, and wherein the analyzed image data are associated with corresponding analyzed first inertial data based on the time stamps as taught by Marti for the advantage of providing an improved system with such as system being able to synchronize in real-time to compensate for differences in time as suggested by Marti, ¶0067.
Allowable Subject Matter
Claim 2 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Note; a change in scope in view of the requested corrections will require further search and consideration.
The following is an examiner’s statement of reasons for allowance of claim 2:
The limitation recited in claim 2 regarding to the features of wherein the tracker comprises a second acceleration sensor configured to generate second inertial data indicative of an acceleration of the tracker, the method further comprising: receiving, from the second acceleration sensor of the tracker, the second inertial data; determining if at least one second predetermined condition is fulfilled or not fulfilled based upon analyzing the received second inertial data the at least one second predetermined condition indicative of at least one of i) the drift of the tracker; and ii) the impact on the tracker, wherein the coordinate system associated with the tracker attached to the patient or associated with the camera system configured to generate the image data for tracking the tracker is re-registered with the coordinate system associated with the image data of the patient when the positional change of the tracker indicative of at least one of the drift of the tracker and the impact on the tracker is identified based on the received image data and the at least one first predetermined condition is not fulfilled, while the at least one second predetermined condition is fulfilled, in combination with the other claimed elements, are not taught or disclosed in the prior arts
While individual aspects of these features may be known in the prior arts, the prior arts do not disclose or suggest a system the performs each of these features in a coordinated manner within a single framework. Although the closest prior arts below disclose aspects of these features individually, the closest prior art below do not disclose a system that integrates these elements in the manner claimed. In addition, the closest prior art below, taken collectively, fail to provide the necessary motivation or teaching to combine these features as claimed.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nicholas Robinson whose telephone number is (571)272-9019. The examiner can normally be reached M-F 9:00AM-5:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at (571) 272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.A.R./Examiner, Art Unit 3798
/PASCAL M BUI PHO/Supervisory Patent Examiner, Art Unit 3798