DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 34-52 are currently pending in the application.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/15/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 34, 38-40 and 43 are rejected under 35 U.S.C. 103 as being unpatentable over Kruger et a. (US 9,208,561 B2) in view of Casas (US 20160191887 A1).
Concerning claim 34, Kruger et al. (hereinafter Kruger) teaches a method for non-tactile patient registration of an object with a position detection system (col. 3, ll. 19-34), the method comprising:
photogrammetrically generating a surface model of the object based on one or more images captured with an image sensor unit positioned at a capturing position (col. 4, ll. 30-39, col. 5, ll. 35-36, l. 50 – performing a plurality of image recordings of a surface of the object or body part; col. 5, ll. 38-40: photogrammetrically generating a surface model of the object from the captured image), wherein the position detection system is an electromagnetic position detection system that has a working space having a predetermined electromagnetic field strength (col. 4, ll. 40-54), and wherein the image sensor unit comprises a position sensor positioned separate from the object (col. 2, ll. 65-67; fig. 2: position sensor 14’; col. 5, ll. 53-60);
determining the capturing position in a coordinate system of the position detection system, based on a sensor signal from the position sensor (col. 3, ll. 9-12 col. 5, ll. 56-60); and
relating the photogrammetrically generated surface model of the object to the position detection system based at least in part on the capturing position (col. 2, ll. 47-52).
Not explicitly taught is the image sensor unit comprising a motion sensor.
Casas, in the same field of endeavor, teaches using at least one 3D scanner imaging device comprising motion tracking devices that is separate from the object (¶0035: Tracking means, such as inertial measurement units, may be attached to the 3D scanner.; ¶0036; ¶¶0083-0084; ¶0137 – structure-from-motion, surface reconstruction, real-time registration of a 3D surface, etc.). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of tracking a position and orientation of an imaging device of Kruger in the manner described by Casas in order to provide real-time data for a more precise location and orientation of images and objects in the common coordinate system used (Casas, ¶0035). The use of motion tracking devices further enhances the system by allowing features such as gesture recognition allowing for interaction of the surgeon or other users with the computer during surgery through the real-time user interface means, without touching anything in the operating room (Casas, ¶0084).
Concerning claim 38, Kruger further teaches the method of claim 34, wherein the image sensor unit comprises one or more stereographic cameras, one or more cameras in a multi-camera arrangement, one or more infrared image sensors (col. 4, ll. 19-24), or a combination thereof.
Concerning claim 39, Kruger further teaches the method of claim 38, wherein the method comprises capturing the one or more images with the image sensor unit position at a capturing position outside of the working space (col. 4, ll. 40-54)
Concerning claim 40, Kruger in view of Casas teaches the method of claim 34, wherein the motion sensor is configured to track position of the image sensor unit along a path on which the image sensor unit is moved (Kruger, now incorporating the teachings of Casas,: fig. 2: position sensor 14’ (now a motion sensor); col. 5, ll. 53-60; Casas ¶0035).
Concerning claim 43, Kruger further teaches the method of claim 34, wherein the object is a face of a subject depicted in a captured image (fig. 3: pattern 22 is image on the surface of a subject’s head (i.e., a face in fig. 3)).
Claims 35-37, 44-49 and 52 are rejected under 35 U.S.C. 103 as being unpatentable over Kruger et a. (US 9,208,561 B2) in view of Casas (US 20160191887 A1), further in view of Pflugi et al. (“Augmented Marker Tracking for Peri-Acetabular Osteotomy Surgery”, 11 July 2017).
Concerning claim 35, Kruger in view of Casas teaches the method of claim 34. Not explicitly taught is the method, wherein, determining the capturing position comprises: detecting motion of the image sensor unit along a path relative to the position detection system, based on the sensor signal from the motion sensor; defining a spatial relationship that relates at least one point on the path in a coordinate system of the motion sensor to a known position in a coordinate system of the position detection system; and determining the capturing position in the coordinate system of the position detection system based on the detected motion of the image sensor along the path and the defined spatial relationship.
Pflugi teaches an augmented marker tracking system, wherein an inertial measurement unit (IMU) is used to measure the system’s orientation (Abstract); and
detecting motion of the image sensor unit along a path relative to the position detection system, based on the sensor signal from the motion sensor (section II-F: motion signals from the IMU are recorded (i.e., recording a path) and fused with the optically derived positions by a Kalman filter);
defining a spatial relationship that relates at least one point on the path in a coordinate system of the motion sensor to a known position in a coordinate system of the position detection system (section II-F; fig. 4: The Kalman filter is initialized using the first pose estimate from marker tracking); and
determining the capturing position in the coordinate system of the position detection system based on the detected motion of the image sensor along the path and the defined spatial relationship (section II-F: The subsequent positions of the augmented marker are then determined by the prediction/update steps of the Kalman filter based on this initial pose estimate and the recorded path. When marker detection fails, the filter does not perform an update step and instead, the final orientation output is the one after the prediction step.).
Taking the teachings of Kruger, Casas and Pflugi together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Kruger in the manner taught by Pflugi by using a motion sensor in order to determine the capturing position in a coordinate system of the position detection system. Such a modification is merely a simple substitution of one known element for another to obtain predictable results.
Concerning claim 36, Pflugi further teaches the method of claim 35, wherein the path starts at the known position in the coordinate system of the position detection system and ends at the capturing position (fig. 4: The Kalman filter is initialized using the first pose estimate from marker tracking (i.e., the start position is known in the coordinate system of the position detection system)).
Concerning claim 37, Pflugi further teaches the method of claim 35, wherein the path starts at the capturing position and ends at the known position in the coordinate system of the position detection system (fig. 4: The Kalman filter being bidirectional allows for data to be run forward and backward (i.e., the start position being the capturing position and ending at the known position in the coordinate system)).
Concerning claim 44, Kruger et al. (hereinafter Kruger) teaches a method for non-tactile patient registration of an object with a position detection system (col. 3, ll. 19-34), the method comprising:
photogrammetrically generating a surface model of the object based on one or more images captured with an image sensor unit positioned at a capturing position (col. 4, ll. 30-39, col. 5, ll. 35-36, l. 50 – performing a plurality of image recordings of a surface of the object or body part; col. 5, ll. 38-40: photogrammetrically generating a surface model of the object from the captured image), wherein the image sensor unit comprises a position sensor positioned separate from the object (col. 2, ll. 65-67; fig. 2: position sensor 14’; col. 5, ll. 53-60);
determining the capturing position in a coordinate system of the position detection system, based on a sensor signal from the position sensor (col. 3, ll. 9-12 col. 5, ll. 56-60); and
relating the photogrammetrically generated surface model of the object to the position detection system based at least in part on the capturing position (col. 2, ll. 47-52). Not explicitly taught is the image sensor unit comprising a motion sensor.
Casas, in the same field of endeavor, teaches using at least one 3D scanner imaging device comprising motion tracking devices that is separate from the object (¶0035: Tracking means, such as inertial measurement units, may be attached to the 3D scanner.; ¶0036; ¶¶0083-0084; ¶0137 – structure-from-motion, surface reconstruction, real-time registration of a 3D surface, etc.). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of tracking a position and orientation of an imaging device of Kruger in the manner described by Casas in order to provide real-time data for a more precise location and orientation of images and objects in the common coordinate system used (Casas, ¶0035). The use of motion tracking devices further enhances the system by allowing features such as gesture recognition allowing for interaction of the surgeon or other users with the computer during surgery through the real-time user interface means, without touching anything in the operating room (Casas, ¶0084).
Not explicitly taught by Kruger or Casas is the method, wherein, determining the capturing position comprises: detecting motion of the image sensor unit along a path relative to the position detection system, based on the sensor signal from the motion sensor; defining a spatial relationship that relates at least one point on the path in a coordinate system of the motion sensor to a known position in a coordinate system of the position detection system; and determining the capturing position in the coordinate system of the position detection system based on the detected motion of the image sensor along the path and the defined spatial relationship.
Pflugi teaches an augmented marker tracking system, wherein an inertial measurement unit (IMU) is used to measure the system’s orientation (Abstract); and
detecting motion of the image sensor unit along a path relative to the position detection system, based on the sensor signal from the motion sensor (section II-F: motion signals from the IMU are recorded (i.e., recording a path) and fused with the optically derived positions by a Kalman filter);
defining a spatial relationship that relates at least one point on the path in a coordinate system of the motion sensor to a known position in a coordinate system of the position detection system (section II-F; fig. 4: The Kalman filter is initialized using the first pose estimate from marker tracking); and
determining the capturing position in the coordinate system of the position detection system based on the detected motion of the image sensor along the path and the defined spatial relationship (section II-F: The subsequent positions of the augmented marker are then determined by the prediction/update steps of the Kalman filter based on this initial pose estimate and the recorded path. When marker detection fails, the filter does not perform an update step and instead, the final orientation output is the one after the prediction step.).
Taking the teachings of Kruger, Casas and Pflugi together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Kruger in the manner taught by Pflugi by using a motion sensor in order to determine the capturing position in a coordinate system of the position detection system. Such a modification is merely a simple substitution of one known element for another to obtain predictable results.
Concerning claim 45, Pflugi further teaches the method of claim 44, wherein the path starts at the known position in the coordinate system of the position detection system and ends at the capturing position (fig. 4: The Kalman filter is initialized using the first pose estimate from marker tracking (i.e., the start position is known in the coordinate system of the position detection system)).
Concerning claim 46, Pflugi further teaches the method of claim 44, wherein the path starts at the capturing position and ends at the known position in the coordinate system of the position detection system (fig. 4: The Kalman filter being bidirectional allows for data to be run forward and backward (i.e., the start position being the capturing position and ending at the known position in the coordinate system)).
Concerning claim 47, Kruger further teaches the method of claim 44, wherein the image sensor unit comprises one or more stereographic cameras, one or more cameras in a multi-camera arrangement, one or more infrared image sensors (col. 4, ll. 19-24), or a combination thereof.
Concerning claim 48, Kruger further teaches the method of claim 44, wherein the position detection system is an electromagnetic position detection system (col. 4, ll. 40-54), and wherein the position detection system has a working space having a predetermined electromagnetic field strength (col. 4, ll. 40-54).
Concerning claim 49, Kruger further teaches the method of claim 48, wherein the method comprises capturing the one or more images with the image sensor unit position at a capturing position outside of the working space (col. 4, ll. 40-54).
Concerning claim 52, Kruger further teaches the method of claim 344, wherein the object is a face of a subject depicted in a captured image (fig. 3: pattern 22 is image on the surface of a subject’s head (i.e., a face in fig. 3)).
Claim 41 is rejected under 35 U.S.C. 103 as being unpatentable over Kruger et a. (US 9,208,561 B2) in view of Casas (US 20160191887 A1), further in view of Lang (US 2017/0258526 A1) and in further view of Drako (US 20180225523 A1).
Concerning claim 41, Kruger in view of Casas teaches the method of claim 34. Not explicitly taught is the method, wherein capturing the one or more images captured with an image sensor unit positioned at a capturing position further comprises: analyzing streaming image data of the image sensor unit; and automatically triggering image capture based upon certain conditions.
Lang teaches methods for computer assisted surgery, wherein capturing the one or more images captured with an image sensor unit positioned at a capturing position further comprises:
analyzing streaming image data of the image sensor unit (¶1220: live stream of data); and
automatically triggering image capture based upon certain conditions (¶¶1270-1271: recognizing an anatomic landmark, optical marker, etc.).
Taking the teachings of Kruger, Casas and Lang together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the Kruger invention to capture and analyze streaming data and automatically triggering image capture based upon certain conditions in order to automatically activate the system. It is noted that Kruger, Casas and Lang fail to explicitly teach automatically triggering image capture based upon recognition of the object in a certain position relative to the image sensor unit.
Drako teaches a system, wherein image capture is triggered when certain postures or orientation of a skeleton are detected (¶0041). Taking the teachings of Kruger, Casas, Lang and Drako together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the Kruger invention to capture and analyze streaming data and automatically triggering image capture based upon recognition of the object in a certain position relative to the image sensor unit in order to automatically activate the system.
Claim 42 is rejected under 35 U.S.C. 103 as being unpatentable over Kruger et a. (US 9,208,561 B2) in view of Casas (US 20160191887 A1), further in view of Lang (US 2017/0258526 A1) and in further view of Zhang et al. (US 20170356824 A1).
Concerning claim 42, Kruger in view of Casas teaches the method of claim 34. Not explicitly taught is the method, wherein capturing the one or more images captured with an image sensor unit positioned at a capturing position further comprises: analyzing streaming image data of the image sensor unit; and automatically triggering image capture based upon certain conditions.
Lang teaches methods for computer assisted surgery, wherein capturing the one or more images captured with an image sensor unit positioned at a capturing position further comprises:
analyzing streaming image data of the image sensor unit (¶1220: live stream of data); and
automatically triggering image capture based upon certain conditions (¶¶1270-1271: recognizing an anatomic landmark, optical marker, etc.).
Taking the teachings of Kruger, Casas and Lang together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the Kruger invention to capture and analyze streaming data and automatically triggering image capture based upon certain conditions in order to automatically activate the system. It is noted that Kruger, Casas and Lang fail to explicitly teach automatically triggering image capture based upon a positional relation between a reference position sensor attached to the object and the image sensor unit.
Zhang et al. (hereinafter Zhang) teaches a system, wherein image capture is triggered based upon a positional relation between a reference position sensor attached to the object and the image sensor unit (¶0027: image capture begins by way of at least one position sensor). Taking the teachings of Kruger, Casas, Lang and Zhang together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the Kruger invention to capture and analyze streaming data and automatically triggering image capture based upon a positional relation between a reference position sensor attached to the object and the image sensor unit in order to automatically activate the system.
Claim 50 is rejected under 35 U.S.C. 103 as being unpatentable over Kruger et a. (US 9,208,561 B2) in view of Casas (US 20160191887 A1), further in view of Pflugi et al. (“Augmented Marker Tracking for Peri-Acetabular Osteotomy Surgery”, 11 July 2017), further in view of Lang (US 2017/0258526 A1) and in further view of Drako (US 20180225523 A1).
Concerning claim 50, Kruger in view of Casas, further in view of Pflugi teaches the method of claim 44. Not explicitly taught is the method, wherein capturing the one or more images captured with an image sensor unit positioned at a capturing position further comprises: analyzing streaming image data of the image sensor unit; and automatically triggering image capture based upon certain conditions.
Lang teaches methods for computer assisted surgery, wherein capturing the one or more images captured with an image sensor unit positioned at a capturing position further comprises:
analyzing streaming image data of the image sensor unit (¶1220: live stream of data); and
automatically triggering image capture based upon certain conditions (¶¶1270-1271: recognizing an anatomic landmark, optical marker, etc.).
Taking the teachings of Kruger, Casas, Pflugi and Lang together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the Kruger invention to capture and analyze streaming data and automatically triggering image capture based upon certain conditions in order to automatically activate the system. It is noted that Kruger, Casas, Pflugi and Lang fail to explicitly teach automatically triggering image capture based upon recognition of the object in a certain position relative to the image sensor unit.
Drako teaches a system, wherein image capture is triggered when certain postures or orientation of a skeleton are detected (¶0041). Taking the teachings of Kruger, Casas, Pflugi, Lang and Drako together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the Kruger invention to capture and analyze streaming data and automatically triggering image capture based upon recognition of the object in a certain position relative to the image sensor unit in order to automatically activate the system.
Claim 51 is rejected under 35 U.S.C. 103 as being unpatentable over Kruger et a. (US 9,208,561 B2) in view of Casas (US 20160191887 A1), further in view of Pflugi et al. (“Augmented Marker Tracking for Peri-Acetabular Osteotomy Surgery”, 11 July 2017), further in view of Lang (US 2017/0258526 A1) and in further view of Zhang et al. (US 20170356824 A1).
Concerning claim 51, Kruger in view of Casas, further in view of Pflugi teaches the method of claim 44. Not explicitly taught is the method, wherein capturing the one or more images captured with an image sensor unit positioned at a capturing position further comprises: analyzing streaming image data of the image sensor unit; and automatically triggering image capture based upon certain conditions.
Lang teaches methods for computer assisted surgery, wherein capturing the one or more images captured with an image sensor unit positioned at a capturing position further comprises:
analyzing streaming image data of the image sensor unit (¶1220: live stream of data); and
automatically triggering image capture based upon certain conditions (¶¶1270-1271: recognizing an anatomic landmark, optical marker, etc.).
Taking the teachings of Kruger, Casas, Pflugi and Lang together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the Kruger invention to capture and analyze streaming data and automatically triggering image capture based upon certain conditions in order to automatically activate the system. It is noted that Kruger, Casas, Pflugi and Lang fail to explicitly teach automatically triggering image capture based upon a positional relation between a reference position sensor attached to the object and the image sensor unit.
Zhang et al. (hereinafter Zhang) teaches a system, wherein image capture is triggered based upon a positional relation between a reference position sensor attached to the object and the image sensor unit (¶0027: image capture begins by way of at least one position sensor). Taking the teachings of Kruger, Casas, Pflugi, Lang and Zhang together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the Kruger invention to capture and analyze streaming data and automatically triggering image capture based upon a positional relation between a reference position sensor attached to the object and the image sensor unit in order to automatically activate the system.
Response to Arguments
Applicant’s arguments, see pages 7-11 (specifically arguments I.(a) and I.(b)) of the remarks, filed 04/15/2026, with respect to the rejection(s) of claim(s) 34-52 under 35 U.S.C. §103 have been fully considered and are persuasive. Therefore, the rejections have been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Casas (US 20160191887 A1).
Conclusion
Applicant's submission of an information disclosure statement under 37 CFR 1.97(c) with the timing fee set forth in 37 CFR 1.17(p) on 04/15/2026 prompted the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 609.04(b). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES M ANDERSON II whose telephone number is (571)270-1444. The examiner can normally be reached Monday - Friday 10AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BRIAN PENDLETON can be reached at 571-272-7527. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/James M Anderson II/Primary Examiner, Art Unit 2425
Prosecution Insights