DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicants’ submission filed on 12 June 2026 has been entered. Claims 1, 2, 7, 8, 10-12, 18 and 20 have been amended. Claims 1-20 are currently pending and have been considered below.
Response to Arguments
Applicants’ arguments with respect to claim(s) 1-20 have been carefully considered but are moot in view of the new grounds of rejection necessitated by Applicants’ amendments.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 10 and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 10 and 20 recite the limitation, “obtaining a specific relative pose of the host relative to a reference coordinate system.” It is unclear if “a reference coordinate system” as recited in claims 10 and 20 is the same “reference coordinate system” as recited in independent claims 1 and 11 or if it is a different “reference coordinate system.”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo, Xiaoting, et al. "Vision and dual IMU integrated attitude measurement system." 2017 International Conference on Optical Instruments and Technology: Optoelectronic Measurement Technology and Systems. Vol. 10621. SPIE, 2018, hereinafter, “Guo”, and further in view of Foxlin, Eric, and Michael Harrington. "Weartrack: A self-referenced head and hand tracker for wearable computers and portable vr." Digest of Papers. Fourth International Symposium on Wearable Computers. IEEE, 2000, hereinafter, “Foxlin 2000”.
As per claim 1, Guo discloses an object tracking method, adapted to a host (Guo pages 5-6, 4. Evaluation Experiments of the Integrated System, fast object motion tracking), and the method comprising:
determining a reference motion state based on a first predicted motion state and a calibration factor (Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as : 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix ... 7) Update quaternion and bias; Guo, pages 3-4, 2.2 Principle of dual IMU system, the calibration of initial rotation matrix is necessary. And two vector determinations method is applied for calibration of the rotation matrix here. Control the rocking base moving to one direction, the motion information expressed in dual IMU can be expressed as 1 ωm and 1 ωs respectively. Control the rocking base moving to another direction, the output of the dual IMU is 2 ωm and 2 ω s respectively. According to these two determinations, two sets of vectors can be obtained ... the rotation matrix calibration is completed and the pure angular rate can be obtained);
obtaining a first motion data and a second motion data (Guo, Abstract, To determination relative attitude between two space objects on a rocking base, an integrated system based on vision and dual IMU (inertial determination unit) is built up. The determination system fuses the attitude information of vision with the angular determinations of dual IMU by extended Kalman filter (EKF) to obtain the relative attitude. One IMU (master) is attached to the measured motion object and the other (slave) to the rocking base);
determining a first relative pose of the [second]-object relative to the [master] based on the first motion data, the second motion data, and the reference motion state (Guo, Abstract, To determination relative attitude between two space objects on a rocking base, an integrated system based on vision and dual IMU (inertial determination unit) is built up. The determination system fuses the attitude information of vision with the angular determinations of dual IMU by extended Kalman filter (EKF) to obtain the relative attitude. One IMU (master) is attached to the measured motion object and the other (slave) to the rocking base. As the determination output of inertial sensor is relative to inertial frame, thus angular rate of the master IMU includes not only motion of the measured object relative to inertial frame but also the rocking base relative to inertial frame … The proposed integrated attitude determination system is tested on practical experimental platform; Guo, pages 3-4, 2.2 Principle of dual IMU system, Based on this theory, under the circumstances of the rocking base swinging at random for master IMU, the angular rate of its gyro output can be interpreted as [Equation 2] ... where m im ω , m ib ω , m bm ω stand for the angular rate of master IMU relative to inertial frame, the angular rate of rocking base relative to inertial frame, and the angular rate of master IMU relative to rocking base respectively. The i, m, b represent the inertial frame, the master IMU, and the rocking base accordingly ... For attitude determination on rocking base, our purpose is to obtain the relative attitude between measured moving object namely the turntable relative to rocking base ... the rotation matrix calibration is completed and the pure angular rate can be obtained); and
determining a specific pose of the [second] object based on the first relative pose (Guo, pages 3-4, 2.2 Principle of dual IMU system, Based on this theory, under the circumstances of the rocking base swinging at random for master IMU, the angular rate of its gyro output can be interpreted as [Equation 2] ... where m im ω , m ib ω , m bm ω stand for the angular rate of master IMU relative to inertial frame, the angular rate of rocking base relative to inertial frame, and the angular rate of master IMU relative to rocking base respectively. The i, m, b represent the inertial frame, the master IMU, and the rocking base accordingly ... For attitude determination on rocking base, our purpose is to obtain the relative attitude between measured moving object namely the turntable relative to rocking base ... the rotation matrix calibration is completed and the pure angular rate can be obtained).
Guo does not explicitly disclose the following limitations as further recited however Foxlin 2000 discloses
wherein the host and the object are in an environment with a reference coordinate system (Foxlin 2000, page 155, 1. Introduction, tracking the orientation and position of users’ heads and hands ... in order to control view parameters for Head-Mounted Displays (HMDs) and allow manual interactions with the virtual world; Foxlin 2000, page 156, 1.1 The concept, hand position measured in head space ... tracking hardware can provide full 6-DOF tracking; Foxlin 2000, pages 156-157, 2.1 Prototype implementation, The FreeD therefore measured the ring position relative to the head-fixed coordinate frame whose orientation was measured by the IS-300 ... The Demo3D program consists of a tracker driver, and a fairly conventional VR rendering environment that expects to receive 6-DOF head and hand tracking data from the tracker driver, as well as button states for the hand tracking device ... The basic functions of the tracker driver, when tracking a single 3-DOF point on the hand, are: ... parse the orientation data from the IS-300 and the position triad from the FreeD; Foxlin 2000, page 158, 3.2.3 Head-motion parallax using anchor beacon, we use the 3-DOF position vector from the user’s head to the hand-mounted beacon to track the position of the hand relative to the head [reference coordinate system is the head coordinate frame; host is the HMD and the object is the hand controller]);
obtaining a first motion data of the host and a second motion data of the object (Foxlin 2000, page 159, 3.3 Demo : VR game in a parking lot, Figure 3 illustrates a user playing a game of 3D asteroids in a parking lot. As he looks around the 360” surrounding space, he may see asteroids flying towards him at any moment from any direction. He must use the ring pointer to aim a virtual laser at the asteroids and destroy them);
determining a first relative pose of the reference-object relative to the host (Foxlin 2000, page 159, 3.3 Demo : VR game in a parking lot, Figure 3 illustrates a user playing a game of 3D asteroids in a parking lot. As he looks around the 360” surrounding space, he may see asteroids flying towards him at any moment from any direction. He must use the ring pointer to aim a virtual laser at the asteroids and destroy them, where the laser direction is calculated as the vector from his headset to his ring pointer. With a 5-DOF wand, it would be possible to draw an object (pistol, sabre, tennis racquet, etc) in his hand that coincides with the position and orientation where he feels like he is holding the wand grip);
determining a specific pose of the object relative to the reference coordinate system based on the first relative pose (Foxlin 2000, page 159, 3.3 Demo : VR game in a parking lot, Figure 3 illustrates a user playing a game of 3D asteroids in a parking lot. As he looks around the 360” surrounding space, he may see asteroids flying towards him at any moment from any direction. He must use the ring pointer to aim a virtual laser at the asteroids and destroy them, where the laser direction is calculated as the vector from his headset to his ring pointer. With a 5-DOF wand, it would be possible to draw an object (pistol, sabre, tennis racquet, etc) in his hand that coincides with the position and orientation where he feels like he is holding the wand grip).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Foxlin 2000 with Guo because they are in the same field of endeavor. One skilled in the art would have been motivated to include the reference coordinate system between the host and the object as taught by Foxlin 2000 in the system of Guo in order to provide an alternative means to determine motion and pose between two objects moving relative to each other (Foxlin 2000, Abstract).
As per claim 2, Guo and Foxlin 2000 disclose the method according to claim 1, further comprising:
obtaining a specific gain, a visual relative pose of the object relative to the host and a motion relative pose of the object relative to the host; determining the calibration factor based on the specific gain, the visual relative pose, and the motion relative pose (Guo, page 2, 2.1 Vision attitude determination, During the whole determination process, the stereo target will be captured by the camera and a series of target images at different pose will be obtained, where each image contains unique attitude information; Guo, pages 3-4, 2.2 Principle of dual IMU system, the calibration of initial rotation matrix is necessary. And two vector determinations method is applied for calibration of the rotation matrix here. Control the rocking base moving to one direction, the motion information expressed in dual IMU can be expressed as 1 ωm and 1 ωs respectively. Control the rocking base moving to another direction, the output of the dual IMU is 2 ωm and 2 ω s respectively. According to these two determinations, two sets of vectors can be obtained ... the rotation matrix calibration is completed and the pure angular rate can be obtained; Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix ... 7) Update quaternion and bias).
As per claim 3, Guo and Foxlin 2000 disclose the method according to claim 2, further comprising:
obtaining a first reference gain factor, the first predicted motion state, and the visual relative pose, and accordingly determining the specific gain (Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix).
As per claim 4, Guo and Foxlin 2000 disclose the method according to claim 3, further comprising:
updating the first reference gain factor based on the specific gain and the first predicted motion state (Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix).
As per claim 5, Guo and Foxlin 2000 disclose the method according to claim 2, wherein the step of determining the calibration factor based on the specific gain, the visual relative pose, and the motion relative pose comprises:
determining a pose difference between the visual relative pose and the motion relative pose (Guo, pages 2-3, 2.1 Vision attitude determination, During the whole determination process, the stereo target will be captured by the camera and a series of target images at different pose will be obtained, where each image contains unique attitude information ... To increase the determination accuracy of vision method, a pose algorithm based on constraint calibration is employed ... The specific calculation process of constraint calibration method can be divided into two steps: calibration and determination. In the calibration procedure, the position of the whole space is collected and calibrated. And a one-to-one correspondence among the reference location, reference attitude and space base vector is established; Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix); and
determining the calibration factor based on the specific gain and the pose difference (Guo, page 9, 5. Conclusions, A new integrated system fusing vision and dual IMU is designed and applied for attitude determination on rocking base. To improve the calculation accuracy, constrained calibration algorithm is used to calculate vision attitude. For dual IMU system, the initial rotation matrix needs to know. And the two vector determinations method is employed to complete the calibration. EKF method is used to fuse inertial and vision data together by filter).
As per claim 6, Guo and Foxlin 2000 disclose the method according to claim 1, wherein the step of determining the reference motion state based on the first predicted motion state and the calibration factor comprises:
determining the reference motion state via combining the first predicted motion state with the calibration factor (Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as : 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix ... 7) Update quaternion and bias; Guo, pages 3-4, 2.2 Principle of dual IMU system, the calibration of initial rotation matrix is necessary. And two vector determinations method is applied for calibration of the rotation matrix here. Control the rocking base moving to one direction, the motion information expressed in dual IMU can be expressed as 1 ωm and 1 ωs respectively. Control the rocking base moving to another direction, the output of the dual IMU is 2 ωm and 2 ω s respectively. According to these two determinations, two sets of vectors can be obtained ... the rotation matrix calibration is completed and the pure angular rate can be obtained).
As per claim 7, Guo and Foxlin 2000 disclose the method according to claim 1, wherein the step of determining the first relative pose of the object relative to the host based on the first motion data, the second motion data, and the reference motion state comprises:
determining a second predicted motion state based on the first motion data, the second motion data, and the reference motion state, wherein the second predicted motion state comprises the first relative pose and parameters associated with the first motion data and the second motion data (Guo, pages 3-4, 2.2 Principle of dual IMU system, the calibration of initial rotation matrix is necessary. And two vector determinations method is applied for calibration of the rotation matrix here. Control the rocking base moving to one direction, the motion information expressed in dual IMU can be expressed as 1 ωm and 1 ωs respectively. Control the rocking base moving to another direction, the output of the dual IMU is 2 ωm and 2 ω s respectively. According to these two determinations, two sets of vectors can be obtained ... the rotation matrix calibration is completed and the pure angular rate can be obtained; Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix ... 7) Update quaternion and bias [this is an iterative algorithm].
As per claim 8, Guo and Foxlin 2000 disclose the method according to claim 7, wherein the first motion data is collected by a first motion detection circuit on the host, the second motion data is collected by a second motion detection circuit on the object (Guo, Abstract, To determination relative attitude between two space objects on a rocking base, an integrated system based on vision and dual IMU (inertial determination unit) is built up. The determination system fuses the attitude information of vision with the angular determinations of dual IMU by extended Kalman filter (EKF) to obtain the relative attitude. One IMU (master) is attached to the measured motion object and the other (slave) to the rocking base; Foxlin 2000, pages 156-157, 2.1 Prototype implementation, The FreeD therefore measured the ring position relative to the head-fixed coordinate frame whose orientation was measured by the IS-300 ... The Demo3D program consists of a tracker driver, and a fairly conventional VR rendering environment that expects to receive 6-DOF head and hand tracking data from the tracker driver, as well as button states for the hand tracking device ... The basic functions of the tracker driver, when tracking a single 3-DOF point on the hand, are: ... parse the orientation data from the IS-300 and the position triad from the FreeD);
wherein the parameters associated with the first motion data comprise intrinsic and extrinsic parameters associated with the first motion detection circuit; wherein the parameters associated with the second motion data comprise intrinsic and extrinsic parameters associated with the second motion detection circuit (Guo, page 3, 2.2 Principle of dual IMU system, the output of inertial sensor is relative to inertial frame. A MEMS IMU usually consists of triaxial accelerometers, triaxial gyroscopes, and sometimes even triaxial magnetometers. Based on this theory, under the circumstances of the rocking base swinging at random for master IMU, the angular rate of its gyro output can be interpreted as [Equation 2]; Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... Considering bias and noise, the output model of gyro can be built up as: [Equation 9]; Guo, Abstract, To determination relative attitude between two space objects on a rocking base).
As per claim 9, Guo and Foxlin 2000 disclose the method according to claim 1, further comprising:
obtaining an updated reference gain factor and a second predicted motion state; determining a second reference gain factor based on the updated reference gain factor and the second predicted motion state (Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix ... 7) Update quaternion and bias [this is an iterative algorithm]).
As per claim 10, Guo and Foxlin 2000 disclose the method according to claim 1, wherein the step of determining the specific pose of the object based on the first relative pose comprises:
obtaining a specific relative pose of the host relative to a reference coordinate system (Guo, pages 3-4, 2.2 Principle of dual IMU system, Based on this theory, under the circumstances of the rocking base swinging at random for master IMU, the angular rate of its gyro output can be interpreted as [Equation 2] ... where m im ω , m ib ω , m bm ω stand for the angular rate of master IMU relative to inertial frame, the angular rate of rocking base relative to inertial frame, and the angular rate of master IMU relative to rocking base respectively. The i, m, b represent the inertial frame, the master IMU, and the rocking base accordingly ... For attitude determination on rocking base, our purpose is to obtain the relative attitude between measured moving object namely the turntable relative to rocking base; Foxlin 2000, pages 156-157, 2.1 Prototype implementation, The basic functions of the tracker driver, when tracking a sing le 3-DOF point on the hand, are: ... parse the orientation data from the IS-300 and the position triad from the FreeD. Package the orientation data with the current head position in world-frame, and output the combined 6-DOF data record for the head to the VR program);
determining the specific pose of the object via combining the specific relative pose with the first relative pose (Guo, Abstract, To determination relative attitude between two space objects on a rocking base, an integrated system based on vision and dual IMU (inertial determination unit) is built up. The determination system fuses the attitude information of vision with the angular determinations of dual IMU by extended Kalman filter (EKF) to obtain the relative attitude. One IMU (master) is attached to the measured motion object and the other (slave) to the rocking base. As the determination output of inertial sensor is relative to inertial frame, thus angular rate of the master IMU includes not only motion of the measured object relative to inertial frame but also the rocking base relative to inertial frame; Foxlin 2000, pages 156-157, 2.1 Prototype implementation, Transform the hand position vector from head frame to world frame by first multiplying by the rotation matrix from head to world frame obtained from the orientation tracker, then adding the current assumed world-frame head position. Output the result to the VR program as a 3-DOF position record for the hand device).
As per claim 11, Guo discloses a [master] comprising:
a non-transitory storage circuit, storing a program code; and a processor, coupled to the non-transitory storage circuit and accessing the program code (Guo, pages 5-6, 4. Evaluation Experiments of the Integrated System, image processing procedure, which improves computational efficiency of the integrated system) to perform:
determining a reference motion state based on a first predicted motion state and a calibration factor (Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as : 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix ... 7) Update quaternion and bias; Guo, pages 3-4, 2.2 Principle of dual IMU system, the calibration of initial rotation matrix is necessary. And two vector determinations method is applied for calibration of the rotation matrix here. Control the rocking base moving to one direction, the motion information expressed in dual IMU can be expressed as 1 ωm and 1 ωs respectively. Control the rocking base moving to another direction, the output of the dual IMU is 2 ωm and 2 ω s respectively. According to these two determinations, two sets of vectors can be obtained ... the rotation matrix calibration is completed and the pure angular rate can be obtained);
obtaining a first motion data of the [master] and a second motion data of the [second] object (Guo, Abstract, To determination relative attitude between two space objects on a rocking base, an integrated system based on vision and dual IMU (inertial determination unit) is built up. The determination system fuses the attitude information of vision with the angular determinations of dual IMU by extended Kalman filter (EKF) to obtain the relative attitude. One IMU (master) is attached to the measured motion object and the other (slave) to the rocking base);
determining a first relative pose of the [second ] object relative to the [master] based on the first motion data, the second motion data, and the reference motion state (Guo, Abstract, To determination relative attitude between two space objects on a rocking base, an integrated system based on vision and dual IMU (inertial determination unit) is built up. The determination system fuses the attitude information of vision with the angular determinations of dual IMU by extended Kalman filter (EKF) to obtain the relative attitude. One IMU (master) is attached to the measured motion object and the other (slave) to the rocking base. As the determination output of inertial sensor is relative to inertial frame, thus angular rate of the master IMU includes not only motion of the measured object relative to inertial frame but also the rocking base relative to inertial frame … The proposed integrated attitude determination system is tested on practical experimental platform; Guo, pages 3-4, 2.2 Principle of dual IMU system, Based on this theory, under the circumstances of the rocking base swinging at random for master IMU, the angular rate of its gyro output can be interpreted as [Equation 2] ... where m im ω , m ib ω , m bm ω stand for the angular rate of master IMU relative to inertial frame, the angular rate of rocking base relative to inertial frame, and the angular rate of master IMU relative to rocking base respectively. The i, m, b represent the inertial frame, the master IMU, and the rocking base accordingly ... For attitude determination on rocking base, our purpose is to obtain the relative attitude between measured moving object namely the turntable relative to rocking base ... the rotation matrix calibration is completed and the pure angular rate can be obtained); and
determining a specific pose of the [second]object based on the first relative pose (Guo, pages 3-4, 2.2 Principle of dual IMU system, Based on this theory, under the circumstances of the rocking base swinging at random for master IMU, the angular rate of its gyro output can be interpreted as [Equation 2] ... where m im ω , m ib ω , m bm ω stand for the angular rate of master IMU relative to inertial frame, the angular rate of rocking base relative to inertial frame, and the angular rate of master IMU relative to rocking base respectively. The i, m, b represent the inertial frame, the master IMU, and the rocking base accordingly ... For attitude determination on rocking base, our purpose is to obtain the relative attitude between measured moving object namely the turntable relative to rocking base ... the rotation matrix calibration is completed and the pure angular rate can be obtained).
Guo does not explicitly disclose the following limitations as further recited however Foxlin 2000 discloses
a host (Foxlin 2000, page 155, 1. Introduction, tracking the orientation and position of users’ heads and hands ... in order to control view parameters for Head-Mounted Displays (HMDs) and allow manual interactions with the virtual world);
wherein the host and an object are in an environment with a reference coordinate system (Foxlin 2000, page 155, 1. Introduction, tracking the orientation and position of users’ heads and hands ... in order to control view parameters for Head-Mounted Displays (HMDs) and allow manual interactions with the virtual world; Foxlin 2000, page 156, 1.1 The concept, hand position measured in head space ... tracking hardware can provide full 6-DOF tracking; Foxlin 2000, pages 156-157, 2.1 Prototype implementation, The FreeD therefore measured the ring position relative to the head-fixed coordinate frame whose orientation was measured by the IS-300 ... The Demo3D program consists of a tracker driver, and a fairly conventional VR rendering environment that expects to receive 6-DOF head and hand tracking data from the tracker driver, as well as button states for the hand tracking device ... The basic functions of the tracker driver, when tracking a single 3-DOF point on the hand, are: ... parse the orientation data from the IS-300 and the position triad from the FreeD; Foxlin 2000, page 158, 3.2.3 Head-motion parallax using anchor beacon, we use the 3-DOF position vector from the user’s head to the hand-mounted beacon to track the position of the hand relative to the head [reference coordinate system is the head coordinate frame; host is the HMD and the object is the hand controller]);
obtaining a first motion data of the host and a second motion data of the object (Foxlin 2000, page 159, 3.3 Demo : VR game in a parking lot, Figure 3 illustrates a user playing a game of 3D asteroids in a parking lot. As he looks around the 360” surrounding space, he may see asteroids flying towards him at any moment from any direction. He must use the ring pointer to aim a virtual laser at the asteroids and destroy them);
determining a first relative pose of the reference-object relative to the host (Foxlin 2000, page 159, 3.3 Demo : VR game in a parking lot, Figure 3 illustrates a user playing a game of 3D asteroids in a parking lot. As he looks around the 360” surrounding space, he may see asteroids flying towards him at any moment from any direction. He must use the ring pointer to aim a virtual laser at the asteroids and destroy them, where the laser direction is calculated as the vector from his headset to his ring pointer. With a 5-DOF wand, it would be possible to draw an object (pistol, sabre, tennis racquet, etc) in his hand that coincides with the position and orientation where he feels like he is holding the wand grip);
determining a specific pose of the reference-object relative to the reference coordinate system based on the first relative pose (Foxlin 2000, page 159, 3.3 Demo : VR game in a parking lot, Figure 3 illustrates a user playing a game of 3D asteroids in a parking lot. As he looks around the 360” surrounding space, he may see asteroids flying towards him at any moment from any direction. He must use the ring pointer to aim a virtual laser at the asteroids and destroy them, where the laser direction is calculated as the vector from his headset to his ring pointer. With a 5-DOF wand, it would be possible to draw an object (pistol, sabre, tennis racquet, etc) in his hand that coincides with the position and orientation where he feels like he is holding the wand grip).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Foxlin 2000 with Guo because they are in the same field of endeavor. One skilled in the art would have been motivated to include the reference coordinate system between the host and the object as taught by Foxlin 2000 in the system of Guo in order to provide an alternative means to determine motion and pose between two objects moving relative to each other (Foxlin 2000, Abstract).
As per claim 12, Guo and Foxlin 2000 disclose the host according to claim 11, wherein the processor further performs:
obtaining a specific gain, a visual relative pose of the object relative to the host and a motion relative pose of the object relative to the host; determining the calibration factor based on the specific gain, the visual relative pose, and the motion relative pose (Guo, page 2, 2.1 Vision attitude determination, During the whole determination process, the stereo target will be captured by the camera and a series of target images at different pose will be obtained, where each image contains unique attitude information; Guo, pages 3-4, 2.2 Principle of dual IMU system, the calibration of initial rotation matrix is necessary. And two vector determinations method is applied for calibration of the rotation matrix here. Control the rocking base moving to one direction, the motion information expressed in dual IMU can be expressed as 1 ωm and 1 ωs respectively. Control the rocking base moving to another direction, the output of the dual IMU is 2 ωm and 2 ω s respectively. According to these two determinations, two sets of vectors can be obtained ... the rotation matrix calibration is completed and the pure angular rate can be obtained; Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix ... 7) Update quaternion and bias).
As per claim 13, Guo and Foxlin 2000 disclose the host according to claim 12, wherein the processor further performs:
obtaining a first reference gain factor, the first predicted motion state, and the visual relative pose, and accordingly determining the specific gain (Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix).
As per claim 14, Guo and Foxlin 2000 disclose the host according to claim 13, wherein the processor further performs:
updating the first reference gain factor based on the specific gain and the first predicted motion state (Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix).
As per claim 15, Guo and Foxlin 2000 disclose the host according to claim 12, wherein the processor performs:
determining a pose difference between the visual relative pose and the motion relative pose (Guo, pages 2-3, 2.1 Vision attitude determination, During the whole determination process, the stereo target will be captured by the camera and a series of target images at different pose will be obtained, where each image contains unique attitude information ... To increase the determination accuracy of vision method, a pose algorithm based on constraint calibration is employed ... The specific calculation process of constraint calibration method can be divided into two steps: calibration and determination. In the calibration procedure, the position of the whole space is collected and calibrated. And a one-to-one correspondence among the reference location, reference attitude and space base vector is established; Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix); and
determining the calibration factor based on the specific gain and the pose difference (Guo, page 9, 5. Conclusions, A new integrated system fusing vision and dual IMU is designed and applied for attitude determination on rocking base. To improve the calculation accuracy, constrained calibration algorithm is used to calculate vision attitude. For dual IMU system, the initial rotation matrix needs to know. And the two vector determinations method is employed to complete the calibration. EKF method is used to fuse inertial and vision data together by filter).
As per claim 16, Guo and Foxlin 2000 disclose the host according to claim 11, wherein the processor performs
determining the reference motion state via combining the first predicted motion state with the calibration factor (Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as : 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix ... 7) Update quaternion and bias; Guo, pages 3-4, 2.2 Principle of dual IMU system, the calibration of initial rotation matrix is necessary. And two vector determinations method is applied for calibration of the rotation matrix here. Control the rocking base moving to one direction, the motion information expressed in dual IMU can be expressed as 1 ωm and 1 ωs respectively. Control the rocking base moving to another direction, the output of the dual IMU is 2 ωm and 2 ω s respectively. According to these two determinations, two sets of vectors can be obtained ... the rotation matrix calibration is completed and the pure angular rate can be obtained).
As per claim 17, Guo and Foxlin 2000 disclose the host according to claim 11, wherein the processor performs:
determining a second predicted motion state based on the first motion data, the second motion data, and the reference motion state, wherein the second predicted motion state comprises the first relative pose and parameters associated with the first motion data and the second motion data. (Guo, pages 3-4, 2.2 Principle of dual IMU system, the calibration of initial rotation matrix is necessary. And two vector determinations method is applied for calibration of the rotation matrix here. Control the rocking base moving to one direction, the motion information expressed in dual IMU can be expressed as 1 ωm and 1 ωs respectively. Control the rocking base moving to another direction, the output of the dual IMU is 2 ωm and 2 ω s respectively. According to these two determinations, two sets of vectors can be obtained ... the rotation matrix calibration is completed and the pure angular rate can be obtained; Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix ... 7) Update quaternion and bias [this is an iterative algorithm]
As per claim 18, Guo and Foxlin 2000 disclose the host according to claim 17, wherein the first motion data is collected by a first motion detection circuit on the host, the second motion data is collected by a second motion detection circuit on the object (Guo, Abstract, To determination relative attitude between two space objects on a rocking base, an integrated system based on vision and dual IMU (inertial determination unit) is built up. The determination system fuses the attitude information of vision with the angular determinations of dual IMU by extended Kalman filter (EKF) to obtain the relative attitude. One IMU (master) is attached to the measured motion object and the other (slave) to the rocking base; Foxlin 2000, pages 156-157, 2.1 Prototype implementation, The FreeD therefore measured the ring position relative to the head-fixed coordinate frame whose orientation was measured by the IS-300 ... The Demo3D program consists of a tracker driver, and a fairly conventional VR rendering environment that expects to receive 6-DOF head and hand tracking data from the tracker driver, as well as button states for the hand tracking device ... The basic functions of the tracker driver, when tracking a single 3-DOF point on the hand, are: ... parse the orientation data from the IS-300 and the position triad from the FreeD);
wherein the parameters associated with the first motion data comprise intrinsic and extrinsic parameters associated with the first motion detection circuit; wherein the parameters associated with the second motion data comprise intrinsic and extrinsic parameters associated with the second motion detection circuit (Guo, page 3, 2.2 Principle of dual IMU system, the output of inertial sensor is relative to inertial frame. A MEMS IMU usually consists of triaxial accelerometers, triaxial gyroscopes, and sometimes even triaxial magnetometers. Based on this theory, under the circumstances of the rocking base swinging at random for master IMU, the angular rate of its gyro output can be interpreted as [Equation 2]; Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... Considering bias and noise, the output model of gyro can be built up as: [Equation 9]; Guo, Abstract, To determination relative attitude between two space objects on a rocking base).
As per claim 19, Guo and Foxlin 2000 disclose the host according to claim 11, wherein the processor further performs:
obtaining an updated reference gain factor and a second predicted motion state; determining a second reference gain factor based on the updated reference gain factor and the second predicted motion state (Guo, pages 4-5, 3. Fusion Attitude Algorithm Based on Error Quaternion EKF, For fusion attitude calculation, when vision data is available, vision data and angular rate are fused by EKF ... With filter state x = [xq, xb] ... initial state, and initial covariance matrix P, the filter process can be described as: 1) Propagation of state vector ... 2) One step prediction of state ... 3) One step prediction of covariance matrix ... 4) Update filter gain ... 5) Update state ...6) Update covariance matrix ... 7) Update quaternion and bias).
As per claim 20, Guo and Foxlin 2000 disclose the host according to claim 11, wherein the processor performs:
obtaining a specific relative pose of the host relative to a reference coordinate system (Guo, pages 3-4, 2.2 Principle of dual IMU system, Based on this theory, under the circumstances of the rocking base swinging at random for master IMU, the angular rate of its gyro output can be interpreted as [Equation 2] ... where m im ω , m ib ω , m bm ω stand for the angular rate of master IMU relative to inertial frame, the angular rate of rocking base relative to inertial frame, and the angular rate of master IMU relative to rocking base respectively. The i, m, b represent the inertial frame, the master IMU, and the rocking base accordingly ... For attitude determination on rocking base, our purpose is to obtain the relative attitude between measured moving object namely the turntable relative to rocking base; Foxlin 2000, pages 156-157, 2.1 Prototype implementation, The basic functions of the tracker driver, when tracking a sing le 3-DOF point on the hand, are: ... parse the orientation data from the IS-300 and the position triad from the FreeD. Package the orientation data with the current head position in world-frame, and output the combined 6-DOF data record for the head to the VR program);
determining the specific pose of the object via combining the specific relative pose with the first relative pose (Guo, Abstract, To determination relative attitude between two space objects on a rocking base, an integrated system based on vision and dual IMU (inertial determination unit) is built up. The determination system fuses the attitude information of vision with the angular determinations of dual IMU by extended Kalman filter (EKF) to obtain the relative attitude. One IMU (master) is attached to the measured motion object and the other (slave) to the rocking base. As the determination output of inertial sensor is relative to inertial frame, thus angular rate of the master IMU includes not only motion of the measured object relative to inertial frame but also the rocking base relative to inertial frame; Foxlin 2000, pages 156-157, 2.1 Prototype implementation, Transform the hand position vector from head frame to world frame by first multiplying by the rotation matrix from head to world frame obtained from the orientation tracker, then adding the current assumed world-frame head position. Output the result to the VR program as a 3-DOF position record for the hand device).
Conclusion
Applicants’ amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRACY MANGIALASCHI whose telephone number is (571)270-5189. The examiner can normally be reached M-F, 9:30AM TO 6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRACY MANGIALASCHI/Primary Examiner, Art Unit 2668
Prosecution Insights