DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on June 29, 2026 has been entered.
Status of Claims
Claims 1-20 are pending.
Information Disclosure Statement
3. The information disclosure statement (IDS) submitted on 07/21/2026 have been entered and considered. Initialed copies of the PTO-1449 by the Examiner are attached.
Response to Arguments
Claim Rejections – 35 USC section § 102/103
Applicant's arguments with respect to the limitation “the first body part is anatomically distinct from the second body part” have been considered but are moot in view of the new ground(s) of rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 10, 12-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over MOUNIER et al. (US 20230048398 A1) hereinafter referenced as MOUNIER in view of Beyhs et al. (US US20210303069 A1).
Regarding claim 1, MOUNIER teaches a controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0063]-MOUNIER discloses the one or more compute components 110 of the XR system 100 can include, for example and without limitation, a central processing unit (CPU) 112, a graphics processing unit (GPU) 114, a digital signal processor (DSP) 116, and/or an image signal processor (ISP) 118. In some examples, the XR system 100 can include other types of processors such as, for example, a computer vision (CV) processor, a neural network processor (NNP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc. The XR system 100 can use the one or more compute components 110 to perform various computing operations such as, for example, extended reality operations (e.g., tracking, localization, object detection, classification, pose estimation, mapping, content anchoring, content rendering, etc.), image/video processing, graphics rendering, machine learning, data processing, modeling, calculations, and/or any other operations. Please also read Paragraphs [0132] and [0153].), adapted to determine a position conversion relationship between a first position of a first body part of a user and a second position of the second body part of the user (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.) (wherein a glove is a wearing part of the portion of the user). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s) (where in hand is a body part of a portion of the user), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Please also read Paragraph [0133].),
wherein the position conversion relationship is configured to convert one point in a three-dimensional space to another point in the three-dimensional space (Figs. 1-4, Paragraph [0139]-MOUNIER discloses in some aspects, tracking the movement of the wearable device can include determining the first position of the wearable device within a first coordinate system of the wearable device; transforming the first coordinate system of the wearable device to a second coordinate system of the electronic device; and determining the second position of the wearable device within the second coordinate system of the electronic device. Please also read Paragraph [0040].), the wearing part is where a wearable device is disposed on the second body part of the user (Figs. 1-4, #150 called a wearable device, Paragraph [0043]-MOUNIER discloses a wearable accessory (or wearable device) that interfaces/interacts with an XR device to aid in tracking, provide power savings, and/or increase user privacy during XR experiences. In some examples, a wearable accessory can be worn by a user of an XR device during an XR experience. The wearable accessory (wherein the wearable accessory is the wearing part) can be worn on a user's finger (or multiple fingers), a wrist, an ankle, and/or any other body part. The wearable accessory can include embedded sensors configured to obtain measurements of a state (e.g., a position, movement, etc.) of the wearable accessory in 3D space, and thus the state of the body part on which the wearable accessory is worn (e.g., a finger, a hand, etc.). Please also read Paragraph [0044].), and the controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0063]), wherein the first position is different from the second position (process 520 can include determining, based on data from the wearable device and/or a command from the wearable device, one or more extended reality (XR) inputs to an XR application on the electronic device. In some examples, the one or more XR inputs can include a modification of a virtual element along multiple dimensions in space, a selection of the virtual element, a navigation event, and/or a request to measure a distance defined by the first position of the wearable device, the second position of the wearable device, and/or the movement of the wearable device – see [p][0150]) is configured to:
obtain a camera data of a previous moment (Fig. 2B, #102 called an image sensor, Paragraph [0114]-MOUNIER discloses at block 422, the XR system 100 can use one or more image sensors that are turned on/enabled to capture an image(s) of the target. The one or more image sensors can include any image sensor on the XR system 100 having visibility to the target. At block 424, the wearable device 150 can also send data to the XR system 100. Further in Paragraph [0115]-MOUNIER discloses at block 426, the XR system 100 can use the image(s) of the target and the data from the wearable device 150 to track the target, as previously described. Please also read Paragraphs [0062] and [0087].) from a camera (Fig. 2B, #102 called an image sensor, Paragraph [0062]), wherein the camera data comprises an image of the wearable device and the first position (Figs. 2A-5B, illustrates imaging of the wearable device, Paragraph [0133]-MOUNIER discloses at block 522, the process 520 can include determining a first position of a wearable device (e.g., wearable device 150, wearable ring 200) in a physical space. In some examples, determining the first position of the wearable device can include receiving, from the wearable device, image data from one or more image sensors on the electronic device and/or data associated with one or more measurements from one or more sensors on the wearable device; and determining the first position of the wearable device based on the image data from the one or more image sensors and/or data associated with the one or more measurements from the one or more sensors. Further in Paragraph [0138]-MOUNIER discloses the process 520 can include tracking, based on the first position and the second position, a movement of the wearable device relative to the electronic device.);
obtain a sensor data of the tracking position of the previous moment (Figs. 1, 2B and 4-5B, #152 called an Inertial Measurement Unit (IMU), Paragraph [0080]-MOUNIER discloses at time T1, the ring device 200 can obtain data 220 based at least partly based on sensor data from one or more sensors (e.g., touchpad 204, sensors 208, etc.) on the ring device 200, and send the data 220 to the XR system 100. Further in Paragraph [0079]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. Please also read Paragraphs [0040] and [0068].) from a sensor of the wearable device (Figs. 1, 2B and 4-5B, #152 called an Inertial Measurement Unit (IMU), Paragraph [0068]-MOUNIER discloses the one or more compute components 110 can receive sensor data (e.g., data from the IMU 152, the ultrasonic sensor 154, the pressure sensor 156, and/or the touch sensor 158) from the wearable device 150, and use such data to track the wearable device 150 (with or without other data from the image sensor 102, the image sensor 104, or the other sensor(s) 106), adjust processing/power operations, etc., as described herein.);
determine the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Please also read Paragraph [0133].) based on the camera data of the previous moment and the sensor data of the previous moment (Figs. 1 and 2B, Paragraph [0080]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. In some examples, the XR system 100 can use the data 220 to determine whether the finger 210 (and/or a hand of the finger 210) is visible to one or more image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100. For example, the XR system 100 can use the data 220 to determine whether the finger 210 (and/or a hand of the finger 210) is within a FOV of one or more image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100 and/or whether a view of the one or more image sensors to the finger 210 (and/or the hand of the finger 210) is obstructed by one or more objects.);
fusion the camera data and the sensor data and output a fusion data of the wearing part of a current moment based on the position conversion relationship (Figs. 1-4, Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Further in Paragraph [0105]-MOUNIER discloses at block 408, the wearable device 150 can send additional data to the XR system 100. At block 410, the XR system 100 can use the data to determine a visibility of one or more image sensors at the XR system 100 to the target. For example, the XR system 100 can use the data from the wearable device 150 to determine the position of the wearable device 150 and the target in 3D space. Please also read Paragraph [0040].);
and determine an operation of the wearing part of the user based on the fusion data (Figs. 2A-2B, 3 and 4, Paragraph [0043]-MOUNIER discloses the wearable accessory can include embedded sensors configured to obtain measurements of a state (e.g., a position, movement, etc.) of the wearable accessory in 3D space, and thus the state of the body part on which the wearable accessory is worn (e.g., a finger, a hand, etc.). The wearable accessory can provide the measurements to the XR device, which can integrate with its tracking system for more robust tracking and accuracy. Please also read Paragraphs [0040] and [0138].).
MOUNIER fails to explicitly teach the first body part is anatomically distinct from the second body part.
However, Beyhs explicitly teaches the first body part is anatomically distinct from the second body part (for e.g., the finger is anatomically different from the palm or the index finger being anatomically difference from the thumb – see [p][0067][0074] and Figs 3A-B and 12A-B )
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of MOUNIER of having a controller, adapted to determine a position conversion relationship between a position of a body part of a portion of a user and a tracking position of a wearing part of the portion of the user, wherein the position conversion relationship is configured to convert one point in a three-dimensional space to another point in the three-dimensional space, the wearing part is where a wearable device is disposed on the user, and the controller is configured to: obtain a camera data of a previous moment from a camera, wherein the camera data comprises an image of the wearable device and the first position, with the teachings of Beyhs of having the first body part is anatomically distinct from the second body part.
Wherein having MOUNIER’s the first body part is anatomically distinct from the second body part.
The motivation behind the modification would have been to obtain a wearable tracking system that provides more robust tracking and accuracy by detecting contact between a first body part and a second body par, since both MOUNIER and Beyhs are systems that relate to gesture detection. Wherein MOUNIER’s wearable system helps to increase the tracking fidelity/accuracy of the XR device and can trigger subsequent processing mode adjustments of any of the cameras of the XR device based on a position and/or motion of the wearable accessory, while Beyhs detecting contact between a first body part and a second body part. Please see MOUNIER et al. (US 20230048398 A1), Paragraphs [0043], [0046], and [0101], and Beyhs et al. (US20210303069A1), Paragraph [0004].
Regarding claim 2, MOUNIER in view of Beyhs teach the controller according to claim 1, MOUNIER further teaches wherein the controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0063]) is further configured to:
determine a camera-identified position of the wearable device of the previous moment as the tracking position based on the camera data (Figs. 1-4 and 5B, Paragraph [0133]-MOUNIER discloses at block 522, the process 520 can include determining a first position of a wearable device (e.g., wearable device 150, wearable ring 200) in a physical space. In some examples, determining the first position of the wearable device can include receiving, from the wearable device, image data from one or more image sensors on the electronic device and/or data associated with one or more measurements from one or more sensors on the wearable device; and determining the first position of the wearable device based on the image data from the one or more image sensors and/or data associated with the one or more measurements from the one or more sensors.);
and determine the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the predetermined position (Figs. 1-4, Paragraph [0133]-MOUNIER discloses at block 522, the process 520 can include determining a first position of a wearable device (e.g., wearable device 150, wearable ring 200) in a physical space. In some examples, determining the first position of the wearable device can include receiving, from the wearable device, image data from one or more image sensors on the electronic device and/or data associated with one or more measurements from one or more sensors on the wearable device; and determining the first position of the wearable device based on the image data from the one or more image sensors and/or data associated with the one or more measurements from the one or more sensors) and the tracking position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position and the second position, a movement of the wearable device relative to the electronic device.).
Regarding claim 3, MOUNIER in view of Beyhs teach the controller according to claim 1, MOUNIER further teaches wherein the controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0063]) is further configured to:
determine a sensor-identified position of the wearing part of the previous moment as the second position based on the sensor data (Figs. 1-4, Paragraph [0080]-MOUNIER discloses in some examples, the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1.);
and determine the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the first position and the second position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position (wherein first position is the predetermined position) and the second position (wherein the second position is the tracking position), a movement of the wearable device relative to the electronic device. Please also read Paragraph [0133].).
Regarding claim 4, MOUNIER in view of Beyhs teach the controller according to claim 1, MOUNIER further teaches wherein the controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0063]) is further configured to:
determine a camera-identified position of the wearing part of the previous moment based on the camera data (Figs. 1-4 and 5B, Paragraph [0133]-MOUNIER discloses at block 522, the process 520 can include determining a first position of a wearable device (e.g., wearable device 150, wearable ring 200) in a physical space. In some examples, determining the first position of the wearable device can include receiving, from the wearable device, image data from one or more image sensors on the electronic device and/or data associated with one or more measurements from one or more sensors on the wearable device; and determining the first position of the wearable device based on the image data from the one or more image sensors and/or data associated with the one or more measurements from the one or more sensors.);
determine a sensor-identified position of the wearing part of the previous moment based on the sensor data (Figs. 1, 2B and 4-5B, Paragraph [0079]-MOUNIER discloses at time T1, the ring device 200 can obtain data 220 based at least partly based on sensor data from one or more sensors (e.g., touchpad 204, sensors 208, etc.) on the ring device 200, and send the data 220 to the XR system 100. Further in Paragraph [0080]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. Please also read Paragraphs [0040] and [0068].);
determine the second position of the wearing part based on the camera-identified position and the sensor-identified position (Figs. 1, 2B and 4-5B, Paragraph [0083]-MOUNIER discloses for example, the wearable device 150 can receive signals from the wearable device 150 to inform a tracking engine in the XR system 100. In some cases, the signals can include data from and/or data based on sensor data from sensors embedded in the wearable device 150. The XR system 100 can use the signals from the wearable device 150 to track a body part (e.g., a finger, hand, wrist, group of fingers, etc.) wearing the wearable device 150 and correlate an estimated location of the body part with a FOV of any image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100.);
and determine the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the first position and the second position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position (wherein first position is the predetermined position) and the second position (wherein the second position is the tracking position), a movement of the wearable device relative to the electronic device. Please also read Paragraph [0133].).
Regarding claim 5, MOUNIER in view of Beyhs teach the controller according to claim 1, MOUNIER further teaches wherein the camera data comprises a posture image of the user with a predetermined posture (Figs. 2A-2B, 4 and 5A-5B, illustrates imaging the wearing part, Paragraph [0067]-MOUNIER discloses the one or more compute components 110 can perform XR processing operations based on data from the image sensor 102, the image sensor 104, the one or more other sensors 106, and/or the wearable device 150. For example, in some cases, the one or more compute components 110 can perform tracking, localization, object detection, object classification, pose estimation, shape estimation, mapping, content anchoring, content rendering, image processing, modeling, content generation, gesture detection, gesture recognition, and/or other operations based on data from the image sensor 102, the image sensor 104, the one or more other sensors 106, and/or the wearable device 150. Further in Paragraph [0068]-MOUNIER discloses the one or more compute components 110 can implement one or more algorithms for tracking and estimating a relative pose of the wearable device 150 and the XR system 100. In some cases, the one or more compute components 110 can receive image data captured by the image sensor 102 and/or the image sensor 104 and perform pose estimation based on the received image data to calculate a relative pose of the wearable device 150 and the XR system 100. Please also read Paragraph [0080].) and the controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0063]) is further configured to:
determine a camera-identified position of the wearing part of the previous moment as the second position (Figs. 2A-2B, 4 and 5A-5B, illustrates imaging the wearing part, Paragraph [0067]-MOUNIER discloses the one or more compute components 110 can perform XR processing operations based on data from the image sensor 102, the image sensor 104, the one or more other sensors 106, and/or the wearable device 150. For example, in some cases, the one or more compute components 110 can perform tracking, localization, object detection, object classification, pose estimation, shape estimation, mapping, content anchoring, content rendering, image processing, modeling, content generation, gesture detection, gesture recognition, and/or other operations based on data from the image sensor 102, the image sensor 104, the one or more other sensors 106, and/or the wearable device 150. Further in Paragraph [0068]-MOUNIER discloses the one or more compute components 110 can implement one or more algorithms for tracking and estimating a relative pose of the wearable device 150 and the XR system 100. In some cases, the one or more compute components 110 can receive image data captured by the image sensor 102 and/or the image sensor 104 and perform pose estimation based on the received image data to calculate a relative pose of the wearable device 150 and the XR system 100. Please also read Paragraph [0080].) based on the posture image (Figs. 1-4, Paragraph [0098]-MOUNIER discloses the XR system 100 can detect a target (e.g., the wearable device 150, a hand associated with the wearable device 150, a finger associated with the wearable device 150, etc.) in an image(s) captured by one or more image sensors of the XR system 100. The one or more image sensors of the XR system 100 can capture the image when the target is within a FOV of the one or more image sensors as determined at least partly based on the data 302. Please also read Paragraph [0068].);
and determine the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the first position and the tracking position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position (wherein first position is the predetermined position) and the second position (wherein the second position is the tracking position), a movement of the wearable device relative to the electronic device. Please also read Paragraph [0133].).
Regarding claim 6, MOUNIER in view of Beyhs teach the controller according to claim 1, MOUNIER further teaches wherein the sensor data comprises a posture parameter of the user with a predetermined posture (Figs. 2A-2B and 4-5B, Paragraph [0053]-MOUNIER discloses the wearable accessory can include an inertial measurement unit (IMU) that can integrate multi-axes, accelerometers, gyroscopes, and/or other sensors to provide the XR device an estimate of the pose of the wearable accessory (and thus a body part wearing the wearable accessory) in physical space. Further in Paragraph [0071]-MOUNIER discloses in some cases, the IMU 152 can detect acceleration by the wearable device 150 and generate acceleration measurements based on the detected acceleration. In some cases, the IMU 152 can additionally or alternatively detect and measure the orientation and angular velocity of the wearable device 150. For example, the IMU 152 can measure the pitch, roll, and yaw of the wearable device 150.) and the controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0063]) is further configured to:
determine a sensor-identified position of the wearing part of the previous moment as the second position (Figs. 1, 2B and 4-5B, Paragraph [0079]-MOUNIER discloses at time T1, the ring device 200 can obtain data 220 based at least partly based on sensor data from one or more sensors (e.g., touchpad 204, sensors 208, etc.) on the ring device 200, and send the data 220 to the XR system 100. Further in Paragraph [0080]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. Please also read Paragraphs [0040] and [0068].) based on the posture parameter (Figs. 2A-2B and 4-5B, Paragraph [0071]-MOUNIER discloses in some cases, the IMU 152 can detect acceleration by the wearable device 150 and generate acceleration measurements based on the detected acceleration. In some cases, the IMU 152 can additionally or alternatively detect and measure the orientation and angular velocity of the wearable device 150. For example, the IMU 152 can measure the pitch, roll, and yaw of the wearable device 150.);
and determine the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the first position and the second position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position (wherein first position is the predetermined position) and the second position (wherein the second position is the tracking position), a movement of the wearable device relative to the electronic device. Please also read Paragraph [0133].).
Regarding claim 7, MOUNIER in view of Beyhs teach the controller according to claim 1, MOUNIER further teaches wherein the camera data comprises a posture image of the user with a predetermined posture (Figs. 1-4, Paragraph [0098]-MOUNIER discloses the XR system 100 can detect a target (e.g., the wearable device 150, a hand associated with the wearable device 150, a finger associated with the wearable device 150, etc.) in an image(s) captured by one or more image sensors of the XR system 100. The one or more image sensors of the XR system 100 can capture the image when the target is within a FOV of the one or more image sensors as determined at least partly based on the data 302. Further in Paragraph [0098]-MOUNIER discloses the XR system 100 can use the data 302 and/or image data from one or more image sensors on the XR system 100 to detect and/or recognize a gesture of the target, modify content (e.g., virtual content, interfaces, controls, etc.) rendered by the XR system 100, generate inputs/interactions with content rendered by the XR system 100, etc. Please also read Paragraphs [0068] and [0080].), the sensor data comprises a posture parameter of the user with a predetermined posture (Figs. 2A-2B and 4-5B, Paragraph [0053]-MOUNIER discloses the wearable accessory can include an inertial measurement unit (IMU) that can integrate multi-axes, accelerometers, gyroscopes, and/or other sensors to provide the XR device an estimate of the pose of the wearable accessory (and thus a body part wearing the wearable accessory) in physical space. Further in Paragraph [0071]-MOUNIER discloses in some cases, the IMU 152 can detect acceleration by the wearable device 150 and generate acceleration measurements based on the detected acceleration. In some cases, the IMU 152 can additionally or alternatively detect and measure the orientation and angular velocity of the wearable device 150. For example, the IMU 152 can measure the pitch, roll, and yaw of the wearable device 150.), and the controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0063]) is further configured to:
determine a camera-identified position of the wearing part of the previous moment (Figs. 1-4 and 5B, Paragraph [0133]-MOUNIER discloses at block 522, the process 520 can include determining a first position of a wearable device (e.g., wearable device 150, wearable ring 200) in a physical space. In some examples, determining the first position of the wearable device can include receiving, from the wearable device, image data from one or more image sensors on the electronic device and/or data associated with one or more measurements from one or more sensors on the wearable device; and determining the first position of the wearable device based on the image data from the one or more image sensors and/or data associated with the one or more measurements from the one or more sensors.) based on the posture image (Figs. 1-4, Paragraph [0098]-MOUNIER discloses the XR system 100 can detect a target (e.g., the wearable device 150, a hand associated with the wearable device 150, a finger associated with the wearable device 150, etc.) in an image(s) captured by one or more image sensors of the XR system 100. The one or more image sensors of the XR system 100 can capture the image when the target is within a FOV of the one or more image sensors as determined at least partly based on the data 302. Please also read Paragraph [0068].);
determine a sensor-identified position of the wearing part of the previous moment (Figs. 1, 2B and 4-5B, Paragraph [0079]-MOUNIER discloses at time T1, the ring device 200 can obtain data 220 based at least partly based on sensor data from one or more sensors (e.g., touchpad 204, sensors 208, etc.) on the ring device 200, and send the data 220 to the XR system 100. Further in Paragraph [0080]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. Please also read Paragraphs [0040] and [0068].) based on the posture parameter (Figs. 2A-2B and 4-5B, Paragraph [0071]-MOUNIER discloses in some cases, the IMU 152 can detect acceleration by the wearable device 150 and generate acceleration measurements based on the detected acceleration. In some cases, the IMU 152 can additionally or alternatively detect and measure the orientation and angular velocity of the wearable device 150. For example, the IMU 152 can measure the pitch, roll, and yaw of the wearable device 150.);
determine the second position of the wearing part based on the camera-identified position and the sensor-identified position (Figs. 1, 2B and 4-5B, Paragraph [0083]-MOUNIER discloses for example, the wearable device 150 can receive signals from the wearable device 150 to inform a tracking engine in the XR system 100. In some cases, the signals can include data from and/or data based on sensor data from sensors embedded in the wearable device 150. The XR system 100 can use the signals from the wearable device 150 to track a body part (e.g., a finger, hand, wrist, group of fingers, etc.) wearing the wearable device 150 and correlate an estimated location of the body part with a FOV of any image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100.);
and determine the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the first position and the second position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position (wherein first position is the predetermined position) and the second position (wherein the second position is the tracking position), a movement of the wearable device relative to the electronic device. Please also read Paragraph [0133].).
Regarding claim 10, MOUNIER in view of Beyhs teach the controller according to claim 1, MOUNIER further teaches wherein the sensor (Figs. 1, 2B and 4-5B, #152 called an Inertial Measurement Unit (IMU), Paragraph [0080]) comprises an inertial measurement unit and the sensor data comprises an inertial measurement value of the wearing part (Figs. 1-4, Paragraph [0053]-MOUNIER discloses the wearable accessory can include an inertial measurement unit (IMU) that can integrate multi-axes, accelerometers, gyroscopes, and/or other sensors to provide the XR device an estimate of the pose of the wearable accessory (and thus a body part wearing the wearable accessory) in physical space. The wearable accessory can include one or more sensors such as ultrasonic sensors and/or microphones used for ranging of the wearable accessory (and the body part wearing the wearable accessory). In some examples, one or more ultrasonic sensors and/or microphones can help determine if the wearable device is close to another object(s), if a user's hands (or other body part) are closer together or farther apart, if any of the user's hands (or other body part) are close to one or more other objects, etc. In some examples, a barometric air pressure sensor in the wearable accessory can determine relative elevation changes associated with the wearable accessory. The wearable accessory can send measurements from one or more sensors to the XR device, which can use the sensor measurements as further described herein.).
Regarding claim 12, MOUNIER in view of Beyhs teach the controller according to claim 1, MOUNIER further teaches wherein the wearing part comprises at least one limb of the user (Figs. 1-4, Paragraph [0043]-MOUNIER discloses the wearable accessory (wherein the wearable accessory is the wearing part) can be worn on a user's finger (or multiple fingers), a wrist, an ankle, and/or any other body part. The wearable accessory can include embedded sensors configured to obtain measurements of a state (e.g., a position, movement, etc.) of the wearable accessory in 3D space, and thus the state of the body part on which the wearable accessory is worn (e.g., a finger, a hand, etc.). Please also read Paragraphs [0044] and [0190].).
Regarding claim 13, MOUNIER in view of Beyhs teach the controller according to claim 1, MOUNIER further teaches wherein the wearing part comprises at least one finger of the user (Figs. 1-4, Paragraph [0044]-MOUNIER discloses the wearable accessory (wherein the wearable accessory is the wearing part) can include, for example and without limitation, a ring that can be worn on a finger, a sleeve of rings that can be worn on multiple fingers, a bracelet that can be worn on a wrist, a glove that can be worn on a hand, etc. For example, the wearable accessory can include a ring device worn on a user's finger. Please also read Paragraph [0043].).
Regarding claim 14, MOUNIER teaches a control method for a wearable device (Figs. 3-5B, Paragraph [0003]-MOUNIER discloses according to at least one example, a method is provided for tracking operations using data received from a wearable device. The method can include determining a first position of a wearable device in a physical space; receiving, from the wearable device, position information associated with the wearable device; determining a second position of the wearable device based on the received position information; and tracking, based on the first position and the second position, a movement of the wearable device relative to the electronic device.), MOUNIER further teaches comprising:
obtaining a camera data of a previous moment from a camera (Fig. 2B, #102 called an image sensor, Paragraph [0114]-MOUNIER discloses at block 422, the XR system 100 can use one or more image sensors that are turned on/enabled to capture an image(s) of the target. The one or more image sensors can include any image sensor on the XR system 100 having visibility to the target. At block 424, the wearable device 150 can also send data to the XR system 100. Further in Paragraph [0115]-MOUNIER discloses at block 426, the XR system 100 can use the image(s) of the target and the data from the wearable device 150 to track the target, as previously described. Please also read Paragraphs [0062] and [0087].), wherein the camera data comprises an image of the wearable device and a first position (Figs. 2A-5B, illustrates imaging of the wearable device, Paragraph [0133]-MOUNIER discloses at block 522, the process 520 can include determining a first position of a wearable device (e.g., wearable device 150, wearable ring 200) in a physical space. In some examples, determining the first position of the wearable device can include receiving, from the wearable device, image data from one or more image sensors on the electronic device and/or data associated with one or more measurements from one or more sensors on the wearable device; and determining the first position of the wearable device based on the image data from the one or more image sensors and/or data associated with the one or more measurements from the one or more sensors. Further in Paragraph [0138]-MOUNIER discloses the process 520 can include tracking, based on the first position and the second position, a movement of the wearable device relative to the electronic device);
obtaining a sensor data of the second position of a second body part of the user of the previous moment (Figs. 1, 2B and 4-5B, #152 called an Inertial Measurement Unit (IMU), Paragraph [0080]-MOUNIER discloses at time T1, the ring device 200 can obtain data 220 based at least partly based on sensor data from one or more sensors (e.g., touchpad 204, sensors 208, etc.) on the ring device 200, and send the data 220 to the XR system 100. Further in Paragraph [0079]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. Please also read Paragraphs [0040] and [0068].) from a sensor of the wearable device, wherein the wearable device is disposed on the second body part of the user (Figs. 1, 2B and 4-5B, #152 called an Inertial Measurement Unit (IMU), Paragraph [0068]-MOUNIER discloses the one or more compute components 110 can receive sensor data (e.g., data from the IMU 152, the ultrasonic sensor 154, the pressure sensor 156, and/or the touch sensor 158) from the wearable device 150, and use such data to track the wearable device 150 (with or without other data from the image sensor 102, the image sensor 104, or the other sensor(s) 106), adjust processing/power operations, etc., as described herein.);
determining a position conversion relationship for improving an accuracy of an operation of the second body part of the user (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Please also read Paragraph [0133]) based on the camera data of the previous moment and the sensor data of the previous moment (Figs. 1 and 2B, Paragraph [0080]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. In some examples, the XR system 100 can use the data 220 to determine whether the finger 210 (and/or a hand of the finger 210) is visible to one or more image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100. For example, the XR system 100 can use the data 220 to determine whether the finger 210 (and/or a hand of the finger 210) is within a FOV of one or more image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100 and/or whether a view of the one or more image sensors to the finger 210 (and/or the hand of the finger 210) is obstructed by one or more objects.),
wherein the position conversion relationship indicates a relationship between a first position of a body part of a portion of a user and a second position of a wearing part of the portion of the user (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.) (wherein a glove is a wearing part of the portion of the user). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s) (where in hand is a body part of a portion of the user), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Please also read Paragraph [0133].) and is configured to convert one point in a three-dimensional space to another point in the three-dimensional space (Figs. 1-4, Paragraph [0139]-MOUNIER discloses in some aspects, tracking the movement of the wearable device can include determining the first position of the wearable device within a first coordinate system of the wearable device; transforming the first coordinate system of the wearable device to a second coordinate system of the electronic device; and determining the second position of the wearable device within the second coordinate system of the electronic device. Please also read Paragraph [0040]) and the wearing part is where the wearable device is disposed on the user (Figs. 1-4, #150 called a wearable device, Paragraph [0043]-MOUNIER discloses a wearable accessory (or wearable device) that interfaces/interacts with an XR device to aid in tracking, provide power savings, and/or increase user privacy during XR experiences. In some examples, a wearable accessory can be worn by a user of an XR device during an XR experience. The wearable accessory (wherein the wearable accessory is the wearing part) can be worn on a user's finger (or multiple fingers), a wrist, an ankle, and/or any other body part. The wearable accessory can include embedded sensors configured to obtain measurements of a state (e.g., a position, movement, etc.) of the wearable accessory in 3D space, and thus the state of the body part on which the wearable accessory is worn (e.g., a finger, a hand, etc.). Please also read Paragraph [0044].);
fusioning the camera data and the sensor data and outputting a fusion data of the wearing part of a current moment based on the position conversion relationship (Figs. 1-4, Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Further in Paragraph [0105]-MOUNIER discloses at block 408, the wearable device 150 can send additional data to the XR system 100. At block 410, the XR system 100 can use the data to determine a visibility of one or more image sensors at the XR system 100 to the target. For example, the XR system 100 can use the data from the wearable device 150 to determine the position of the wearable device 150 and the target in 3D space. Please also read Paragraph [0040]), wherein the first position is different from the second position (process 520 can include determining, based on data from the wearable device and/or a command from the wearable device, one or more extended reality (XR) inputs to an XR application on the electronic device. In some examples, the one or more XR inputs can include a modification of a virtual element along multiple dimensions in space, a selection of the virtual element, a navigation event, and/or a request to measure a distance defined by the first position of the wearable device, the second position of the wearable device, and/or the movement of the wearable device – see [p][0150]);
and determining an operation of the wearing part of the user based on the fusion data (Figs. 2A-2B, 3 and 4, Paragraph [0043]-MOUNIER discloses the wearable accessory can include embedded sensors configured to obtain measurements of a state (e.g., a position, movement, etc.) of the wearable accessory in 3D space, and thus the state of the body part on which the wearable accessory is worn (e.g., a finger, a hand, etc.). The wearable accessory can provide the measurements to the XR device, which can integrate with its tracking system for more robust tracking and accuracy. Please also read Paragraphs [0040] and [0138].).
MOUNIER fails to explicitly teach the first body part is anatomically distinct from the second body part.
However, Beyhs explicitly teaches the first body part is anatomically distinct from the second body part (for e.g., the finger is anatomically different from the palm or the index finger being anatomically difference from the thumb – see [p][0067][0074] and Figs 3A-B and 12A-B )
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of MOUNIER of having a controller, adapted to determine a position conversion relationship between a position of a body part of a portion of a user and a tracking position of a wearing part of the portion of the user, wherein the position conversion relationship is configured to convert one point in a three-dimensional space to another point in the three-dimensional space, the wearing part is where a wearable device is disposed on the user, and the controller is configured to: obtain a camera data of a previous moment from a camera, wherein the camera data comprises an image of the wearable device and the first position, with the teachings of Beyhs of having the first body part is anatomically distinct from the second body part.
Wherein having MOUNIER’s the first body part is anatomically distinct from the second body part.
The motivation behind the modification would have been to obtain a wearable tracking system that provides more robust tracking and accuracy by detecting contact between a first body part and a second body par, since both MOUNIER and Beyhs are systems that relate to gesture detection. Wherein MOUNIER’s wearable system helps to increase the tracking fidelity/accuracy of the XR device and can trigger subsequent processing mode adjustments of any of the cameras of the XR device based on a position and/or motion of the wearable accessory, while Beyhs detecting contact between a first body part and a second body part. Please see MOUNIER et al. (US 20230048398 A1), Paragraphs [0043], [0046], and [0101], and Beyhs et al. (US20210303069A1), Paragraph [0004].
Regarding claim 15, MOUNIER in view of Beyhs teach the control method for the wearable device according to claim 14, MOUNIER further teaches further comprising:
determining a camera-identified position of the wearing part of the previous moment based on the camera data (Figs. 1-4 and 5B, Paragraph [0133]-MOUNIER discloses at block 522, the process 520 can include determining a first position of a wearable device (e.g., wearable device 150, wearable ring 200) in a physical space. In some examples, determining the first position of the wearable device can include receiving, from the wearable device, image data from one or more image sensors on the electronic device and/or data associated with one or more measurements from one or more sensors on the wearable device; and determining the first position of the wearable device based on the image data from the one or more image sensors and/or data associated with the one or more measurements from the one or more sensors.);
determining a sensor-identified position of the wearing part of the previous moment based on the sensor data (Figs. 1, 2B and 4-5B, Paragraph [0079]-MOUNIER discloses at time T1, the ring device 200 can obtain data 220 based at least partly based on sensor data from one or more sensors (e.g., touchpad 204, sensors 208, etc.) on the ring device 200, and send the data 220 to the XR system 100. Further in Paragraph [0080]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. Please also read Paragraphs [0040] and [0068].);
determining the second position of the wearing part based on the camera-identified position and the sensor-identified position (Figs. 1, 2B and 4-5B, Paragraph [0083]-MOUNIER discloses for example, the wearable device 150 can receive signals from the wearable device 150 to inform a tracking engine in the XR system 100. In some cases, the signals can include data from and/or data based on sensor data from sensors embedded in the wearable device 150. The XR system 100 can use the signals from the wearable device 150 to track a body part (e.g., a finger, hand, wrist, group of fingers, etc.) wearing the wearable device 150 and correlate an estimated location of the body part with a FOV of any image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100.);
and determining the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the first position and the second position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position (wherein first position is the predetermined position) and the second position (wherein the second position is the tracking position), a movement of the wearable device relative to the electronic device. Please also read Paragraph [0133].).
Regarding claim 16, MOUNIER in view of Beyhs teach the control method for the wearable device according to claim 14, MOUNIER further teaches further comprising:
obtaining a posture image of the user with a predetermined posture from the camera data (Figs. 2A-2B, 4 and 5A-5B, illustrates imaging the wearing part, Paragraph [0067]-MOUNIER discloses the one or more compute components 110 can perform XR processing operations based on data from the image sensor 102, the image sensor 104, the one or more other sensors 106, and/or the wearable device 150. For example, in some cases, the one or more compute components 110 can perform tracking, localization, object detection, object classification, pose estimation, shape estimation, mapping, content anchoring, content rendering, image processing, modeling, content generation, gesture detection, gesture recognition, and/or other operations based on data from the image sensor 102, the image sensor 104, the one or more other sensors 106, and/or the wearable device 150. Further in Paragraph [0068]-MOUNIER discloses the one or more compute components 110 can implement one or more algorithms for tracking and estimating a relative pose of the wearable device 150 and the XR system 100. In some cases, the one or more compute components 110 can receive image data captured by the image sensor 102 and/or the image sensor 104 and perform pose estimation based on the received image data to calculate a relative pose of the wearable device 150 and the XR system 100. Please also read Paragraph [0080].);
determining a camera-identified position of the wearing part of the previous moment as the second position based on the posture image (Figs. 2A-2B, 4 and 5A-5B, illustrates imaging the wearing part, Paragraph [0067]-MOUNIER discloses the one or more compute components 110 can perform XR processing operations based on data from the image sensor 102, the image sensor 104, the one or more other sensors 106, and/or the wearable device 150. For example, in some cases, the one or more compute components 110 can perform tracking, localization, object detection, object classification, pose estimation, shape estimation, mapping, content anchoring, content rendering, image processing, modeling, content generation, gesture detection, gesture recognition, and/or other operations based on data from the image sensor 102, the image sensor 104, the one or more other sensors 106, and/or the wearable device 150. Further in Paragraph [0068]-MOUNIER discloses the one or more compute components 110 can implement one or more algorithms for tracking and estimating a relative pose of the wearable device 150 and the XR system 100. In some cases, the one or more compute components 110 can receive image data captured by the image sensor 102 and/or the image sensor 104 and perform pose estimation based on the received image data to calculate a relative pose of the wearable device 150 and the XR system 100. Please also read Paragraph [0080].);
and determining the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the first position and the tracking position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position (wherein first position is the predetermined position) and the second position (wherein the second position is the tracking position), a movement of the wearable device relative to the electronic device. Please also read Paragraph [0133].).
Regarding claim 17, MOUNIER in view of Beyhs teach the control method for the wearable device according to claim 14, MOUNIER further teaches further comprising:
obtaining a posture parameter of the user with a predetermined posture from the sensor data (Figs. 2A-2B and 4-5B, Paragraph [0053]-MOUNIER discloses the wearable accessory can include an inertial measurement unit (IMU) that can integrate multi-axes, accelerometers, gyroscopes, and/or other sensors to provide the XR device an estimate of the pose of the wearable accessory (and thus a body part wearing the wearable accessory) in physical space. Further in Paragraph [0071]-MOUNIER discloses in some cases, the IMU 152 can detect acceleration by the wearable device 150 and generate acceleration measurements based on the detected acceleration. In some cases, the IMU 152 can additionally or alternatively detect and measure the orientation and angular velocity of the wearable device 150. For example, the IMU 152 can measure the pitch, roll, and yaw of the wearable device 150.);
determining a sensor-identified position of the wearing part of the previous moment as the second position (Figs. 1, 2B and 4-5B, Paragraph [0079]-MOUNIER discloses at time T1, the ring device 200 can obtain data 220 based at least partly based on sensor data from one or more sensors (e.g., touchpad 204, sensors 208, etc.) on the ring device 200, and send the data 220 to the XR system 100. Further in Paragraph [0080]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. Please also read Paragraphs [0040] and [0068].) based on the posture parameter (Figs. 2A-2B and 4-5B, Paragraph [0071]-MOUNIER discloses in some cases, the IMU 152 can detect acceleration by the wearable device 150 and generate acceleration measurements based on the detected acceleration. In some cases, the IMU 152 can additionally or alternatively detect and measure the orientation and angular velocity of the wearable device 150. For example, the IMU 152 can measure the pitch, roll, and yaw of the wearable device 150.);
and determining the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the first position and the second position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position (wherein first position is the predetermined position) and the second position (wherein the second position is the tracking position), a movement of the wearable device relative to the electronic device. Please also read Paragraph [0133].).
Regarding claim 18, MOUNIER in view of Beyhs teach the control method for the wearable device according to claim 14, MOUNIER further teaches further comprising:
obtaining a posture image of the user with a predetermined posture from the camera data (Figs. 1-4, Paragraph [0098]-MOUNIER discloses the XR system 100 can detect a target (e.g., the wearable device 150, a hand associated with the wearable device 150, a finger associated with the wearable device 150, etc.) in an image(s) captured by one or more image sensors of the XR system 100. The one or more image sensors of the XR system 100 can capture the image when the target is within a FOV of the one or more image sensors as determined at least partly based on the data 302. Further in Paragraph [0098]-MOUNIER discloses the XR system 100 can use the data 302 and/or image data from one or more image sensors on the XR system 100 to detect and/or recognize a gesture of the target, modify content (e.g., virtual content, interfaces, controls, etc.) rendered by the XR system 100, generate inputs/interactions with content rendered by the XR system 100, etc. Please also read Paragraphs [0068] and [0080].);
obtaining a posture parameter of the user with the predetermined posture from the sensor data (Figs. 2A-2B and 4-5B, Paragraph [0053]-MOUNIER discloses the wearable accessory can include an inertial measurement unit (IMU) that can integrate multi-axes, accelerometers, gyroscopes, and/or other sensors to provide the XR device an estimate of the pose of the wearable accessory (and thus a body part wearing the wearable accessory) in physical space. Further in Paragraph [0071]-MOUNIER discloses in some cases, the IMU 152 can detect acceleration by the wearable device 150 and generate acceleration measurements based on the detected acceleration. In some cases, the IMU 152 can additionally or alternatively detect and measure the orientation and angular velocity of the wearable device 150. For example, the IMU 152 can measure the pitch, roll, and yaw of the wearable device 150.);
determining a camera-identified position of the wearing part of the previous moment (Figs. 1-4 and 5B, Paragraph [0133]-MOUNIER discloses at block 522, the process 520 can include determining a first position of a wearable device (e.g., wearable device 150, wearable ring 200) in a physical space. In some examples, determining the first position of the wearable device can include receiving, from the wearable device, image data from one or more image sensors on the electronic device and/or data associated with one or more measurements from one or more sensors on the wearable device; and determining the first position of the wearable device based on the image data from the one or more image sensors and/or data associated with the one or more measurements from the one or more sensors.) based on the posture image (Figs. 1-4, Paragraph [0098]-MOUNIER discloses the XR system 100 can detect a target (e.g., the wearable device 150, a hand associated with the wearable device 150, a finger associated with the wearable device 150, etc.) in an image(s) captured by one or more image sensors of the XR system 100. The one or more image sensors of the XR system 100 can capture the image when the target is within a FOV of the one or more image sensors as determined at least partly based on the data 302. Please also read Paragraph [0068].);
determining a sensor-identified position of the wearing part of the previous moment (Figs. 1, 2B and 4-5B, Paragraph [0079]-MOUNIER discloses at time T1, the ring device 200 can obtain data 220 based at least partly based on sensor data from one or more sensors (e.g., touchpad 204, sensors 208, etc.) on the ring device 200, and send the data 220 to the XR system 100. Further in Paragraph [0080]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. Please also read Paragraphs [0040] and [0068].) based on the posture parameter (Figs. 2A-2B and 4-5B, Paragraph [0071]-MOUNIER discloses in some cases, the IMU 152 can detect acceleration by the wearable device 150 and generate acceleration measurements based on the detected acceleration. In some cases, the IMU 152 can additionally or alternatively detect and measure the orientation and angular velocity of the wearable device 150. For example, the IMU 152 can measure the pitch, roll, and yaw of the wearable device 150.);
determining the second position of the wearing part based on the camera-identified position and the sensor-identified position (Figs. 1, 2B and 4-5B, Paragraph [0083]-MOUNIER discloses for example, the wearable device 150 can receive signals from the wearable device 150 to inform a tracking engine in the XR system 100. In some cases, the signals can include data from and/or data based on sensor data from sensors embedded in the wearable device 150. The XR system 100 can use the signals from the wearable device 150 to track a body part (e.g., a finger, hand, wrist, group of fingers, etc.) wearing the wearable device 150 and correlate an estimated location of the body part with a FOV of any image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100.);
and determining the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the first position and the tracking position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position (wherein first position is the predetermined position) and the second position (wherein the second position is the tracking position), a movement of the wearable device relative to the electronic device. Please also read Paragraph [0133].).
Regarding claim 20, MOUNIER teaches a wearable tracking system (Figs. 1-4, illustrates a wearable tracking system, Paragraph [0003]-MOUNIER discloses FIG. 1 is a diagram illustrating an example of an extended reality system and a wearable device used for extended reality experiences and functionalities, in accordance with some examples of the present disclosure; Further in Paragraph [0045]-MOUNIER discloses the wearable accessory can help reduce power consumption at the XR device by triggering the XR device to modify an operation of the cameras at the XR device and/or a tracking system used by the XR device. Please also read Paragraph [0043].), comprising:
a camera (Fig. 2B, #102 called an image sensor, Paragraph [0062]), configured to obtain a camera data of a previous moment (Fig. 2B, Paragraph [0114]-MOUNIER discloses at block 422, the XR system 100 can use one or more image sensors that are turned on/enabled to capture an image(s) of the target. The one or more image sensors can include any image sensor on the XR system 100 having visibility to the target. At block 424, the wearable device 150 can also send data to the XR system 100. Further in Paragraph [0115]-MOUNIER discloses at block 426, the XR system 100 can use the image(s) of the target and the data from the wearable device 150 to track the target, as previously described. Please also read Paragraphs [0062] and [0087].), wherein the camera data comprises an image of a wearable device and a first position of a first body part of a user (Figs. 2A-5B, illustrates imaging of the wearable device, Paragraph [0133]-MOUNIER discloses at block 522, the process 520 can include determining a first position of a wearable device (e.g., wearable device 150, wearable ring 200) in a physical space. In some examples, determining the first position of the wearable device can include receiving, from the wearable device, image data from one or more image sensors on the electronic device and/or data associated with one or more measurements from one or more sensors on the wearable device; and determining the first position of the wearable device based on the image data from the one or more image sensors and/or data associated with the one or more measurements from the one or more sensors. Further in Paragraph [0138]-MOUNIER discloses the process 520 can include tracking, based on the first position (wherein first position is a predetermined position) and the second position, a movement of the wearable device relative to the electronic device.);
the wearable device (Figs. 1-4, #150 called a wearable device, Paragraph [0055]), comprising a sensor (Figs. 1-4, #152 called an Inertial Measurement Unit (IMU), Paragraph [0059]), wherein the sensor is configured to obtain a sensor data of the second position of a second body part of the user of the previous moment, wherein the wearable device is disposed on the second body part of the user (Figs. 1, 2B and 4-5B, Paragraph [0080]-MOUNIER discloses at time T1, the ring device 200 can obtain data 220 based at least partly based on sensor data from one or more sensors (e.g., touchpad 204, sensors 208, etc.) on the ring device 200, and send the data 220 to the XR system 100. Further in Paragraph [0079]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. Please also read Paragraphs [0040] and [0068].);
and a controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0063]), wherein the controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0153]-MOUNIER discloses in some examples, the process 500 or the process 520 may be performed by one or more computing devices or apparatuses. In one illustrative example, the process 500 can be performed by the XR system 100 and/or the wearable device 150 shown in FIG. 1 and/or one or more computing devices with the computing device architecture 600 shown in FIG. 6. In another illustrative example, the process 520 can be performed by the XR system 100 and/or one or more computing devices with the computing device architecture 600 shown in FIG. 6. In some cases, such a computing device or apparatus may include a processor, microprocessor, microcomputer, or other component of a device that is configured to carry out the steps of the process 500 or the process 520. Please also read Paragraph [0063].) is configured to:
determine a position conversion relationship for improving an accuracy of an operation of an operation of the second body part of the user (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Please also read Paragraph [0133].) based on the camera data of the previous moment and the sensor data of the previous moment (Figs. 1 and 2B, Paragraph [0080]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. In some examples, the XR system 100 can use the data 220 to determine whether the finger 210 (and/or a hand of the finger 210) is visible to one or more image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100. For example, the XR system 100 can use the data 220 to determine whether the finger 210 (and/or a hand of the finger 210) is within a FOV of one or more image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100 and/or whether a view of the one or more image sensors to the finger 210 (and/or the hand of the finger 210) is obstructed by one or more objects.),
wherein the position conversion relationship indicates a relationship between the first position of a body part (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.) (wherein a glove is a wearing part of the portion of the user). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s) (where in hand is a body part of a portion of the user), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Please also read Paragraph [0133]) and is configured to convert one point in a three-dimensional space to another point in the three-dimensional space (Figs. 1-4, Paragraph [0139]-MOUNIER discloses in some aspects, tracking the movement of the wearable device can include determining the first position of the wearable device within a first coordinate system of the wearable device; transforming the first coordinate system of the wearable device to a second coordinate system of the electronic device; and determining the second position of the wearable device within the second coordinate system of the electronic device. Please also read Paragraph [0040].)
fusion the camera data and the sensor data and output a fusion data of the second body part of a current moment based on the position conversion relationship (Figs. 1-4, Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Further in Paragraph [0105]-MOUNIER discloses at block 408, the wearable device 150 can send additional data to the XR system 100. At block 410, the XR system 100 can use the data to determine a visibility of one or more image sensors at the XR system 100 to the target. For example, the XR system 100 can use the data from the wearable device 150 to determine the position of the wearable device 150 and the target in 3D space. Please also read Paragraph [0040]), wherein the first position is different from the second position (process 520 can include determining, based on data from the wearable device and/or a command from the wearable device, one or more extended reality (XR) inputs to an XR application on the electronic device. In some examples, the one or more XR inputs can include a modification of a virtual element along multiple dimensions in space, a selection of the virtual element, a navigation event, and/or a request to measure a distance defined by the first position of the wearable device, the second position of the wearable device, and/or the movement of the wearable device – see [p][0150]);
and determine the operation of the second body part of the user based on the fusion data (Figs. 2A-2B, 3 and 4, Paragraph [0043]-MOUNIER discloses the wearable accessory can include embedded sensors configured to obtain measurements of a state (e.g., a position, movement, etc.) of the wearable accessory in 3D space, and thus the state of the body part on which the wearable accessory is worn (e.g., a finger, a hand, etc.). The wearable accessory can provide the measurements to the XR device, which can integrate with its tracking system for more robust tracking and accuracy. Please also read Paragraphs [0040] and [0138]).
MOUNIER fails to explicitly teach the first body part is anatomically distinct from the second body part.
However, Beyhs explicitly teaches the first body part is anatomically distinct from the second body part (for e.g., the finger is anatomically different from the palm or the index finger being anatomically difference from the thumb – see [p][0067][0074] and Figs 3A-B and 12A-B )
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of MOUNIER of having a controller, adapted to determine a position conversion relationship between a position of a body part of a portion of a user and a tracking position of a wearing part of the portion of the user, wherein the position conversion relationship is configured to convert one point in a three-dimensional space to another point in the three-dimensional space, the wearing part is where a wearable device is disposed on the user, and the controller is configured to: obtain a camera data of a previous moment from a camera, wherein the camera data comprises an image of the wearable device and the first position, with the teachings of Beyhs of having the first body part is anatomically distinct from the second body part.
Wherein having MOUNIER’s the first body part is anatomically distinct from the second body part.
The motivation behind the modification would have been to obtain a wearable tracking system that provides more robust tracking and accuracy by detecting contact between a first body part and a second body par, since both MOUNIER and Beyhs are systems that relate to gesture detection. Wherein MOUNIER’s wearable system helps to increase the tracking fidelity/accuracy of the XR device and can trigger subsequent processing mode adjustments of any of the cameras of the XR device based on a position and/or motion of the wearable accessory, while Beyhs detecting contact between a first body part and a second body part. Please see MOUNIER et al. (US 20230048398 A1), Paragraphs [0043], [0046], and [0101], and Beyhs et al. (US20210303069A1), Paragraph [0004].
Claims 8-9 and 19, are rejected under 35 U.S.C. 103 as being unpatentable over MOUNIER et al. (US 20230048398 A1) hereinafter referenced as MOUNIER in view of Beyhs et al. (US US20210303069 A1) as applied to claims 1 and 14 in view of ERIVANTCEV et al. (US 20230011082 A1).
Regarding claim 8, MOUNIER teaches the controller according to claim 1, Although MOUNIER further teaches wherein the controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0063]) is further configured to:
determine the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the first position and the second position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position (wherein first position is the predetermined position) and the second position (wherein the second position is the tracking position), a movement of the wearable device relative to the electronic device. Please also read Paragraph [0133].);
and obtain the fusion data of the current moment based on the position conversion relationship (Figs. 1-4, Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Further in Paragraph [0105]-MOUNIER discloses at block 408, the wearable device 150 can send additional data to the XR system 100. At block 410, the XR system 100 can use the data to determine a visibility of one or more image sensors at the XR system 100 to the target. For example, the XR system 100 can use the data from the wearable device 150 to determine the position of the wearable device 150 and the target in 3D space. Please also read Paragraph [0040].).
MOUNIER in view of Beyhs fail to explicitly teach obtain a fusion position of the previous moment as the second position by fusing the camera data and the sensor data based on a Kalman filter.
However, ERIVANTCEV explicitly teaches obtain a fusion position of the previous moment as the second position (Figs. 1-5, Paragraph [0138]-ERIVANTCEV discloses at block 333, the sensor module (e.g., 113) computes, based on the inputs from the inertial measurement unit (e.g., 131), first positions and first orientations of the sensor module (e.g., 113) at a first time interval during a first period of time containing multiple of the first time interval (e.g., at a rate of hundreds per second). Further in Paragraph [0140]-ERIVANTCEV discloses at block 335, at least one camera is used to capture images of the sensor module (e.g., 113) at a second time interval, larger than the first time interval, during the first period of time containing multiple of the second interval (e.g., at a rate of 30 to 60 per second).) by fusing the camera data and the sensor data based on a Kalman filter (Figs. 1-5, Paragraph [0038]-ERIVANTCEV discloses an optical-based tracking system can use one or more cameras to capture images of a sensor module marked using optical markers and analyze the images to compute the position and/or orientation of the part. For example, an inertial-based tracking system can use a sensor module having an inertial measurement unit to determine its position and/or orientation and thus the position and/or orientation of the part of the user wearing the sensor module. Further in Paragraph [0039]-ERIVANTCEV discloses the system can dynamically combine the measurements from the optical-based tracking system and the inertial-based tracking system using a Kalman-type filter approach for improved accuracy and/or efficiency. Please also read Paragraph [0148].).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of MOUNIER as modified by Beyhs of having a controller, adapted to determine a position conversion relationship between a position of a body part of a portion of a user and a second position of a wearing part of the portion of the user, wherein the position conversion relationship is configured to convert one point in a three-dimensional space to another point in the three-dimensional space, the wearing part is where a wearable device is disposed on the user, and the controller is configured to: obtain a camera data of a previous moment from a camera, wherein the camera data comprises an image of the wearable device and the first position, with the teachings of ERIVANTCEV of having obtain a fusion position of the previous moment as the second position by fusing the camera data and the sensor data based on a Kalman filter.
Wherein having MOUNIER’s wearable second system of obtain a fusion position of the previous moment as the second position by fusing the camera data and the sensor data based on a Kalman filter.
The motivation behind the modification would have been to obtain a wearable tracking system that provides more robust tracking and accuracy, since both MOUNIER and ERIVANTCEV are systems that relate to extended reality. Wherein MOUNIER’s wearable system helps to increase the tracking fidelity/accuracy of the XR device and can trigger subsequent processing mode adjustments of any of the cameras of the XR device based on a position and/or motion of the wearable accessory, while ERIVANTCEV's system provides estimates that are improved via the inputs from the optical-based tracking system, the improved estimates can be used to calibrate the inertial-based tracking system and thus remove the accumulated drifts in the inertial-based tracking system. Please see MOUNIER et al. (US 20230048398 A1), Paragraphs [0043], [0046], and [0101], and ERIVANTCEV et al. (US 20230011082 A1), Paragraph [0023].
Regarding claim 9, MOUNIER teaches the controller according to claim 1, Although MOUNIER further teaches wherein the controller (Figs. 1-4 and 6, #110 called one or more compute components, Paragraph [0063]) is further configured to:
determine a camera-predicted position of the wearing part of the current moment by an extrapolation based on the camera data (Figs. 2A-4, Paragraph [0080]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. In some examples, the XR system 100 can use the data 220 to determine whether the finger 210 (and/or a hand of the finger 210) is visible to one or more image sensors (e.g., image sensor 102, image sensor 104) on the XR system 100. Further in Paragraph [0081]-MOUNIER discloses at time Tn, the ring device 200 can obtain data 230 based at least partly based on sensor data from the one or more sensors on the ring device 200, and send the data 230 to the XR system 100. The XR system 100 can use the data 230 to track the ring device 200 at Tn. In some cases, the XR system 100 can use the data 230 to estimate a position of the ring device 200 at one or more time steps after Tn. In some examples, the XR system 100 can use the data 230 to determine whether the finger 210 (and/or a hand of the finger 210) is visible to one or more image sensors on the XR system 100, as previously explained.);
determine a sensor-predicted position of the wearing part of the current moment (Figs. 2A-4, Paragraph [0080]-MOUNIER discloses the XR system 100 can use the data 220 to track the ring device 200 at T1. In some cases, the XR system 100 can use the data 220 to estimate a position of the ring device 200 at one or more time steps after T1. Further in Paragraph [0081]-MOUNIER discloses at time Tn, the ring device 200 can obtain data 230 based at least partly based on sensor data from the one or more sensors on the ring device 200, and send the data 230 to the XR system 100. The XR system 100 can use the data 230 to track the ring device 200 at Tn. In some cases, the XR system 100 can use the data 230 to estimate a position of the ring device 200 at one or more time steps after Tn.) based on an acceleration value or displacement value of the sensor data (Figs. 2A-4, Paragraph [0069]-MOUNIER discloses the one or more other sensors 106 can detect acceleration by the XR system 100 and generate acceleration measurements based on the detected acceleration. In some cases, the one or more other sensors 106 can additionally or alternatively detect and measure the orientation and angular velocity of the XR system 100. For example, the one or more other sensors 106 can measure the pitch, roll, and yaw of the XR system 100. In some examples, the XR system 100 can use measurements obtained by the one or more other sensors 106 to calculate the relative pose of the XR system 100. In some cases, the XR system 100 can additionally or alternatively use sensor data from the wearable device 150 to perform tracking, pose estimation, and/or other operations. Further in Paragraph [0070]-MOUNIER discloses the wearable device 150 can use the IMU 152, the ultrasonic sensor 154, and/or the pressure sensor 156 to obtain tracking measurements for the wearable device 150. The tracking measurements can include, for example and without limitation, position measurements, velocity/motion measurements, range/distance measurements, elevation measurements, etc.);
and obtain the fusion data of the current moment based on the fusion-predicted position and the position conversion relationship (Figs. 1-4, Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Further in Paragraph [0105]-MOUNIER discloses at block 408, the wearable device 150 can send additional data to the XR system 100. At block 410, the XR system 100 can use the data to determine a visibility of one or more image sensors at the XR system 100 to the target. For example, the XR system 100 can use the data from the wearable device 150 to determine the position of the wearable device 150 and the target in 3D space. Please also read Paragraph [0040].).
MOUNIER in view of Beyhs fail to explicitly teach obtain a fusion-predicted position of the current moment by fusing the camera-predicted position and the sensor-predicted position based on a Kalman filter.
However, ERIVANTCEV explicitly teaches obtain a fusion-predicted position of the current moment (Figs. 1-5, Paragraph [0129]-ERIVANTCEV discloses the Kalman-type filter 309 is configured to generate a new estimate based on a prior estimate and a new measurement (e.g., 305 or 306). The Kalman-type filter 309 includes estimates for state parameters (e.g., position and orientation) and rates of the state parameters (e.g., velocity). A filter parameter α provides a weight for a change from the prior estimate of the state parameters to the new measurements of the state parameters for adding to the prior estimate of the state parameters.) by fusing the camera-predicted position and the sensor-predicted position based on a Kalman filter (Figs. 1-5, Paragraph [0127]-ERIVANTCEV discloses once a measurement 305 of position and orientation is determined at block 303 to be available in the inertial-based tracking system 301, the measurement 305 is provided to a Kalman-type filter 309 to update its position and orientation estimate 311 for the sensor module. The estimate 311 identifies the real-time position and orientation of the sensor module, in view of the measurement 305 from the inertial-based tracking system. Further in Paragraph [0127]-ERIVANTCEV discloses similarly, once a measurement 306 of position and orientation is determined at block 304 to be available in the optical-based tracking system 302, the measurement 306 is provided to the Kalman-type filter 309 to update its position and orientation estimate 311 for the sensor module. The estimate 311 identifies the real-time position and orientation of the sensor module, in view of the measurement 305 from the inertial-based tracking system.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of MOUNIER as modified by Beyhs of having a controller, adapted to determine a position conversion relationship between a position of a body part of a portion of a user and a tracking position of a wearing part of the portion of the user, wherein the position conversion relationship is configured to convert one point in a three-dimensional space to another point in the three-dimensional space, the wearing part is where a wearable device is disposed on the user, and the controller is configured to: obtain a camera data of a previous moment from a camera, wherein the camera data comprises an image of the wearable device and the first position, with the teachings of ERIVANTCEV of having obtain a fusion-predicted position of the current moment by fusing the camera-predicted position and the sensor-predicted position based on a Kalman filter.
Wherein having MOUNIER’s wearable tracking system of obtain a fusion-predicted position of the current moment by fusing the camera-predicted position and the sensor-predicted position based on a Kalman filter.
The motivation behind the modification would have been to obtain a wearable tracking system that provides more robust tracking and accuracy, since both MOUNIER and ERIVANTCEV are systems that relate to extended reality. Wherein MOUNIER’s wearable system helps to increase the tracking fidelity/accuracy of the XR device and can trigger subsequent processing mode adjustments of any of the cameras of the XR device based on a position and/or motion of the wearable accessory, while ERIVANTCEV's system provides estimates that are improved via the inputs from the optical-based tracking system, the improved estimates can be used to calibrate the inertial-based tracking system and thus remove the accumulated drifts in the inertial-based tracking system. Please see MOUNIER et al. (US 20230048398 A1), Paragraphs [0043], [0046], and [0101], and ERIVANTCEV et al. (US 20230011082 A1), Paragraph [0023].
Regarding claim 19, MOUNIER in view of Beyhs teach the control method for the wearable device according to claim 14, Although MOUNIER further teaches further comprising:
determining the position conversion relationship (Figs. 1-4, Paragraph [0040]-MOUNIER discloses an XR device can implement a tracking algorithm that uses sensor data to track the position of an object in three-dimensional (3D) space, such as a hand, a finger, an input device (e.g., a controller, a stylus, a joystick, a glove, etc.). For example, a tracking algorithm can use measurements from various sensors, such as image sensors and inertial measurement units (IMUs), a pose of a controller (or data/measurements thereof), and/or a motion model for a user's hand(s) to estimate a hand/controller pose used by the XR device during an XR experience. In some cases, the tracking algorithm can use sensor data to predict the location of a user's hand(s), a location of a camera of the XR device, and/or a state of the XR device. The XR device can measure the location of the XR device, camera, and/or user's hand(s), and use such measurement to update the state of the XR device. Further in Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100.) based on the first position and the second position (Figs. 1-4, Paragraph [0138]-MOUNIER discloses at block 528, the process 520 can include tracking, based on the first position (wherein first position is the predetermined position) and the second position (wherein the second position is the tracking position), a movement of the wearable device relative to the electronic device. Please also read Paragraph [0133].);
and obtaining the fusion data of the current moment based on the position conversion relationship (Figs. 1-4, Paragraph [0084]-MOUNIER discloses the XR system 100 can improve a tracking robustness by combining signals from the wearable device 150 with image data from one or more image sensors on the XR system 100. Further in Paragraph [0105]-MOUNIER discloses at block 408, the wearable device 150 can send additional data to the XR system 100. At block 410, the XR system 100 can use the data to determine a visibility of one or more image sensors at the XR system 100 to the target. For example, the XR system 100 can use the data from the wearable device 150 to determine the position of the wearable device 150 and the target in 3D space. Please also read Paragraph [0040].).
MOUNIER in view of Beyhs fail to explicitly teach obtaining a fusion position of the previous moment as the second position by fusing the camera data and the sensor data based on a Kalman filter.
However, ERIVANTCEV explicitly teaches obtaining a fusion position of the previous moment as the second position (Figs. 1-5, Paragraph [0138]-ERIVANTCEV discloses at block 333, the sensor module (e.g., 113) computes, based on the inputs from the inertial measurement unit (e.g., 131), first positions and first orientations of the sensor module (e.g., 113) at a first time interval during a first period of time containing multiple of the first time interval (e.g., at a rate of hundreds per second). Further in Paragraph [0140]-ERIVANTCEV discloses at block 335, at least one camera is used to capture images of the sensor module (e.g., 113) at a second time interval, larger than the first time interval, during the first period of time containing multiple of the second interval (e.g., at a rate of 30 to 60 per second).) by fusing the camera data and the sensor data based on a Kalman filter (Figs. 1-5, Paragraph [0038]-ERIVANTCEV discloses an optical-based tracking system can use one or more cameras to capture images of a sensor module marked using optical markers and analyze the images to compute the position and/or orientation of the part. For example, an inertial-based tracking system can use a sensor module having an inertial measurement unit to determine its position and/or orientation and thus the position and/or orientation of the part of the user wearing the sensor module. Further in Paragraph [0039]-ERIVANTCEV discloses the system can dynamically combine the measurements from the optical-based tracking system and the inertial-based tracking system using a Kalman-type filter approach for improved accuracy and/or efficiency. Please also read Paragraph [0148].).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of MOUNIER as modified by Beyhs of having a control method for a wearable device, comprising: obtaining a camera data of a previous moment from a camera, wherein the camera data comprises an image of the wearable device and a first position; obtaining a sensor data of the second position of the previous moment from a sensor of the wearable device, with the teachings of ERIVANTCEV of having obtain a fusion-predicted position of the current moment by fusing the camera-predicted position and the sensor-predicted position based on a Kalman filter.
Wherein having MOUNIER’s wearable tracking system of obtaining a fusion position of the previous moment as the tracking position by fusing the camera data and the sensor data based on a Kalman filter.
The motivation behind the modification would have been to obtain a wearable tracking system that provides more robust tracking and accuracy, since both MOUNIER and ERIVANTCEV are systems that relate to extended reality. Wherein MOUNIER’s wearable system helps to increase the tracking fidelity/accuracy of the XR device and can trigger subsequent processing mode adjustments of any of the cameras of the XR device based on a position and/or motion of the wearable accessory, while ERIVANTCEV's system provides estimates that are improved via the inputs from the optical-based tracking system, the improved estimates can be used to calibrate the inertial-based tracking system and thus remove the accumulated drifts in the inertial-based tracking system. Please see MOUNIER et al. (US 20230048398 A1), Paragraphs [0043], [0046], and [0101], and ERIVANTCEV et al. (US 20230011082 A1), Paragraph [0023].
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over MOUNIER et al. (US 20230048398 A1) hereinafter referenced as MOUNIER in view of Beyhs et al. (US US20210303069 A1) as applied to claim 1 in view of DE NARDI et al. (US 20200150753 A1).
Regarding claim 11, Although MOUNIER teaches the controller according to claim 10, MOUNIER in view Beyhs fail to explicitly teach wherein the inertial measurement value comprises changes in six degrees of freedom and the six degrees of freedom comprises three translation values corresponding to three perpendicular axes and three rotation values corresponding to the three perpendicular axes.
However, DE NARDI explicitly teaches wherein the inertial measurement value comprises changes in six degrees of freedom and the six degrees of freedom comprises three translation values corresponding to three perpendicular axes and three rotation values corresponding to the three perpendicular axes (Figs. 1 and 5, Paragraph [0133]-MOUNIER discloses the bracelet 105 includes a position sensor 120. There may be any number of position sensors 120 located at various points on the bracelet 105. The one or more position sensors may be located external to an inertial measurement unit (IMU) (not shown), internal to an IMU, or some combination thereof. The position sensor 120 may be any sensor capable of determining a position of the bracelet 105 and generating a signal in response to movement of the bracelet 105. Since the bracelet 105 is worn on a user's wrist, the position sensors 120 therefore provide position signals that result from the movement of a user's arm. In some embodiments, the position sensor 120 tracks the position of the bracelet 105 over time, using a previous location data point to determine subsequent positions. In some embodiments, the position sensor 120 may be an accelerometer that measures translational motion (e.g., forward/back, up/down, left/right). Additionally or alternatively, the position sensor 120 is a gyroscope that measures rotational motion (e.g., pitch, yaw, and roll). In some embodiments, the multiple accelerometers and multiple gyroscopes located on the bracelet 105 together provide position signals indicating movement of the bracelet 105 in six degrees of freedom. The position sensor 120 may be a MEMS device.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of MOUNIER as modified by Beyhs of having a controller, adapted to determine a position conversion relationship between a position of a body part of a portion of a user and a tracking position of a wearing part of the portion of the user, wherein the position conversion relationship is configured to convert one point in a three-dimensional space to another point in the three-dimensional space, the wearing part is where a wearable device is disposed on the user, and the controller is configured to: obtain a camera data of a previous moment from a camera, wherein the camera data comprises an image of the wearable device and the first position, with the teachings of DE NARDI of having wherein the inertial measurement value comprises changes in six degrees of freedom and the six degrees of freedom comprises three translation values corresponding to three perpendicular axes and three rotation values corresponding to the three perpendicular axes.
Wherein having MOUNIER’s wearable tracking system of wherein the inertial measurement value comprises changes in six degrees of freedom and the six degrees of freedom comprises three translation values corresponding to three perpendicular axes and three rotation values corresponding to the three perpendicular axes.
The motivation behind the modification would have been to obtain a wearable tracking system that provides enhanced tracking, power savings, and privacy functionalities, since both MOUNIER and DE NARDI are systems that relate to extended reality. Wherein MOUNIER’s wearable system helps to increase the tracking fidelity/accuracy of the XR device and can trigger subsequent processing mode adjustments of any of the cameras of the XR device based on a position and/or motion of the wearable accessory, while DE NARDI's system leverages signals collected across multiple devices to provide a more immersive artificial reality system that better responds to a user's interaction with the artificial reality. Please see MOUNIER et al. (US 20230048398 A1), Paragraphs [0043], [0046], and [0101], and DE NARDI et al. (US 20200150753 A1), Paragraph [0015].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Yoo (US Patent No.: US12164710B2) discloses a method of controlling a wearable device worn on a finger of a user, includes: sensing a contact by a second finger of the user via an outer surface electrode located on an outer circumferential surface of the wearable device worn on a first finger of the user; based on the sensing of the contact, measuring an impedance between the outer surface electrode and an inner surface electrode that is in contact with the first finger of the user; identifying a type of the second finger based on the impedance; and controlling an operation of the wearable device based on the type of the second finger.
Beyhs et al (US Patent No.: US11397466B2) discloses contact or movement gestures between a first body part and a second body part can be detected. Sense circuitry can be configured to sense a signal at the sense electrode (e.g., configured to contact the second body part) in response to a drive signal applied to the drive electrode (e.g., configured to contact the first body part). Processing circuitry can be configured to detect contact in accordance with a determination that one or more criteria are met (e.g., an amplitude criterion and a non-distortion criterion). Additionally or alternatively, processing circuitry can be configured to detect a movement gesture in accordance with a determination that one or more criteria are met (e.g., a contact criterion and a movement criterion).
Segil et al (Pub No.: US20250117089A1) discloses a wearable electronic device includes an electrode array for transdermal stimulation of a sensory nerve that itself innervates a body part distal to a worn location of the wearable electronic device. The transdermal stimulation is configured to induce an electrical current or voltage that evokes a sensory impression at an area innervated by the sensory nerve, not at the stimulated sensory nerve itself. In one implementation, the wearable electronic device takes a finger ring form factor worn on a proximal phalanx of an index finger. In this configuration, the wearable electronic device can stimulate a portion of a branch of the median nerve extending through the index finger. Upon stimulation of the median nerve, a user wearing the finger ring may perceive pressure applied to the user's fingertip.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDRAE S ALLISON whose telephone number is (571)270-1052. The examiner can normally be reached Monday-Friday 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANDRAE S ALLISON can be reached on (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or (571) 272-1000.
/ANDRAE S ALLISON/Primary Examiner, Art Unit 2673
July 25, 2026
Prosecution Insights