DETAILED ACTION
Notice of AIA Status
The present application is being examined under the AIA the first inventor to file provisions.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 02/21/2025, 11/12/2025 and 07/10/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 1-3, 5, 8-10, 12, and 15-19, are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Reddy et al. (US 10664993 B1) hereafter referenced as Reddy.
Regarding claim 1, Reddy explicitly teaches a method of operating a wearable system comprising a headset and a controller (Fig. 5, Column 21, Lines [0014-17]- Reddy discloses illustrates the image system 502 utilizing a headset device 506 it should be understood that in some examples the image system 502 may operate as other wearable electronics or as hand held electronic devices.),
the method comprising: causing a set of fiducials of the controller to flash (Fig. 11, Column 34, Lines [0033-35]- Reddy discloses the constellation points 1102-1112 may be static LEDs, color LEDs, flashing LEDs, or other active components.),
the set of fiducials being arranged in a known geometry (Fig. 1, Column 4, Lines [0026-31]- Reddy discloses the object may be marked with a predetermined pattern or constellation. For instance, an object may be equipped with a number of active components, such as light emitting diodes (LEDs), arranged according to the predetermined pattern or constellation);
tracking a pose of the controller by (Fig. 1, Column 9, Lines [0016-20]- Reddy discloses the image system may be in a tracking state or otherwise have a prior or past 6DOF pose of the object or controller that may be used to assist with determining a current 6DOF pose of the object or controller within a newly captured image.):
capturing a set of headset images using a headset camera of the headset (Fig. 36, Column 58, Lines [0024-29]- Reddy discloses an image system may track the 6DOF pose of a controller, a headset device, a user, or a user's body part using cameras positioned both at the controller (e.g., in the hands of the user) and on the headset device to provide for a more robust user experience.);
identifying the set of fiducials in the set of headset images (Fig. 10 Column 31, Lines [0025-36]- Reddy discloses in order to determine the physical pose of the controller 1000, the image system may include an image component to capture images of the controller 1000 may be processed to identify image points corresponding to constellation points 1004 on the exterior of the object 1000. For instance, in the illustrated example, the constellation points 1004 may be formed from LED lights, generally indicated by LED lights 1006-1012. In general, the LEDs 1006-1012 from the constellation on the object and may be detected within a monochrome image captured by the headset device or other image device of the image system.);
and tracking the pose of the controller based on the identified set of fiducials in the set of headset images and based on a pose of the headset (Fig. 32, Column 55, Lines [0005-9]- Reddy discloses thus, with known 6DOF pose of the headset device, 6DOF pose of the controller, and shared physical space via the shared scene the position of the controller may be determined.);
while tracking the pose of the controller, capturing a set of controller images using a controller camera of the controller (Fig. 1, Column 7, Lines [0043-49]- Reddy discloses the controller may also capture images of the physical environment including image data of the headset device or user themselves. Both the headset device and the controller may perform a simultaneous locations and mapping (SLAM) technique to generate a shared map or virtual scene of the physical environment.);
identifying a set of two-dimensional feature points in each controller image of the set of controller images (Fig. 4, Column 20, Lines [0008-15]- Reddy discloses for example, the tags 408-414 may be patterned or otherwise distinguishable by the image system when is captured of the environment 400. Thus, in a manner, similar to detecting an object as discussed herein, the system may detect features of the environment, such as the wall 416, the door 418, or the table 420 using constellation points formed by the tracking enhancement tags or stickers 408-414.);
and determining a set of three-dimensional map points based on the set of two-dimensional feature points and the pose of the controller (Fig. 4, Column 20, Lines [0001-8]- Reddy discloses the headset device 404 and/or the controller 406 may be utilized to generate a shared map of the physical environment 400. In some cases, the 6DOF pose of the headset device 404 and/or the controller 406 may be relative to the shared map. In this example, to assist with generating the shared map and for determining a 6DOF pose of either the headset device 304, the controller 306, or the user 402 (e.g., the user 402 or a body part of the user 402), the physical environment 400 may be equipped with tracking enhancement tags, such as tags 408-414.).
Regarding claim 2, Reddy teaches the method of claim 1, Reddy further teaches wherein the wearable system comprises (Fig. 5, Column 21, Lines [0014-17]- Reddy discloses illustrates the image system 502 utilizing a headset device 506 it should be understood that in some examples the image system 502 may operate as other wearable electronics or as hand held electronic devices.):
the headset comprising: the headset camera (Fig. 36, Column 58, Lines [0024-29]- Reddy discloses an image system may track the 6DOF pose of a controller, a headset device, a user, or a user's body part using cameras positioned both at the controller (e.g., in the hands of the user) and on the headset device to provide for a more robust user experience.);
and a headset inertial measurement unit (Fig. 6, Column 21, Lines [0041-48]- Reddy discloses thus, the headset device 600 may include one or more measurement units 604 to determine the orientation data of the headset device 600 (e.g., acceleration, angular momentum, pitch, roll, yaw, etc. of the headset device 600). The measurement units 604 may include one or more IMUs, one or more accelerometers, one or more gyroscopes, one or more magnetometers, and/or one or more pressure sensors, as well as other sensors.);
and the controller comprising: the set of fiducials arranged in the known geometry (Fig. 1, Column 4, Lines [0026-31]- Reddy discloses the object may be marked with a predetermined pattern or constellation. For instance, an object may be equipped with a number of active components, such as light emitting diodes (LEDs), arranged according to the predetermined pattern or constellation);
the controller camera (Fig. 10, Column 32, Lines [0006-11]- Reddy discloses the controller 1000 may also include one or more image components 1020. The image components 1020 may be of various sizes and quality, for instance, the image components 1020 may include one or more wide screen cameras, 3D cameras, high definition cameras, video cameras, among other types of cameras.);
and a controller inertial measurement unit (Fig. 1, Column 9, Lines [0022-29]- Reddy discloses the controller may be equipped with an IMU, accelerometers, gyroscopes, magnetometers, or a combination thereof. In these instances, the orientation data of the controller may be used to generate one or more forward predictive 6DOF poses of the object or controller (e.g., an extrapolative 6DOF pose of the object that corresponds to the 6DOF pose of the object at the time the newly capture image was taken).);
wherein the wearable system is configured to determine a position or orientation of the headset or the controller based on data (Fig. 1, Column 7 Lines [0049-53]- Reddy discloses the headset device may also utilize the image date to determine and track the 6DOF pose of the controller and the controller may utilize the image data to determine and track the 6DOF pose of the headset device or the user.)
captured by the headset camera (Fig. 3, Column 19 Lines [0014-22]- Reddy discloses the headset device 404 may capture at least one image or frame including data representative of the controller 406. Within the image data, a number of constellation points of the controller 406 may be visible and detected by the image system. For example, the image system may perform operations associated with image point detection and then determine a 6DOF pose of the controller 406.),
the controller camera (Fig. 4, Column 19 Lines [0059-65]- Reddy discloses the controller 406 may capture at least one image or frame including data representative of the headset device 404. Within the image data, a number of constellation points of the headset device 406 may be visible and detected by the image system. For example, the image system may perform operations associated with image point detection and then determine a 6DOF pose of the headset device 404.),
the headset inertial measurement unit (Fig. 2, Column 7 Lines [0034-41]- Reddy discloses the image system and/or the object itself (e.g., the headset 206) may be equipped with a measurement unit (e.g., an IMU, gyroscope, accelerometer, magnetometer, other device capable of capturing changes in orientation, position, or momentum of the object, or a combination thereof) to capture orientation data (e.g., acceleration, angular momentum, rotation data, pitch, roll, yaw, etc.)), or
the controller inertial measurement unit (Fig. 10, Column 31 Lines [0037-50]- Reddy discloses the image system may utilize orientation data 1014 associated with the controller 1000 to determine the 6DOF pose of the controller 1000. In these cases, the controller 1000 may be equipped with one or more measurement units 1016 and one or more communication interfaces 1018. The measurement units 1016 may be configured to collect data (e.g., the orientation data 1014) associated with the movement of the controller 1000, such as acceleration data, angular momentum data, pitch data, roll data, yaw data, etc. In one example, the measurement units 1016 may include one or more IMUs, one or more accelerometers, one or more gyroscopes, one or more magnetometers, and/or one or more pressure sensors, as well as other sensors).
Regarding claim 3, Reddy teaches the method of claim 1, Reddy further teaches further comprising: generating a map using the set of three-dimensional map points (Column 68 Lines [0005-12]- Reddy discloses the tracking enhancement tag can optionally encode a unique identifier that can be computed or read by the headset; encode orientation information so that its orientation can be determined uniquely once detected; have information encoded at both high frequency (e.g. a unique identifier) and low frequency (e.g. orientation); and be configured to provide more standard visual features such as corners for SLAM.),
wherein each three-dimensional map point of the set of three-dimensional map points corresponds to a particular feature in an environment of the wearable system (Fig. 1, Column 4 Lines [0063-64]- Reddy discloses each constellation point and the corresponding model point associated with an object may be assigned a unique identifier or ID which may be used to limit or restrain the number of candidates (e.g., sets of image point to model point relationships that may be used to generate a 6DOF pose) associated with the object based on the image points detected.).
Regarding claim 5, Reddy teaches the method of claim 3, Reddy further teaches wherein the map comprises a list of keyframe poses (Fig. 4, Column 20, Lines [0026-29]- Reddy discloses the headset device 404 and/or the controller 406 may locate the 6DOF pose of each of the tracking enhancement tags 408-414 within the 3D model of the physical environment 400.)
and a corresponding list of the set of two-dimensional feature points for each three-dimensional map point of the set of three-dimensional map points (Fig. 4, Column 20, Lines [0008-15]- Reddy discloses for example, the tags 408-414 may be patterned or otherwise distinguishable by the image system when is captured of the environment 400. Thus, in a manner, similar to detecting an object as discussed herein, the system may detect features of the environment, such as the wall 416, the door 418, or the table 420 using constellation points formed by the tracking enhancement tags or stickers 408-414. Further in Fig. 1, Column 7 Lines [0062-67]- Reddy discloses the object models may be utilized in conjunction with images including the object having the constellations to determine the 6DOF pose and/or identify of the object by utilizing the 3D-2D point correspondence between model points and image points.).
Regarding claim 8, Reddy explicitly teaches a wearable system comprising (Fig. 5, Column 21, Lines [0014-17]- Reddy discloses illustrates the image system 502 utilizing a headset device 506 it should be understood that in some examples the image system 502 may operate as other wearable electronics or as hand held electronic devices.):
a headset (Fig. 5, Column 21, Lines [0014-17]- Reddy discloses illustrates the image system 502 utilizing a headset device 506 it should be understood that in some examples the image system 502 may operate as other wearable electronics or as hand held electronic devices.);
a controller (Fig. 1, Column 11, Lines [0058-60]- Reddy discloses FIG. 1 illustrates an example physical environment 100 including a user 102 of an image system 104 interacting with a first physical object or controller 106.);
and one or more processors configured to perform operations comprising (Fig. 6, Column 22, Lines [0016-22]- Reddy discloses the headset device 600 may also include one or more processors 608, such as at least one or more access components, control logic circuits, central processing units, or processors, as well as one or more computer-readable media 610 to perform the function associated with the virtual environment. Additionally, each of the processors 608 may itself comprise one or more processors or processing cores.):
causing the set of fiducials of the controller to flash (Fig. 11, Column 34, Lines [0033-35]- Reddy discloses the constellation points 1102-1112 may be static LEDs, color LEDs, flashing LEDs, or other active components.),
the set of fiducials being arranged in a known geometry (Fig. 1, Column 4, Lines [0026-31]- Reddy discloses the object may be marked with a predetermined pattern or constellation. For instance, an object may be equipped with a number of active components, such as light emitting diodes (LEDs), arranged according to the predetermined pattern or constellation);
tracking a pose of the controller by (Fig. 1, Column 9, Lines [0016-20]- Reddy discloses the image system may be in a tracking state or otherwise have a prior or past 6DOF pose of the object or controller that may be used to assist with determining a current 6DOF pose of the object or controller within a newly captured image.):
capturing a set of headset images using a headset camera of the headset (Fig. 36, Column 58, Lines [0024-29]- Reddy discloses an image system may track the 6DOF pose of a controller, a headset device, a user, or a user's body part using cameras positioned both at the controller (e.g., in the hands of the user) and on the headset device to provide for a more robust user experience.);
identifying the set of fiducials in the set of headset images (Fig. 10 Column 31, Lines [0025-36]- Reddy discloses in order to determine the physical pose of the controller 1000, the image system may include an image component to capture images of the controller 1000 may be processed to identify image points corresponding to constellation points 1004 on the exterior of the object 1000. For instance, in the illustrated example, the constellation points 1004 may be formed from LED lights, generally indicated by LED lights 1006-1012. In general, the LEDs 1006-1012 from the constellation on the object and may be detected within a monochrome image captured by the headset device or other image device of the image system.);
and tracking the pose of the controller based on the identified set of fiducials in the set of headset images and based on a pose of the headset (Fig. 32, Column 55, Lines [0005-9]- Reddy discloses thus, with known 6DOF pose of the headset device, 6DOF pose of the controller, and shared physical space via the shared scene the position of the controller may be determined.);
while tracking the pose of the controller, capturing a set of controller images using a controller camera of the controller (Fig. 1, Column 7, Lines [0043-49]- Reddy discloses the controller may also capture images of the physical environment including image data of the headset device or user themselves. Both the headset device and the controller may perform a simultaneous locations and mapping (SLAM) technique to generate a shared map or virtual scene of the physical environment.);
identifying a set of two-dimensional feature points in each controller image of the set of controller images (Fig. 4, Column 20, Lines [0008-15]- Reddy discloses for example, the tags 408-414 may be patterned or otherwise distinguishable by the image system when is captured of the environment 400. Thus, in a manner, similar to detecting an object as discussed herein, the system may detect features of the environment, such as the wall 416, the door 418, or the table 420 using constellation points formed by the tracking enhancement tags or stickers 408-414.);
and determining a set of three-dimensional map points based on the set of two-dimensional feature points and the pose of the controller (Fig. 4, Column 20, Lines [0001-8]- Reddy discloses the headset device 404 and/or the controller 406 may be utilized to generate a shared map of the physical environment 400. In some cases, the 6DOF pose of the headset device 404 and/or the controller 406 may be relative to the shared map. In this example, to assist with generating the shared map and for determining a 6DOF pose of either the headset device 304, the controller 306, or the user 402 (e.g., the user 402 or a body part of the user 402), the physical environment 400 may be equipped with tracking enhancement tags, such as tags 408-414.).
Regarding claim 9, Reddy teaches the wearable system of claim 8, Reddy further teaches wherein: the headset comprises: the headset camera (Fig. 36, Column 58, Lines [0024-29]- Reddy discloses an image system may track the 6DOF pose of a controller, a headset device, a user, or a user's body part using cameras positioned both at the controller (e.g., in the hands of the user) and on the headset device to provide for a more robust user experience.);
and a headset inertial measurement unit (Fig. 6, Column 21, Lines [0041-48]- Reddy discloses thus, the headset device 600 may include one or more measurement units 604 to determine the orientation data of the headset device 600 (e.g., acceleration, angular momentum, pitch, roll, yaw, etc. of the headset device 600). The measurement units 604 may include one or more IMUs, one or more accelerometers, one or more gyroscopes, one or more magnetometers, and/or one or more pressure sensors, as well as other sensors.);
and the controller comprises: the set of fiducials arranged in the known geometry (Fig. 1, Column 4, Lines [0026-31]- Reddy discloses the object may be marked with a predetermined pattern or constellation. For instance, an object may be equipped with a number of active components, such as light emitting diodes (LEDs), arranged according to the predetermined pattern or constellation);
the controller camera (Fig. 10, Column 32, Lines [0006-11]- Reddy discloses the controller 1000 may also include one or more image components 1020. The image components 1020 may be of various sizes and quality, for instance, the image components 1020 may include one or more wide screen cameras, 3D cameras, high definition cameras, video cameras, among other types of cameras.);
and a controller inertial measurement unit (Fig. 1, Column 9, Lines [0022-29]- Reddy discloses the controller may be equipped with an IMU, accelerometers, gyroscopes, magnetometers, or a combination thereof. In these instances, the orientation data of the controller may be used to generate one or more forward predictive 6DOF poses of the object or controller (e.g., an extrapolative 6DOF pose of the object that corresponds to the 6DOF pose of the object at the time the newly capture image was taken).);
and the wearable system is configured to determine a position or orientation of the headset or the controller based on data (Fig. 1, Column 7 Lines [0049-53]- Reddy discloses the headset device may also utilize the image date to determine and track the 6DOF pose of the controller and the controller may utilize the image data to determine and track the 6DOF pose of the headset device or the user.)
captured by the headset camera (Fig. 3, Column 19 Lines [0014-22]- Reddy discloses the headset device 404 may capture at least one image or frame including data representative of the controller 406. Within the image data, a number of constellation points of the controller 406 may be visible and detected by the image system. For example, the image system may perform operations associated with image point detection and then determine a 6DOF pose of the controller 406.),
the controller camera (Fig. 4, Column 19 Lines [0059-65]- Reddy discloses the controller 406 may capture at least one image or frame including data representative of the headset device 404. Within the image data, a number of constellation points of the headset device 406 may be visible and detected by the image system. For example, the image system may perform operations associated with image point detection and then determine a 6DOF pose of the headset device 404.),
the headset inertial measurement unit (Fig. 2, Column 7 Lines [0034-41]- Reddy discloses the image system and/or the object itself (e.g., the headset 206) may be equipped with a measurement unit (e.g., an IMU, gyroscope, accelerometer, magnetometer, other device capable of capturing changes in orientation, position, or momentum of the object, or a combination thereof) to capture orientation data (e.g., acceleration, angular momentum, rotation data, pitch, roll, yaw, etc.)), or
the controller inertial measurement unit (Fig. 10, Column 31 Lines [0037-50]- Reddy discloses the image system may utilize orientation data 1014 associated with the controller 1000 to determine the 6DOF pose of the controller 1000. In these cases, the controller 1000 may be equipped with one or more measurement units 1016 and one or more communication interfaces 1018. The measurement units 1016 may be configured to collect data (e.g., the orientation data 1014) associated with the movement of the controller 1000, such as acceleration data, angular momentum data, pitch data, roll data, yaw data, etc. In one example, the measurement units 1016 may include one or more IMUs, one or more accelerometers, one or more gyroscopes, one or more magnetometers, and/or one or more pressure sensors, as well as other sensors).
Regarding claim 10, Reddy teaches the wearable system of claim 8, Reddy further teaches wherein the operations further comprise: generating a map using the set of three-dimensional map points (Column 68 Lines [0005-12]- Reddy discloses the tracking enhancement tag can optionally encode a unique identifier that can be computed or read by the headset; encode orientation information so that its orientation can be determined uniquely once detected; have information encoded at both high frequency (e.g. a unique identifier) and low frequency (e.g. orientation); and be configured to provide more standard visual features such as corners for SLAM.),
wherein each three-dimensional map point of the set of three-dimensional map points corresponds to a particular feature in an environment of the wearable system (Fig. 1, Column 4 Lines [0063-64]- Reddy discloses each constellation point and the corresponding model point associated with an object may be assigned a unique identifier or ID which may be used to limit or restrain the number of candidates (e.g., sets of image point to model point relationships that may be used to generate a 6DOF pose) associated with the object based on the image points detected.).
Regarding claim 12, Reddy teaches the wearable system of claim 10, Reddy further teaches wherein the map comprises a list of keyframe poses (Fig. 4, Column 20, Lines [0026-29]- Reddy discloses the headset device 404 and/or the controller 406 may locate the 6DOF pose of each of the tracking enhancement tags 408-414 within the 3D model of the physical environment 400.)
and a corresponding list of the set of two-dimensional feature points for each three-dimensional map point of the set of three-dimensional map points (Fig. 4, Column 20, Lines [0008-15]- Reddy discloses for example, the tags 408-414 may be patterned or otherwise distinguishable by the image system when is captured of the environment 400. Thus, in a manner, similar to detecting an object as discussed herein, the system may detect features of the environment, such as the wall 416, the door 418, or the table 420 using constellation points formed by the tracking enhancement tags or stickers 408-414. Further in Fig. 1, Column 7 Lines [0062-67]- Reddy discloses the object models may be utilized in conjunction with images including the object having the constellations to determine the 6DOF pose and/or identify of the object by utilizing the 3D-2D point correspondence between model points and image points.).
Regarding claim 15, Reddy explicitly teaches a non-transitory computer-readable medium comprising instructions that (Fig. 7, Column 24, Lines [0040-46]- Reddy discloses depending on the configuration, the computer-readable media 714 may be an example of tangible non-transitory computer storage media and may include volatile and nonvolatile memory and/or removable and non-removable media implemented in any type of technology for storage of information such as computer-readable instructions or modules, data structures, program modules or other data.),
when executed by one or more processors (Fig. 7, Column 24, Lines [0033-38]- Reddy discloses the image system 700 may also include one or more processors 712, such as at least one or more access components, control logic circuits, central processing units, or processors, as well as one or more computer-readable media 714 to perform the function associated with the virtual environment.),
cause the one or more processors to perform operations for operating a wearable system comprising a headset and a controller (Fig. 6, Column 22, Lines [0045-47]- Reddy discloses the pose estimation instructions 612 when executed by the processor 308 may cause the processor 308 to perform operations associated with image point detection.),
the operations comprising: causing a set of fiducials of the controller to flash (Fig. 11, Column 34, Lines [0033-35]- Reddy discloses the constellation points 1102-1112 may be static LEDs, color LEDs, flashing LEDs, or other active components.),
the set of fiducials being arranged in a known geometry (Fig. 1, Column 4, Lines [0026-31]- Reddy discloses the object may be marked with a predetermined pattern or constellation. For instance, an object may be equipped with a number of active components, such as light emitting diodes (LEDs), arranged according to the predetermined pattern or constellation);
tracking a pose of the controller by (Fig. 1, Column 9, Lines [0016-20]- Reddy discloses the image system may be in a tracking state or otherwise have a prior or past 6DOF pose of the object or controller that may be used to assist with determining a current 6DOF pose of the object or controller within a newly captured image.):
capturing a set of headset images using a headset camera of the headset (Fig. 36, Column 58, Lines [0024-29]- Reddy discloses an image system may track the 6DOF pose of a controller, a headset device, a user, or a user's body part using cameras positioned both at the controller (e.g., in the hands of the user) and on the headset device to provide for a more robust user experience.);
identifying the set of fiducials in the set of headset images (Fig. 10 Column 31, Lines [0025-36]- Reddy discloses in order to determine the physical pose of the controller 1000, the image system may include an image component to capture images of the controller 1000 may be processed to identify image points corresponding to constellation points 1004 on the exterior of the object 1000. For instance, in the illustrated example, the constellation points 1004 may be formed from LED lights, generally indicated by LED lights 1006-1012. In general, the LEDs 1006-1012 from the constellation on the object and may be detected within a monochrome image captured by the headset device or other image device of the image system.);
and tracking the pose of the controller based on the identified set of fiducials in the set of headset images and based on a pose of the headset (Fig. 32, Column 55, Lines [0005-9]- Reddy discloses thus, with known 6DOF pose of the headset device, 6DOF pose of the controller, and shared physical space via the shared scene the position of the controller may be determined.);
while tracking the pose of the controller, capturing a set of controller images using a controller camera of the controller (Fig. 1, Column 7, Lines [0043-49]- Reddy discloses the controller may also capture images of the physical environment including image data of the headset device or user themselves. Both the headset device and the controller may perform a simultaneous locations and mapping (SLAM) technique to generate a shared map or virtual scene of the physical environment.);
identifying a set of two-dimensional feature points in each controller image of the set of controller images (Fig. 4, Column 20, Lines [0008-15]- Reddy discloses for example, the tags 408-414 may be patterned or otherwise distinguishable by the image system when is captured of the environment 400. Thus, in a manner, similar to detecting an object as discussed herein, the system may detect features of the environment, such as the wall 416, the door 418, or the table 420 using constellation points formed by the tracking enhancement tags or stickers 408-414.);
and determining a set of three-dimensional map points based on the set of two-dimensional feature points and the pose of the controller (Fig. 4, Column 20, Lines [0001-8]- Reddy discloses the headset device 404 and/or the controller 406 may be utilized to generate a shared map of the physical environment 400. In some cases, the 6DOF pose of the headset device 404 and/or the controller 406 may be relative to the shared map. In this example, to assist with generating the shared map and for determining a 6DOF pose of either the headset device 304, the controller 306, or the user 402 (e.g., the user 402 or a body part of the user 402), the physical environment 400 may be equipped with tracking enhancement tags, such as tags 408-414.).
Regarding claim 16, Reddy teaches the non-transitory computer-readable medium of claim 15, Reddy further teaches wherein the wearable system comprises (Fig. 5, Column 21, Lines [0014-17]- Reddy discloses illustrates the image system 502 utilizing a headset device 506 it should be understood that in some examples the image system 502 may operate as other wearable electronics or as hand held electronic devices.):
the headset comprising: the headset camera (Fig. 36, Column 58, Lines [0024-29]- Reddy discloses an image system may track the 6DOF pose of a controller, a headset device, a user, or a user's body part using cameras positioned both at the controller (e.g., in the hands of the user) and on the headset device to provide for a more robust user experience.);
and a headset inertial measurement unit (Fig. 6, Column 21, Lines [0041-48]- Reddy discloses thus, the headset device 600 may include one or more measurement units 604 to determine the orientation data of the headset device 600 (e.g., acceleration, angular momentum, pitch, roll, yaw, etc. of the headset device 600). The measurement units 604 may include one or more IMUs, one or more accelerometers, one or more gyroscopes, one or more magnetometers, and/or one or more pressure sensors, as well as other sensors.);
and the controller comprising: the set of fiducials arranged in the known geometry (Fig. 1, Column 4, Lines [0026-31]- Reddy discloses the object may be marked with a predetermined pattern or constellation. For instance, an object may be equipped with a number of active components, such as light emitting diodes (LEDs), arranged according to the predetermined pattern or constellation);
the controller camera (Fig. 10, Column 32, Lines [0006-11]- Reddy discloses the controller 1000 may also include one or more image components 1020. The image components 1020 may be of various sizes and quality, for instance, the image components 1020 may include one or more wide screen cameras, 3D cameras, high definition cameras, video cameras, among other types of cameras.);
and a controller inertial measurement unit (Fig. 1, Column 9, Lines [0022-29]- Reddy discloses the controller may be equipped with an IMU, accelerometers, gyroscopes, magnetometers, or a combination thereof. In these instances, the orientation data of the controller may be used to generate one or more forward predictive 6DOF poses of the object or controller (e.g., an extrapolative 6DOF pose of the object that corresponds to the 6DOF pose of the object at the time the newly capture image was taken).);
wherein the wearable system is configured to determine a position or orientation of the headset or the controller based on data (Fig. 1, Column 7 Lines [0049-53]- Reddy discloses the headset device may also utilize the image date to determine and track the 6DOF pose of the controller and the controller may utilize the image data to determine and track the 6DOF pose of the headset device or the user.)
captured by the headset camera (Fig. 3, Column 19 Lines [0014-22]- Reddy discloses the headset device 404 may capture at least one image or frame including data representative of the controller 406. Within the image data, a number of constellation points of the controller 406 may be visible and detected by the image system. For example, the image system may perform operations associated with image point detection and then determine a 6DOF pose of the controller 406.),
the controller camera (Fig. 4, Column 19 Lines [0059-65]- Reddy discloses the controller 406 may capture at least one image or frame including data representative of the headset device 404. Within the image data, a number of constellation points of the headset device 406 may be visible and detected by the image system. For example, the image system may perform operations associated with image point detection and then determine a 6DOF pose of the headset device 404.),
the headset inertial measurement unit (Fig. 2, Column 7 Lines [0034-41]- Reddy discloses the image system and/or the object itself (e.g., the headset 206) may be equipped with a measurement unit (e.g., an IMU, gyroscope, accelerometer, magnetometer, other device capable of capturing changes in orientation, position, or momentum of the object, or a combination thereof) to capture orientation data (e.g., acceleration, angular momentum, rotation data, pitch, roll, yaw, etc.)), or
the controller inertial measurement unit (Fig. 10, Column 31 Lines [0037-50]- Reddy discloses the image system may utilize orientation data 1014 associated with the controller 1000 to determine the 6DOF pose of the controller 1000. In these cases, the controller 1000 may be equipped with one or more measurement units 1016 and one or more communication interfaces 1018. The measurement units 1016 may be configured to collect data (e.g., the orientation data 1014) associated with the movement of the controller 1000, such as acceleration data, angular momentum data, pitch data, roll data, yaw data, etc. In one example, the measurement units 1016 may include one or more IMUs, one or more accelerometers, one or more gyroscopes, one or more magnetometers, and/or one or more pressure sensors, as well as other sensors).
Regarding claim 17, Reddy teaches the non-transitory computer-readable medium of claim 15, Reddy further teaches wherein the operations further comprise: generating a map using the set of three-dimensional map points (Column 68 Lines [0005-12]- Reddy discloses the tracking enhancement tag can optionally encode a unique identifier that can be computed or read by the headset; encode orientation information so that its orientation can be determined uniquely once detected; have information encoded at both high frequency (e.g. a unique identifier) and low frequency (e.g. orientation); and be configured to provide more standard visual features such as corners for SLAM.),
wherein each three-dimensional map point of the set of three-dimensional map points corresponds to a particular feature in an environment of the wearable system (Fig. 1, Column 4 Lines [0063-64]- Reddy discloses each constellation point and the corresponding model point associated with an object may be assigned a unique identifier or ID which may be used to limit or restrain the number of candidates (e.g., sets of image point to model point relationships that may be used to generate a 6DOF pose) associated with the object based on the image points detected.).
Regarding claim 19, Reddy teaches the non-transitory computer-readable medium of claim 17, Reddy further teaches wherein the map comprises a list of keyframe poses (Fig. 4, Column 20, Lines [0026-29]- Reddy discloses the headset device 404 and/or the controller 406 may locate the 6DOF pose of each of the tracking enhancement tags 408-414 within the 3D model of the physical environment 400.)
and a corresponding list of the set of two-dimensional feature points for each three-dimensional map point of the set of three-dimensional map points (Fig. 4, Column 20, Lines [0008-15]- Reddy discloses for example, the tags 408-414 may be patterned or otherwise distinguishable by the image system when is captured of the environment 400. Thus, in a manner, similar to detecting an object as discussed herein, the system may detect features of the environment, such as the wall 416, the door 418, or the table 420 using constellation points formed by the tracking enhancement tags or stickers 408-414. Further in Fig. 1, Column 7 Lines [0062-67]- Reddy discloses the object models may be utilized in conjunction with images including the object having the constellations to determine the 6DOF pose and/or identify of the object by utilizing the 3D-2D point correspondence between model points and image points.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4, 11, and 18 are rejected under 35 U.S.C 103 as being unpatentable over Reddy et al. (US 10664993 B1) hereafter referenced as Reddy in view of Dalal et al. (US 20210244254 A1) hereafter referenced as Dalal.
Regarding claim 4, Reddy teaches the method of claim 3, Reddy further teaches wherein the map comprises a map point identifier (Fig. 1, Column 4 Lines [0063-64]- Reddy discloses each constellation point and the corresponding model point associated with an object may be assigned a unique identifier or ID which may be used to limit or restrain the number of candidates (e.g., sets of image point to model point relationships that may be used to generate a 6DOF pose) associated with the object based on the image points detected.)
Reddy fails to explicitly teach and a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points.
However, Dalal explicitly teaches and a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points (Fig. 5, Paragraph [0105]- Dalal discloses the mapping module 500 may tag each of 4 chairs around a table as “chair” and the table as “table” and may include unique identifiers for each object (i.e., “chair A” and “chair B”).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Reddy of having a method of operating a wearable system comprising a headset and a controller, the method comprising: causing a set of fiducials of the controller to flash, the set of fiducials being arranged in a known geometry; tracking a pose of the controller by: capturing a set of headset images using a headset camera of the headset with the teachings of Dalal a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points.
Wherein having Reddy’s system for tracking of a pose and position of a headset and controller wherein a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points.
The motivation behind the modification would have been to allow for a more accurate rendering of the environment, since both Reddy and Dalal are both systems that perform mapping of a 3d environment from images. Wherein Dalal’s system wherein improved the accuracy of localization of the user, while Dalal’s system improved accuracy of rendering of the environment. Please see Reddy et al. (US 10664993 B1), Column 59 Lines [0020-31] and Dalal et al. (US 20210244254 A1) Paragraph [0011].
Regarding claim 11, Reddy teaches the wearable system of claim 10, Reddy further teaches wherein the map comprises a map point identifier (Fig. 1, Column 4 Lines [0063-64]- Reddy discloses each constellation point and the corresponding model point associated with an object may be assigned a unique identifier or ID which may be used to limit or restrain the number of candidates (e.g., sets of image point to model point relationships that may be used to generate a 6DOF pose) associated with the object based on the image points detected.)
Reddy fails to explicitly teach and a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points.
However, Dalal explicitly teaches and a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points (Fig. 5, Paragraph [0105]- Dalal discloses the mapping module 500 may tag each of 4 chairs around a table as “chair” and the table as “table” and may include unique identifiers for each object (i.e., “chair A” and “chair B”).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Reddy of having a wearable system comprising: a headset; a controller; and one or more processors configured to perform operations comprising: causing the set of fiducials of the controller to flash, the set of fiducials being arranged in a known geometry; tracking a pose of the controller by: capturing a set of headset images using a headset camera of the headset with the teachings of Dalal a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points.
Wherein having Reddy’s system for tracking of a pose and position of a headset and controller wherein a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points.
The motivation behind the modification would have been to allow for a more accurate rendering of the environment, since both Reddy and Dalal are both systems that perform mapping of a 3d environment from images. Wherein Dalal’s system wherein improved the accuracy of localization of the user, while Dalal’s system improved accuracy of rendering of the environment. Please see Reddy et al. (US 10664993 B1), Column 59 Lines [0020-31] and Dalal et al. (US 20210244254 A1) Paragraph [0011].
Regarding claim 18, Reddy teaches the non-transitory computer-readable medium of claim 17, Reddy further teaches wherein the map comprises a map point identifier (Fig. 1, Column 4 Lines [0063-64]- Reddy discloses each constellation point and the corresponding model point associated with an object may be assigned a unique identifier or ID which may be used to limit or restrain the number of candidates (e.g., sets of image point to model point relationships that may be used to generate a 6DOF pose) associated with the object based on the image points detected.)
Reddy fails to explicitly teach and a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points.
However, Dalal explicitly teaches and a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points (Fig. 5, Paragraph [0105]- Dalal discloses the mapping module 500 may tag each of 4 chairs around a table as “chair” and the table as “table” and may include unique identifiers for each object (i.e., “chair A” and “chair B”).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Reddy of having a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations for operating a wearable system comprising a headset and a controller, the operations comprising: causing a set of fiducials of the controller to flash, the set of fiducials being arranged in a known geometry; tracking a pose of the controller by: capturing a set of headset images using a headset camera of the headset with the teachings of Dalal a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points.
Wherein having Reddy’s system for tracking of a pose and position of a headset and controller wherein a feature identifier associated with each three-dimensional map point of the set of three-dimensional map points.
The motivation behind the modification would have been to allow for a more accurate rendering of the environment, since both Reddy and Dalal are both systems that perform mapping of a 3d environment from images. Wherein Dalal’s system wherein improved the accuracy of localization of the user, while Dalal’s system improved accuracy of rendering of the environment. Please see Reddy et al. (US 10664993 B1), Column 59 Lines [0020-31] and Dalal et al. (US 20210244254 A1) Paragraph [0011].
Claims 6-7, 13-14, and 20 are rejected under 35 U.S.C 103 as being unpatentable over Reddy et al. (US 10664993 B1) hereafter referenced as Reddy in view of Chen et al. (US 20170011553 A1) hereafter referenced as Chen.
Regarding claim 6, Reddy teaches the method of claim 1, Reddy fails to explicitly teach further comprising: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured; and associating the set of transforms with the set of controller images.
However, Chen explicitly teaches further comprising: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured (Fig. 1, Paragraph [0041]- Chen discloses once the location of the HMD 100 relative to the handheld electronic device 102 is determined, at block 540, the inverse transform may be calculated, at block 550, to determine the location of the handheld electronic device 102 relative to the HMD 100. In some implementations, the handheld electronic device 102 may collect and/or process this image data substantially continuously, or at a rate at which the markers 115 may be detected and movement of the handheld electronic device 102 may be tracked without any noticeable interruption.);
and associating the set of transforms with the set of controller images (Fig. 1, Paragraph [0033]- Chen discloses the handheld electronic device 102 may transmit the inverted matrix to the HMD 100 for each frame captured.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Reddy of having a method of operating a wearable system comprising a headset and a controller, the method comprising: causing a set of fiducials of the controller to flash, the set of fiducials being arranged in a known geometry; tracking a pose of the controller by: capturing a set of headset images using a headset camera of the headset with the teachings of Chen further comprising: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured; and associating the set of transforms with the set of controller images.
Wherein having Reddy’s system for tracking of a pose and position of a headset and controller wherein further comprising: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured; and associating the set of transforms with the set of controller images.
The motivation behind the modification would have been to allow for improved interaction with the virtual environment, since both Reddy and Chen are both systems for tracking of a headset and handheld device. Wherein Reddy’s system wherein improved the accuracy of localization of the user, while Chen’s system improved interaction with the virtual environment. Please see Reddy et al. (US 10664993 B1), Column 59 Lines [0020-31] and Chen et al. (US 20170011553 A1) Paragraph [0016].
Regarding claim 7, Reddy in view of Chen teaches the method of claim 6, Reddy fails to explicitly teach further comprising: determining a location of a three-dimensional map point of the set of three-dimensional map points by triangulating a two-dimensional feature point of the set of two-dimensional feature points in each of the set of controller images using the set of transforms.
However, Chen explicitly teaches further comprising: determining a location of a three-dimensional map point of the set of three-dimensional map points by triangulating a two-dimensional feature point of the set of two-dimensional feature points in each of the set of controller images using the set of transforms (Fig. 1, Paragraph [0033]- Chen discloses the handheld electronic device 102 may transmit the inverted matrix to the HMD 100 for each frame captured. The HMD 100 may then use this matrix of numbers, for example, 16 numbers, to render the handheld electronic device 102 so that it appears to be at the same location as in the real world, and render movement of the handheld electronic device based on actual movement.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Reddy of having a method of operating a wearable system comprising a headset and a controller, the method comprising: causing a set of fiducials of the controller to flash, the set of fiducials being arranged in a known geometry; tracking a pose of the controller by: capturing a set of headset images using a headset camera of the headset with the teachings of Chen further comprising: determining a location of a three-dimensional map point of the set of three-dimensional map points by triangulating a two-dimensional feature point of the set of two-dimensional feature points in each of the set of controller images using the set of transforms.
Wherein having Reddy’s system for tracking of a pose and position of a headset and controller wherein further comprising: determining a location of a three-dimensional map point of the set of three-dimensional map points by triangulating a two-dimensional feature point of the set of two-dimensional feature points in each of the set of controller images using the set of transforms.
The motivation behind the modification would have been to allow for improved interaction with the virtual environment, since both Reddy and Chen are both systems for tracking of a headset and handheld device. Wherein Reddy’s system wherein improved the accuracy of localization of the user, while Chen’s system improved interaction with the virtual environment. Please see Reddy et al. (US 10664993 B1), Column 59 Lines [0020-31] and Chen et al. (US 20170011553 A1) Paragraph [0016].
Regarding claim 13, Reddy teaches the wearable system of claim 8, Reddy fails to explicitly teach wherein the operations further comprise: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured; and associating the set of transforms with the set of controller images.
However, Chen explicitly teaches wherein the operations further comprise: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured (Fig. 1, Paragraph [0041]- Chen discloses once the location of the HMD 100 relative to the handheld electronic device 102 is determined, at block 540, the inverse transform may be calculated, at block 550, to determine the location of the handheld electronic device 102 relative to the HMD 100. In some implementations, the handheld electronic device 102 may collect and/or process this image data substantially continuously, or at a rate at which the markers 115 may be detected and movement of the handheld electronic device 102 may be tracked without any noticeable interruption.);
and associating the set of transforms with the set of controller images (Fig. 1, Paragraph [0033]- Chen discloses the handheld electronic device 102 may transmit the inverted matrix to the HMD 100 for each frame captured.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Reddy of having a wearable system comprising: a headset; a controller; and one or more processors configured to perform operations comprising: causing the set of fiducials of the controller to flash, the set of fiducials being arranged in a known geometry; tracking a pose of the controller by: capturing a set of headset images using a headset camera of the headset with the teachings of Chen wherein the operations further comprise: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured; and associating the set of transforms with the set of controller images.
Wherein having Reddy’s system for tracking of a pose and position of a headset and controller wherein the operations further comprise: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured; and associating the set of transforms with the set of controller images.
The motivation behind the modification would have been to allow for improved interaction with the virtual environment, since both Reddy and Chen are both systems for tracking of a headset and handheld device. Wherein Reddy’s system wherein improved the accuracy of localization of the user, while Chen’s system improved interaction with the virtual environment. Please see Reddy et al. (US 10664993 B1), Column 59 Lines [0020-31] and Chen et al. (US 20170011553 A1) Paragraph [0016].
Regarding claim 14, Reddy in view of Chen teaches the wearable system of claim 13, Reddy fails to explicitly teach wherein the operations further comprise: determining a location of a three-dimensional map point of the set of three-dimensional map points by triangulating a two-dimensional feature point of the set of two-dimensional feature points in each of the set of controller images using the set of transforms.
However, Chen explicitly teaches wherein the operations further comprise: determining a location of a three-dimensional map point of the set of three-dimensional map points by triangulating a two-dimensional feature point of the set of two-dimensional feature points in each of the set of controller images using the set of transforms (Fig. 1, Paragraph [0033]- Chen discloses the handheld electronic device 102 may transmit the inverted matrix to the HMD 100 for each frame captured. The HMD 100 may then use this matrix of numbers, for example, 16 numbers, to render the handheld electronic device 102 so that it appears to be at the same location as in the real world, and render movement of the handheld electronic device based on actual movement.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Reddy of having a wearable system comprising: a headset; a controller; and one or more processors configured to perform operations comprising: causing the set of fiducials of the controller to flash, the set of fiducials being arranged in a known geometry; tracking a pose of the controller by: capturing a set of headset images using a headset camera of the headset with the teachings of Chen wherein the operations further comprise: determining a location of a three-dimensional map point of the set of three-dimensional map points by triangulating a two-dimensional feature point of the set of two-dimensional feature points in each of the set of controller images using the set of transforms.
Wherein having Reddy’s system for tracking of a pose and position of a headset and controller wherein the operations further comprise: determining a location of a three-dimensional map point of the set of three-dimensional map points by triangulating a two-dimensional feature point of the set of two-dimensional feature points in each of the set of controller images using the set of transforms.
The motivation behind the modification would have been to allow for improved interaction with the virtual environment, since both Reddy and Chen are both systems for tracking of a headset and handheld device. Wherein Reddy’s system wherein improved the accuracy of localization of the user, while Chen’s system improved interaction with the virtual environment. Please see Reddy et al. (US 10664993 B1), Column 59 Lines [0020-31] and Chen et al. (US 20170011553 A1) Paragraph [0016].
Regarding claim 20, Reddy teaches the non-transitory computer-readable medium of claim 15, Reddy fails to explicitly teach wherein the operations further comprise: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured; and associating the set of transforms with the set of controller images.
However, Chen explicitly teaches wherein the operations further comprise: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured (Fig. 1, Paragraph [0041]- Chen discloses once the location of the HMD 100 relative to the handheld electronic device 102 is determined, at block 540, the inverse transform may be calculated, at block 550, to determine the location of the handheld electronic device 102 relative to the HMD 100. In some implementations, the handheld electronic device 102 may collect and/or process this image data substantially continuously, or at a rate at which the markers 115 may be detected and movement of the handheld electronic device 102 may be tracked without any noticeable interruption.);
and associating the set of transforms with the set of controller images (Fig. 1, Paragraph [0033]- Chen discloses the handheld electronic device 102 may transmit the inverted matrix to the HMD 100 for each frame captured.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Reddy of having a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations for operating a wearable system comprising a headset and a controller, the operations comprising: causing a set of fiducials of the controller to flash, the set of fiducials being arranged in a known geometry; tracking a pose of the controller by: capturing a set of headset images using a headset camera of the headset with the teachings of Chen wherein the operations further comprise: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured; and associating the set of transforms with the set of controller images.
Wherein having Reddy’s system for tracking of a pose and position of a headset and controller wherein the operations further comprise: generating a set of transforms representing a position and an orientation of the controller relative to the headset at a set of time instances corresponding to when each of the set of controller images were captured; and associating the set of transforms with the set of controller images.
The motivation behind the modification would have been to allow for improved interaction with the virtual environment, since both Reddy and Chen are both systems for tracking of a headset and handheld device. Wherein Reddy’s system wherein improved the accuracy of localization of the user, while Chen’s system improved interaction with the virtual environment. Please see Reddy et al. (US 10664993 B1), Column 59 Lines [0020-31] and Chen et al. (US 20170011553 A1) Paragraph [0016].
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered
pertinent to applicant`s disclosure.
Yitzhak et al. (US 10838515 B1)- A virtual reality (VR) system tracks the location and position of a controller using image sensors on a headset and a controller. The headset and controller provide a first and second view of a user's environment. Using its camera, the headset generates a map of the environment and identifies its location within it. Based headset's location, the VR system generates a simulated world of the environment and displays it to the user. The headset also estimates the location of the controller. Based on the estimated location, the headset sends a portion of the map to the controller. The controller determines its pose using the portion of the map, the image sensors, and additional sensors. The controller sends its pose and an updated portion of the map to the headset. Based on the controller's pose and updated portion of the map, the VR system modifies the content displayed to the user....................Please see Fig. 1. Abstract.
Bosworth et al. (US 11507203 B1)- In one embodiment, a computing system may determine a pose of a controller held by a user based on sensor data captured by the controller. The system may determine positions of a first set of keypoints associated with a first portion of a body of the user based on images captured by the controller cameras and (2) the controller pose. The system may determine a pose of a headset worn by the user based on sensor data captured by the headset. The system may determine positions of a second set of keypoints associated with a second portion of the body of the user based on images captured by the headset cameras and (2) the pose of the headset. The system may determine a full body pose of the user based at least on the positions of the first set and the second set of keypoints.....................Please see Fig. 1. Abstract.
Lee et al. (US 20160232715 A1)- Systems and methods for generating an action in a virtual reality or augmented reality environment based on position or movement of a mobile device in the real world are disclosed. A particular embodiment includes: displaying an optical marker on a display device of a motion-tracking controller; receiving a set of reference data from the motion-tracking controller; receiving captured marker image data from an image capturing subsystem of an eyewear system; comparing reference marker image data with the captured marker image data, the reference marker image data corresponding to the optical marker; generating a transformation matrix using the reference marker image data and the captured marker image data, the transformation matrix corresponding to a position and orientation of the motion-tracking controller relative to the eyewear system; and generating an action in a virtual world, the action corresponding to the transformation matrix......................Please see Fig. 1. Abstract.
Linde et al. (US 20190110039 A1)- A head-mounted display (HMD) is configured to capture images and/or video of a local area. The HMD includes an imaging assembly and a controller. The imaging assembly includes a plurality of cameras positioned at different locations on the HMD and oriented to capture images of different portions of a local area surrounding the HMD. The controller generates imaging instructions for each camera using image information. The imaging instructions cause respective midpoints of exposure times for each camera to occur at a same time value for each of the captured images. The cameras capture images of the local area in accordance with the imaging instructions. The controller determines a location of the HMD in the local area using the captured images and updates a model that represents a mapping function of the depth and exposure settings of the local area........................Please see Fig. 1. Abstract.
Xu et al. (US 20190206116 A1)- A method for simultaneous localization and mapping. The method includes the step of detecting two-dimensional (2D) feature points from a current frame captured by a camera; matching the 2D feature points from the current frame directly to three-dimensional (3D) map points in a 3D map, so as to obtain correspondence between the 2D feature points and the 3D map points; and computing a current pose of the camera based on the obtained correspondence. Each of the 2D feature points and the 3D map points has a feature descriptor. The step of matching is performed by comparing the feature descriptors of the 2D feature points and the feature descriptors of the 3D map points........................Please see Fig. 1. Abstract.
Steedly et al. (US 20200333878 A1)- A head-mounted display (HMD) is configured to capture images and/or video of a local area. The HMD includes an imaging assembly and a controller. The imaging assembly includes a plurality of cameras positioned at different locations on the HMD and oriented to capture images of different portions of a local area surrounding the HMD. The controller generates imaging instructions for each camera using image information. The imaging instructions cause respective midpoints of exposure times for each camera to occur at a same time value for each of the captured images. The cameras capture images of the local area in accordance with the imaging instructions. The controller determines a location of the HMD in the local area using the captured images and updates a model that represents a mapping function of the depth and exposure settings of the local area........................Please see Fig. 1. Abstract.
Wu et al. (US 20220374072 A1)- A head-mounted display system and a 6-degree-of-freedom tracking method and apparatus thereof are disclosed. The method includes: controlling two channels of tracking cameras so that central moments of exposure durations of each frame of the two channels of tracking cameras are same, and controlling so that a lightening moment of the LED lights on the handle controller is synchronized with the middle moment of the exposure duration of an even-number frame of the two channels of tracking cameras; calculating 6-degree-of-freedom tracking data of the head-mounted display device in a three-dimensional space in real time according to acquired image data of an odd-number frame of the two channels of tracking cameras; and calculating 6-degree-of-freedom tracking data of the handle controller in the three-dimensional space in real time according to acquired image data of an even-number frame of the two channels of tracking cameras........................Please see Fig. 1. Abstract.
Liu et al. (US 20210365064 A1)- A head-mounted display (HMD) is configured to capture images and/or video of a local area. The HMD includes an imaging assembly and a controller. The imaging assembly includes a plurality of cameras positioned at different locations on the HMD and oriented to capture images of different portions of a local area surrounding the HMD. The controller generates imaging instructions for each camera using image information. The imaging instructions cause respective midpoints of exposure times for each camera to occur at a same time value for each of the captured images. The cameras capture images of the local area in accordance with the imaging instructions. The controller determines a location of the HMD in the local area using the captured images and updates a model that represents a mapping function of the depth and exposure settings of the local area........................Please see Fig. 1. Abstract.
Miller et al. (US 9766703 B2)- An augmented reality display system comprises a passable world model data comprises a set of map points corresponding to one or more objects of the real world. The augmented reality system also comprises a processor to communicate with one or more individual augmented reality display systems to pass a portion of the passable world model data to the one or more individual augmented reality display systems, wherein the piece of the passable world model data is passed based at least in part on respective locations corresponding to the one or more individual augmented reality display systems........................Please see Fig. 1. Abstract.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LUCIUS C.G. ALLEN whose telephone number is (703)756-5987. The examiner can normally be reached Mon - Fri 8-5pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571)272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LUCIUS CAMERON GREEN ALLEN/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights