DETAILED ACTION
Claims 1-10, 16-25 are currently pending and have been examined in this application. Claims 11-15 have been canceled. This is the first action on the merits.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is in response to the Response to Election/Restriction Requirement filed 09/08/2026.
Election/Restrictions
Applicant’s election without traverse of Species A in the reply filed on 09/08/2026 is acknowledged.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3, 8-10, 16-18, 21-22, 25 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhang (US20220381898).
Claim 1:
Zhang explicitly teaches:
An apparatus for rendering an extended reality (XR) environment, the apparatus comprising:
at least one memory; and at least one processor coupled to the at least one memory and configured to:
(Zhang) – “Disclosed are systems and techniques for extended reality optimizations using radio frequency (RF) sensing.” (Abstract)
“According to at least one example, an apparatus is provided for controlling device and/or processing settings based on RF sensing. The apparatus can include memory and one or more processors coupled to the memory, the one or more processors being configured to obtain radio frequency (RF) sensing data; determine, based on the RF sensing data, one or more reflected paths of one or more reflected RF signals, wherein each reflected RF signal comprises a reflection of a transmitted RF signal from one or more objects in a physical space; compare the one or more reflected paths to a field-of-view (FOV) of an image capturing device associated with the apparatus; and based on the comparison, trigger an action by at least one of the image capturing device and the apparatus.” (Para 0007)
generate, based on a received radio frequency (RF) signal, a local map representing an area in which a first device is located;
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“RF sensing can use RF data to generate a map of a space, such as a space in a scene or an indoor space. In some cases, using a monostatic configuration, a wireless device (e.g., XR device, smartphone, etc.) can obtain RF sensing data using wireless interfaces that are capable of performing transmit and receive functions. In some examples, the wireless device can implement a Wi-Fi radar to obtain the RF sensing data. The Wi-Fi radar can implement RF interfaces capable of performing transmit and receive functions. The Wi-Fi radar (and/or the wireless device implementing the Wi-Fi radar) can use the signal strength of radio transmissions to determine a location/position of one or more objects. For example, a wireless device may utilize a wireless interface (e.g., of a Wi-Fi radar) to transmit an RF signal and capture any signals that reflect from objects in the surrounding environment. The wireless interface can also receive leakage signals that are coupled directly from the transmitter's antenna to the receiver's antenna without reflecting from any objects. In some examples, the wireless device may gather RF sensing data in the form of channel state information (CSI) data relating to the direct paths (leakage signals) of the transmitted signal and data relating to the reflected paths of the signals received that correspond to the transmitted signal. In some cases, a bistatic configuration can be used where the transmit and receive functions are performed by different devices. For instance, a first device can transmit wireless signals that reflect from one or more objects in the scene, and a wireless interface of a second device can receive the reflected signals and/or signals directly from the first device. In some cases, the signals can be omnidirectional signals transmitted using an omnidirectional antenna, and the signals can be transmitted in a 360-degree radiation pattern.” (Para 0057)
align features of the local map with features of a reference map associated with a predetermined location;
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
Examiner Note: Any map or coordinate system may correspond with reference map.
determine, based on the aligned features, the first device is located at the predetermined location; and
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
render a virtual representation of the aligned local map and the aligned reference map at the predetermined location.
(Zhang) – “The mobile device 704 can receive the DL data 710 and send DL data 712 to the XR device 706, which the XR device 706 can use for an XR presentation/experience. In some examples, the DL data 712 can include rendered virtual content. For example, the mobile device 704 can render virtual content from the destination device 702 and send the rendered virtual content to the XR device 706. In some cases, the DL data 712 can include XR-related data such as, for example, a depth map, eye buffer (e.g., render texture, eye buffer resolution, etc.), tracking information, and/or any other XR data.” (Para 0151)
“The XR device 706 can obtain sensor data, such as inertial measurements, image data, etc., and provide such data to the mobile device 704. For example, the XR device 706 can send to the mobile device 704 UL data 722 including data from one or more inertial measurement units (IMUs) and a tracking frame from an image capturing device at the XR device 706. In some cases, the mobile device 704 can use the UL data 722 to render virtual content according to a position of the XR device 706 and/or one or more objects in a scene.” (Para 0152)
“In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
Examiner Note: Per BRI, any presentation in any way related to the local map and the reference map may correspond with the claimed rendering.
Claim 2:
Zhang teaches the respective limitations of Claim 1. Zhang further teaches:
wherein the local map is generated based on channel state information (CSI) associated with propagation characteristics of the received RF signal.
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“RF sensing can use RF data to generate a map of a space, such as a space in a scene or an indoor space. In some cases, using a monostatic configuration, a wireless device (e.g., XR device, smartphone, etc.) can obtain RF sensing data using wireless interfaces that are capable of performing transmit and receive functions. In some examples, the wireless device can implement a Wi-Fi radar to obtain the RF sensing data. The Wi-Fi radar can implement RF interfaces capable of performing transmit and receive functions. The Wi-Fi radar (and/or the wireless device implementing the Wi-Fi radar) can use the signal strength of radio transmissions to determine a location/position of one or more objects. For example, a wireless device may utilize a wireless interface (e.g., of a Wi-Fi radar) to transmit an RF signal and capture any signals that reflect from objects in the surrounding environment. The wireless interface can also receive leakage signals that are coupled directly from the transmitter's antenna to the receiver's antenna without reflecting from any objects. In some examples, the wireless device may gather RF sensing data in the form of channel state information (CSI) data relating to the direct paths (leakage signals) of the transmitted signal and data relating to the reflected paths of the signals received that correspond to the transmitted signal. In some cases, a bistatic configuration can be used where the transmit and receive functions are performed by different devices. For instance, a first device can transmit wireless signals that reflect from one or more objects in the scene, and a wireless interface of a second device can receive the reflected signals and/or signals directly from the first device. In some cases, the signals can be omnidirectional signals transmitted using an omnidirectional antenna, and the signals can be transmitted in a 360-degree radiation pattern.” (Para 0057)
Claim 3:
Zhang teaches the respective limitations of Claim 1. Zhang further teaches:
wherein the received RF signal is a reflection of an RF signal transmitted by the first device.
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“RF sensing can use RF data to generate a map of a space, such as a space in a scene or an indoor space. In some cases, using a monostatic configuration, a wireless device (e.g., XR device, smartphone, etc.) can obtain RF sensing data using wireless interfaces that are capable of performing transmit and receive functions. In some examples, the wireless device can implement a Wi-Fi radar to obtain the RF sensing data. The Wi-Fi radar can implement RF interfaces capable of performing transmit and receive functions. The Wi-Fi radar (and/or the wireless device implementing the Wi-Fi radar) can use the signal strength of radio transmissions to determine a location/position of one or more objects. For example, a wireless device may utilize a wireless interface (e.g., of a Wi-Fi radar) to transmit an RF signal and capture any signals that reflect from objects in the surrounding environment. The wireless interface can also receive leakage signals that are coupled directly from the transmitter's antenna to the receiver's antenna without reflecting from any objects. In some examples, the wireless device may gather RF sensing data in the form of channel state information (CSI) data relating to the direct paths (leakage signals) of the transmitted signal and data relating to the reflected paths of the signals received that correspond to the transmitted signal. In some cases, a bistatic configuration can be used where the transmit and receive functions are performed by different devices. For instance, a first device can transmit wireless signals that reflect from one or more objects in the scene, and a wireless interface of a second device can receive the reflected signals and/or signals directly from the first device. In some cases, the signals can be omnidirectional signals transmitted using an omnidirectional antenna, and the signals can be transmitted in a 360-degree radiation pattern.” (Para 0057)
Claim 8:
Zhang teaches the respective limitations of Claim 1. Zhang further teaches:
wherein the generation of the local map is based on a reflection of the received RF signal and images generated using one or more cameras of the first device.
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“In some examples, cameras can be implemented in electronic devices for extended reality (XR). XR technologies can include augmented reality (AR), virtual reality (VR), mixed reality (MR), and the like. XR technologies can combine real environments from the physical world (e.g., the real world) and virtual environments or content to provide users with XR experiences.” (Para 0046)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
Claim 9:
Zhang teaches the respective limitations of Claim 1. Zhang further teaches:
wherein the first device is an extended reality (XR) head mounted device (HMD), and wherein the received RF signal is provided by a Wi-Fi access point.
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“RF sensing can use RF data to generate a map of a space, such as a space in a scene or an indoor space. In some cases, using a monostatic configuration, a wireless device (e.g., XR device, smartphone, etc.) can obtain RF sensing data using wireless interfaces that are capable of performing transmit and receive functions. In some examples, the wireless device can implement a Wi-Fi radar to obtain the RF sensing data. The Wi-Fi radar can implement RF interfaces capable of performing transmit and receive functions. The Wi-Fi radar (and/or the wireless device implementing the Wi-Fi radar) can use the signal strength of radio transmissions to determine a location/position of one or more objects. For example, a wireless device may utilize a wireless interface (e.g., of a Wi-Fi radar) to transmit an RF signal and capture any signals that reflect from objects in the surrounding environment. The wireless interface can also receive leakage signals that are coupled directly from the transmitter's antenna to the receiver's antenna without reflecting from any objects. In some examples, the wireless device may gather RF sensing data in the form of channel state information (CSI) data relating to the direct paths (leakage signals) of the transmitted signal and data relating to the reflected paths of the signals received that correspond to the transmitted signal. In some cases, a bistatic configuration can be used where the transmit and receive functions are performed by different devices. For instance, a first device can transmit wireless signals that reflect from one or more objects in the scene, and a wireless interface of a second device can receive the reflected signals and/or signals directly from the first device. In some cases, the signals can be omnidirectional signals transmitted using an omnidirectional antenna, and the signals can be transmitted in a 360-degree radiation pattern.” (Para 0057)
“In some aspects, the apparatus is or is part of a mobile device (e.g., a mobile telephone or “smart phone” or other mobile device), a wearable device (e.g., a head mounted display), an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device), a tablet, a personal computer, a laptop computer, a server computer, a wireless access point, or other any other device having an RF interface. In some aspects, the apparatuses described above can include a camera or multiple cameras for capturing one or more images. In some aspects, the apparatus further includes a display for displaying one or more images, notifications, and/or other displayable data. In some aspects, the apparatuses described above can include one or more sensors, which can be used for determining a location of the apparatuses, an orientation of the apparatuses, and/or for any other purpose.” (Para 0029)
“In general, a UE may be any wireless communication device (e.g., a mobile phone, router, tablet computer, laptop computer, wearable device (e.g., smart watch, glasses, an extended reality (XR) device such as a virtual reality (VR) headset, an augmented reality (AR) headset or glasses, or a mixed reality (MR) headset, etc.), vehicle (e.g., automobile, motorcycle, bicycle, etc.), Internet of Things (IoT) device, etc.) used by a user to communicate over a wireless communications network.” (Para 0068)
Claim 10:
Zhang teaches the respective limitations of Claim 1. Zhang further teaches:
wherein the at least one processor is further configured to: determine, based on a second received RF signal, an object in motion within a predetermined distance of the first device;
(Zhang) – “In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine that the one or more objects are outside of the FOV of the image capturing device and moving towards a portion of a scene corresponding to the FOV of the image capturing device; and in response to determining that the one or more objects are outside of the FOV of the image capturing device and moving towards the portion of the scene corresponding to the FOV of the image capturing device, switch an active camera setting from the image capturing device to a different image capturing device. In some cases, the switched active camera setting can trigger the mobile device to use the different image capturing device to capture one or more images.” (Para 0016)
“Changes in certain CSI properties can be used to detect motion, estimate a change in location, determine a change in a motion pattern, etc.” (Para 0058)
“In some aspects, the process 800 can include determining that the one or more objects are outside of the FOV of the image capturing device and moving towards a portion of a scene corresponding to the FOV of the image capturing device; and in response to determining that the one or more objects are outside of the FOV of the image capturing device and moving towards the portion of the scene corresponding to the FOV of the image capturing device, switching an active camera setting from the image capturing device to a different image capturing device, the switched active camera setting triggering the apparatus to use the different image capturing device to capture one or more images.” (Para 0175)
activate a camera of the first device based on the determination; and
(Zhang) – “In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine that the one or more objects are outside of the FOV of the image capturing device and moving towards a portion of a scene corresponding to the FOV of the image capturing device; and in response to determining that the one or more objects are outside of the FOV of the image capturing device and moving towards the portion of the scene corresponding to the FOV of the image capturing device, switch an active camera setting from the image capturing device to a different image capturing device. In some cases, the switched active camera setting can trigger the mobile device to use the different image capturing device to capture one or more images.” (Para 0016)
“Changes in certain CSI properties can be used to detect motion, estimate a change in location, determine a change in a motion pattern, etc.” (Para 0058)
“In some aspects, the process 800 can include determining that the one or more objects are outside of the FOV of the image capturing device and moving towards a portion of a scene corresponding to the FOV of the image capturing device; and in response to determining that the one or more objects are outside of the FOV of the image capturing device and moving towards the portion of the scene corresponding to the FOV of the image capturing device, switching an active camera setting from the image capturing device to a different image capturing device, the switched active camera setting triggering the apparatus to use the different image capturing device to capture one or more images.” (Para 0175)
generate an additional reference map associated with the location of the first device using the second received RF signal and the activated camera.
(Zhang) – “In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine that the one or more objects are outside of the FOV of the image capturing device and moving towards a portion of a scene corresponding to the FOV of the image capturing device; and in response to determining that the one or more objects are outside of the FOV of the image capturing device and moving towards the portion of the scene corresponding to the FOV of the image capturing device, switch an active camera setting from the image capturing device to a different image capturing device. In some cases, the switched active camera setting can trigger the mobile device to use the different image capturing device to capture one or more images.” (Para 0016)
“Changes in certain CSI properties can be used to detect motion, estimate a change in location, determine a change in a motion pattern, etc.” (Para 0058)
“In some examples, the XR device can use the RF sensing data to determine whether a detected hand is occluded by one or more objects and/or whether the detected hand is outside of a FOV of the camera of the XR device. For example, the XR device can determine whether the detected hand is under a table that occludes the hand from the camera's FOV. As another example, the XR device can determine whether the hand is behind the user's back and outside of the camera's FOV. In some cases, when the hand is determined to be occluded and/or outside of the camera's FOV, the XR device can control a power setting of the camera to reduce a power consumption of the XR device. For example, the XR device can turn off the camera or place the camera in a lower power mode while the hand is occluded or outside of the camera's FOV. The XR device can turn the camera back on or place the camera in a higher power mode if it later determines that the hand is no longer occluded or outside of the camera's FOV. In some cases, the XR device can use RF sensing to detect a movement of a hand(s) and prepare the camera to take one or more images to capture a hand gesture if the movement indicates that the hand(s) is approaching the camera's FOV. In some cases, when the hand is determined to be occluded or otherwise outside of the camera's FOV, the XR device can use the RF sensing data to generate a sparse map of the physical space associated with the XR device.”(Para 0065)
“In some aspects, the process 800 can include determining that the one or more objects are outside of the FOV of the image capturing device and moving towards a portion of a scene corresponding to the FOV of the image capturing device; and in response to determining that the one or more objects are outside of the FOV of the image capturing device and moving towards the portion of the scene corresponding to the FOV of the image capturing device, switching an active camera setting from the image capturing device to a different image capturing device, the switched active camera setting triggering the apparatus to use the different image capturing device to capture one or more images.” (Para 0175)
Claim 16:
Zhang explicitly teaches:
A method for rendering an extended reality (XR) environment, the method comprising:
(Zhang) – “Disclosed are systems and techniques for extended reality optimizations using radio frequency (RF) sensing.” (Abstract)
“In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
generating, based on a received radio frequency (RF) signal, a local map representing an area in which a first device is located;
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“RF sensing can use RF data to generate a map of a space, such as a space in a scene or an indoor space. In some cases, using a monostatic configuration, a wireless device (e.g., XR device, smartphone, etc.) can obtain RF sensing data using wireless interfaces that are capable of performing transmit and receive functions. In some examples, the wireless device can implement a Wi-Fi radar to obtain the RF sensing data. The Wi-Fi radar can implement RF interfaces capable of performing transmit and receive functions. The Wi-Fi radar (and/or the wireless device implementing the Wi-Fi radar) can use the signal strength of radio transmissions to determine a location/position of one or more objects. For example, a wireless device may utilize a wireless interface (e.g., of a Wi-Fi radar) to transmit an RF signal and capture any signals that reflect from objects in the surrounding environment. The wireless interface can also receive leakage signals that are coupled directly from the transmitter's antenna to the receiver's antenna without reflecting from any objects. In some examples, the wireless device may gather RF sensing data in the form of channel state information (CSI) data relating to the direct paths (leakage signals) of the transmitted signal and data relating to the reflected paths of the signals received that correspond to the transmitted signal. In some cases, a bistatic configuration can be used where the transmit and receive functions are performed by different devices. For instance, a first device can transmit wireless signals that reflect from one or more objects in the scene, and a wireless interface of a second device can receive the reflected signals and/or signals directly from the first device. In some cases, the signals can be omnidirectional signals transmitted using an omnidirectional antenna, and the signals can be transmitted in a 360-degree radiation pattern.” (Para 0057)
aligning features of the local map with features of a reference map associated with a predetermined location;
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
Examiner Note: Any map or coordinate system may correspond with reference map.
determining, based on the aligned features, the first device is located at the predetermined location; and
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
rendering a virtual representation of the aligned local map and the aligned reference map at the predetermined location.
(Zhang) – “The mobile device 704 can receive the DL data 710 and send DL data 712 to the XR device 706, which the XR device 706 can use for an XR presentation/experience. In some examples, the DL data 712 can include rendered virtual content. For example, the mobile device 704 can render virtual content from the destination device 702 and send the rendered virtual content to the XR device 706. In some cases, the DL data 712 can include XR-related data such as, for example, a depth map, eye buffer (e.g., render texture, eye buffer resolution, etc.), tracking information, and/or any other XR data.” (Para 0151)
“The XR device 706 can obtain sensor data, such as inertial measurements, image data, etc., and provide such data to the mobile device 704. For example, the XR device 706 can send to the mobile device 704 UL data 722 including data from one or more inertial measurement units (IMUs) and a tracking frame from an image capturing device at the XR device 706. In some cases, the mobile device 704 can use the UL data 722 to render virtual content according to a position of the XR device 706 and/or one or more objects in a scene.” (Para 0152)
“In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
Examiner Note: Per BRI, any presentation in any way related to the local map and the reference map may correspond with the claimed rendering.
Claim 17:
Rejected for the same reasons as Claim 2
Claim 18:
Rejected for the same reasons as Claim 3
Claim 21:
Zhang explicitly teaches:
A non-transitory computer-readable medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to:
(Zhang) – “Disclosed are systems and techniques for extended reality optimizations using radio frequency (RF) sensing.” (Abstract)
“According to at least one example, an apparatus is provided for controlling device and/or processing settings based on RF sensing. The apparatus can include memory and one or more processors coupled to the memory, the one or more processors being configured to obtain radio frequency (RF) sensing data; determine, based on the RF sensing data, one or more reflected paths of one or more reflected RF signals, wherein each reflected RF signal comprises a reflection of a transmitted RF signal from one or more objects in a physical space; compare the one or more reflected paths to a field-of-view (FOV) of an image capturing device associated with the apparatus; and based on the comparison, trigger an action by at least one of the image capturing device and the apparatus.” (Para 0007)
generate, based on a received radio frequency (RF) signal, a local map representing an area in which a first device is located;
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“RF sensing can use RF data to generate a map of a space, such as a space in a scene or an indoor space. In some cases, using a monostatic configuration, a wireless device (e.g., XR device, smartphone, etc.) can obtain RF sensing data using wireless interfaces that are capable of performing transmit and receive functions. In some examples, the wireless device can implement a Wi-Fi radar to obtain the RF sensing data. The Wi-Fi radar can implement RF interfaces capable of performing transmit and receive functions. The Wi-Fi radar (and/or the wireless device implementing the Wi-Fi radar) can use the signal strength of radio transmissions to determine a location/position of one or more objects. For example, a wireless device may utilize a wireless interface (e.g., of a Wi-Fi radar) to transmit an RF signal and capture any signals that reflect from objects in the surrounding environment. The wireless interface can also receive leakage signals that are coupled directly from the transmitter's antenna to the receiver's antenna without reflecting from any objects. In some examples, the wireless device may gather RF sensing data in the form of channel state information (CSI) data relating to the direct paths (leakage signals) of the transmitted signal and data relating to the reflected paths of the signals received that correspond to the transmitted signal. In some cases, a bistatic configuration can be used where the transmit and receive functions are performed by different devices. For instance, a first device can transmit wireless signals that reflect from one or more objects in the scene, and a wireless interface of a second device can receive the reflected signals and/or signals directly from the first device. In some cases, the signals can be omnidirectional signals transmitted using an omnidirectional antenna, and the signals can be transmitted in a 360-degree radiation pattern.” (Para 0057)
align features of the local map with features of a reference map associated with a predetermined location;
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
Examiner Note: Any map or coordinate system may correspond with reference map.
determine, based on the aligned features, the first device is located at the predetermined location; and
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
render a virtual representation of the aligned local map and the aligned reference map at the predetermined location.
(Zhang) – “The mobile device 704 can receive the DL data 710 and send DL data 712 to the XR device 706, which the XR device 706 can use for an XR presentation/experience. In some examples, the DL data 712 can include rendered virtual content. For example, the mobile device 704 can render virtual content from the destination device 702 and send the rendered virtual content to the XR device 706. In some cases, the DL data 712 can include XR-related data such as, for example, a depth map, eye buffer (e.g., render texture, eye buffer resolution, etc.), tracking information, and/or any other XR data.” (Para 0151)
“The XR device 706 can obtain sensor data, such as inertial measurements, image data, etc., and provide such data to the mobile device 704. For example, the XR device 706 can send to the mobile device 704 UL data 722 including data from one or more inertial measurement units (IMUs) and a tracking frame from an image capturing device at the XR device 706. In some cases, the mobile device 704 can use the UL data 722 to render virtual content according to a position of the XR device 706 and/or one or more objects in a scene.” (Para 0152)
“In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
Examiner Note: Per BRI, any presentation in any way related to the local map and the reference map may correspond with the claimed rendering.
Claim 22:
Rejected for the same reasons as Claim 2
Claim 25:
Rejected for the same reasons as Claim 10
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4-7, 19-20, 23-24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang (US20220381898) in view of Berliner (US20240427471).
Claim 4:
Zhang teaches the respective limitations of Claim 1. Zhang further teaches:
wherein the generation of the local map associated with the first device is based on the received RF signal and a second received RF signal used to [triangulate] a location of the first device.
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“RF sensing can use RF data to generate a map of a space, such as a space in a scene or an indoor space. In some cases, using a monostatic configuration, a wireless device (e.g., XR device, smartphone, etc.) can obtain RF sensing data using wireless interfaces that are capable of performing transmit and receive functions. In some examples, the wireless device can implement a Wi-Fi radar to obtain the RF sensing data. The Wi-Fi radar can implement RF interfaces capable of performing transmit and receive functions. The Wi-Fi radar (and/or the wireless device implementing the Wi-Fi radar) can use the signal strength of radio transmissions to determine a location/position of one or more objects. For example, a wireless device may utilize a wireless interface (e.g., of a Wi-Fi radar) to transmit an RF signal and capture any signals that reflect from objects in the surrounding environment. The wireless interface can also receive leakage signals that are coupled directly from the transmitter's antenna to the receiver's antenna without reflecting from any objects. In some examples, the wireless device may gather RF sensing data in the form of channel state information (CSI) data relating to the direct paths (leakage signals) of the transmitted signal and data relating to the reflected paths of the signals received that correspond to the transmitted signal. In some cases, a bistatic configuration can be used where the transmit and receive functions are performed by different devices. For instance, a first device can transmit wireless signals that reflect from one or more objects in the scene, and a wireless interface of a second device can receive the reflected signals and/or signals directly from the first device. In some cases, the signals can be omnidirectional signals transmitted using an omnidirectional antenna, and the signals can be transmitted in a 360-degree radiation pattern.” (Para 0057)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
Examiner Note: Bracketed text not explicitly taught by primary reference, but is taught by non-primary reference later in the rejection.
Zhang does not explicitly teach:
triangulate
Berliner, in the same field of endeavor of extended reality, teaches:
triangulate
(Berliner) – “Some disclosed embodiments involve receiving a wireless signal captured by a receiver included in the extended reality appliance, and analyzing the wireless signal to determine the indication of the current physical location of the extended reality appliance. A wireless signal refers to any electromagnetic wave configured to carry information without the need for physical wires or cables. Examples of a wireless signal include a Wi-Fi, Bluetooth, or radio frequency signal. In the example shown in FIG. 17, processor 1700 may receive a Bluetooth signal 1734. A receiver included in the extended reality appliance may include any device or component placed, inserted, installed, integrated, or otherwise incorporated in the extended reality appliance and configured to capture wireless signals. Examples of a receiver included in the extended reality appliance include a radio frequency, Bluetooth, Wi-Fi, radar, or Near Field Communication receiver. In the example shown in FIG. 17, Bluetooth signal 1734 may be captured and transmitted to processor 1700 via a Bluetooth receiver 1732. Analyzing the wireless signal to determine the indication of the current physical location of the extended reality appliance refers to considering, evaluating, manipulating, or otherwise using the wireless signal to determine the indication of the current physical location of the extended reality appliance. Examples of analyzing the wireless signal to determine the indication of the current physical location of the extended reality appliance include W-Fi positioning, Bluetooth positioning, cellular-based positioning, or GPS positioning. For example, Bluetooth signals may be transmitted from devices, such as beacons, near or in the current physical location and a Bluetooth receiver may measure the strength and known locations of the beacons using those signals to perform triangulation and estimate the current physical location. In the example shown in FIG. 17, a Bluetooth receiver 1732 may be configured to perform triangulation and estimate the current physical location.” (Para 0081)
Therefore, it would be obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the techniques for extended reality optimizations of Zhang with the method for extended reality of Berliner. One of ordinary skill in the art would have been motivated to make these modifications for the purpose of “providing a new approach for solving the productivity dilemma, one that uses extended reality (XR) to provide a mobile environment that enables users to experience the comfort of a stationary workspace anywhere they want by providing virtual desktop-like screens.” (Berliner Para 0004)
Claim 5:
Zhang in combination with the references relied upon in Claim 4 teach those respective limitations. Zhang further teaches:
wherein the generation of the local map is based on the received RF signal including an angle of arrival (AoA) of the received RF signal and a round trip time (RTT) associated with an amount of time to receive the received RF signal from when the received RF signal was transmitted.
(Zhang) – “In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“RF sensing can use RF data to generate a map of a space, such as a space in a scene or an indoor space. In some cases, using a monostatic configuration, a wireless device (e.g., XR device, smartphone, etc.) can obtain RF sensing data using wireless interfaces that are capable of performing transmit and receive functions. In some examples, the wireless device can implement a Wi-Fi radar to obtain the RF sensing data. The Wi-Fi radar can implement RF interfaces capable of performing transmit and receive functions. The Wi-Fi radar (and/or the wireless device implementing the Wi-Fi radar) can use the signal strength of radio transmissions to determine a location/position of one or more objects. For example, a wireless device may utilize a wireless interface (e.g., of a Wi-Fi radar) to transmit an RF signal and capture any signals that reflect from objects in the surrounding environment. The wireless interface can also receive leakage signals that are coupled directly from the transmitter's antenna to the receiver's antenna without reflecting from any objects. In some examples, the wireless device may gather RF sensing data in the form of channel state information (CSI) data relating to the direct paths (leakage signals) of the transmitted signal and data relating to the reflected paths of the signals received that correspond to the transmitted signal. In some cases, a bistatic configuration can be used where the transmit and receive functions are performed by different devices. For instance, a first device can transmit wireless signals that reflect from one or more objects in the scene, and a wireless interface of a second device can receive the reflected signals and/or signals directly from the first device. In some cases, the signals can be omnidirectional signals transmitted using an omnidirectional antenna, and the signals can be transmitted in a 360-degree radiation pattern.” (Para 0057)
“In some cases, the XR device may determine and store device position data and device orientation data. In some instances, the XR device position data and device orientation data can be used to adjust the calculations for the distance and angle of reflection of reflected signals (determined using the CSI data) if the device is in motion. For example, the position and orientation data can be used to correlate one or more reflected signals with their corresponding transmitted signal. In some examples, device position or location data can be gathered using techniques that measure round trip time (RTT), Passive Positioning, Angle of Arrival (AoA), Received Signal Strength Indicator (RSSI), using the CSI data, using any other suitable technique, or any combination thereof. The device orientation data can be obtained from electronic sensors on the XR device, such as one or more gyroscopes, accelerometers, compasses, any other suitable sensor, or any combination thereof.” (Para 0066)
Claim 6:
Zhang in combination with the references relied upon in Claim 4 teach those respective limitations. Zhang further teaches:
wherein the second received RF signal was transmitted by a wearable device.
(Zhang) – “In some aspects, the apparatus is or is part of a mobile device (e.g., a mobile telephone or “smart phone” or other mobile device), a wearable device (e.g., a head mounted display), an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device), a tablet, a personal computer, a laptop computer, a server computer, a wireless access point, or other any other device having an RF interface. In some aspects, the apparatuses described above can include a camera or multiple cameras for capturing one or more images. In some aspects, the apparatus further includes a display for displaying one or more images, notifications, and/or other displayable data. In some aspects, the apparatuses described above can include one or more sensors, which can be used for determining a location of the apparatuses, an orientation of the apparatuses, and/or for any other purpose.” (Para 0029)
“In some examples, at least one of the one or more objects comprises a hand of a user of the mobile device. In some aspects, the method, non-transitory computer-readable medium, and apparatuses described above can determine, based on the RF sensing data and the one or more reflected paths, at least one of a map of the physical space and a hand gesture associated with the hand of the user.” (Para 0026)
“To provide realistic XR experiences, XR technologies can integrate virtual content with the physical world. In some examples, this can involve generating a map of the real-world environment and determining or calculating a particular pose of the user's XR device relative to the map of the real-world environment in order to anchor virtual content to the real-world environment in a convincing manner. The pose information can be used to match virtual content with the user's perceived motion and the spatio-temporal state of the real-world environment.” (Para 0047)
“RF sensing can use RF data to generate a map of a space, such as a space in a scene or an indoor space. In some cases, using a monostatic configuration, a wireless device (e.g., XR device, smartphone, etc.) can obtain RF sensing data using wireless interfaces that are capable of performing transmit and receive functions. In some examples, the wireless device can implement a Wi-Fi radar to obtain the RF sensing data. The Wi-Fi radar can implement RF interfaces capable of performing transmit and receive functions. The Wi-Fi radar (and/or the wireless device implementing the Wi-Fi radar) can use the signal strength of radio transmissions to determine a location/position of one or more objects. For example, a wireless device may utilize a wireless interface (e.g., of a Wi-Fi radar) to transmit an RF signal and capture any signals that reflect from objects in the surrounding environment. The wireless interface can also receive leakage signals that are coupled directly from the transmitter's antenna to the receiver's antenna without reflecting from any objects. In some examples, the wireless device may gather RF sensing data in the form of channel state information (CSI) data relating to the direct paths (leakage signals) of the transmitted signal and data relating to the reflected paths of the signals received that correspond to the transmitted signal. In some cases, a bistatic configuration can be used where the transmit and receive functions are performed by different devices. For instance, a first device can transmit wireless signals that reflect from one or more objects in the scene, and a wireless interface of a second device can receive the reflected signals and/or signals directly from the first device. In some cases, the signals can be omnidirectional signals transmitted using an omnidirectional antenna, and the signals can be transmitted in a 360-degree radiation pattern.” (Para 0057)
“The XR device can perform a camera calibration to align the camera image coordinate system with the RF sensing coordinate system. The camera image coordinate system can be from the field-of-view (FOV) of a camera at the XR device based on the camera location and pose of the XR device. In some examples, the XR device can use sensor data (with or without other data), such as data from an inertial measurement unit (IMU), to calculate a pose of the XR device. In some cases, the XR device can use the pose of the XR device to calculate reference coordinates of the XR device's FOV and/or refine an object's position (e.g., hand position, input device position, etc.) calculated using RF sensing.” (Para 0063)
“In general, a UE may be any wireless communication device (e.g., a mobile phone, router, tablet computer, laptop computer, wearable device (e.g., smart watch, glasses, an extended reality (XR) device such as a virtual reality (VR) headset, an augmented reality (AR) headset or glasses, or a mixed reality (MR) headset, etc.), vehicle (e.g., automobile, motorcycle, bicycle, etc.), Internet of Things (IoT) device, etc.) used by a user to communicate over a wireless communications network.” (Para 0068)
Claim 7:
Zhang in combination with the references relied upon in Claim 6 teach those respective limitations. Zhang further teaches:
wherein the wearable device is at least one of a haptic vest, haptic gloves, a smartwatch, or haptic shoes.
(Zhang) – “In some aspects, the apparatus is or is part of a mobile device (e.g., a mobile telephone or “smart phone” or other mobile device), a wearable device (e.g., a head mounted display), an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device), a tablet, a personal computer, a laptop computer, a server computer, a wireless access point, or other any other device having an RF interface. In some aspects, the apparatuses described above can include a camera or multiple cameras for capturing one or more images. In some aspects, the apparatus further includes a display for displaying one or more images, notifications, and/or other displayable data. In some aspects, the apparatuses described above can include one or more sensors, which can be used for determining a location of the apparatuses, an orientation of the apparatuses, and/or for any other purpose.” (Para 0029)
“In general, a UE may be any wireless communication device (e.g., a mobile phone, router, tablet computer, laptop computer, wearable device (e.g., smart watch, glasses, an extended reality (XR) device such as a virtual reality (VR) headset, an augmented reality (AR) headset or glasses, or a mixed reality (MR) headset, etc.), vehicle (e.g., automobile, motorcycle, bicycle, etc.), Internet of Things (IoT) device, etc.) used by a user to communicate over a wireless communications network.” (Para 0068)
Claim 19:
Rejected for the same reasons as Claim 4
Claim 20:
Rejected for the same reasons as Claim 5
Claim 23:
Rejected for the same reasons as Claim 4
Claim 24:
Rejected for the same reasons as Claim 5
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Cohen (US12442915) teaches a method for determining the orientation of a tracked device.
Mandl (US20250292493) teaches hand-based light estimation for extended reality.
Gupte (US20180000025) teaches a similar extended reality system.
Elhadad (US20250190049) teaches a similar wearable extended reality appliance.
Forutanpour (US20220405959) teaches a similar extended reality device.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID RUBEN PEDERSEN whose telephone number is (571)272-9696. The examiner can normally be reached M-Th: 07:00 -16:00 Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramon Mercado can be reached at (571) 270-5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID RUBEN PEDERSEN/Examiner, Art Unit 3658