DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 29 July 2026 have been fully considered but they are not persuasive.
The Applicant argues that Burgoyne et al. fails to disclose “adjust[ing] the zoom level based at least in part on the measured pace of the measurement of the user.” The Examiner respectfully disagrees. Burgoyne et al. discloses a computer-generated reality (CGR), and more specifically to techniques for moving in a CGR environment (paragraph [0002]). Computer-generated reality (CGR) is a broad term for any experience where the environment or information is created or enhanced by computer technology, replacing or augmenting the real world. This can include virtual reality (VR), where the entire world is simulated, and augmented reality (AR), where digital content is overlaid on the real world. In VR, the user is fully immersed in a simulated world, while in AR, the real world remains visible but enhanced with digital overlays. Extended reality (XR) is the umbrella term that encompasses all forms of computer-generated reality- including VR, AR, and mixed reality (MR) – as well as any technology that fuses real and virtual elements. Furthermore, Burgoyne et al. discloses in Figs. 1A and 1B an exemplary system 100 for use in various computer-generated reality technologies (paragraph [0025]). In some embodiments, elements of system 100 are implemented in a base station device (e.g., a computing device, such as a remote server, mobile device, or a laptop) and other elements of the system 100 are implemented in a head-mounted display (HMD) device designed to be worn by the user, where the HMD device is in communication with the base station. In some embodiments, device 100a is implemented in a base station device of a HMD device (paragraph [0027]). In some embodiments, system 100 is a mobile device. In some embodiments, system 100 is a head-mounted display (HMD) device. In some embodiments, system 100 is a wearable HUD device (paragraph [0029]). Moreover, system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of system 100 and/or display(s) 120. For example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment. Orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers (paragraph [0036]). The accelerometer can estimate the pace (speed) of movement by measuring the magnitude and frequency of acceleration signals over time. The information from the accelerometer can be used to effectively create the dolly zoom technique. The embodiments illustrated in Figs. 2E-2F also illustrate an example of a dolly zoom technique. According to the dolly zoom technique, system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view. In some embodiments, system 200 changes the magnification, field of view, and position simultaneously (paragraphs [0056] - [0058]). Therefore, as can be seen from the cited portions above of the Burgoyne et al. reference, Burgoyne et al. meets the claimed limitations and the rejection is maintained.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 73-76, 78-80, 82-86, 88-90, 92, 133, and 134 are rejected under 35 U.S.C. 103 as being unpatentable over Park (U.S. Patent Application Publication 2024/0312147) in view of Burgoyne et al. (U.S. Patent Application Publication 2020/0273146).
Regarding claim 73, Park discloses an extended reality (XR) apparatus (paragraph [0042] – augmented reality (AR)/extended reality (XR) applications may render the virtual/mixed environment with spatial aspects (e.g., locations, distances, etc.) – in such implementations, head positioning and/or gaze inference may be used to affect the user’s experience in virtualized/mixed space) comprising: a sensor (paragraph [0037] – the exemplary smart glasses combine a high-resolution sensor with a telephoto lens to capture images beyond the user’s natural vision – in other variants, the images may be locally processed with onboard computer-vision for e.g., region-of-interest (ROI) detection, text recognition, facial recognition, object-recognition, and/or other data extraction (e.g., QR codes, visual markers, etc.)); a telephoto camera (paragraph [0037] – the exemplary smart glasses combine a high-resolution sensor with a telephoto lens to capture images beyond the user’s natural vision – in other variants, the images may be locally processed with onboard computer-vision for e.g., region-of-interest (ROI) detection, text recognition, facial recognition, object-recognition, and/or other data extraction (e.g., QR codes, visual markers, etc.)); control circuitry configured to: determine, based on sensor data captured by the sensor, a distance between the XR apparatus and an object in a region of interest (ROI) in a real-world environment (paragraph [0042] – augmented reality (AR)/extended reality (XR) applications may render the virtual/mixed environment with spatial aspects (e.g., locations, distances, etc.) – in such implementations, head positioning and/or gaze inference may be used to affect the user’s experience in virtualized/mixed space; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102); and cause the telephoto camera to capture a zoomed-in view of the ROI at a zoom level (paragraph [0042] – augmented reality (AR)/extended reality (XR) applications may render the virtual/mixed environment with spatial aspects (e.g., locations, distances, etc.) – in such implementations, head positioning and/or gaze inference may be used to affect the user’s experience in virtualized/mixed space; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102; paragraph [0054] – in this example, since the user is fixated on a person at a great distance, the facial detection logic of the smart glasses may identify the location and/or presence of a face (but not an identity) at the gaze point 106 – the face data may be cropped out of the telephoto image to create the “region of interest” (ROI) image 108). However, Park fails to disclose detecting, based at least in part on the distance, whether a user of the XR apparatus is moving towards the object within the real-world environment; determine, based on at least in part on the distance, a pace of a movement of the user within the real-world environment; adjust the zoom level based at least in part on the detecting; and cause display of the zoomed-in view at the adjusted zoom level and the determined pace.
Referring to the Burgoyne et al. reference, Burgoyne et al. discloses an extended reality (XR) apparatus comprising: control circuitry configured to: detect, based on at least in part on the distance, whether a user of the XR apparatus is moving towards the object within the real-world environment (paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same); determine, based at least in part on the distance, a pace of a movement of the user within the real-world environment (paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same; Burgoyne et al. discloses the dolly effect, which would take the accelerometer into consideration when determining the level of zoom required based on the change of position of the user); adjust the zoom level based at least in part on the detecting and the determined pace (paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same; Burgoyne et al. discloses the dolly effect, which would take the accelerometer into consideration when determining the level of zoom required based on the change of position of the user); and cause display of the zoomed-in view at the adjusted zoom level (paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have had detected the movement of the XR apparatus and adjusted the zoom level accordingly as disclosed by Burgoyne et al. in the apparatus disclosed by Park in order to compensate for any movement by the XR apparatus and to maintain focus on the region of interest.
Regarding claim 74, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 73 including that wherein the sensor is a LiDAR sensor (Park: paragraph [0042] – augmented reality (AR)/extended reality (XR) applications may render the virtual/mixed environment with spatial aspects (e.g., locations, distances, etc.) – in such implementations, head positioning and/or gaze inference may be used to affect the user’s experience in virtualized/mixed space; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102; LiDAR would be included in the well-known range finding techniques mentioned by Park).
Regarding claim 75, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 73 including that wherein the control circuitry is further configured to: determine that the user is moving towards the object (Burgoyne et al.: paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same); and based at least in part on determining that the user is moving towards the object, adjusting the zoom level by zooming out with respect to the object (Burgoyne et al.: paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same).
Regarding claim 76, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 73 including that wherein the control circuitry is further configured to: determine that the user is moving away from the object (Burgoyne et al.: paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same; Burgoyne et al. discloses the dolly effect, which is as the camera moves toward or away from a subject, the lens zooms in the opposite direction, therefore as the user moves away from the object, the lens would zoom in); and based at least in part on determining that the user is moving away from the object, adjusting the zoom level by zooming in on the object (Burgoyne et al.: paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same; Burgoyne et al. discloses the dolly effect, which is as the camera moves toward or away from a subject, the lens zooms in the opposite direction, therefore as the user moves away from the object, the lens would zoom in).
Regarding claim 78, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 73 including that wherein the control circuitry is further configured to: determine the pace of the movement of the user is below a threshold; and based at least in part on the determining that the pace of the movement of the user is below the threshold, adjust the zoom level by zooming in on the object (Burgoyne et al.: paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same; Burgoyne et al. discloses the dolly effect, which would take the accelerometer into consideration when determining the level of zoom required based on the change of position of the user – the threshold can be zero, meaning no movement or movement in one direction detected vs movement in the other direction).
Regarding claim 79, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 73 including that wherein the control circuitry is further configured to determine the ROI in the real-world environment by: collecting facial data of the user using an inward-facing infrared (IR) sensor (Park: paragraph [0038] – during gaze inference, the smart glasses can track eye movement more frequently (and potentially with infrared (IR) illumination) to infer the presence of a gaze point; Burgoyne et al.: paragraph [0034] – image sensor(s) also optionally include one or more infrared (IR) sensor(s), such as a passive IR sensor or an active IR sensor, for detecting infrared light from the environment; paragraph [0042] – in some embodiments, the position of region 206 on display 202 is based on a gaze position (e.g., region 206 is centered on the positions of a user’s gaze) – in some embodiments, a gaze position is determined based on data collected by sensors included in system 200); and determining a gaze vector based at least in part on the collected facial data (Park: paragraph [0038] – during gaze inference, the smart glasses can track eye movement more frequently (and potentially with infrared (IR) illumination) to infer the presence of a gaze point; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – gaze vectors - the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102); Burgoyne et al.: paragraph [0034] – image sensor(s) also optionally include one or more infrared (IR) sensor(s), such as a passive IR sensor or an active IR sensor, for detecting infrared light from the environment; paragraph [0042] – in some embodiments, the position of region 206 on display 202 is based on a gaze position (e.g., region 206 is centered on the positions of a user’s gaze) – in some embodiments, a gaze position is determined based on data collected by sensors included in system 200).
Regarding claim 80, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claims 73 and 79 including that wherein the control circuitry is further configured to cause the telephoto camera to capture the zoomed-in view of the ROI by: detecting, based at least in part on sensor data detected by the inward facing IR sensor, a facial action (Park: paragraph [0038] – during gaze inference, the smart glasses can track eye movement more frequently (and potentially with infrared (IR) illumination) to infer the presence of a gaze point; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – gaze vectors - the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102); Burgoyne et al.: paragraph [0034] – image sensor(s) also optionally include one or more infrared (IR) sensor(s), such as a passive IR sensor or an active IR sensor, for detecting infrared light from the environment; paragraph [0042] – in some embodiments, the position of region 206 on display 202 is based on a gaze position (e.g., region 206 is centered on the positions of a user’s gaze) – in some embodiments, a gaze position is determined based on data collected by sensors included in system 200); and based at least in part on detecting the facial action, causing the telephoto camera to zoom-in based at least in part on the direction of the gaze vector (Park: paragraph [0038] – during gaze inference, the smart glasses can track eye movement more frequently (and potentially with infrared (IR) illumination) to infer the presence of a gaze point; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – gaze vectors - the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102); paragraph [0054] – in this example, since the user is fixated on a person at a great distance, the facial detection logic of the smart glasses may identify the location and/or presence of a face (but not an identity) at the gaze point 106 – the face data may be cropped out of the telephoto image to create the “region of interest” (ROI) image 108; Burgoyne et al.: paragraph [0034] – image sensor(s) also optionally include one or more infrared (IR) sensor(s), such as a passive IR sensor or an active IR sensor, for detecting infrared light from the environment; paragraph [0042] – in some embodiments, the position of region 206 on display 202 is based on a gaze position (e.g., region 206 is centered on the positions of a user’s gaze) – in some embodiments, a gaze position is determined based on data collected by sensors included in system 200).
Regarding claim 82, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 73 including that the XR apparatus further comprises determining that the object has moved from a first location to a second location; and causing the object at the second location to remain within a field of view (FOV) of the telephoto camera by adjusting a capture angle of the telephoto camera, based at least in part on the second location (Park: paragraph [0038] – during gaze inference, the smart glasses can track eye movement more frequently (and potentially with infrared (IR) illumination) to infer the presence of a gaze point; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – gaze vectors - the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102); paragraph [0054] – in this example, since the user is fixated on a person at a great distance, the facial detection logic of the smart glasses may identify the location and/or presence of a face (but not an identity) at the gaze point 106 – the face data may be cropped out of the telephoto image to create the “region of interest” (ROI) image 108; paragraph [0079] – wide angle camera lenses (or even ultra-wide camera lenses) may be used to track the location of the face relative to the user – once a high-resolution image is captured, the facial detection information may be used to crop to an ROI for facial recognition; as the user’s gaze moves, the camera lens and FOV would change accordingly; Burgoyne et al.: paragraph [0034] – image sensor(s) also optionally include one or more infrared (IR) sensor(s), such as a passive IR sensor or an active IR sensor, for detecting infrared light from the environment; paragraph [0042] – in some embodiments, the position of region 206 on display 202 is based on a gaze position (e.g., region 206 is centered on the positions of a user’s gaze) – in some embodiments, a gaze position is determined based on data collected by sensors included in system 200; the camera angles would change accordingly to the user’s gaze).
Regarding claim 83, Park discloses a method comprising: a sensor to determine, based at least in part on sensor data captured by the sensor, a distance between an extended reality (XR) apparatus and an object in a region of interest (ROI) in a real-world environment (paragraph [0042] – augmented reality (AR)/extended reality (XR) applications may render the virtual/mixed environment with spatial aspects (e.g., locations, distances, etc.) – in such implementations, head positioning and/or gaze inference may be used to affect the user’s experience in virtualized/mixed space; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102); and causing a telephoto camera to capture a zoomed-in view of the ROI at a zoom level (paragraph [0042] – augmented reality (AR)/extended reality (XR) applications may render the virtual/mixed environment with spatial aspects (e.g., locations, distances, etc.) – in such implementations, head positioning and/or gaze inference may be used to affect the user’s experience in virtualized/mixed space; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102; paragraph [0054] – in this example, since the user is fixated on a person at a great distance, the facial detection logic of the smart glasses may identify the location and/or presence of a face (but not an identity) at the gaze point 106 – the face data may be cropped out of the telephoto image to create the “region of interest” (ROI) image 108). However, Park fails to disclose detecting, based at least in part on the distance, whether a user of the XR apparatus is moving towards the object within the real-world environment; determining, based at least in part on the distance, a pace of a movement of the user within the real-world environment; adjusting the zoom level based at least in part on the detecting and the determined pace; and causing display of the zoomed-in view at the adjusted zoom level.
Referring to the Burgoyne et al. reference, Burgoyne et al. discloses a method comprising: detecting, based at least in part on the distance, whether a user of the XR apparatus is moving towards the object within the real-world environment (paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same); determining, based at least in part on the distance, a pace of movement of the user within the real-world environment (paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same; Burgoyne et al. discloses the dolly effect, which would take the accelerometer into consideration when determining the level of zoom required based on the change of position of the user); adjusting the zoom level based at least in part on the detecting and the determined pace (paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same; Burgoyne et al. discloses the dolly effect, which would take the accelerometer into consideration when determining the level of zoom required based on the change of position of the user); and causing display of the zoomed-in view at the adjusted zoom level (paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have had detected the movement of the XR apparatus and adjusted the zoom level accordingly as disclosed by Burgoyne et al. in the method disclosed by Park in order to compensate for any movement by the XR apparatus and to maintain focus on the region of interest.
Regarding claim 84, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 83 including that the sensor is a LiDAR sensor (Park: paragraph [0042] – augmented reality (AR)/extended reality (XR) applications may render the virtual/mixed environment with spatial aspects (e.g., locations, distances, etc.) – in such implementations, head positioning and/or gaze inference may be used to affect the user’s experience in virtualized/mixed space; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102; LiDAR would be included in the well-known range finding techniques mentioned by Park).
Regarding claim 85, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 83 including that the method further comprises: determining that the user is moving towards the object(Burgoyne et al.: paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same); and based at least in part on determining that the user is moving towards the object, adjusting the zoom level by zooming out with respect to the object (Burgoyne et al.: paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same).
Regarding claim 86, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 83 including that the method further comprises: determining that the user is moving away from the object (Burgoyne et al.: paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same; Burgoyne et al. discloses the dolly effect, which is as the camera moves toward or away from a subject, the lens zooms in the opposite direction, therefore as the user moves away from the object, the lens would zoom in); and based at least in part on determining that the user is moving away from the object, adjusting the zoom level by zooming in on the object (Burgoyne et al.: paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same; Burgoyne et al. discloses the dolly effect, which is as the camera moves toward or away from a subject, the lens zooms in the opposite direction, therefore as the user moves away from the object, the lens would zoom in).
Regarding claim 88, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 83 including that the method further comprises: determining the pace of the movement of the user is below a threshold; and based at least in part on the determining that the pace of the movement of the user is below the threshold, adjusting the zoom level by zooming in on the object (Burgoyne et al.: paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same; Burgoyne et al. discloses the dolly effect, which would take the accelerometer into consideration when determining the level of zoom required based on the change of position of the user – the threshold can be zero, meaning no movement or movement in one direction detected vs movement in the other direction).
Regarding claim 89, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 83 including that wherein determining the ROI in the real-world environment further comprises: collecting facial data of the user using an inward-facing infrared (IR) sensor (Park: paragraph [0038] – during gaze inference, the smart glasses can track eye movement more frequently (and potentially with infrared (IR) illumination) to infer the presence of a gaze point; Burgoyne et al.: paragraph [0034] – image sensor(s) also optionally include one or more infrared (IR) sensor(s), such as a passive IR sensor or an active IR sensor, for detecting infrared light from the environment; paragraph [0042] – in some embodiments, the position of region 206 on display 202 is based on a gaze position (e.g., region 206 is centered on the positions of a user’s gaze) – in some embodiments, a gaze position is determined based on data collected by sensors included in system 200); and determining a gaze vector based at least in part on the collected facial data (Park: paragraph [0038] – during gaze inference, the smart glasses can track eye movement more frequently (and potentially with infrared (IR) illumination) to infer the presence of a gaze point; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – gaze vectors - the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102); Burgoyne et al.: paragraph [0034] – image sensor(s) also optionally include one or more infrared (IR) sensor(s), such as a passive IR sensor or an active IR sensor, for detecting infrared light from the environment; paragraph [0042] – in some embodiments, the position of region 206 on display 202 is based on a gaze position (e.g., region 206 is centered on the positions of a user’s gaze) – in some embodiments, a gaze position is determined based on data collected by sensors included in system 200).
Regarding claim 90, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claims 83 and 89 including that the method further comprises: detecting, based at least in part on sensor data detected by the inward facing IR sensor, a facial action (Park: paragraph [0038] – during gaze inference, the smart glasses can track eye movement more frequently (and potentially with infrared (IR) illumination) to infer the presence of a gaze point; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – gaze vectors - the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102); Burgoyne et al.: paragraph [0034] – image sensor(s) also optionally include one or more infrared (IR) sensor(s), such as a passive IR sensor or an active IR sensor, for detecting infrared light from the environment; paragraph [0042] – in some embodiments, the position of region 206 on display 202 is based on a gaze position (e.g., region 206 is centered on the positions of a user’s gaze) – in some embodiments, a gaze position is determined based on data collected by sensors included in system 200); and based at least in part on detecting the facial action, causing the telephoto camera to zoom- in based at least in part on the direction of the gaze vector (Park: paragraph [0038] – during gaze inference, the smart glasses can track eye movement more frequently (and potentially with infrared (IR) illumination) to infer the presence of a gaze point; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – gaze vectors - the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102); paragraph [0054] – in this example, since the user is fixated on a person at a great distance, the facial detection logic of the smart glasses may identify the location and/or presence of a face (but not an identity) at the gaze point 106 – the face data may be cropped out of the telephoto image to create the “region of interest” (ROI) image 108; Burgoyne et al.: paragraph [0034] – image sensor(s) also optionally include one or more infrared (IR) sensor(s), such as a passive IR sensor or an active IR sensor, for detecting infrared light from the environment; paragraph [0042] – in some embodiments, the position of region 206 on display 202 is based on a gaze position (e.g., region 206 is centered on the positions of a user’s gaze) – in some embodiments, a gaze position is determined based on data collected by sensors included in system 200).
Regarding claim 92, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 83 including that the method further comprises: determining that the object has moved from a first location to a second location; and causing the object at the second location to remain within a field of view (FOV) of the telephoto camera by adjusting a capture angle of the telephoto camera, based at least in part on the second location (Park: paragraph [0038] – during gaze inference, the smart glasses can track eye movement more frequently (and potentially with infrared (IR) illumination) to infer the presence of a gaze point; paragraph [0045] – once triggered, the camera assembly captures a high-resolution image – in one specific embodiment, the high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer vision analysis of the user’s gaze point; paragraph [0048] – gaze vectors - the convergence in space provides a distance and direction from the user which can be used to select an appropriate camera lens – in other variants, the smart glasses may incorporate range finding techniques (e.g., time-of-flight (TOF), stereo vision, laser, infrared, etc.) to determine when telephoto vision is desired – in other words, when the user’s gaze point 106 is at long range (greater than 100ft), the smart glasses uses the telephoto camera assembly 102); paragraph [0054] – in this example, since the user is fixated on a person at a great distance, the facial detection logic of the smart glasses may identify the location and/or presence of a face (but not an identity) at the gaze point 106 – the face data may be cropped out of the telephoto image to create the “region of interest” (ROI) image 108; paragraph [0079] – wide angle camera lenses (or even ultra-wide camera lenses) may be used to track the location of the face relative to the user – once a high-resolution image is captured, the facial detection information may be used to crop to an ROI for facial recognition; as the user’s gaze moves, the camera lens and FOV would change accordingly; Burgoyne et al.: paragraph [0034] – image sensor(s) also optionally include one or more infrared (IR) sensor(s), such as a passive IR sensor or an active IR sensor, for detecting infrared light from the environment; paragraph [0042] – in some embodiments, the position of region 206 on display 202 is based on a gaze position (e.g., region 206 is centered on the positions of a user’s gaze) – in some embodiments, a gaze position is determined based on data collected by sensors included in system 200; the camera angles would change accordingly to the user’s gaze).
Regarding claims 133 and 134, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claims 73 and 78, but fails to explicitly disclose wherein the threshold is a non-zero value indicating slow user movement or wherein the threshold is a value indicating a user is stationary. Official Notice is taken that both the concept and advantages of setting thresholds that indicate different movement speeds of the user is well-known in the art. For example, the different thresholds could indicate the user is running, jogging, walking, moving slowly, or is stationary. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have included different thresholds indicating different movements of the user in the apparatus disclosed by Park in view of Burgoyne et al. in order to help determine how the extended reality apparatus should compensate for the user’s movements.
Claims 135 and 136 are rejected under 35 U.S.C. 103 as being unpatentable over Park in view of Burgoyne et al. as applied to claim 73 above, and further in view of Jobanputra et al. (U.S. Patent Application Publication 2020/0073385).
Regarding claim 135, Park in view of Burgoyne et al. discloses all of the limitations as previously discussed with respect to claim 73, but fails to disclose that the control circuitry is further configured to predict a movement path of the object based at least in part on the sensor data; and the adjusting is based at least in part on the predicting.
Referring to the Jobanputra et al. reference, Jobanputra et al. discloses am apparatus, comprising: control circuitry that is configured to predict a movement path of the object based at least in part on the sensor data (Fig. 26; paragraph [0059] – the tracking system 140 may communicate with the motion planning system 130, for example, to maneuver the UAV 100 based on measured, estimated, and/or predicted positions, orientations, and/or trajectories of objects in the physical environment – for example, the tracking system 140 may communicate a navigation objective to the motion planning system 130 to maintain a particular separation distance to a tracked object that is in motion; paragraph [0060] – the tracking system 140, operating separately or in conjunction with the motion planner 130, is further configured to generate control commands configured to cause one or more stabilization/tracking devices 152 to adjust an orientation and/or position of any image capture devices 114/115 relative to the body of the UAV 100 based on the motion of the UAV 100 and/or the tracking of one or more objects – such stabilization/tracking devices 152 may include a mechanical gimbal or a hybrid digital-mechanical gimbal, as previously described – for example, while tracking an object in motion relative to the UAV 100, the tracking system 140 may generate control commands configured to adjust an orientation of an image capture device 115 so as to keep the tracked object centered in the field of view (FOV) of the image capture device 115 while the UAV 100 is in motion – similarly, the tracking system 140 may generate commands or output data to a digital image processor (e.g., that is part of a hybrid digital-mechanical gimbal) to transform images captured by the image capture device 115 to keep the tracked object centered in the FOV of the image capture device 115 while the UAV 100 is in motion; paragraph [0189] – Fig. 26 shows an example view 2600 of a physical environment 2610 as presented at a display of an AR device – as shown in Fig. 26, one or more augmentations may be presented to the user in the form of augmenting graphical overlays 2620a, 2622a, 2624a, 2626a, and 2620b associated with objects (e.g., bikers 2640a and 2640b) in the physical environment 2610; paragraph [0194] – in some embodiments, augmentations may include visual effects that track or interact with tracked objects – for example, Fig. 26 shows an augmenting graphical overlay 2624a in the form of a projection of a 3D trajectory (e.g., current, past, and/or future) associated with biker 2640a; paragraph [0230] – object detections – the combination of these rays and planes over time can be fused into an accurate prediction of the 3D position and velocity trajectory of the object over time); and the adjusting is based at least in part on the predicting (Fig. 26; paragraph [0059] – the tracking system 140 may communicate with the motion planning system 130, for example, to maneuver the UAV 100 based on measured, estimated, and/or predicted positions, orientations, and/or trajectories of objects in the physical environment – for example, the tracking system 140 may communicate a navigation objective to the motion planning system 130 to maintain a particular separation distance to a tracked object that is in motion; paragraph [0060] – the tracking system 140, operating separately or in conjunction with the motion planner 130, is further configured to generate control commands configured to cause one or more stabilization/tracking devices 152 to adjust an orientation and/or position of any image capture devices 114/115 relative to the body of the UAV 100 based on the motion of the UAV 100 and/or the tracking of one or more objects – such stabilization/tracking devices 152 may include a mechanical gimbal or a hybrid digital-mechanical gimbal, as previously described – for example, while tracking an object in motion relative to the UAV 100, the tracking system 140 may generate control commands configured to adjust an orientation of an image capture device 115 so as to keep the tracked object centered in the field of view (FOV) of the image capture device 115 while the UAV 100 is in motion – similarly, the tracking system 140 may generate commands or output data to a digital image processor (e.g., that is part of a hybrid digital-mechanical gimbal) to transform images captured by the image capture device 115 to keep the tracked object centered in the FOV of the image capture device 115 while the UAV 100 is in motion; paragraph [0189] – Fig. 26 shows an example view 2600 of a physical environment 2610 as presented at a display of an AR device – as shown in Fig. 26, one or more augmentations may be presented to the user in the form of augmenting graphical overlays 2620a, 2622a, 2624a, 2626a, and 2620b associated with objects (e.g., bikers 2640a and 2640b) in the physical environment 2610; paragraph [0194] – in some embodiments, augmentations may include visual effects that track or interact with tracked objects – for example, Fig. 26 shows an augmenting graphical overlay 2624a in the form of a projection of a 3D trajectory (e.g., current, past, and/or future) associated with biker 2640a; paragraph [0230] – object detections – the combination of these rays and planes over time can be fused into an accurate prediction of the 3D position and velocity trajectory of the object over time).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have had predicted a movement path of the object based at least in part on the sensor data; and adjusted settings based at least in part on the predicting as disclosed by Jobanputra et al. in the apparatus disclosed by Park in view of Burgoyne et al. in order to compensate for any movement by the object.
Regarding claim 136, Park in view of Burgoyne et al. in view of Jobanputra et al. discloses all of the limitations as previously discussed with respect to claims 73 and 135 including that wherein the control circuitry is further configured to cause display of a trajectory overlay within the zoomed-in view, wherein the trajectory overlay includes indication of the predicted movement path of the object (Burgoyne et al.: paragraph [0036] – system 100 includes orientation sensor(s) 110 for detecting orientation and/or movement of the system 100 and/or display(s) 120 – for example, system 100 uses orientation sensor(s) 110 to track changes in the position and/or orientation of system 100 and/or display(s) 120, such as with respect to physical objects in the real environment – orientation sensor(s) 110 optionally include one or more gyroscopes and/or one or more accelerometers; paragraph [0056] – Figs. 2E-2F also illustrate an example of a dolly zoom effect technique – according to the dolly zoom technique, the system 200 reduces the magnification and enlarges the field of view within region 206 as the position moves forward (e.g., translates) along the center of the field of view – in some embodiments, system 200 changes the magnification, field of view, and position simultaneously; paragraph [0057] – in some embodiments, system 200 performs the dolly zoom such that the determined focus object remains displayed at the same size throughout the dolly zoom – in the illustrated example, system 200 determines deer 208 to be the focus object and performs the dolly zoom such that the displayed size of the deer 208 remains the same; Burgoyne et al. discloses the dolly effect, which would take the accelerometer into consideration when determining the level of zoom required based on the change of position of the user; Jobanputra et al.: Fig. 26; paragraph [0059] – the tracking system 140 may communicate with the motion planning system 130, for example, to maneuver the UAV 100 based on measured, estimated, and/or predicted positions, orientations, and/or trajectories of objects in the physical environment – for example, the tracking system 140 may communicate a navigation objective to the motion planning system 130 to maintain a particular separation distance to a tracked object that is in motion; paragraph [0060] – the tracking system 140, operating separately or in conjunction with the motion planner 130, is further configured to generate control commands configured to cause one or more stabilization/tracking devices 152 to adjust an orientation and/or position of any image capture devices 114/115 relative to the body of the UAV 100 based on the motion of the UAV 100 and/or the tracking of one or more objects – such stabilization/tracking devices 152 may include a mechanical gimbal or a hybrid digital-mechanical gimbal, as previously described – for example, while tracking an object in motion relative to the UAV 100, the tracking system 140 may generate control commands configured to adjust an orientation of an image capture device 115 so as to keep the tracked object centered in the field of view (FOV) of the image capture device 115 while the UAV 100 is in motion – similarly, the tracking system 140 may generate commands or output data to a digital image processor (e.g., that is part of a hybrid digital-mechanical gimbal) to transform images captured by the image capture device 115 to keep the tracked object centered in the FOV of the image capture device 115 while the UAV 100 is in motion; paragraph [0189] – Fig. 26 shows an example view 2600 of a physical environment 2610 as presented at a display of an AR device – as shown in Fig. 26, one or more augmentations may be presented to the user in the form of augmenting graphical overlays 2620a, 2622a, 2624a, 2626a, and 2620b associated with objects (e.g., bikers 2640a and 2640b) in the physical environment 2610; paragraph [0194] – in some embodiments, augmentations may include visual effects that track or interact with tracked objects – for example, Fig. 26 shows an augmenting graphical overlay 2624a in the form of a projection of a 3D trajectory (e.g., current, past, and/or future) associated with biker 2640a; paragraph [0230] – object detections – the combination of these rays and planes over time can be fused into an accurate prediction of the 3D position and velocity trajectory of the object over time).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HEATHER R JONES whose telephone number is (571)272-7368. The examiner can normally be reached Mon. - Fri.: 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached at (571)272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HEATHER R JONES/Primary Examiner, Art Unit 2481
October 3, 2026
