DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The present application is being examined under the claims filed 12/24/2025. Claims 18 and 29 have been canceled. Claims 30 and 31 have been added. Claims 1-17, 19-28, and 30-31 are pending. Claims 1-17, 19-28, and 30-31 are rejected.
Response to Arguments
I. Applicant's arguments filed 12/24/2025 have been fully considered but they are not persuasive.
II. Regarding claim 1, applicant argues that “Chen's policies applied by the execution module to directly control the components is not the same as ‘placing a restriction on at least one of the plurality of subsystems based on the identified user context,’ as recited in claim 1, which would allow the components to operate independently within the restriction.” Additionally, applicant argues that Peri does not cure the deficiencies of Chen because “Peri's operational mode directly places the hardware components in a fixed state.”
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., “the components to operate independently within the restriction”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
III. Regarding claim 11, applicant argues “Chen's evaluation of a ‘real-time power consumption deterioration’ is, at best, based on an estimated power consumption level for a ‘predicted’ user context, but not ‘based on a current battery level and the estimated power consumption level for the predicted user context,’ as recited in amended claim 11.” Examiner respectfully disagrees.
Chen’s deterioration evaluation trigger is based on both the current battery level and the power consumption in the current user context, because the trigger is based on the current battery level being outside a range and the range is determined by the amount of power consumed in the current context (Chen e.g., par. 127, trigger if minimum available power in a current time segment is 80%, minimum available power in a next time segment is 20%, and a power change is 60%; and Chen par. 129, historical statistics, a current location, and a current application scene [i.e., estimated power consumption level for the predicted context] is used to determine the power level at which to send the alarm).
IV. Regarding claim 11, applicant argues “Chen does not disclose any localization subsystem, let alone, "limiting resource usage of the localization algorithm." Examiner respectfully disagrees.
Chen discloses utilizing GPS data to determine the location of the device (Chen par. 91, GPS location information obtained through collection is clustered, a home location and an office location are identified based on geographical locations that appear most frequently at different times, a business trip, a tourist trip, or the like is determined based on a current time [i.e., location is determined using an algorithm run on GPS data]) and limiting resource usage [of background application management and computer resource scheduling] based on a current battery level and the estimated power consumption level for the predicted user context (Chen par. 228, deterioration alarm causes power draw adjustment through new scheduling policy [see par. 217, device’s goal is to maintain 20% charge to avoid entering a power saving mode, i.e., trigger deterioration alarm based on current battery level]; and Chen par. 159-160, power consumption deterioration alarm [i.e., based on battery level] triggers new scheduling policy selection for device, which may include a CPU/GPU/DDR frequency, computing resource scheduling, background application management and control, peripheral device management and control [i.e., limiting resource usage, see e.g., par. 242, an increase and a decrease in a frequency of a big core resource are possible search directions for a policy to implement and par. 243, information about a resource power consumption table may also be used as input for filtering the resource scheduling policies]; and par. 132, power consumption deterioration alarm sent based on estimated power consumption). Peri is used to further teach limiting resource usage of the localization algorithm (Peri par. 242, op-mode engine 510 can control to turn off all unused hardware components 506 (e.g., the camera 506 a, the sensors 506 b, etc.) in order to save power [i.e., resource usage is limited by disabling the components]; and Peri par. 96, each op-mode can define power and performance quality management for multiple tasks 514 related to image rendering and/or computer vision calculations in the XR device 508 (e.g., pose, planes, hands, gestures, objects, and the like) [i.e., localizing the device in reference to the environment, localization algorithm]; and Peri par. 104, op-mode engine 510 can detect a change of the real-time system performance of the XR device 508, such as a change in battery level… in response, the op-mode engine 510 changes the current op-mode 601 to a different op-mode 601 that is compliant with the detected change [i.e., change mode based on battery level to different mode).
V. Regarding amended claim 20, applicant argues that Chen nor Peri disclose a gaze tracking subsystem. While Corson has been added to address specifically a device that explicitly comprises a gaze tracking subsystem, Peri also discloses tracking the user’s gaze (Peri par. 91, heads up display (HUD) elements [see par. 102, HUD mode is an op-mode i.e., user context] are elements wherein users can make small head movements to gaze or look directly at various application (app) elements without moving the HUD elements container or UI panel in the display view [i.e., gaze is tracked in HUD mode]; and Peri par. 242, op-mode engine 510 can control to turn off all unused hardware components 506 (e.g., the camera 506 a, the sensors 506 b, etc.) in order to save power [i.e., frequency is limited by disabling the components]; also see par. 101, op-mode engine 510 is also responsible for setting an appropriate op-mode 601 for the XR device 508 based on the performance or system load and available power]), as well as managing the power and performance quality of tasks based on the operation mode (Peri par. 96, each op-mode can define power and performance quality management for multiple tasks 514 related to image rendering and/or computer vision calculations in the XR device 508 (e.g., pose, planes, hands, gestures, objects, and the like)).
VI. Regarding claim 20, applicant argues “turning an "eye tracking" hardware component on or off is not the same as reducing a frequency of such component” and that reducing a frequency of the gaze tracking subsystem can still allow the gaze tracking subsystem to operate at a lower rate. Examiner respectfully disagrees.
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., “limit the frequency to still allow the gaze tracking subsystem to operate at a lower rate”) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5, 8, and 30-31 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et. al. (US 2021/0232201 A1) [previously cited] in view of Corson (US 2021/0183114 A1).
Regarding Claim 1, Chen discloses a method (Chen FIG. 2A and 2B) of operating an electronic device (Chen FIG 1., smartphone 100) having a plurality of subsystems (Chen par. 88, sub-modules [i.e., subsystems] aid in specific implementation of the functional modules [see par. 79-87 for example functional modules]; also see FIG. 11 depicting additional modules [e.g., user input 830, output 840, communications 810] and FIG. 1 with hardware 130 and applications 110 used as input to resource management and control modules), the method comprising:
with one or more cameras of the plurality of subsystems, capturing images of a scene (Chen par. 254, audio/video input module 860 is configured to input an audio signal or a video signal [i.e., the audio/visual subsystem captures video inputs], audio/video input module 860 may include a camera and a microphone);
[…];
identifying a user context based on a behavior of the plurality of subsystems (Chen FIG. 9 and par. 6 and par. 174-181, on determining 'usage scene' [i.e., user context] of the device, which uses subsystems such as location services [par. 177] and an activity manager service [par. 179]); and
with a centralized user experience manager operable to control the plurality of subsystems (Chen FIG. 3 and par. 86-88, decision-making module 250 and execution module 260 used to determine a resource management and control policy [i.e., control the plurality of subsystems] and implement the policy respectively), placing a restriction on at least one of the plurality of subsystems based on the identified user context (Chen FIG. 2 step S202 and par. 56, policy scheduling has different weighting of priorities based on usage scenes [i.e., user contexts] [see par. 73, policy affects operation of both hardware and software subsystems]; and e.g., par. 242, GPU frequency is decreased [i.e., restricted] and par. 70, severity of this decision is based on policy [gaming favors performance over maintaining temperature or conserving power]).
Chen does not explicitly disclose:
with one or more displays of the plurality of subsystems, outputting a passthrough video feed based on the captured images;
In the analogous art of operating a head-mounted device, Corson teaches:
with one or more displays of the plurality of subsystems, outputting a passthrough video feed based on the captured images (Corson FIG. 8A-2, and par. 89, HMD 102 is shown to include image capture devices 802A and 802B [i.e., video capture subsystem]… video from 802A or 802B can be presented to the user to provide the user with a “video see-through” ability while wearing the HMD 102);
Therefore, it would have been obvious of one of ordinary skill in the art, having the teachings of Chen and Corson before him, before the effective filing date of the claimed invention, to combine Chen’s method of power saving on a device containing multiple subsystems with Corson’s usage of a video capture subsystem to display a video feed to the user, the motivation being to consider subsystem usage in various device models (Corson FIG. 8A-2 and par. 86, the electronic device is a PlayStation® VR headset [also see Chen par. 57, device may be an intelligent wearable device such as smart glasses]) and provide a functional equivalent of being able to see the environment external to a head-mounted device type (Corson par. 89, video can be presented to the user to provide the user with a “video see-through” ability while wearing the device).
Regarding Claim 2, Chen in view of Corson discloses the method of claim 1, wherein identifying a user context comprises:
receiving an assertion from one or more subsystems in the plurality of subsystems (Chen FIG. 9 and par. 176-182, scene awareness module receives signals [i.e., assertion, see instant app. par. 56] from systems monitoring device location and activity [see par. 88, sub-modules [i.e., subsystems] are used to achieve module performance]); and
identifying a user context based on the received assertion (Chen par. 181, scene information is encapsulated into a usage scene represented by a scene data type [i.e., user context] (also see par. 62, the application scene includes information about the time, space, a foreground application type, and in-application usage).
Regarding Claim 3, Chen in view of Corson discloses the method of claim 1, wherein the plurality of subsystems further comprises:
application-level subsystems comprising one or more of:
a media player subsystem (Chen par. 22, user may watch videos on WeChat [e.g., par. 62, use of device may be "watching videos at home"]), a media streaming subsystem (Chen FIG. 1, applications 110 includes YouTube (a media streaming service) [also par. 22, user may watch videos on WeChat]), a multiuser communication session subsystem (Chen par. 249, device includes a communications module 810, which enables device to communicate with a communications network [i.e., multiuser] or another computer system; or par. 93, user may hold a video chat), and a voice-controlled automated assistant subsystem (Chen par. 57, device may be an intelligent voice device (for example, a smart speaker), which requires a voice controlled assistant subsystem);
processing units comprising one or more of:
a graphics rendering subsystem (Chen FIG. 1, hardware 130 includes graphics processing unit), a dynamic foveation subsystem, and a scene understanding subsystem (Chen FIG. 3, scene awareness module 210 (see par. 62-65 and 176-182)); and
sensor subsystems comprising one or more of:
an ambient light sensor, a face and body tracking subsystem, a hands tracking subsystem (Corson par. 46, using hand gestures that are made and captured by one or more cameras, it is possible to interface, control, maneuver, interact with, and participate in the virtual reality environment [i.e., hands are tracked to control the user interface]), a gaze tracking subsystem (Corson par. 97, a gaze tracking camera 912 is included in the HMD 102 to enable tracking of the gaze of the user), an audio sensor (Chen FIG. 11, audio/video input module 860 includes microphone), and a temperature sensor (Chen FIG. 3 and par. 84, heat control model 240 uses temperature of a hot zone as parameter (and par. 151, input for ambient temperature prediction is a collected temperature value)).
Regarding Claim 4, Chen in view of Corson discloses the method of claim 3 wherein identifying a user context comprises one or more of:
identifying that the media player subsystem is being used to launch an immersive mode, identifying that the multiuser communication session subsystem is being used to launch a multiuser communication session call, or identifying that the electronic device is being operated in a spatial capture or recording mode (Chen par. 93, in-application scene identification includes operation of a user on the application (referred to as a user action) and an action of a process in the application [e.g., par. 93, user identified as having a video chat [which involves spatial capture via the camera]).
Regarding Claim 5, Chen in view of Corson discloses the method of claim 1,
wherein placing a restriction on at least one of the plurality of subsystems based on the identified user context comprises one or more of limiting a frequency (Chen par. 160, policy may include a CPU/GPU/DDR frequency [see par. 242, assuming that a dominant hot zone is a GPU, a frequency of the GPU may be pertinently reduced to quickly control a temperature rise]), limiting a resolution, and disabling a function of the at least one of the plurality of subsystems based on the identified user context.
Regarding Claim 8, Chen in view of Corson discloses the method of claim 1, further comprising:
with the centralized user experience manager, performing thermal mitigation operations by adjusting one or more of the plurality of subsystems in response to a detecting rising temperature within the electronic device (Chen par. 226, a hot zone that contributes the most to the housing temperature rise is used as the dominant hot zone (for example, a GPU); and par. 242, assuming that a dominant hot zone is a GPU, a frequency of the GPU may be pertinently reduced to quickly control a temperature rise).
Regarding Claim 30, Chen in view of Corson discloses the method of claim 1, further comprising:
allowing the at least one subsystem in the plurality of subsystems to make adjustments within the restriction (Chen par. 242, an increase and a decrease in a frequency of a big core resource are possible search directions, and the search directions are determined by subsequent overall user experience [i.e., subsystem may be adjusted within restrictions based on user experience criteria]).
Regarding Claim 31, Chen in view of Corson discloses the method of claim 1, further comprising:
launching the identified user context (Chen par. 93, in-application scene identification identifies details of a scene in an application [e.g., launching a game]), wherein placing the restriction on at least one of the plurality of subsystems is performed in response to launching the identified user context (Chen par. 101, the scene awareness module 210 may be configured to identify a scene, and then the performance model 220 queries the database 301 to obtain a target performance indicator of the scene [i.e., restrictions on subsystems are based on current scene, see par. 245]).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Corson, further in view of Thompson et. al. (US 2016/0091950 A1) [previously cited].
Regarding Claim 6, Chen in view of Corson discloses the method of claim 1 further comprising:
allowing a given subsystem in the plurality of subsystems to dynamically scale […] its operation (Chen FIG. 10 and par. 246, update parameters in real-time and dynamically update the optimal policy based on the response); and
with the centralized user experience manager, setting a constraint that limits a […] behavior of the given subsystem (Chen par. 53, example in which power consumption budget [i.e., constraint] is applied to a source of heat ['hot zone', see par. 17-18] to prevent temperature rise, which is determined by heat control model 240 [see par. 84, indication sent when heat above threshold]); also see par. 242, example where a frequency of the GPU may be reduced to quickly control a temperature rise).
[Note: Chen states that the operations are controlled by policies that dictate the parameters of the subsystems (Chen par. 189, policy includes a target framerate of 57fps), which indicates autonomy of the subsystem to dynamically adjust to meet the specified policy. Additionally, the policies are able to set constraints for the behavior of the system (Chen par. 242, examples where DDR resources need to be increased [i.e., reducing constraints on DDR] or GPU may be reduced [i.e., increasing constraints on GPU])]
Chen in view of Corson does not explicitly teach:
allowing a given subsystem in the plurality of subsystems to dynamically scale back its operation;
with the centralized user experience manager, setting a constraint that limits a dynamic behavior of the given subsystem.
In the analogous art of managing operations of subsystems Thompson teaches:
allowing a given subsystem in the plurality of subsystems (Thompson FIG. 3 and par. 31, subsystem 108 [and par. 38, computing devices may have multiple subsystems]) to dynamically scale its operation (Thomson par. 22, subsystem is monitored by current detector and throttled if demand current has reached a threshold value [also see par. 24, power management system has a dynamic current range for supporting the variable current demands of the subsystem])
with the centralized manager, setting a constraint that limits a dynamic behavior of the given subsystem (Thomson FIG. 3 and par. 25, throttle signal 110 [sent by power management system 106] can be configured to cause the subsystem 108 to transition out of a current operating state, processing state, clock rate, or other suitable state to reduce a current demand of the subsystem 108 [i.e., change the dynamic behavior of the subsystem])
Therefore, it would have been obvious of one of ordinary skill in the art, having the teachings of Chen, Corson, and Thomson before them, before the effective filing date of the claimed invention, to combine Chen and Corson’s system including dynamically managing and adjusting subsystems using control policies with Thomson’s subsystems with control over their own dynamic behavior, the motivation being to respond quickly to adverse conditions (Thomson par. 20, disclosure for limiting the amount of time a subsystem of a computing device can spend at or above a threshold current limit [also see par. 36, damage to the power supply may result from being above power threshold]), while still gaining the user experience improvement of complete device context of Chen (Chen par. 3).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Corson, further in view of Fullam (US 2016/0080720 A1) [previously cited].
Regarding Claim 7, Chen in view of Corson discloses the method of claim 1.
Chen in view of Corson does not explicitly disclose:
with the centralized user experience manager, adjusting one or more of the plurality of subsystems in response to detecting a scenario indicative of user discomfort by:
monitoring gaze or breathing of a user, detecting dropped frames or unstable frame rates at a display of the electronic device, detecting movement in a periphery of a user’s field of view, or determining whether a latency associated with one or more of the plurality of subsystems exceeds a predetermined threshold level.
[Note: Chen discusses making decisions in order to maximize a user’s experience, which is partially comprised of device performance factors (Chen par. 236, user experience in a performance dimension is measured, and par. 246 user experience used to select policy), such as frame rate (Chen par. 103-108, the frame loss rate, the low frame rate percentage, and the fluency are tracked and adjusted based on policy [e.g., par 189, with target frame rate 57). Chen also notes that the embodiment of the device may be smart glasses (Chen par. 57).]
In the analogous art of adjusting device subsystem parameters (Fullam FIG. 1 and par. 14, device 12A includes sensory subsystem 26A), Fullam teaches:
with the centralized user experience manager (Fullam FIG. 1 and par. 14 and 17, controller 20A processes data from subsystems [e.g., from eye-imaging camera] and controls output to displays), adjusting one or more of the plurality of subsystems in response to detecting a scenario indicative of user discomfort (Fullam par. 27, to address the eye-discomfort issues [see par. 18-19], controller 20 may be configured to adjust one or more operating parameters of display system 12 in response to an ocular condition of a viewer of the display system [also see par. 32 for adjustable parameters]) by:
monitoring gaze or breathing of a user (Fullam par 29, output from ocular sensor 60 may indicate that the viewer is experiencing eye discomfort using frequency with which the viewer's focal point is shifted [i.e., gaze], and par. 34, sensory subsystem has a respiration sensor [i.e., breathing sensor]), detecting dropped frames or unstable frame rates at a display of the electronic device, detecting movement in a periphery of a user’s field of view, or determining whether a latency associated with one or more of the plurality of subsystems exceeds a predetermined threshold level.
Therefore, it would have been obvious of one of ordinary skill in the art, having the teachings of Chen and Fullam before them, before the effective filing date of the claimed invention, to combine Chen’s method of subsystem management taking account user experience with Fullam’s sensors and method for detecting user discomfort, the motivation being to improve the user experience by mitigating discomfort caused by the device (Fullam par. 18).
Claims 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Corson, further in view of Tavakoli et. al. (US 2018/0329465 A1) [previously cited].
Regarding Claim 9, Chen in view of Corson discloses the method of claim 8. Chen in view of Corson does not explicitly disclose:
wherein performing the thermal mitigation operations comprises adjusting thermal mitigation knobs from a list to control the plurality of subsystems, and wherein the thermal mitigation knobs in the list are ordered based on an amount of impact each of the thermal mitigation knobs has on the identified user context.
[Note: Chen generally discloses the desire to maintain optimal overall user experience with policy selection (Chen par. 159-160 and par. 244-245). Therefore, the policies implemented in Chen ideally have the least impact on user context, which is found by weighing the impacts of each factor (Chen par. 245, calculate overall user experience of each resource scheduling policy based on a weight coefficient [also see par. 229, weight coefficient of each user experience factor varies with different application scenes]). The selected policy may include the thermal mitigation operations (Chen par. 242, performance bottleneck and the dominant hot zone are provided by a performance model and a heat control model, a frequency of the GPU may be pertinently reduced to quickly control a temperature rise). While the operations are weighted based on user experience, Chen does not discuss formatting the knobs in an ordered list.]
In the analogous art of managing thermals of a head-mounted device Tavakoli teaches:
wherein performing the thermal mitigation operations comprises adjusting thermal mitigation knobs from a list to control the plurality of subsystems, and wherein the thermal mitigation knobs in the list are ordered based on an amount of impact each of the thermal mitigation knobs has on the identified user context (Tavakoli FIG. 3 and par. 42, certain performance settings may be rated “high,” “medium,” “low” or “none” for impact (power consumption vs. user experience), and the performance settings knobs may be adjusted down to provide the most impact on power consumption for the least cost on user experience [see par. 43 and 44 for example where eye buffer resolution and textures level of detail are parameters that lower load on GPU]; also see FIG. 2 and par. 27, use case affects active components and power consumption).
Therefore, it would have been obvious of one of ordinary skill in the art, having the teachings of Chen, Corson, and Tavakoli before them, before the effective filing date of the claimed invention, to combine Chen and Corson’s system for maintaining optimal overall user experience with policy selection with Tavakoli’s explicit weighing of each factor, the motivation being to have the ability to easily look up specific adjustments that have maximum impact on power consumption of a specific component and least impact on user experience (Tavakoli par. 41-44, look up table used to store data on how to adjust for maximum impact on power consumption and least impact on user experience, based on components that should be affected).
Regarding Claim 10, Chen in view of Corson, further in view of Tavakoli teaches the method of claim 9. Chen in view of Tavakoli further teaches wherein the thermal mitigation knobs in the list comprise one or more of:
a knob for limiting a frame rate of a graphics rendering subsystem in the plurality of subsystems (Tavakoli FIG. 3, showing knob for display FPS; and Chen par. 99, target performance indicators include target frame rate, a frame loss rate, a low frame rate percentage), a knob for limiting automatic speech recognition (ASR), text-to-speech (TTS), and dictation quality of a voice-controlled automated assistant subsystem in the plurality of subsystems, a knob for limiting a streaming quality of a media streaming subsystem in the plurality of subsystems, a knob for limiting a frequency of a face and body tracking subsystem in the plurality of subsystems (Tavakoli FIG. 3, showing knob for 6DOF camera FPS), a knob for limiting a frequency of a hands tracking subsystem in the plurality of subsystems, and a knob for limiting a frequency of a gaze tracking subsystem in the plurality of subsystems.
Claims 11-17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Peri et. al. (US 2022/0382356 A1) [previously cited].
Regarding Claim 11, Chen discloses a method (Chen FIG. 2A and 2B) of operating an electronic device (Chen FIG 1., smartphone 100) having a plurality of subsystems (Chen par. 88, sub-modules [i.e., subsystems] aid in specific implementation of the functional modules [see par. 79-87 for example functional modules]; also see FIG. 11 depicting additional modules [e.g., user input 830, output 840, communications 810] and FIG. 1 with hardware 130 and applications 110 [i.e., hardware and software subsystems] used as input to resource management and control modules), the method comprising:
with a localization subsystem of the plurality of subsystems (Chen par. 91, GPS location is utilized [i.e., localization subsystem]), running a localization algorithm (Chen par. 91, GPS location information obtained through collection is clustered, a home location and an office location are identified based on geographical locations that appear most frequently at different times, a business trip, a tourist trip, or the like is determined based on a current time [i.e., location is determined using an algorithm run on GPS data]);
monitoring one or more states of the plurality of subsystems (Chen par. 122, statistics on remaining power of the user at each time point are collected; Chen par. 119, user, usage duration, a charging start moment and a charging end moment, a charging location, average power consumption of the application, and other information, and is stored persistently in a file or database form; Chen FIG. 9 and par. 175-181, usage scene of the user (that is, an application scene of the mobile phone) is detected using subsystems depicted in FIG. 9);
predicting a user context based on the monitored states of the plurality of subsystems (Chen par. 122, application usage rule [i.e., the predicted user context based on historical usage] is calculated using applications commonly used by the user in different locations and different time segments [e.g., average usage duration, average power consumption of the applications, and the user location]);
estimating a power consumption level for the predicted user context (Chen par. 132, usage duration of a foreground application and estimated power consumption in a next time segment are predicted using the application usage rule [i.e., predicted user context] and the current foreground application); and
with a centralized user experience manager operable to control the plurality of subsystems (Chen FIG. 3 and par. 86-88, decision-making module 250 and execution module 260 used to determine a resource management and control policy [i.e., control device subsystems] and implement the policy respectively; and par. 172, awareness modules monitor and report subsystem states to combined scheduling center), limiting resource usage [of background application management and computer resource scheduling] based on a current battery level and the estimated power consumption level for the predicted user context (Chen par. 228, deterioration alarm causes power draw adjustment through new scheduling policy [see par. 217, device’s goal is to maintain 20% charge to avoid entering a power saving mode, i.e., trigger deterioration alarm based on current battery level]; and Chen par. 159-160, power consumption deterioration alarm [i.e., based on battery level] triggers new scheduling policy selection for device, which may include a CPU/GPU/DDR frequency, computing resource scheduling, background application management and control, peripheral device management and control [i.e., limiting resource usage, see e.g., par. 242, an increase and a decrease in a frequency of a big core resource are possible search directions for a policy to implement and par. 243, information about a resource power consumption table may also be used as input for filtering the resource scheduling policies]; and Chen e.g., par. 127, trigger if minimum available power in a current time segment is 80%, minimum available power in a next time segment is 20%, and a power change is 60%; and Chen par. 129, historical statistics, a current location, and a current application scene [i.e., estimated power consumption level for the predicted context] is used to determine the power level at which to send the alarm).
Chen does not explicitly disclose:
limiting resource usage of the localization algorithm based on a current battery level and the estimated power consumption level for the predicted user context.
In the analogous art of managing power in an electronic device by modifying subsystem behavior based on user context, Peri teaches:
limiting resource usage of the localization algorithm based on a current battery level and the estimated power consumption level for the predicted user context (Peri par. 242, op-mode engine 510 can control to turn off all unused hardware components 506 (e.g., the camera 506 a, the sensors 506 b, etc.) in order to save power [i.e., resource usage is limited by disabling the components]; and Peri par. 96, each op-mode can define power and performance quality management for multiple tasks 514 related to image rendering and/or computer vision calculations in the XR device 508 (e.g., pose, planes, hands, gestures, objects, and the like) [i.e., localizing the device in reference to the environment, localization algorithm]; and Peri par. 104, op-mode engine 510 can detect a change of the real-time system performance of the XR device 508, such as a change in battery level… in response, the op-mode engine 510 changes the current op-mode 601 to a different op-mode 601 that is compliant with the detected change [i.e., change mode based on battery level to different mode]).
Therefore, it would have been obvious of one of ordinary skill in the art, having the teachings of Chen and Peri before him, before the effective filing date of the claimed invention, to combine Chen’s method of power saving on a device containing multiple subsystems with Peri’s limiting resource usage of an algorithm, the motivation being to conserve additional power based on the use case of the device (Peri par. 33 and 110).
Regarding Claim 12, Chen in view of Peri discloses the method of claim 11, further comprising:
maintaining historical data of past monitored states of the plurality of subsystems (Chen FIG. 4 and par. 12 and 98, database 301 stores correspondences between application scenes and target performance indicators; and par. 118-119, collected and recorded information includes an application used by the user, usage duration, a charging start moment and a charging end moment, a charging location, average power consumption of the application, and other information, and stored persistently in a file or database form).
Regarding Claim 13, Chen in view of Peri discloses the method of claim 11, further comprising:
monitoring a usage of one or more applications running on the electronic device (Chen par. 118-119, collected data includes applications used by the user and usage duration); and
predicting the user context based on the monitored usage of one or more applications running on the electronic device (Chen par. 122, application usage rule [i.e., the predicted user context based on historical usage] is calculated using applications commonly used by the user in different locations and different time segments [e.g., average usage duration, average power consumption of the applications, and the user location]).
Regarding Claim 14, Chen in view of Peri discloses the method of claim 13, further comprising:
maintaining historical data of past monitored usage of one or more applications running on the electronic device (Chen par. 118-119, collected data includes applications used by the user and usage duration and is stored persistently in a file or database form).
Regarding Claim 15, Chen in view of Peri discloses the method of claim 11,
wherein predicting the user context comprises predicting the user context based on the monitored states of the plurality of subsystems using a machine learning model trained on data associated with a plurality of users (Chen par. 21, machine learning or a dynamic planning method is used in the process of determining the resource management and control policy; and Chen par. 100, database with optimal policies found by clustering reported big data [i.e., plurality of users]).
Regarding Claim 16, Chen in view of Peri discloses the method of claim 11, further comprising:
predicting the user context based on calendar information (Chen FIG. 9 and par. 91 and 176, system calendar is queried to contribute to scene information; and par. 132, usage duration of a foreground application and estimated power consumption [i.e., user context] in a next time segment are predicted using the application usage rule and the current foreground application [see par. 122, application usage rule accounts for applications commonly used by the user in different locations and different time segments).
Regarding Claim 17, Chen in view of Peri discloses the method of claim 11, further comprising:
deactivating the localization algorithm to conserve charge on a battery in the electronic device (Peri par. 96, each op-mode can define power and performance quality management for multiple tasks 514 related to image rendering and/or computer vision calculations in the XR device 508 (e.g., pose, planes, hands, gestures, objects, and the like) [i.e., localizing the device in reference to the environment, localization algorithm]; and Peri par. 242, op-mode engine 510 can control to turn off all unused hardware components 506 (e.g., the camera 506 a, the sensors 506 b, etc.) in order to save power [i.e., resource usage is limited by disabling the components]; also see Peri par. 104, op-mode engine 510 can detect a change of the real-time system performance of the XR device 508, such as a change in battery level… in response, the op-mode engine 510 changes the current op-mode 601 to a different op-mode 601 that is compliant with the detected change).
Regarding Claim 19, Chen in view of Peri discloses the method of claim 11, further comprising:
with the centralized user experience manager, predicting a time when the electronic device will be charged to charge a battery in the electronic device (Chen par. 209-211, method to predict time of the user from current charging to next charging); and
with the centralized user experience manager, selectively adjusting at least one of the plurality of subsystems based on the predicted time of when the electronic device will be charged (Chen par. 217, amount of charge needed between charging intervals is calculated, trigger a deterioration alarm if insufficient charge [see par. 71 and 228, deterioration alarm indicates that the current resource management and control policy cannot meet overall user experience and new resource management and control policy starts to be determined] [i.e., create new policy to readjust subsystems]).
Claims 20-22, 25, and 27-28 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Corson and Peri.
Regarding Claim 20, Chen discloses a method (Chen FIG. 2A and 2B) of operating an electronic device (Chen FIG 1., smartphone 100) having a plurality of subsystems (Chen par. 88, sub-modules [i.e., subsystems] aid in specific implementation of the functional modules [see par. 79-87 for example functional modules]; also see FIG. 11 depicting additional modules [for example, user input 830, output 840, communications 810] and FIG. 1 with hardware 130 and applications 110 used as input to resource management and control modules) and a battery (Chen par. 122-126, device is charged by the user; Chen par. 127, referencing charge level of 60%), the method comprising:
[…];
monitoring one or more states of the plurality of subsystems (Chen par. 122, statistics on remaining power of the user at each time point are collected; Chen par. 119, user, usage duration, a charging start moment and a charging end moment, a charging location, average power consumption of the application, and other information, and is stored persistently in a file or database form; Chen FIG. 9 and par. 175-181, usage scene of the user (that is, an application scene of the mobile phone) is detected using subsystems depicted in FIG. 9);
identifying a user context based on the monitored states of the plurality of subsystems (Chen FIG.9, par. 90-93 and par. 175-181, determine the scene [i.e., user context] using spatial-temporal awareness of the user, application type awareness, and in-application scene identification [i.e., monitored states]);
estimating a power consumption level of the identified user context (Chen par. 117, power consumption model is used to evaluate a degree of impact on the standby duration or available power based on the current application scene and the current application); and
with a centralized user experience manager operable to control the plurality of subsystems (Chen FIG. 3 and par. 86-88, decision-making module 250 and execution module 260 used to determine a resource management and control policy [i.e., control device subsystems] and implement the policy respectively; and par. 172, awareness modules monitor and report subsystem states to combined scheduling center), adjusting power drawn from the battery by adjusting at least one of the plurality of subsystems reducing a frequency of [a] subsystem based on the estimated power consumption level of the identified user context (Chen par. 228, deterioration alarm causes power draw adjustment through new scheduling policy [see par. 217, device’s goal is to maintain 20% charge to avoid entering a power saving mode based on user context, i.e., trigger deterioration alarm based on current battery level]; and Chen par. 159-160, power consumption deterioration alarm [i.e., based on battery level and identified user context] triggers new scheduling policy selection for device, which may include a CPU/GPU/DDR frequency, computing resource scheduling, background application management and control, peripheral device management and control [i.e., limiting resource usage, see e.g., par. 242, an increase and a decrease in a frequency of a big core resource are possible search directions for a policy to implement).
Chen does not explicitly teach:
with a gaze tracking subsystem of the plurality of subsystems, gathering gaze information;
adjusting power drawn from the battery by reducing a frequency of the gaze tracking subsystem based on the estimated power consumption level of the identified user context.
In the analogous art of operating a head-mounted device, Corson teaches:
with a gaze tracking subsystem of the plurality of subsystems, gathering gaze information (Corson par. 97, a gaze tracking camera 912 is included in the HMD 102 to enable tracking of the gaze of the user [i.e., gaze tracking subsystem]… gaze tracking camera captures images of the user's eyes [i.e., gaze information], which are analyzed to determine the gaze direction of the user);
adjusting power drawn from the battery by [adjusting] a frequency of the gaze tracking subsystem […] (Corson par. 97, a user's eyes are determined to be looking in a specific direction, then the video rendering for that direction can be prioritized or emphasized, such as by providing greater detail or faster updates in the region where the user is looking [i.e., greater detail video rendering in one area is an increased frequency in the gaze tracking subsystem]).
Therefore, it would have been obvious of one of ordinary skill in the art, having the teachings of Chen and Corson before him, before the effective filing date of the claimed invention, to combine Chen’s method of power saving on a device containing multiple subsystems with Corson’s usage of a gaze subsystem, the motivation being to consider subsystem usage in various device models (Corson FIG. 8A-2 and par. 86, the electronic device is a PlayStation® VR headset [also see Chen par. 57, device may be an intelligent wearable device such as smart glasses]) and optimize resource usage based on the user’s gaze direction (Corson par. 97, a user's eyes are determined to be looking in a specific direction, then the video rendering for that direction can be prioritized or emphasized, such as by providing greater detail or faster updates in the region where the user is looking).
Chen in view of Corson does not explicitly disclose:
adjusting power drawn from the battery by reducing a frequency of the gaze tracking subsystem based on the estimated power consumption level of the identified user context.
In the analogous art of managing power in an electronic device by modifying subsystem behavior based on user context, Peri teaches:
adjusting power drawn from the battery by reducing a frequency of the gaze tracking subsystem based on the estimated power consumption level of the identified user context (Peri par. 91, heads up display (HUD) elements [see par. 102, HUD mode is an op-mode i.e., user context] are elements wherein users can make small head movements to gaze or look directly at various application (app) elements without moving the HUD elements container or UI panel in the display view [i.e., gaze is tracked in HUD mode]; and Peri par. 242, op-mode engine 510 can control to turn off all unused hardware components 506 (e.g., the camera 506 a, the sensors 506 b, etc.) in order to save power [i.e., frequency is limited by disabling the components]; also see Peri par. 104, op-mode engine 510 can detect a change of the real-time system performance of the XR device 508, such as a change in battery level… in response, the op-mode engine 510 changes the current op-mode 601 to a different op-mode 601 that is compliant with the detected change [i.e., change mode based on battery level to different mode, also see par. 101, op-mode engine 510 is also responsible for setting an appropriate op-mode 601 for the XR device 508 based on the performance or system load and available power]).
Therefore, it would have been obvious of one of ordinary skill in the art, having the teachings of Chen, Corson, and Peri before him, before the effective filing date of the claimed invention, to combine Chen and Corson’s method of power saving on a device containing multiple subsystems with Peri’s adjusting of a gaze tracking subsystem, the motivation being to conserve additional power based on the use case of the device (Peri par. 33 and 110).
Regarding Claim 21, Chen in view of Corson and Peri discloses the method of claim 20, further comprising:
gathering information with one or more sensors in the plurality of subsystems (Chen par. 88, sub-modules [i.e., subsystems] aid in specific implementation of the functional modules [see par. 79-87 for example functional modules]; also see FIG. 11 depicting additional modules [for example, user input 830, output 840, communications 810] and FIG. 1 with hardware 130 and applications 110 used as input to resource management and control modules; or Peri par. 33, AR devices have a number of sensors and solutions that allow the device to perform 6 degree of freedom (DoF) head and hand tracking, fully map the environment, as well as perform AI based object, face and sometimes body detection);
adjusting power drawn from the battery by reducing a frequency of the one or more sensors based on the estimated power consumption level of the identified user context (Peri par. 108, op-mode engine 810 facilitates switching of op-modes based on environmental factors [e.g., automatic switch at par. 111 due to surrounding lighting condition, i.e., user context]; and Peri par. 112, 6-DoF pose solution and sensors will be limited or turned off in response to media consumption use case [to save power i.e., adjust power drawn from the battery, see par. 110] and par. 34, increase the tracking resolution [i.e., frequency] is the opposite of saving power, so decreasing tracking resolution saves power based on mode [also see FIG. 6 for modes]);
Regarding Claim 22, Chen in view of Corson and Peri discloses the method of claim 20, further comprising:
running an algorithm for processing the gaze information (Corson par. 97, a gaze tracking camera 912 is included in the HMD 102 to enable tracking of the gaze of the user [i.e., gaze tracking subsystem]… gaze tracking camera captures images of the user's eyes [i.e., gaze information], which are analyzed to determine the gaze direction of the use [i.e., algorithm for processing the gaze information]; also see Peri par. 91, heads up display (HUD) elements are elements wherein users can make small head movements to gaze or look directly at various application (app) elements [i.e., gaze information must be processed to use for HUD]); and
deactivating or limiting resource usage of the algorithm for processing the gaze information (Peri par. 34, it is desired to only execute those systems that are required for the use case employed at the time; also see Peri par. 36, if the application is calling for a simple HUD like display, then HMD camera tracking systems can be turned off, image transfer resolution and color depth can be lowered, and hand tracking algorithms can be turned off and Peri par. 112, 6-DoF pose solution and sensors will be limited or turned off in response to media consumption use case [i.e., example of unneeded algorithm being turned off, and in a media consumption case, 6-DoF sensors (including head tracking, see par. 99 HMD optical tracking system used in 6-DoF mode)]);
Regarding Claim 25, Chen in view of Corson and Peri discloses the method of claim 20, further comprising:
controlling a display brightness of the electronic device (Chen par. 4 and 167, a display backlight is adjusted [to reduce heat]);
rendering virtual content with a graphics rendering subsystem in the plurality of subsystems (Chen FIG. 1, GPU [example virtual content includes par. 90 (hardware-decoded video) and par. 183 (game)]);
identifying one or more objects in a scene with a scene understanding subsystem in the plurality of subsystems (Chen par. 93, in-application scene identification identifies details of a scene in an application [e.g., launching a game or in choosing heroes phase]);
establishing a wireless connection with one or more external devices using a communications subsystem in the plurality of subsystems (Chen FIG. 9, fetching data for application type dimension includes “Identifying and querying on a cloud side” as a step to determining application scene); and
adjusting at least some of the plurality of subsystems based on the estimated power consumption level of the identified user context (Chen par. 132 and 217, power deterioration alarm is triggered by insufficient power determined by estimates and par. 159-160 power deterioration alarm causing a combined scheduling model to recalculate best policy [i.e., adjusting subsystems]) by performing one or more of:
reducing the display brightness of the electronic device (Chen par. 4, a display backlight is limited [i.e., reduced] due to heat; Chen par. 8, relationship between power consumption, performance, and temperature is a positive correlation);
constraining or deactivating the graphics rendering subsystem (Chen par. 197-202, weight graphics rendering factors based on the scene [see par. 204, power consumption in the game scene is faster than that in a chat scene]; Chen par. 8, relationship between power consumption, performance, and temperature is a positive correlation);
constraining or deactivating the graphics rendering subsystem (Chen par. 197-202, weight graphics rendering factors based on the scene [see par. 204, power consumption in the game scene is faster than that in a chat scene]; Chen par. 8, relationship between power consumption, performance, and temperature is a positive correlation);
constraining or deactivating the scene understanding subsystem;
limiting a connectivity of the communications subsystem; and
reducing a pull rate with which the electronic device checks for application notifications.
Regarding Claim 27, Chen in view of Corson and Peri discloses the method of claim 20, further comprising:
with the centralized user experience manager, adjusting the plurality of subsystems in accordance with a plurality of customizable user experience policies associated with different user classifications or geographical locations (Chen par. 209, charging info is affected by space information [see par. 65 and 91, space dimension [e.g.,, a home location and an office location] found using GPS location] and par. 66 power consumption is factor for control policy [i.e., power needs factor in user classifications or geographical locations, which is used to control policy]).
Regarding Claim 28, Chen in view of Corson and Peri discloses the method of claim 20, further comprising:
with one or more cameras, capturing images of a scene (Peri FIG. 4 and par. 93, embodiment 4D includes non-see-through display 440 and includes a camera or camera input configured to capture real-world information and display; also see Chen par. 254, audio/video input module 860 is configured to input an audio signal or a video signal [i.e., the audio/visual subsystem captures video inputs], audio/video input module 860 may include a camera and a microphone);
with one or more displays, outputting the captured images as a passthrough video feed (Peri par. 93, camera is configured to capture real-world information and display, via the non-see-through display 440, real-world information [e.g., par. 111, a user is examining the mechanics of a car in a garage and his device is in Mode 5 (Room MR) to support full comprehension and tracking]; also see Corson FIG. 8A-2, and par. 89, HMD 102 is shown to include image capture devices 802A and 802B);
determining whether the passthrough video feed is being displayed to a user (Peri FIG. 6 and par. 102, op-mode engine determines the application 501 has only requested Mode 1 (HUD) [which does not use passthrough capabilities, see FIG. 6 and par. 36]); and
in response to determining that the passthrough video feed is not being displayed to the user (Peri par. 36, in response to the application calling for a simple HUD like display, the HMD camera tracking systems can be turned off, image transfer resolution and color depth can be lowered, and hand tracking algorithms can be turned off [also see par. 102, in response to an application only requesting a HUD, op-mode engine 510 can control to turn off all unused hardware components 506 (e.g., the camera 506 a, the sensors 506 b, etc.)]), adjusting at least one of the plurality of subsystems associated with processing the passthrough video feed by throttling down one or more of:
a scene understanding subsystem (Peri par. 102, in response to an application only requesting a HUD, op-mode engine 510 can control to turn off all unused hardware components 506 (e.g., the camera 506 a, the sensors 506 b, etc.) and Peri par. 108, the CV system 800 includes multiple sensors 801-803, multiple software modules supporting different levels of XR tracking and scene comprehension capabilities 804-809 [see e.g., par. 109, CV system 800 turns off the sensors 801 and 802, keeps the sensor 803 active and only provides the 3-DoF tracking capability 804 in response from switching from mixed reality to desktop augmented reality]), a subsystem configured to perform point-of-view correction, and a subsystem configured to model environment lighting.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Corson and Peri, further in view of Yang (US 2016/0063850 A1) [previously cited].
Regarding Claim 23, Chen in view of Corson and Peri discloses the method of claim 20. Chen in view of Corson and Peri does not explicitly teach:
in response to estimating the power consumption level of the identified user context, operating the electronic device in a feedback mode during which the electronic device outputs only audio or haptic alerts without outputting any visual alerts.
[Note: Chen generally discloses optimizing a policy with power consumption as a parameter (Chen par. 244) and that backlight increases heat (and therefore power consumption (Chen par. 4, display backlight may be an excessive heat source [and Chen par. 8, relationship between power consumption, performance, and temperature is positive]) of the computing device. Therefore, reducing or disabling the backlight would result in power saving. Chen also notes its device has the output module 840 that may further include an audio output module, an alarm, and a tactile module (Chen par. 253).]
In the analogous art of conserving power on an electronic device, Yang teaches:
in response to estimating the power consumption level of the identified user context (Yang par. 83-84, based on the current battery level, the alert mode is selected based on the power that may be consumed during an alert output; also see par. 30, select an alert mode that conserves power or reduced peak power usage), operating the electronic device in a feedback mode during which the electronic device outputs only audio or haptic alerts without outputting any visual alerts (Yang par. 173, alert embodiment with a first audio and first haptic component, but no visual component; and par. 83-84, an alert mode is selected that uses less power as compared to some other alert modes [backlighting/visuals require power])
Therefore, it would have been obvious of one of ordinary skill in the art, having the teachings of Chen, Corson, Peri, and Yang before them, before the effective filing date of the claimed invention, to combine Chen, Corson, and Peri’s device with policy selection with consideration for power use with Yang’s audio and haptic only alerts, the motivation being to further conserve power by limiting alert components that consume a large amount of energy (Yang par. 83-84).
Claim 24 is rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Corson and Peri, further in view of Munteanu et. al. (US 2010/0085289 A1) [previously cited].
Regarding Claim 24, Chen in view of Corson and Peri discloses the method of claim 20. Chen does not explicitly disclose:
in response to estimating the power consumption level of the identified user context, operating the electronic device in a single color mode during which the electronic device displays only black and white or grayscale content.
[Note: Chen generally discusses that the display backlight increases heat (and therefore power consumption (Chen par. 4, display backlight may be an excessive heat source [and Chen par. 8, relationship between power consumption, performance, and temperature is positive]) and that the display of the device may be OLED (Chen par. 253).]
In the analogous art of implementing methods to reduce power use of a system (Munteanu par. 12 and 15, a field-sequential display can operate in one of a color mode or a grayscale mode [i.e., low-power mode, see par. 12 display controller can take advantage of the enhanced contrast provided by the grayscale image content to reduce or disable backlighting to save power]), Munteanu teaches:
in response to estimating the power consumption level of the identified user context (Munteanu par. 15, operation occurs due to signal 122 instructing change to a low-power mode [also see par. 3 on power demands of display] and par. 18, signal is triggered based on user context [e.g., minimum inactive period has occurred]), operating the electronic device in a single color mode during which the electronic device displays only black and white or grayscale content (Munteanu par. 22, in the low-power mode rather than displaying multiple-color image data, the timing controller 112 instead generates grayscale image data)
Therefore, it would have been obvious of one of ordinary skill in the art, having the teachings of Chen, Corson, Peri, and Munteanu before them, before the effective filing date of the claimed invention, to combine Chen, Corson, and Peri’s device with policy selection with consideration for power use with Munteanu’s methodology to conserve power by running a device in grayscale mode, the motivation being to increase adjustable parameters that can be used to save power (Munteanu par. 12, grayscale operation allows images to be displayed at a lower frame rate and with a dimmer backlight, saving power).
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Corson and Peri, further in view of Browy et. al. (US 2020/0380793 A1) [previously cited].
Regarding Claim 26, Chen in view of Corson and Peri discloses the method of claim 20. Chen does not explicitly teach:
with the centralized user experience manager, directing a hands tracking subsystem in the plurality of subsystems to operate at a first frequency in response to identifying a first user context based on the monitored states of the plurality of subsystems (Chen FIG. 11 and par. 252, user input module is configured to receive contactless gesture; and Peri FIG. 8 and par. 108, computer vision (CV) system [also see par. 96, XR device 508 can determine hand and gesture information]; and Peri par. 111-113, device switches modes based on user context and par. 34, instead of saving power, it is desired or necessary to increase the tracking resolution [i.e., frame rate] is changed based on mode [also see FIG. 6 for modes]);
with the centralized user experience manager, directing the hands tracking subsystem in the plurality of subsystems to operate at a second frequency greater than the first frequency in response to identifying a second user context based on the monitored states of the plurality of subsystems (Peri par. 111-113, device switches modes based on user context [four separate cases are given, for example resolution and frame rate increased from mode 2 to mode 3] and par. 34, instead of saving power, it is desired or necessary to increase the tracking resolution [i.e., frame rate] is changed based on mode [also see FIG. 6 for modes]); and
with the hands tracking subsystem, detecting a [hand gestures] during the second user context (Peri par. 96, XR device 508 can determine hand and gesture information).
Chen in view of Corson and Peri does not explicitly teach:
with the hands tracking subsystem, detecting a sign language during the second user context;
In the analogous art of electronic devices configured to provide services to the user via subsystems, Browy teaches:
with the hands tracking subsystem (Browy FIG. 9 and par. 112, hand/gestures totem), detecting a sign language during the second user context (Browy par. 139, wearable system 200 can interpret sign language by, for example, detecting gestures that may constitute sign language [also see par. 191, device mode determines whether to capture the user’s own sign language [i.e., user context]])
Therefore, it would have been obvious of one of ordinary skill in the art, having the teachings of Chen, Corson, Peri, and Browy before them, before the effective filing date of the claimed invention, to combine Chen, Corson, and Peri’s device with mode switching including gesture recognition sensors with Browy’s system for interpreting gestures as sign language, the motivation being to permit greater interaction among differently-abled persons (Browy par. 4).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to COLE JIAWEI WENTZEL whose telephone number is (703) 756-4762. The examiner can normally be reached 9:30am-5:30pm ET (Mon-Fri).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Jung can be reached on (571) 270-3779. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.J.W./Examiner, Art Unit 2175
/Paul Yen/Primary Examiner, Art Unit 2175
Prosecution Insights