DETAILED ACTION
Claims 1-15 rejected under 35 USC § 103.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Diefenbaugh et al., U.S. PG-Publication No. 2019/0096023 A1 (hereinafter DIEFENBAUGH), in view of Maciocco et al., U.S. PG-Publication No. 2020/0142735 A1 (hereinafter MACIOCCO).
Claim 1
DIEFENBAUGH discloses an electronic device. ¶ 0175: A host graphics system communicated with a head-mounted display (HMD) having its own graphics engines.
DIEFENBAUGH discloses a communication module comprising communication circuitry. ¶ 0180: Transmitter and receiver modules support wired or wireless host/HMD communication.
DIEFENBAUGH discloses at least one processor, comprising processing circuitry. ¶ 0033: The processing system includes processors and graphics processors.
DIEFENBAUGH discloses memory storing instructions that, when executable by at least one the processor individually or collectively, cause the electronic device to. ¶ 0038: system memory stores instructions executed by the processor.
DIEFENBAUGH discloses connect to an augmented reality (AR) device through the communication module. ¶ 0038: The HMD supports augmented-reality applications. ¶ 0171: Real imagery and rendered virtual imagery are combined. ¶ 0176: The connected HMD reports its processing capabilities to the host.
DIEFENBAUGH discloses perform inter-device load balancing such that the AR jobs are processed in a distributed manner in the AR device and the electronic device according to processing entities of respective AR jobs. ¶ 0177: Host and HMD controllers “manage the distribution of processing” according to policies. ¶ 0176: The host selects headset operations and enables or disables features to shift execution between devices.
DIEFENBAUGH discloses based on an inter-device load balancing changing condition occurring to one of AR jobs currently processed in the AR device, reconfigure a first AR job to which the changing condition occurred, among the AR jobs currently processed in the AR device, so as to be switched and processed in the electronic device. ¶ 0179: Reaching the headset’s thermal budget causes the controller to disable headset postprocessing and “enable post processing by the host.” ¶ 0183: After headset execution and performance monitoring, the host onloads an operation and deactivates it on the headset.
DIEFENBAUGH discloses receive data related to the first AR job from the AR device and process the data. ¶ 0147: Headset tracking information is sent to the host graphics system. ¶¶ 0155-156: Time warp processing transforms a frame using the headset’s latest sensor measurements. ¶ 0174: Asynchronous time warp is among the operations subject to dynamic host/headset reassignment.
DIEFENBAUGH does not expressly disclose connecting so as to receive a basic performance preset comprising an inter-device load balancing policy and a configuration parameter related to AR jobs based on providing an AR service; and performing load-balancing based on the basic performance preset.
MAIOCCO discloses connecting so as to receive a basic performance preset comprising an inter-device load balancing policy and a configuration parameter related to AR jobs based on providing an AR service. ¶ 0087: The policy controller controls “receipt and storage of policy data” supplied by users, developers, or managers. ¶ 0030: Policies specify latency, power consumption, CPU cycle, and CPU temperature requirements. ¶ 0149: A performance threshold corresponds to the policy data and specifies minimum acceptable workload performance. ¶ 0111: Separate edge platform orchestrators exchange policy information to decide service offloading. ¶ 0022: Supported workloads include augmented reality processing.
MAIOCCO discloses performing load-balancing based on the basic performance preset. ¶ 0087: The controller queries received policies and compares them with telemetry when deciding whether to offload a workload.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the policy provisioning for host/headset job allocation of DIEFENBAUDH to incorporate the receipt and storage of workload allocation policies and associated performance or power parameters taught by MACIOCCO. One of ordinary skill in the art would be motivated to integrate that policy provisioning into DIEFENBAUGH, with a reasonable expectation of success, in order to select execution resources that supplied performance requirements or reduce power consumption, thereby maintaining workload quality or saving batter life, as taught by MACIOCCO ¶¶ 87, 0148-0155.
Claim 2
DIEFENBAUGH discloses wherein the AR jobs comprise at least one of a head tracking job, a hand tracking job, an eye tracking job, and a scene analysis or rendering job. ¶ 0171: Virtual images are “rendered by a 3D graphics pipeline” and combined with real imagery to provide augmented reality. ¶ 0175: The headset graphics engines may implement the disclosed graphics techniques and provide resources for dynamically distributing graphics processing between the host and headset.
Claim 10
Claim 10 is rejected utilizing the rationale for claim 1; the claim is directed to a method performed by the system.
Claims 3-5 and 11-13 are rejected under 35 U.S.C. 103 as being unpatentable over DIEFENBAUGH, in view of MACIOCCO, further in view of Hui et al., U.S. PG-Publication No. 2017/0004019 A1 (hereinafter HUI).
Claim 3
DIEFENBAUGH discloses collect first device performance data. ¶ 0181: The host GPU monitor measures processor load and detects increases that could compromise rendering time.
DIEFENBAUGH discloses while performing the AR service, and receive second device performance data from the AR device. ¶ 0183: Following activation and execution of headset operations, headset monitors “send performance characteristics to the host.” ¶ 0181: Host and headset processing loads are monitored during operation.
DIEFENBAUGH discloses monitor whether the load balancing changing condition is changed with regard to at least one of the AR jobs, based on the first device performance data, the second device performance data. ¶ 0178: Allocation controllers use policies and “feedback received from different monitors on the host and HMD.” ¶ 0181: Increasing host load triggers movement of processing to headset; headset processing may be disabled when the host can handle the workload again. ¶ 0183: The host compares reported headset performance with offload policies before moving an operation to the host.
DIEFENBAUGH does not expressly disclose a sensor module, comprising at least one sensor; collect AR use environment data through the sensor module, and monitor whether the load balancing changing condition is changes with regard to the AR use environment data.
HUI discloses a sensor module, comprising at least one sensor. ¶ 0041: the collected context includes location information generated using GPS1. ¶ 0036.
HUI discloses collect AR use environment data through the sensor module. ¶ 0038. Profiling components collect mobile network conditions, application information, and the local device’s current load. ¶ 0040: Context gathering is “overall and collective,” including centralized collection stage. ¶ 0041: Network context includes network types, bandwidth, and GPS location information. ¶ 0044: Runtime interfaces and registered listeners obtain network conditions, processor workloads, and battery usage.
HUI discloses monitor whether the load balancing changing condition is changes with regard to the AR use environment data. ¶ 0037: The system monitors changes in network, device, and application conditions and allocates tasks using current context. ¶ 0048: The network profiler evaluates transmission delay, while the device profiler determines available resources and processor usage for partition decisions.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the performance monitoring and allocation controller of the combination of DIEFENBAUGH-MACIOCCO to incorporate the runtime collection of environment context and its use in task allocation taught by HUI. One of ordinary skill in the art would be motivated to integrate that environment profiling and evaluation into the combination of DIEFENBAUGH-MACIOCCO, with a reasonable expectation of success, in order to reduce AR task delay by selecting processing resources according to current device and communication conditions, as taught by HUI ¶¶ 0033, 0037, and 0048.
Claim 4
DIEFENBAUGH discloses wherein the second device performance data comprises at least one of the AR device GPU clock and CPU load, the AR device GPU clock and GPU load, the AR device display frequency, the AR device operating temperature, the AR device refresh rate, the AR device screen brightness, the AR device limit temperature, network traffic, tracking frequency, tracking coordinates, tracking latency or tracking accuracy, tracking input image quality, scene analysis result, and scene analysis accuracy. ¶ 0179: Thermal budget monitor 2148 monitors “the temperature of the HMD” or its components and sends an alert when the thermal budget is reached.
DIEFENBAUGH discloses wherein the AR use environment data comprises at least one of … the AR device position information. ¶ 0166: Motion, acceleration, and other sensor data are used to “predict the position and orientation” of the user’s head and eyes.
HUI discloses wherein the first device performance data comprises at least one of the electronic device GPU clock and CPU load, the electronic device GPU clock and GPU load, the network type, the type of a tethering network between electronic device and the AR device, tethering network signal intensity, rendering frame per second (FPS), the electronic device surface temperature, the electronic device limit temperature, and electronic device resolution. ¶ 0041: The profiler obtains network context including network types such as 3G, Wifi, Wimax, and Bluetooth.
HUI discloses wherein the AR use environment data comprises at least one of the electronic device position information and the AR device position information, error process information during an AR service, AR app use time, and peripheral environment information. ¶ 0041: Network context also includes location information generated using GPS. ¶ 0040: Context is gathered collectively before tasks are partitioned between processing resources. ¶ 0037: The system monitors context changes and allocates tasks according to current context.
Claim 5
DIEFENBAUGH discloses based on a change return condition occurring to the first AR job currently processed in the electronic device, reconfigure the first AR job so as to be processed in the AR device again. ¶ 0184: The host executes an operation, evaluates operating conditions, and controls reassignment and activation on the headset. ¶ 0182: A headset backlog causes the controller to enable processing on the host “until the HMD has reduced its backlog.” ¶ 0176: The host dynamically enables and disables processing features to shift their execution between the host and headset in response to operating conditions. ¶ 0184: The host offloads an operation to the headset, activates it there, and the headset executes it.
DIEFENBAUGH discloses transfer data related to the currently processed first AR job to the AR device. ¶ 0180: When enabling headset processing, the host changes the transmitted data format and sends “undistorted frames” and “sensor metadata” needed for the operation.
Claims 11-13
Claims 11-13 are rejected utilizing the rationale for claims 3-5; the claims are directed to a method performed by the system.
Claims 6 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over DIEFENBAUGH, in view of MACIOCCO, further in view of HUI, further in view of Ehsan et al., U.S. PG-Publication No. 2018/0157315 A1 (hereinafter EHSAN).
Claim 6
HUI discloses estimate AR context corresponding to a current situation, based on a combination of the electronic device, the AR device, and an AR app. ¶ 0038: Device, network, and application profilers collectively obtain current operating information; the decision component determines the appropriate resource under the circumstances. ¶ 0042: Application context includes previous execution times, hardware requirements, and application urgency. ¶ 0040: Context gathering is “overall and collective” before task partitioning.
HUI discloses perform performance evaluation corresponding to the estimated AR context, based on the first device performance data, the second device performance data, and the AR use environment data. ¶ 0048: The network profiler calculates transmission delay, while the device profiler determines available resources and processor usage. The decision procedure may match the current situation to performance history.
HUI does not expressly disclose generate a personalized performance reset by adjusting at least a part of configuration parameters of the basic performance preset, based on the performance evaluation result; and apply the generated personalized performance preset to the AR service.
EHSAN discloses generate a personalized performance reset by adjusting at least a part of configuration parameters of the basic performance preset, based on the performance evaluation result. ¶ 0045: Recorded operating conditions and throttling events are used to “iteratively adjust power supply thresholds” for an application use case. ¶ 0049: The manager selects records approximating the active use case and uses them to adjust maximum power settings; it may interpolate between records. ¶ 0051: The manager uses temperature, power, and latency information to select performance settings whose adjustment offers the desired power and user experience tradeoff.
EHSAN discloses apply the generated personalized performance preset to the AR service. ¶ 0074: The manager sets or caps the maximum power setting and monitors application execution under the resulting conditions. ¶ 0055: The manager may reduce the active eye buffer resolution to save GPU power while preserving CPU processing of time warp.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the performance configuration control of DIEFENBAUGH-MACIOCCO-HUI to incorporate the adjustment and application of existing performance settings based on evaluation of the current application use case taught by EHSAN. One of ordinary skill in the art would be motivated to integrate that adjustment procedure into DIEFENBAUGH-MACIOCCO-HUI, with a reasonable expectation of success, in order to avoid thermal throttling and resulting frame drops while maintaining consistent application performance, as taught by EHSAN ¶¶ 0041 and 0045.
Claim 14
Claim 14 is rejected utilizing the rationale for claim 6; the claim is directed to a method performed by the system.
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over DIEFENBAUGH, in view of MACIOCCO, further in view of HUI, further in view of North et al., U.S. PG-Publication No. 2022/0350720 A1 (hereinafter NORTH).
Claim 7
MACIOCCO discloses wherein the communication module is connected to a server. ¶ 0040: The cloud environment includes servers that execute centralized application and respond to client requests. ¶ 0041: The cloud service facilitates generation and retrieval of capability and policy data associated with the cloud, edge, and endpoint environments.
NORTH discloses wherein the instructions cause the electronic device to: perform control such that the electronic device attribute information, the AR device attribute information, and an AR app attribute information are transmitted to the server device through the communication module. ¶ 0052: The client reporting engine identifies the application and retrieves the client device configuration. ¶ 0053: The reporting engine communicates through the client and server communication systems so that the server identifies the client configuration and the application the client is configured to provide.
NORTH discloses a basic performance preset appropriate for a combination of at least one of the electronic device, the AR device, and the AR app is received from the server device. ¶ 0054: The server identifies the application, obtains its associated performance profile, and generates configuration changes from that profile. ¶ 0055: The server compares the target device configuration with the application profile and identifies settings to change; profile selection may depend on the current operating mode. ¶ 0056: The returned changes may concern CPU, GPU, network, operating system, and application settings. ¶ 0057: The server transmits the configuration communication through the network, and the client receives it through its communication system. ¶ 0060: The client may “automatically implement recommended settings changes.”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the preset provisioning mechanism of DIEFENBAUGH-MACIOCCO-HUI to incorporate the reporting of device and application configuration and server delivery of corresponding settings taught by NORTH. One of ordinary skill in the art would be motivated to integrate that reporting and provisioning into DIEFENBAUGH-MACIOCCO-HUI, with a reasonable expectation of success, in order to select settings suited to the actual processing configuration and application, thereby improving application performance without consuming resources in way that impair overall execution, as taught by NORTH ¶¶ 0003, 0049, and 0062.
Claim 8
NORTH discloses control the electronic device to transmit the first device performance data, the second device performance data, and the AR use environment data to the server device. ¶ 0041: The client reporting engine retrieves performance and configuration information and transmits it through its communication system to the optimization system. ¶ 0042: Reported information includes the “percentage of time the application was used,” power consumption, and changes in operating state. ¶ 0043: The information may be transmitted during operation rather than only in periodic batches.
NORTH discloses receive a personalized performance preset appropriate for a combination of the first device performance data, the second device performance data, and the AR use environment data from the server device. ¶ 0047: The server analyzes reported performance and configuration information and generates application profiles identifying configurations that provide improved performance. ¶ 0049: Profiles differ according to the computing platform and operating mode. ¶ 0055: The server compares the target configuration with the application profile and selects changes appropriate to the target’s current operating mode. ¶ 0057: The server transmits the resulting configuration communication, which the client receives through tis communication system.
Claims 9 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over DIEFENBAUGH, in view of MACIOCCO, further in view of HUI, further in view of North et al., U.S. PG-Publication No. 2022/0350720 A1 (hereinafter NORTH), further in view of Cao et al., U.S. PG-Publication No. 2021/0081760 A1 (hereinafter CAO).
Claim 9
MACIOCCO discloses communicate with the server device through the communication module.
NORTH discloses so as to classify the first device performance data, the second device performance data, and the AR use environment data into data for performance evaluation. ¶ 0041: The client transmits performance and configuration information to the optimization system, optionally formatting it to facilitate comparison. ¶ 0046: The server organizes information “into groups for the purposes of performing the analysis,” including application startup, gameplay, and concurrent application activity. ¶ 0047: The server analyzes the reported information to generate application performance profiles identifying suitable configurations.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the performance data handling of DIEFENBAUGH-MACIOCCO-HUI to incorporate the server reporting and classification of operating information taught by NORTH. One of ordinary skill in the art would be motivated to integrate that reporting and classification into DIEFENBAUGH-MACIOCCO-HUI, with a reasonable expectation of success, in order to improve application performance by deriving configuration choices from analyzed execution information, as taught by NORTH ¶¶ 0046-0047 and 0062.
MACIOCCO-NORTH does not expressly disclose calculate a performance evaluation score with regard to each AR performance evaluation factor; select at least one first parameter requiring micro-adjustment among configuration parameters included in the basic performance preset, based on the performance evaluation score; and generate the personalized performance parameter by micro-adjusting the selected at least one first parameter.
CAO discloses calculate a performance evaluation score with regard to each AR performance evaluation factor. ¶ 0067: Each performance characteristic is compared with a corresponding threshold and normalized to 1 or 0. ¶ 0068: A normalized CPU usage value identifies whether the application is CPU intensive. ¶ 0086: The application being tuned may use the same normalized performance vector as the training applications.
CAO discloses select at least one first parameter requiring micro-adjustment among configuration parameters included in the basic performance preset, based on the performance evaluation score. ¶ 0090: The model uses the current performance vector to infer parameter correlation coefficients and identify effective tunable parameters. ¶ 0093: Parameters are selected by comparing their inferred correlation coefficients with corresponding thresholds. ¶ 0059: Tunable parameters include hardware, firmware, operating system, and middleware settings affecting application performance.
CAO discloses generate the personalized performance parameter by micro-adjusting the selected at least one first parameter. ¶ 0056: Selected parameters are supplied to an automatic tuning system that obtains “optimized values.” ¶ 0058: The identification procedure may operate on a server and be used with automatic tuning in distributed applications.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the parameter selection and configuration procedure of DIEFENBAUGH-MACIOCCO-HUI-NORTH to incorporate the numerical evaluation of performance characteristics, identification of influential parameters, and automatic tuning taught by CAO. One of ordinary skill in the art would be motivated to integrate that evaluation and tuning procedure into DIEFENBAUGH-MACIOCCO-HUI-NORTH, with a reasonable expectation of success, in order to suppress insignificant measurement fluctuations and reduce the effort t and delay associated with manually selecting parameters or indiscriminately tuning parameters having little effect on application performance, as taught by CAO ¶¶ 0053-0054 and 0067.
Claim 15
Claim 15 is rejected utilizing the rationale for claims 7 and 9; the claim is directed to a method performed by the system.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See Hutten et al., US PG-Publication No. 2022/0182467 A1. HUTTEN discloses distributing AR workloads between local and remote processors based on processing capacity and latency requirements, monitoring processor resource consumption, and sending a workload to a remote processor when the local processor fails to satisfy a workload parameter (¶¶ 0045-0054, 0065-0067).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FRANK D MILLS whose telephone number is (571)270-3194. The examiner can normally be reached M-F 9-5:30 CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KEVIN YOUNG can be reached at (571)270-3180. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FRANK D MILLS/Primary Examiner, Art Unit 2194 September 19, 2026
1 Compare with present specification ¶¶ 0089, 0098, 0120, 0230: AR use environment data includes “GPS-based position information”
Prosecution Insights