Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
CLAIM INTERPRETATION
The following is a quotation of 35 U.S.C. 112(f): (FP 7.30.03)
(f) ELEMENT IN CLAIM FOR A COMBINATION.—An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as "configured to" or "so that"; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. (FP 7.30.05)
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) (Claims 1-15) is/are:
activation controller that controls activation of the different graphics processor boosting modules in claim 1-5, 7-8, 10-11, 13, 15;
information analysis module configured to analyze and judge the system information in claim 11;
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. (FP 7.30.06)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 13-14, 16 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al (US20170262955) in view of Qawami et al (US20190050049).
Regarding Claim 1. Lin teaches A computing system, comprising:
a graphics processing unit controlling a display;
a code memory, storing instructions; and
a processor, operating the graphics processing unit to control the display, and configured to execute the instructions retrieved from the code memory
to implement a plurality of graphics processor boosting modules for the
graphics processing unit (Lin, abstract, the invention describes apparatus and methods for managing power consumption of a graphics processing system. Specifically, the method adaptively adjusts the performance level of the graphics processing system based on scene information in each frame. A scene-aware power manager is configured to receive scene information and adaptively control performance
of the GPU according to the received scene information. The power manager compares the early indicators of the current frame with the early indicators of a previous frame to determine a level of scene change and to assign a set of initial performance settings based on the determined level of scene change.
[0014] FIG. 4 is a block diagram of the graphics processing system that includes CPU, GPU, memory, and display devices.
[0026] The graphics processing system 100 adaptively adjusts the performance settings to the performance controller 130 based on events reported by the event reporter 140. In some embodiments, the power manager 110 receives the identity of a reported event and a time stamp associated with the event from the event reporter 140 and determines whether the performance settings provided to the performance controller 130 is adequate (not too fast or too slow). In some embodiments, the event reporter 140 is configurable, i.e., the types of events that are to be monitored and/or reported can be configured by the user to suit the type of application running on the CPU and the GPU.); wherein;
the processor further executes the instructions retrieved from the code memory to implement an activation controller that controls activation of the
different graphics processor boosting modules through different configuration interfaces (Lin, [0029] The performance controller 130 controls a collection of control data or signals to various modules or circuits in the graphics processing system 100. The modules or circuits can include one or more of the following: a CPU, a GPU, a memory device, a display device, bus fabric, and other types of circuits that together constitute the graphics processing system 100. The control data can include signals sent by the power manager 110 directly to the various circuits of the graphics processing system 100 and/or control data stored in memory structures by the power manager 110. The performance controller 130 handles control data or signals that control the performance of circuits and/or devices in the graphics processing system. The performance controller 130 controls settings that control clock frequency (e.g., frequency setting 131) and operating voltage (e.g., voltage setting 132). The performance controller 130 also controls settings that control display frame rate, display response time to user interaction, or other settings that may affect the performance or the power usage of the graphics processing system 100. In some embodiments, a set of performance settings can be said to achieve a particular performance metric (e.g., a particular operating frequency or a particular data rate).
Further see Fig 1 setting modules 131 and 132 which are equivalent to boosting modules.).
Lin fails to explicitly teach, however, Qawami teaches with balances between the different graphics processor boosting modules (Qawami, abstract the invention describes methods and apparatus for managing operation of variable-state computing devices using artificial intelligence. An example computing device includes a hardware platform. The example computing device also includes an artificial intelligence (AI) engine to: determine a context of the device; and adjust an operation of the hardware platform based on an expected change in the context of the device. The adjustment modifies at least one of a computational efficiency of the device, a power efficiency of the device, or a memory response time of the device.
[0031] The above examples enable the rendering of a video feed at sufficient resolutions to serve the intended purpose of the device 102 without invoking any unnecessary processing of the video stream, thereby reducing the processing capacity and, thus, the power consumption of the system 100. In other examples, the goal of operating the AI engine 116 may be to improve the efficiency of some target metric other than power conservation. For instance, in other examples not necessarily related to video surveillance, the hardware usage manager 306 may adapt the operation of the components of the device 102 to improve (e.g., optimize) the performance or computational efficiency of the device 102. As a specific example, when the current context of the device 102 as determined by the context determiner 304 indicates a CPU is handling a heavy workload, the hardware usage manager 306 may cause some of the workload to be handled by the graphics engine 112 to maintain high speed performance of the system as a whole. In other examples, the hardware usage manager 306 may tailor operational parameters of the components of the device to improve (e.g., optimize) the memory response time of the device 102 … In other examples, the hardware usage manager 306 may adjust operational parameters to strike a suitable balance between multiple different target metrics (e.g., memory speed, computation efficiency, power efficiency, etc.).
Lin, [0026] The graphics processing system 100 adaptively adjusts the performance settings to the performance controller 130 based on events reported by the event reporter 140. In some embodiments, the power manager 110 receives the identity of a reported event and a time stamp associated with the event from the event reporter 140 and determines whether the performance settings provided to the performance controller 130 is adequate (not too fast or too slow). In some embodiments, the event reporter 140 is configurable, i.e., the types of events that are to be monitored and/or reported can be configured by the user to suit the type of application running on the CPU and the GPU.
To make sure the performance is not too fast or too slow is also another way of balancing.).
Lin and Qawami are analogous art because they both teach method of improve/maintain computer device performance by adaptively adjusting settings/parameters based on collected device performing status data. Qawami further teaches adjust operational parameters to balance between multiple different target performance metrics. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the adaptive parameter adjusting method for computer device performance improvement (taught in Lin), to further balance between different target performance metrics (taught in Qawami), so as to improve the computer device performance as a whole without compromising user’s experience.
Regarding Claim 2. The combination of Lin and Qawami further teaches The computing system as claimed in claim 1, wherein:
the activation controller provides an activation control logic that selects target modules to activate from the graphics processor boosting modules (Lin, [0126] FIG. 7 illustrates the data flow in the graphics processing system 100 when it performs fine-grain performance settings adjustment based on event timestamps. As illustrated, the event reporter 140 reports the detected event (e.g., the GPU having completed 10,000 triangles) to the power manager 110 by sending an event identifier 701 and a timestamp 702 of the event. (In some embodiments, the power manager supplies the timestamp when it receives a reported event). The timestamp of the event allows the power manager to identify the actual time of the event. The power manager 110 then uses the received event ID 701 to lookup an expected time for the event (illustrated as an expected time 711 retrieved from a lookup table 710). The power manager 110 compares the expected time 711 with the actual time based on the timestamp 702 for the event to determine whether the event is within an acceptable threshold of the expected time. If not, the power manager sends adjusted performance settings to the various circuits of the graphics processing system 100, including the CPU 410, the GPU 420, the memory controller 435, and display controller 455, and other devices 490. In some embodiments, the amount of the fine-grain adjustment is provided by the performance LUT 150 by a lookup based on the event ID 701 and the difference between the actual time 702 and the expected time 711 of the event.); and the activation controller takes balances between the different graphics processor
boosting modules by selection of the target modules (Qawami, [0031] The above examples enable the rendering of a video feed at sufficient resolutions to serve the intended purpose of the device 102 without invoking any unnecessary processing of the video stream, thereby reducing the processing capacity and, thus, the power consumption of the system 100. In other examples, the goal of operating the AI engine 116 may be to improve the efficiency of some target metric other than power conservation. For instance, in other examples not necessarily related to video surveillance, the hardware usage manager 306 may adapt the operation of the components of the device 102 to improve (e.g., optimize) the performance or computational efficiency of the device 102. As a specific example, when the current context of the device 102 as determined by the context determiner 304 indicates a CPU is handling a heavy workload, the hardware usage manager 306 may cause some of the workload to be handled by the graphics engine 112 to maintain high speed performance of the system as a whole. In other examples, the hardware usage manager 306 may tailor operational parameters of the components of the device to improve (e.g., optimize) the memory response time of the device 102 … In other examples, the hardware usage manager 306 may adjust operational parameters to strike a suitable balance between multiple different target metrics (e.g., memory speed, computation efficiency, power efficiency, etc.).).
The reasoning for combination of Lin and Qawami is the same as described in Claim 1.
Regarding Claim 3. The combination of Lin and Qawami further teaches The computing system as claimed in claim 2, wherein:
the activation control logic is further configured to control activation timing of the target modules; and the activation controller takes balances between the different graphics processor boosting modules by controlling the activation timing of the target modules (Lin, [0120] As mentioned, the power manager not only supplies an initial performance settings for each frame based on the frame's early indicators, but also performs fine-grain adjustment of the performance settings for the processing of frame after the processing of the frame has already started. In some embodiments, such adjustments take place at specific events during the processing of the frame by the GPU. The power manager uses these events to evaluate the adequacy of the performance settings and adjusts accordingly. In some embodiments, the graphics processing system includes an event reporter such as the event reporter 140 to report these events, by e.g., reporting the identity of each event together with a timestamp for the occurrence of the event. The power manager 110 in turn uses the reported event and timestamp to identify an expected time for the event in order to determine if the performance settings are too high or too low. For example, the power manager 110 in some embodiments monitors the GPU for when it completes computing 10,000 triangles for a frame. The power manager 110 uses the timestamp associated with the event to determine how quickly the GPU finishes the task and whether to increase performance or decrease performance based on a comparison between the timestamp and an expected runtime for the event.).
Regarding Claim 4. The combination of Lin and Qawami further teaches The computing system as claimed in claim 3, wherein:
the activation controller further provides a strength control logic that controls strength of the target modules (Lin, [0092] The following are examples of performance settings that are controlled by the scene-aware power manager: (as initial performance settings or fine-grain adjustment)
[0093] Switch of power source of the GPU or its subinstances;
[0094] Slow-down/speed-up of GPU/CPU and its subinstance frequency and voltage;
[0095] Early wake-up or early speed-up of devices (CPU, GPU, etc.);
[0096] Adjustment of memory bandwidth and arbitration policy (e.g., the main memory 405 and/or the GPU memory 440); and
[0097] Display frame-rate and deadline strategy.
[0098] In some embodiments, the fine-grain adjust of performance settings includes budget and step and correction. Such budget and step correction can be applied to some or all of the following settings of the graphics processing system:
[0099] Switch external shader/sub-modules/SRAM power source PMIC/LDO/MTCMOS;
[0100] Slow down/speed-up active shaders/sub-module/ SRAM frequency and even voltage;
[0101] Early wake-up or early speed-up by prediction to reduce performance drop;
[0102] CPU loading allocation for GPU process;
[0103] DRAM bandwidth allocation; and
[0104] Display strategy and deadline policy.); and the activation controller takes balances between the different graphics processor boosting modules by controlling the strength of the target modules (Qawami, [0033] In the example of FIG. 3, the device activity classifier 308 identifies and classifies different device activities or combinations of device activities based on an analysis of the usage data monitored by the usage monitor 302 in a similar manner to the operations of the context determiner 304 described above. In some examples, the context determiner 304 and the device activity classifier 308 are integrated to operate as a unit. As mentioned above, a device activity refers to any operation or function performed by one or more hardware components of the device 102.
[0034]… Based on these assumptions, in some examples, the device profile generator 310 of FIG. 3 combines the different device activities identified by the device activity classifier 308 to generate a profile or signature for the device 102. A device profile may be represented as a state diagram that indicates the different expected states of the device as well as the probabilities of transitions between different ones of the states. An example device profile 400 is illustrated in FIG. 4 that includes six states 402, 404, 406, 408, 410, 412 interconnected by seven edges or transitions 414, 416, 418, 420, 422, 424, 426…. Further, the device profile generator 310 may track the transitions 414, 416, 418, 420, 422, 424, 426 between different ones of the states over time. After monitoring the changes from one state to another in this manner, the device profile generator 310 may assign different weights to different ones of the transitions 414, 416, 418, 420, 422, 424, 426 to indicate a probability of the device transitioning from any particular state to any other state. In some examples, the transition probabilities may depend on the duration that a device remains within a particular state. Accordingly, in some examples, the device profile generator 310 monitors the duration that the device 102 continues operating within a particular state and assigns the transition probabilities based on the timing information.).
The reasoning for combination of Lin and Qawami is the same as described in Claim 1.
Regarding Claim 5. The combination of Lin and Qawami further teaches The computing system as claimed in claim 1, wherein: the activation controller controls the activation of the different graphics processor boosting modules based on system information collected by the processor (Lin, [0119] As illustrated in FIG. 5b, the power manager uses the level of scene change as an index to lookup the LUT 150 and retrieves a set of performance settings, including frequency, voltage, and frame rate. In the illustrated example, the quantified level of scene change is "3", and the LUT correspondingly produces a set of performance settings that includes frequency of 400 MHz, voltage of 2.4V, and frame rate of 27 per second. As mentioned, a set of performance settings can have many more parameters, such as different sets of frequencies and voltages for multiple different circuits or modules, as well as parameters such as display response deadline, memory access arbitration policy, etc.
[0120] As mentioned, the power manager not only supplies an initial performance settings for each frame based on the frame's early indicators, but also performs fine-grain adjustment of the performance settings for the processing of frame after the processing of the frame has already started. In some embodiments, such adjustments take place at specific events during the processing of the frame by the GPU. The power manager uses these events to evaluate the adequacy of the performance settings and adjusts accordingly. In some embodiments, the graphics processing system includes an event reporter such as the event reporter 140 to report these events, by e.g., reporting the identity of each event together with a timestamp for the occurrence of the event. The power manager 110 in turn uses the reported event and timestamp to identify an expected time for the event in order to determine if the performance settings are too high or too low. For example, the power manager 110 in some embodiments monitors the GPU for when it completes computing 10,000 triangles for a frame. The power manager 110 uses the timestamp associated with the event to determine how quickly the GPU finishes the task and whether to increase performance or decrease performance based on a comparison between the timestamp and an expected runtime for the event.
Therefore, the collected information of those events are the information collected by the processor.).
Regarding Claim 6. The combination of Lin and Qawami further teaches The computing system as claimed in claim 5, wherein: the system information is power consumption information, or temperature information, or an operational frequency of the processor, or an operational frequency of the graphics processing unit, or an operational frequency of a system memory of the computing system, or a frame rate, or a counting result of a hardware counter of the processor (Lin, [0119] As illustrated in FIG. 5b, the power manager uses the level of scene change as an index to lookup the LUT 150 and retrieves a set of performance settings, including frequency, voltage, and frame rate.).
Regarding Claim 7. The combination of Lin and Qawami further teaches The computing system as claimed in claim 5, wherein: the activation controller controls the activation of the different graphics processor boosting modules based on scene moving information about a scenario movie displayed on the display (Lin, [0118] As illustrated in FIG. 5a, the power manager 110 receives scene information and/or indicators (including early indicators) from the CPU 410, the memory controllers 435, the display controller 455, the GPU 420, as well as other devices 490 that includes bus fabric components. The power manager 110 uses the received scene information to look up a performance settings from the LUT 150. To generate the initial performance setting, the power manager compares the early indicators of the upcoming frame with the early indicators of the previous frame (illustrated as being stored in a storage 510) and quantifies the difference as "level of scene change".
[0119] As illustrated in FIG. 5b, the power manager uses the level of scene change as an index to lookup the LUT 150 and retrieves a set of performance settings, including frequency, voltage, and frame rate. In the illustrated example, the quantified level of scene change is "3", and the LUT correspondingly produces a set of performance settings that includes frequency of 400 MHz, voltage of 2.4V, and frame rate of 27 per second.).
Regarding Claim 13. The combination of Lin and Qawami further teaches The computing system as claimed in claim 1, wherein: the activation controller is implemented by a share library, or a driver module, or a process controlling a driver of the computing system (Lin, [0031] The performance LUT 150 is a lookup table for mapping scene information from the framer analyzer 120 into performance settings to the performance controller 130. The performance LUT 150 may include entries for directly mapping scene information to performance settings. The performance LUT 150 may also include entries for mapping derived parameters to the performance settings.).
Regarding Claim 14. The combination of Lin and Qawami further teaches The computing system as claimed in claim 1, wherein: each of the configuration interfaces is implemented by a combination of callback functions, system variables, and configuration files (Lin, [0026] The graphics processing system 100 adaptively adjusts the performance settings to the performance controller 130 based on events reported by the event reporter 140. In some embodiments, the power manager 110 receives the identity of a reported event and a time stamp associated with the event from the event reporter 140 and determines whether the performance settings provided to the performance controller 130 is adequate (not too fast or too slow). In some embodiments, the event reporter 140 is configurable, i.e., the types of events that are to be monitored and/or reported can be configured by the user to suit the type of application running on the CPU and the GPU.
[0031] The performance LUT 150 is a lookup table for mapping scene information from the framer analyzer 120 into performance settings to the performance controller 130. The performance LUT 150 may include entries for directly mapping scene information to performance settings. The performance LUT 150 may also include entries for mapping derived parameters to the performance settings.
[0059] The scene information is used to predict the optimal level of power settings for the graphics processing system, because they are indicative of the amount of work load that needs to be performed in order to generate the data necessary for display or camera recording. The scene information collected from different processes are taken together and jointly analyzed (e.g., summed) for level of scene change. The following are some examples of scene information that the power manager collects from various devices of the graphics processing system: [0080] API trace of GPU endering/compute standard (OpenGL, OpenCL, Vulkan, etc.), including attribute, state
and parameter of each API function call;).
Claim 16 is similar in scope as Claim 1, and thus is rejected under same rationale.
Claims 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al (US20170262955) in view of Qawami et al (US20190050049) further in view of Lawrence et al (US20190043154).
Regarding Claim 8. The combination of Lin and Qawami fails to explicitly teach, however, Lawrence teaches The computing system as claimed in claim 7, wherein:
the activation controller controls the activation of the different graphics processor boosting modules based on user hints (Lawrence, abstract, the invention describes a method to determine user-related concentration information, and adjust one or more parameters of a graphics subsystem based on the user-related concentration information.
[0024] Turning now to FIGS. 3A to 3C, an embodiment of a method 30 of adjusting a graphics parameter may include determining user-related concentration information at block 31, and adjusting one or more parameters of a graphics subsystem based on the user-related concentration information at block 32. For example, the method 30 may include determining the user-related concentration information based on measured EEG information at block 33. Some embodiments of the method 30 may also include adjusting the one or more parameters to adjust a render quality of the
graphics subsystem based on the user-related concentration information at block 34, and/or adjusting the one or more parameters to adjust one or more of an encode and a decode quality of the graphics subsystem based on the user-related concentration information at block 35. For example, the method 30 may include determining an increase in a level of concentration at block 36, and adjusting the one or more parameters to increase one or more of a bandwidth, a resolution, and a frame rate of the graphics subsystem based on the increased level of concentration at block 37. In some embodiments, the graphics subsystem may include at least one of a render pipeline, an encode pipeline, and a decode pipeline for stereo virtual reality content at block 38.
Lin, [0059] The scene information is used to predict the optimal level of power settings for the graphics processing system, because they are indicative of the amount of workload that needs to be performed in order to generate the data necessary for display or camera recording. The scene information collected from different processes are taken together and jointly analyzed (e.g., summed) for level of scene change. The following are some examples of scene information that the power manager collects from various devices of the graphics processing system:
[0077] External user event.
[0089] User interface event;
Therefore, a user indicates his/her focus/concentration when trigger event (such as touch, click, point, gesture, moving eye to a specific location) through a user interface (such as a touch panel, mouse, joystick, camera/sensor that sensing the eye focus). The system detects user’s focus and adaptively adjust parameters for optimal system performance.).
Lin, Qawami and Lawrence are analogous art because they all teach method of improve/maintain computer device performance by adaptively adjusting settings/parameters based on collected device performing status data. Lawrence further teaches adjust operational parameters when detecting user’s concentration. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the adaptive parameter adjusting method for computer device performance improvement (taught in Lin and Qawami), to further include considering user’s concentration (taught in Qawami), so as to improve the computer device performance based on user’s focus. For example, only increase the video resolution at user’s focus area so as to save computing resource.
Regarding Claim 9. The combination of Lin, Qawami and Lawrence further teaches The computing system as claimed in claim 8, wherein: the user hints are obtained from a touch panel on the display (Lin, [0059] The scene information is used to predict the optimal level of power settings for the graphics processing system, because they are indicative of the amount of workload that needs to be performed in order to generate the data necessary for display or camera recording. The scene information collected from different processes are taken together and jointly analyzed (e.g., summed) for level of scene change. The following are some examples of scene information that the power manager collects from various devices of the graphics processing system:
[0077] External user event.
[0089] User interface event;
Therefore, a user indicates his/her focus/concentration when trigger event (such as touch, click, point, gesture, moving eye to a specific location) through a user interface (such as a touch panel, mouse, joystick, camera/sensor that sensing the eye focus). For example, when user uses finger to zoom in an image on a touch screen, it indicates user is focusing on the image. The system detects user’s focus and adaptively adjust parameters for optimal system performance.).
Claims 10-12, 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al (US20170262955) in view of Qawami et al (US20190050049), Lawrence et al (US20190043154) further in view of Schluessler et al (US20180284872).
Regarding Claim 10. The combination of Lin, Qawami and Lawrence fails to explicitly teach, however, Schluessler teaches The computing system as claimed in claim 8, wherein:
the activation controller controls the activation of the different graphics
processor boosting modules based on programmable hints; and the programmable hints are managed in a list wherein different applications are
related to different parameter thresholds, and the parameter thresholds
are provided to judge the system information, the user hints, or the scene
moving information (Schluessler, abstract, the invention describes an embodiment may including an application processor, persistent storage media coupled to the application processor, and a graphics subsystem coupled to the application processor.
The system may further include any of a performance analyzer to analyze a performance of the graphics subsystem to provide performance analysis information, a content-based depth analyzer to analyze content to provide content-based depth analysis information, a focus analyzer to analyze a focus area to provide focus analysis information, an edge analyzer to provide edge analysis information, a frame analyzer to provide frame analysis information, and/or a variance analyzer to analyze respective amounts of variance for the frame. The system may further include a workload adjuster to adjust a workload of the graphics subsystem based on the analysis information.
[0168] Some embodiment may monitor the frame rate and if the frame rate drops then decrease the workload until the frame rate increases or returns to a target frame rate. For example, the workload may include computational load or memory bandwidth. The workload may be decreased by dropping the resolution in a portion of the screen. For example, the portion of the screen may be the outside edges of the screen. For example, the portion of the screen may be a non-foveated portion of the screen. The workload may additionally, or alternatively, be adjusted by change a shading rate. For example, if the frame rate is too low, the system may change how the system processes edge thresholds or other performance parameters (e.g. for higher quality results). The selected portions of the screen and/or the adjusted performance parameters may be user-specific (e.g. based on a setup process and/or based on user calibration). For example, if the user is colorblind or less sensitive to different colors, the system may change those parameters first. Those skilled in the art will appreciate that there may be many other criteria for selecting the portion of screen and/or adjusting the workload.
Lin, [0092] The following are examples of performance settings that are controlled by the scene-aware power manager: (as initial performance settings or fine-grain adjustment)
[0096] Adjustment of memory bandwidth and arbitration policy ( e.g., the main memory 405 and/or the GPU memory 440); and
[0100] Slow down/speed-up active shaders/sub-module/ SRAM frequency and even voltage;
[0102] CPU loading allocation for GPU process;
[0103] DRAM bandwidth allocation; and
[0117] As mentioned, in some embodiments, the power manager uses a LUT to look up performance settings based on scene information (including early indicators.) FIGS. 5a-b illustrates the flow of data in the graphics processing).
Lin, Qawami, Lawrence and Schluessler are analogous art because they all teach method of improve/maintain computer device performance by adaptively adjusting settings/parameters based on collected device performing status data. Schluessler further teaches adjusting operational parameters based on respective application/device computational capability/limit/load. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the adaptive parameter adjusting method for computer device performance improvement (taught in Lin, Qawami and Lawrence), to further include considering respective application/device computational capability/limit/load (taught in Schluessler), so as to improve the computer device performance without overloading respective application/device.
Regarding Claim 11. The combination of Lin, Qawami, Lawrence and Schluessler further teaches The computing system as claimed in claim 10, wherein:
the activation controller includes an information analysis module and a control module;
the information analysis module is configured to analyze and judge the system information, the user hints, or the scene moving information based on
the programmable hints (Schluessler, [0145] For example, the system 600 may include a performance analyzer 614 communicatively coupled to the graphics subsystem 613 to analyze a performance of the graphics subsystem 613 to provide performance analysis information (e.g. as described in more detail below), and a
workload adjuster 615 communicatively coupled to the graphics subsystem 613 and the performance analyzer 614 to adjust a workload of the graphics subsystem 613 based on the performance analysis information. In some embodiments, the workload adjuster 615 may not adjust the content of the workload, but rather may adjust other parameters/
settings/configuration properties/etc. such as the resolution, shading frequency, or areas where different resolution or shading frequencies are performed, that adjust how the workload is processed.); and
according to an analysis result of the information analysis module, the control module selects target modules to activate from the graphics processor
boosting modules, controls activation timing of the target modules (Lin, [0120] As mentioned, the power manager not only supplies an initial performance settings for each frame based on the frame's early indicators, but also performs fine-grain adjustment of the performance settings for the processing of frame after the processing of the frame has already started. In some embodiments, such adjustments take place at specific events during the processing of the frame by the GPU. The power manager uses these events to evaluate the adequacy of the performance settings and adjusts accordingly. In some embodiments, the graphics processing system includes an event reporter such as the event reporter 140 to report these events, by e.g., reporting the identity of each event together with a timestamp for the occurrence of the event. The power manager 110 in turn uses the reported event and timestamp to identify an expected time for the event in order to determine if the performance settings are too high or too low. For example, the power manager 110 in some embodiments monitors the GPU for when it completes computing 10,000 triangles for a frame. The power manager 110 uses the timestamp associated with the event to determine how quickly the GPU finishes the task and whether to increase performance or decrease performance based on a comparison between the timestamp and an expected runtime for the event.), and controls strength of the target modules (Lin, [0092] The following are examples of performance settings that are controlled by the scene-aware power manager: (as initial performance settings or fine-grain adjustment)
[0093] Switch of power source of the GPU or its subinstances;
[0094] Slow-down/speed-up of GPU/CPU and its subinstance frequency and voltage;
[0095] Early wake-up or early speed-up of devices (CPU, GPU, etc.);
[0096] Adjustment of memory bandwidth and arbitration policy (e.g., the main memory 405 and/or the GPU memory 440); and
[0097] Display frame-rate and deadline strategy.
[0098] In some embodiments, the fine-grain adjust of performance settings includes budget and step and correction. Such budget and step correction can be applied to some or all of the following settings of the graphics processing system:
[0099] Switch external shader/sub-modules/SRAM power source PMIC/LDO/MTCMOS;
[0100] Slow down/speed-up active shaders/sub-module/ SRAM frequency and even voltage;
[0101] Early wake-up or early speed-up by prediction to reduce performance drop;
[0102] CPU loading allocation for GPU process;
[0103] DRAM bandwidth allocation; and
[0104] Display strategy and deadline policy.).
The reasoning for combination of Lin, Qawami, Lawrence and Schluessler is the same as described in Claim 10.
Regarding Claim 12. The combination of Lin, Qawami, Lawrence and Schluessler further teaches The computing system as claimed in claim 11, wherein:
the control module issues on and off settings to the configuration interfaces to activate the target modules and control activation timing of the target modules (Lin, [0092] The following are examples of performance settings that are controlled by the scene-aware power manager: (as initial performance settings or fine-grain adjustment)
[0093] Switch of power source of the GPU or its subinstances;
[0094] Slow-down/speed-up of GPU/CPU and its subinstance frequency and voltage;
[0095] Early wake-up or early speed-up of devices (CPU, GPU, etc.);
[0096] Adjustment of memory bandwidth and arbitration policy (e.g., the main memory 405 and/or the GPU memory 440); and
[0097] Display frame-rate and deadline strategy.
[0098] In some embodiments, the fine-grain adjust of performance settings includes budget and step and correction. Such budget and step correction can be applied to some or all of the following settings of the graphics processing system:
[0099] Switch external shader/sub-modules/SRAM power source PMIC/LDO/MTCMOS;.
Qawami, [0012] Over time, the AI engine learns the particular operations and interactions of the device hardware and the type(s) of context that commonly occur to more accurately identify the context of the device in new situations. Similarly, over time the AI engine learns how to adjust or tune operational parameters associated with the device hardware based on any particular context to enable the device to function in a manner that provides a satisfying user experience while also achieving one or more target metrics such as, for example, power efficiency, computational efficiency, memory (cache versus storage) response time, etc. Thus, examples disclosed herein are not limited to adjusting the operation of a graphics engine but may apply to any hardware platform and may involve any suitable adjustment to its operation. As some examples, the Al engine may cause a hardware platform to locate data, tum on or off a component of the platform (e.g., a processor core, a memory bank, etc.), initialize a processor core, purge data from memory, start a virtual machine, terminate a virtual machine, move data between memories, and so forth.); and
the control module issues strength settings to the configuration interfaces to control strength of the target modules (Lin, [0092] The following are examples of performance settings that are controlled by the scene-aware power manager: (as initial performance settings or fine-grain adjustment)
[0093] Switch of power source of the GPU or its subinstances;
[0094] Slow-down/speed-up of GPU/CPU and its subinstance frequency and voltage;
[0095] Early wake-up or early speed-up of devices (CPU, GPU, etc.);
[0096] Adjustment of memory bandwidth and arbitration policy (e.g., the main memory 405 and/or the GPU memory 440); and
[0097] Display frame-rate and deadline strategy.
[0098] In some embodiments, the fine-grain adjust of performance settings includes budget and step and correction. Such budget and step correction can be applied to some or all of the following settings of the graphics processing system:
[0099] Switch external shader/sub-modules/SRAM power source PMIC/LDO/MTCMOS;
[0100] Slow down/speed-up active shaders/sub-module/ SRAM frequency and even voltage;
[0101] Early wake-up or early speed-up by prediction to reduce performance drop;
[0102] CPU loading allocation for GPU process;
[0103] DRAM bandwidth allocation; and
[0104] Display strategy and deadline policy.).
The reasoning for combination of Lin, Qawami, Lawrence and Schluessler is the same as described in Claim 10.
Claim 17 is similar in scope as Claim 7, 10&11, and thus is rejected under same rationale.
Claim 18 is similar in scope as Claim 6. 9 and thus is rejected under same rationale.
Claim 19 is similar in scope as Claim 12, and thus is rejected under same rationale.
Allowable Subject Matter
Claim 15 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reason for the indication of allowable subject matter:
Regarding Claim 15, it recites “The computing system as claimed in claim 4, wherein:
the target modules include a shading rate control module and a resolution
adjusting module;
in response to an extremely heavy loading status of the computing system that requires more system resources than a normal loading status of the computing system, the activation controller activates the shading rate control module at a stronger level in comparison with the resolution adjusting module; and
in response to a lightly heavy loading status of the computing system that
requires more system resources than the normal loading status but fewer system resources than the extremely heavy loading status, the activation controller activates the resolution adjusting module at a stronger level in comparison with the shading rate control module.” in the context of Claim 15.
The prior arts of record either alone or in combination fails to teach or suggest the above quoted limitation of Claim 15. Therefore, Claim 15 is allowable over prior art.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIN SHENG whose telephone number is (571)272-5734. The examiner can normally be reached M-F 9:30AM-3:30PM 6:00PM-8:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 5712723022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Xin Sheng/Primary Examiner, Art Unit 2619