Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Allowable Subject Matter
Claims 8-10, and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 6-7, 11-14, and 18-19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Li et al. (NPL Doc, “ReTriple: Reduction of Redundant Rendering on Android Devices for Performance and Energy Optimizations”).
As per claim 1, Li teaches the claimed:
1. An interface display method, comprising:
determining a static subtree from a render tree based on interface information of a to-be-displayed interface (On page 3, in section III.A, 2nd paragraph where they refer to “Suppose the subtree rooted at View1 keeps unchanged over many frames”. Since this subtree is unchanged over many frames it corresponds to a “static subtree”. The render tree is shown in figure 6. Figure 4 shows that the render tree is also based on user interface information of a to-be-displayed interface), wherein the render tree is used to obtain a current interface through rendering (The top of the 2nd column on page 3 indicates that the render tree receives an update in View6. Thus, the current interface is obtained by way of updates through rendering, e.g. it recites “When a new frame caused by the change in View6 is to be rendered, the update and optimization of the DL Tree will only involve DL6, its ancestors (DL2 and DL0) and 3 additional DL operations”), the static subtree comprises an invariant node, and a node corresponding to content that is not updated on the to-be-displayed interface comprises the invariant node (The top of the 2nd column on page 3, e.g. it recites “When a new frame caused by the change in View6 is to be rendered, the update and optimization of the DL Tree will only involve DL6, its ancestors (DL2 and DL0) and 3 additional DL operations that instruct the GPU to reuse the bitmap of View1 subtree cached on the GPU side”.
In this instance, the static subtree in View1 includes one or more invariant nodes: nodes View3, View4, and View5 as shown in figure 6. A node corresponding to content that is not updated on the to-be-displayed interface comprises the invariant node because it refers to “that instruct the GPU to reuse the bitmap of View1 subtree cached on the GPU side”. The content in the View1 subtree is not updated because it is reused from the cache on the GPU side);
updating the static subtree to a static node, wherein the static node comprises data of each node in the static subtree (Section III, 2nd paragraph states that the entire user interface and all its nodes are rendered initially once by the CPU, and then for later frames, only nodes in the interface that are changed are rendered by the GPU. Unchanged nodes are re-used and, thus, not rendered again for later frames. This is done using the render tree shown in figure 6 which indicates which nodes are changed or unchanged from the previous frames (changed nodes have a dashed outline). The system would have to have a step where static nodes that are part of the static subtree in figure 6 are updated to indicate their unchanged or static status. This is because, in the later frames, the system uses the render tree in figure 6 to determine which nodes are changed and thus need to be rendered by the GPU (e.g. in figure 6 the nodes labelled “View0”, “View2”, and “View6” are changed).
Figure 6 also shows that static nodes labelled “View1”, “View3”, “View4”, and “View5” comprise data of each node in the static subtree);
updating data of a variant node in the render tree based on the interface information (Please see the top of the 2nd col on the first page where it states: “Our work is motivated by the observation that common user interactions (e.g., screen swipes) often result in updates on only a subset of UI elements, but the rendering is performed for all UI elements, including those unchanged ones … we perform an assessment on the reusing potential of rendering results of Views.”
Also, please see the render tree in figure 6 where the common user interaction, e.g. a screen swipe, results in only the nodes labelled “View0”, “View2”, and “View6” needing to be changed or updated. Thus, the nodes labelled “View0”, “View2”, and “View6” correspond to variant nodes), wherein the variant node is a node other than the static node in the render tree (In figure 6 in the render tree where the changed nodes or variant nodes labelled “View0”, “View2”, and “View6” are nodes other than the static nodes. Figure 6 also shows that static nodes labelled as “View1”, “View3”, “View4”, and “View5”);
drawing the to-be-displayed interface based on an updated render tree (In section III, 2nd paragraph “… When a new frame caused by the change in View6 is to be rendered, the update and optimization of the DL Tree will only involve DL6, its ancestors (DL2 and DL0)” Figure 6 shows that DL6 and its ancestors DL2 and DL0 are determined using the updated render tree that indicating the changed nodes or variant nodes labelled “View0”, “View2”, and “View6”); and
displaying the drawn to-be-displayed interface (In figure 2 on the bottom row shows the actually displaying step occurring. Also, please see figure 5(b) where it shows the screen being displayed with the interface).
As per claim 2, Li teaches the claimed:
2. The method according to claim 1, wherein the determining the static subtree from the
render tree comprises:
adding 1 to an invariance count of the node corresponding to the content that is not updated, to obtain an updated invariance count (Please see the pseudo-code in algorithm 1 in section IV. The invariance count is shown as the PRT count. PRT stands for past render times (top of the 1st col on the 2nd page). For each time, content is reused for a later frame for content that is not updated, the PRT count has 1 added onto it (end of the 3rd paragraph in section IV));
determining, as the invariant node, a node whose updated invariance count is greater than a preset count (In algorithm 1 in section IV in line 6); and
determining connected invariant nodes as the static subtree (This is determined in figure 6, e.g. it shows that static/invariant nodes labelled “View1”, “View3”, “View4”, and “View5” are determined to be connected as the static subtree).
As per claim 6, Li teaches the claimed:
6. The method according to claim 1, wherein the drawing the to-be-displayed interface based on the updated render tree comprises:
obtaining, based to the updated render tree, a render instruction corresponding to the variant node and a static render result corresponding to the static node (In section III.A, 2nd paragraph “…Instead of drawing the subtree together with other parts of the DL Tree, we perform rendering for the subtree of View1 entirely on the CPU side and transmit the resulting bitmap to GPU. When a new frame caused by the change in View6 is to be rendered, the update and optimization of the DL Tree will only involve DL6, its ancestors (DL2 and DL0), and 3 additional DL operations that instruct the GPU to reuse the bitmap of View1 subtree cached on the GPU side. GPU commands generated from the optimized partial DL operations will be transmitted to the GPU, where it will execute the normal GPU commands, and assemble its rendering results with the bitmaps cached in the GPU to produce the Frame Buffer.”
In this instance, the render commands to render node “View6” (which is changed) in figure 6 corresponds to the claimed “a render instruction corresponding to the variant node”.
Also in this instance, the instruction “on the GPU to reuse the bitmap of View1 subtree cached on the GPU side” corresponds to the claimed “a render instruction corresponding to a static render results corresponding to the static node”); and
obtaining a target render result based on the render instruction corresponding to the variant node and the static render result, wherein the target render result is the to-be-displayed interface (In section III.A, 2nd paragraph “… When a new frame caused by the change in View6 is to be rendered, the update and optimization of the DL Tree will only involve DL6, its ancestors (DL2 and DL0), and 3 additional DL operations that instruct the GPU to reuse the bitmap of View1 subtree cached on the GPU side. GPU commands generated from the optimized partial DL operations will be transmitted to the GPU, where it will execute the normal GPU commands, and assemble its rendering results with the bitmaps cached in the GPU to produce the Frame Buffer.” In this instance, a target render result is obtained based on the rendering instructions corresponding to the variant node (View6) and the static render result (reused bitmap from View1 subtree). The target render result is the to-be-displayed interface, e.g. please see figure 5(b) which shows this interface).
As per claim 7, Li teaches the claimed:
7. The method according to claim 6, the method further comprising:
storing the static render result (In section III.A, 2nd paragraph “… Suppose the subtree rooted at View1 keeps unchanged over many frames … When a new frame caused by the change in View6 is to be rendered, the update and optimization of the DL Tree will only involve DL6, its ancestors (DL2 and DL0), and 3 additional DL operations that instruct the GPU to reuse the bitmap of View1 subtree cached on the GPU side.”
In this instance, the reused bitmap cached on the GPU side corresponds to the claimed “storing the static render result”); and
obtaining the stored static render result when a first interface is displayed, wherein the first interface is an interface to be displayed after the to-be-displayed interface, and content corresponding to the static node does not need to be updated when the first interface is displayed (In section III.A, 2nd paragraph “When a new frame caused by the change in View6 is to be rendered, the update and optimization of the DL Tree will only involve DL6, its ancestors (DL2 and DL0), and 3 additional DL operations that instruct the GPU to reuse the bitmap of View1 subtree cached on the GPU side. GPU commands generated from the optimized partial DL operations will be transmitted to the GPU, where it will execute the normal GPU commands, and assemble its rendering results with the bitmaps cached in the GPU to produce the Frame Buffer. Therefore the processing of the View1 subtree is avoided on both CPU and GPU.”
In this instance, when a later frame of the interface needs to be rendered (e.g. a first interface to be displayed after the to-be-displayed interface), the content of the View1 node (static node) does not need to be updated because they reused the stored bitmap).
As per claim 11, the reasons and rationale used in the rejection of claims 1 and 6 are incorporated herein.
Li teaches the claimed:
11. An interface display method, comprising:
determining an invariant node from a render tree (Figure 6 shows a render tree with that includes one or more determined invariant (static) nodes: nodes View3, View4, and View5) based on interface information of a to-be-displayed interface (Figure 5 shows that the graphical information relates to interface information), wherein a node corresponding to content that is not updated on the to-be-displayed interface comprises the invariant node (On page 3, in section III.A, 2nd paragraph where they refer to “Suppose the subtree rooted at View1 keeps unchanged over many frames”. Since this subtree is unchanged over many frames it corresponds to a “static subtree” with invariant nodes. The render tree is shown in figure 6. Figure 4 shows that the render tree is also based on user interface information of a to-be-displayed interface);
updating data of a variant node in the render tree based on the interface information (Please see the top of the 2nd col on the first page where it states: “Our work is motivated by the observation that common user interactions (e.g., screen swipes) often result in updates on only a subset of UI elements, but the rendering is performed for all UI elements, including those unchanged ones … we perform an assessment on the reusing potential of rendering results of Views.”
Also, please see the render tree in figure 6 where the common user interaction, e.g. a screen swipe, results in only the nodes labelled “View0”, “View2”, and “View6” needing to be changed or updated. Thus, the nodes labelled “View0”, “View2”, and “View6” correspond to variant nodes), wherein the variant node is a node other than a static node in the render tree (In figure 6 in the render tree where the changed nodes or variant nodes labelled “View0”, “View2”, and “View6” are nodes other than the static nodes. Figure 6 also shows that static nodes labelled as “View1”, “View3”, “View4”, and “View5”);
obtaining, based on an updated render tree, first render information corresponding to the variant node and second render information corresponding to the invariant node, wherein render information comprises any one of a render instruction, a render result, and a layer (In section III.A, 2nd paragraph “…Instead of drawing the subtree together with other parts of the DL Tree, we perform rendering for the subtree of View1 entirely on the CPU side and transmit the resulting bitmap to GPU. When a new frame caused by the change in View6 is to be rendered, the update and optimization of the DL Tree will only involve DL6, its ancestors (DL2 and DL0), and 3 additional DL operations that instruct the GPU to reuse the bitmap of View1 subtree cached on the GPU side. GPU commands generated from the optimized partial DL operations will be transmitted to the GPU, where it will execute the normal GPU commands, and assemble its rendering results with the bitmaps cached in the GPU to produce the Frame Buffer.”
In this instance, the render commands (render instruction or render result) to render node “View6” (which is changed) in figure 6 corresponds to the claimed “first render information corresponding to the variant node”.
Also in this instance, the instruction “on the GPU to reuse the bitmap of View1 subtree cached on the GPU side” corresponds to the claimed “second render information corresponding to the invariant node”); and
obtaining the to-be-displayed interface (Figure 5b shows an example of the to-be-displayed interface) based on the first render information and the second render information (In section III.A, 2nd paragraph “… When a new frame caused by the change in View6 is to be rendered, the update and optimization of the DL Tree will only involve DL6, its ancestors (DL2 and DL0), and 3 additional DL operations that instruct the GPU to reuse the bitmap of View1 subtree cached on the GPU side. GPU commands generated from the optimized partial DL operations will be transmitted to the GPU, where it will execute the normal GPU commands, and assemble its rendering results with the bitmaps cached in the GPU to produce the Frame Buffer.” In this instance, the to-be-display interface is obtained based on the rendering instructions corresponding to the variant node (View6) (i.e. the first render information) and the static render result (reused bitmap from View1 subtree) (i.e. the second render information)).
As per claim 12, this claim is similar in scope to limitations recited in claim 2, and thus is rejected under the same rationale.
As per claim 13, the reasons and rationale for the rejection of claim 1 is incorporated herein.
Li teaches the claimed:
at least one processor and a non-transitory memory storing a computer code for executing by the at least one processor (Section V.A, 1st paragraph “We evaluate ReTriple on FSPAD-733, an open-source Pad equipped with Mali-400 MP2 GPU and Allwinner Quad-core A33 CPU running Android 5.0 with Linux kernel 3.4” The system of Li would have to have some type of memory present in order to store the executable instructions in order for their system to run on a computer-based system).
As per claim 14, this claim is similar in scope to limitations recited in claim 2, and thus is rejected under the same rationale.
As per claims 18 and 19, these claims are similar in scope to limitations recited in claims 6 and 7, respectively, and thus are rejected under the same rationale.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3-5 and 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Li et al. (NPL Doc, “HB-ReTriple: Mobile rendering optimization based on efficient history reusing”, herein referred to as “Li_2”).
As per claim 3, Li alone does not explicitly teach the claimed limitations.
However, Li in combination with Li_2 teaches the claimed:
3. The method according to claim 1, wherein the static subtree comprises a root node and at least one child node (Li_2 in figure 7(a) where the nodes for View2 and View3 make up a static subtree where View2 is the root node and View3 is at least one child node. Li_2 in figure 7(b) shows that this subtree is static because it is reused as a bitmap later on), and the updating the static subtree to the static node comprises:
updating data of the at least one child node to the root node; removing the at least one child node from the render tree; and determining the root node as the static node (Li_2 where the render tree from figure 7(a) in Li_2 is updated such that figure 7(b) shows that the child node View3 is removed and updated to the root node View2. The root node View2 is determined as the static node because it re-uses a bitmap for its display).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the updated render tree as taught by Li_2 with the system of Li in order to improve the rendering performance of the interface over time. Rendering performance is improved because additional portions of the interface are able to reuse a previously rendered bitmap for static portions over subsequent frames.
As per claim 4, Li alone does not explicitly teach the claimed limitations.
However, Li in combination with Li_2 teaches the claimed:
4. The method according to claim 1, wherein the drawing the to-be-displayed interface based on the updated render tree comprises:
traversing the updated render tree to generate a first render instruction corresponding to the variant node and a second render instruction corresponding to the static node (Li_2 where the render tree is updated from figure 5(b) to figure 7(b). In Li_2 in figure 7(b), the updated render tree is traversed to generate a first render instruction corresponding to the variant node View6 and to generate a second render instruction corresponding to the static node View3. Also see in Li_2 in section 3.2, 3rd paragraph); and
executing the first render instruction and the second render instruction to obtain the to-be-displayed interface (Li_2 in figure 7(b) and in section 3.2, 3rd paragraph where the GPU executes the first and second render instructions to obtain the to-be-displayed interface).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to traverse the updated render tree and execute the rendering instructions as taught by Li_2 with the system of Li in order to improve the rendering performance. This occurs because less rendering instructions are needed for static portions of the render tree that are updated to indicate that their portions may reuse bitmaps for subsequent frames.
As per claim 5, Li alone does not explicitly teach the claimed limitations.
However, Li in combination with Li_2 teaches the claimed:
5. The method according to claim 4, the method further comprising: storing the second render instruction; and obtaining the stored second render instruction when a first interface is displayed, wherein the first interface is an interface to be displayed after the to-be-displayed interface, and content corresponding to the static node does not need to be updated when the first interface is displayed (Li_2 in section 3.2, 3rd paragraph “Therefore, if the CPU instead of the GPU renders a View into a bitmap and uploads it to the GPU for once, we can avoid rendering it again if it keeps unchanged in subsequent frames. For example, in Fig. 7, we let the CPU render the sub-tree rooted at View2 into a bitmap and upload it to the GPU. In the next frame, although this unchanged sub-tree is in the dirty region, it will be excluded from rendering optimizations (DL reordering or merging) as it is already a rasterized image. The CPU will simply send the corresponding bitmap drawing instructions to the GPU, and the GPU will reuse the cached bitmap when composing the new frame.”)
In this instance, the “second render instruction” that is stored and obtained is the instruction to reuse the bitmap (corresponding to the static node) for subsequent later frames (e.g. for the first interface being displayed after the to-be-display interface). The bitmap can be reused because it corresponding to a static node and since it does not need to be updated).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to store and obtain the second render instruction as taught by Li_2 with the system of Li. The motivation of claim 4 is incorporated herein.
As per claims 15-17, these claims are similar in scope to limitations recited in claims 3-5, respectively, and thus are rejected under the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL F HAJNIK whose telephone number is (571) 272-7642. The examiner can normally be reached Mon-Fri 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL F HAJNIK/Supervisory Patent Examiner, Art Unit 2616