Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-30 are rejected under 35 U.S.C. 103 as being unpatentable over LI et al (WO 2021134462 A1) in view of ZHOU et al (WO 2021203286 A1).
As per claim 1, Li teaches the claimed “apparatus for display processing,” comprising: “a memory; and at least one processor coupled to the memory and, based at least in part on information stored in the memory” (Li, [0027] - As described herein, one or more techniques may refer to an application, i.e., software, being configured to perform one or more functions. In such examples, the application may be stored on a memory, e.g., on-chip memory of a processor, system memory, or any other memory. Hardware described herein, such as a processor may be configured to execute the application), the at least one processor is configured to: “obtain an indication of at least one frame including a plurality of layers, wherein the at least one frame is associated with the display processing” (Li, [0032] - For the example, the device 100 may be configured to render a plurality of layers associated with respective graphical data for a frame, …); “map each of the plurality of layers for processing at a display processing unit (DPU) or a graphics processing unit (GPU)” (Li, [0029] - Example techniques may improve visual quality of regions of the frame associated with layers of interest and/or reduce the load of a processing unit (e.g., any processing unit configured to perform one or more techniques disclosed herein, such as an application processor, a CPU, a graphics processor, a GPU, a display processor, a DPU, and the like)); “divide a frame buffer into a set of regions of interest (ROIs) for each non-rotation animation layer in the plurality of layers” (Li, [0047]-[0049] - The example region of interest determining component 144 may be configured to facilitate the identifying of a layer of interest and the determining of the coordinates for a region of interest based on the identified layer of interest. For example, the region of interest determining component 144 may receive (or retrieve) the layer identifiers associated with the plurality of layers for a frame included in the layer buffer 132. The region of interest determining component 144 may use the layer identifiers to query the layer of interest list 146 to determine if any of the layer identifiers correspond to a layer of interest); and “assign at least one second per-layer processor at the DPU to each of the set of ROIs in the frame buffer for each non-rotation animation layer in the plurality of layers” (Li, [0057]-[0059] – the region of interest display processing component 148 may receive (or retrieve) a composited frame (or an identifier or memory location at the frame buffer 134 associated with the composited frame) from the region of interest determining component 144. The region of interest display processing component 148 may also receive (or retrieve) coordinates associated with a region of interest of the composited frame. For example, the region of interest may correspond to the layer of interest identified by the region of interest determining component 144. In some such examples, the region of interest display processing component 148 may perform the one or more display processing techniques on the composited frame at the region bounded by the received coordinates). It is noted that Li only teaches the display process of non-rotation animation layer, but Li does not explicitly teach the display process of rotation animation layer; however, Zhou teaches “detect whether each of the plurality of layers is a rotation animation layer or a non-rotation animation layer based on the mapping for each of the plurality of layers” (Zhou, [0043]-[0044] - Initially, and in response to determining that the electronic device is being rotated, the WMS 204 may start a screen rotation process by requesting that the GPU 106 utilize the animation transformation module 206 to generate a screen rotation animation, and requesting a screen capture buffer (e.g., HDR buffer 220 and nonHDR buffer 222) from the compositor 208 of the DPU 104. That is, the DPU 104 receives an indication that a frame rotation animation process for video playback has been initiated); and “assign a first per-layer processor at the DPU to each rotation animation layer in the plurality of layers” (Zhou, [0048]-[0059] - the frame used by the GPU 106 to generate the rotation animation is an HDR format frame… In this example, the operations 400 start at a first step 402 receiving, by a display processor, an indication that a frame rotation animation process for video playback has been initiated, the display processor comprising a display processor pipeline). Thus, it would have been obvious, in view of Zhou, to configure Li’s apparatus as claimed by detecting and performing the frame rotation animation process by the GPU. The motivation is to avoid a mismatch of mapping data between the DPU and the GPU, and improve the visual representation.
Claim 2 adds into claim 1 “wherein the first per-layer processor corresponds to a first composition stage for each rotation animation layer in the plurality of layers” (Zhou, [0041] - As the electronic device 300 is rotated, a rotation animation presented on the display, showing a rotation of the video playback layer 304 and the background layer 302 into a landscape mode display. In the example shown, the background layer 302 is gradually eliminated during the transition from portrait mode to landscape mode, and video playback layer 304 is gradually resized to fit the entire display... As such, the rotation animation is configured to present to the user an animation of the resizing of the video playback layer 304 to accommodate the orientation of the electronic device, as well as a gradual introduction or elimination of a background layer 302), and “wherein the at least one second per-layer processor corresponds to a second composition stage for each of the set of ROIs in the frame buffer” (Li, [0057]-[0059] – the region of interest display processing component 148 may receive (or retrieve) a composited frame (or an identifier or memory location at the frame buffer 134 associated with the composited frame) from the region of interest determining component 144. The region of interest display processing component 148 may also receive (or retrieve) coordinates associated with a region of interest of the composited frame. For example, the region of interest may correspond to the layer of interest identified by the region of interest determining component 144. In some such examples, the region of interest display processing component 148 may perform the one or more display processing techniques on the composited frame at the region bounded by the received coordinates). Thus, it would have been obvious, in view of Zhou, to configure Li’s apparatus as claimed by performing the animation frame process on the ROI of the frame. The motivation is to avoid a mismatch of mapping data between the DPU and the GPU, and improve the visual representation.
Claim 3 adds into claim 2 “process each non-rotation animation layer in the plurality of layers based on the second composition stage for each of the set of ROIs in the frame buffer” (Li, [0061] – The set of rendered layers 210 may be generated by the graphics processor 120 of FIG. 1 and stored in the example layer buffer 132. As mentioned above, the compositing component 142 of FIG. 1 may be configured to blend or stitch the different rendered layers (e.g., the rendered layers 210) to generate the composited frame 250. For example, the compositing component 142 may retrieve the set of rendered layers 210 from the layer buffer 132 of the memory 130 and then perform blending and stitching of the pixel data of the rendered layers 210 to generate the composited frame 250).
Claim 4 adds into claim 3 “wherein to process each non-rotation animation layer in the plurality of layers based on the second composition stage, the at least one processor is configured to: blend each non-rotation animation layer in the plurality of layers based on the second composition stage” (Li, [0061] – The set of rendered layers 210 may be generated by the graphics processor 120 of FIG. 1 and stored in the example layer buffer 132. As mentioned above, the compositing component 142 of FIG. 1 may be configured to blend or stitch the different rendered layers (e.g., the rendered layers 210) to generate the composited frame 250. For example, the compositing component 142 may retrieve the set of rendered layers 210 from the layer buffer 132 of the memory 130 and then perform blending and stitching of the pixel data of the rendered layers 210 to generate the composited frame 250).
Claim 5 adds into claim 3 “wherein each non-rotation animation layer in the plurality of layers is processed if at least one of: (i) the non-rotation animation layer is on top of a regional layer stack and the non-rotation animation layer has a constant blending alpha value, or (ii) the non-rotation animation layer is not on top of the regional layer stack and the non-rotation animation layer is covered by one or more transparent layers” which would have been obvious in view of Li’s blending of rendered layers (e.g., Li, [0061] - For example, the compositing component 142 may retrieve the set of rendered layers 210 from the layer buffer 132 of the memory 130 and then perform blending and stitching of the pixel data of the rendered layers 210 to generate the composited frame 250) in which the blending layers arranged on the stack are assigned with blending alpha values (see also Zhou, [0037] - The mixer 216 may receive image data processed by the SSPP 214 and may perform blending and mixing of the image with one or more other surfaces. For example, mixer 216 may perform alpha blending, color generation, setting of a transparency color key, blending of surfaces in arbitrary order, and blending in linear space). Thus, it would have been obvious, in view of Zhou, to configure Li’s apparatus as claimed by blending multiple layers of an animated frame according to the assigned blending alpha values. The motivation is to avoid a mismatch of mapping data between the DPU and the GPU, and improve the visual representation.
Claim 6 adds into claim 3 “wherein each non-rotation animation layer in the plurality of layers is associated with a color configuration adjustment or a color-related adjustment” (Li, [0014] - In some examples, disclosed techniques may include an assertive display adjustment engine to facilitate adjusting gamma or color for each pixel in a region to adapt for variable lighting conditions. In some examples, disclosed techniques may employ additional or alternative display processing techniques, such as saturation adjustment techniques, hue adjustment techniques, and/or temperature adjustment techniques, to improve visual aspects of the displayed frame).
Claim 7 adds into claim 2 “wherein each of the set of ROIs is assigned a higher composition priority compared to other ROIs in the set of ROIs for each non-rotation animation layer in the plurality of layers” (Li, [0107] - In some examples, the region of interest coordinates determining component 798 may be configured to identify layer of interest by comparing respective identifiers associated with each layer of the plurality of layers with a list of priority identifiers. In some examples, the region of interest coordinates determining component 798 may be configured to generate a modified list of priority identifiers based on the list of priority identifiers. Further, in some examples, the region of interest coordinates determining component 798 may be configured to compare the respective identifiers associated with each layer of the plurality of layers with the list of priority identifiers and the modified list of priority identifiers).
Claim 8 adds into claim 1 “wherein to detect whether each of the plurality of layers is the rotation animation layer or the non-rotation animation layer, the at least one processor is configured to: identify whether a set of coordinates and shapes for each of the plurality of layers includes an adjustment” (Zhou, [0041] - As the electronic device 300 is rotated, a rotation animation presented on the display, showing a rotation of the video playback layer 304 and the background layer 302 into a landscape mode display. In the example shown, the background layer 302 is gradually eliminated during the transition from portrait mode to landscape mode, and video playback layer 304 is gradually resized to fit the entire display... As such, the rotation animation is configured to present to the user an animation of the resizing of the video playback layer 304 to accommodate the orientation of the electronic device, as well as a gradual introduction or elimination of a background layer 302). Thus, it would have been obvious, in view of Zhou, to configure Li’s apparatus as claimed by detecting and performing an adjustment on the coordinates and shapes of layers. The motivation is to avoid a mismatch of mapping data between the DPU and the GPU, and improve the visual representation.
Claim 9 adds into claim 8 “wherein each rotation animation layer in the plurality of layers includes the set of coordinates and shapes with the adjustment” (Zhou, [0041] - As the electronic device 300 is rotated, a rotation animation presented on the display, showing a rotation of the video playback layer 304 and the background layer 302 into a landscape mode display. In the example shown, the background layer 302 is gradually eliminated during the transition from portrait mode to landscape mode, and video playback layer 304 is gradually resized to fit the entire display... As such, the rotation animation is configured to present to the user an animation of the resizing of the video playback layer 304 to accommodate the orientation of the electronic device, as well as a gradual introduction or elimination of a background layer 302), and “wherein each non-rotation animation layer in the plurality of layers includes the set of coordinates and shapes without the adjustment” (Li, [0108] - In some examples, the region of interest coordinates determining component 798 may be configured to reuse coordinates defining a boundary of the identified layer of interest as the coordinates for the region of interest). Thus, it would have been obvious, in view of Zhou, to configure Li’s apparatus as claimed by detecting and performing an adjustment on the shapes of layers on a rotated frame and not perform an adjustment on a non-rotated whole frame. The motivation is to avoid a mismatch of mapping data between the DPU and the GPU, and improve the visual representation.
Claim 10 adds into claim 1 “wherein the rotation animation layer is a layer that rotates during an animation at the DPU or the GPU” (Zhou, [0044] - Referring now to FIG. 2, techniques for improved HDR video rotation animation will be described. Initially, and in response to determining that the electronic device is being rotated, the WMS 204 may start a screen rotation process by requesting that the GPU 106 utilize the animation transformation module 206 to generate a screen rotation animation, and requesting a screen capture buffer ( e.g., HDR buffer 220 and nonHDR buffer 222) from the compositor 208 of the DPU 104. That is, the DPU 104 receives an indication that a frame rotation animation process for video playback has been initiated), and “wherein the non-rotation animation layer is a layer that does not rotate during the animation at the DPU or the GPU” (Li, [0029] - In general, examples disclosed herein provide techniques for performing regional processing of a frame. For example, examples disclosed herein provide techniques for performing one or more display processing techniques to improve visual aspects of a region of interest of a frame and avoiding performing one or more display processing techniques on the full frame. Example techniques may improve visual quality of regions of the frame associated with layers of interest and/or reduce the load of a processing unit (e.g., any processing unit configured to perform one or more techniques disclosed herein, such as an application processor, a CPU, a graphics processor, a GPU, a display processor, a DPU, and the like)) (Noted: Li’s avoiding performing one or more display processing techniques on the full frame means the DPU 104 does not perform the rotation of a whole frame during the animation). Thus, it would have been obvious, in view of Zhou, to configure Li’s apparatus as claimed by detecting and performing the frame rotation animation process by the GPU. The motivation is to avoid a mismatch of mapping data between the DPU and the GPU, and improve the visual representation.
Claim 11 adds into claim 1 “wherein the at least one processor is further configured to: mark, based on mapping each of the plurality of layers for the processing at the DPU or the GPU, each of the plurality of layers that are mapped for the processing at the DPU” (Zhou, [0033] - The DPU 104 may perform one or more image processing operations on the frames, and output the processed frames to the display device 224 for display. Such image processing operations may include format converting, scaling, rotation, blending, and compositing, layering of the image with additional graphics, and the like; [0044] - in response to determining that the electronic device is being rotated, the WMS 204 may start a screen rotation process by requesting that the GPU 106 utilize the animation transformation module 206 to generate a screen rotation animation... That is, the DPU 104 receives an indication that a frame rotation animation process for video playback has been initiated; [0048] - In this example, the frame used by the GPU 106 to generate the rotation animation is an HDR format frame. Thus, the rotation animation data is in an HDR format. Accordingly, by closing one or more components of the DPU 104, less power is consumed for image/video processing. Moreover, because the rotation animation is already in an HDR format, by bypassing components of the display processing pipeline, the rotation animation does not experience degradation due to a mismatch of mapping data between the DPU 104 and the GPU 106). Thus, it would have been obvious, in view of Zhou, to configure Li’s apparatus as claimed by marking and performing the frame rotation animation process by the GPU. The motivation is to avoid a mismatch of mapping data between the DPU and the GPU, and improve the visual representation.
Claim 12 adds into claim 1 “wherein the first per-layer processor at the DPU is assigned to each rotation animation layer in the plurality of layers” (Zhou, [0033] - The DPU 104 may perform one or more image processing operations on the frames, and output the processed frames to the display device 224 for display. Such image processing operations may include format converting, scaling, rotation, blending, and compositing, layering of the image with additional graphics, and the like) “if an available layer size at the DPU is greater than a layer size threshold” which is obvious in order for the DPU being able to perform the rotation operation on the regions of interest (ROIs) of a frame including multiple layers. Thus, it would have been obvious, in view of Zhou, to configure Li’s apparatus as claimed to require the DPU’s available layer size being greater than a layer size threshold. The motivation is to allow the DPU performing the rotation of the regions of interest (ROIs) of a frame including multiple layers.
Claim 13 adds into claim 12 “wherein the layer size threshold is configurable or adjustable by a central processing unit (CPU)” (Li, [0074], [0081] - In some examples, the region of interest determining component 144 may modify the positional information to determine the coordinates for the region of interest. For example, the region of interest determining component 144 may select a sub-region based on the positional information, may scale-up the positional information to increase the size of the region of interest relative to the size of the layer of interest, or may scale-down the positional information to decrease the size of the region of interest relative to the size of the layer of interest).
Claim 14 adds into claim 1 “wherein the plurality of layers is associated with a layer stack or a regional layer stack” (Li, [0042] - The rendered layer may also be associated with an overlay position that defines a position in an overlay stack corresponding to the rendered layer. For example, the overlay stack may include the plurality of layers of a frame, with each rendered layer corresponding to a different layer level).
Claim 15 adds into claim 1 “at least one of an antenna or a transceiver coupled to the at least one processor, wherein the at least one processor is configured to obtain the indication of the at least one frame via at least one of the antenna or the transceiver” (Li, [0104], Figure 7 - The receiver 728 and the transmitter 730 may be combined into a transceiver 732. In such examples, the transceiver 732 may be configured to perform any receiving function and/or transmitting function described herein with respect to the device 704), and wherein the frame buffer corresponds to a composition output at the GPU” (Li, [0098] - In examples in which the display client 731 is operating in the command mode, the processing unit 720 (and/or the display processor) may write the graphical content of a frame to a buffer. In some examples, one or more aspects of the buffer may be implemented by the frame buffer 134 of FIG. 1; [0103] - The processing unit 720 may be a central processing unit (CPU), an application processor, a graphics processing unit (GPU), a graphics processor, a general purpose GPU (GPGPU), a display processing unit (DPU), a display processor, or any other processing unit that may be configured to perform display or graphics processing).
Claim 16 adds into claim 1 “transmit a second indication of the assignment of the first per-layer processor at the DPU to each rotation animation layer in the plurality of layers” (Zhou, [0044] - in response to determining that the electronic device is being rotated, the WMS 204 may start a screen rotation process by requesting that the GPU 106 utilize the animation transformation module 206 to generate a screen rotation animation... That is, the DPU 104 receives an indication that a frame rotation animation process for video playback has been initiated; [0048] - In this example, the frame used by the GPU 106 to generate the rotation animation is an HDR format frame. Thus, the rotation animation data is in an HDR format. Accordingly, by closing one or more components of the DPU 104, less power is consumed for image/video processing. Moreover, because the rotation animation is already in an HDR format, by bypassing components of the display processing pipeline, the rotation animation does not experience degradation due to a mismatch of mapping data between the DPU 104 and the GPU 106) or “the at least one second per-layer processor at the DPU to each of the set of ROIs in the frame buffer for each non-rotation animation layer in the plurality of layers” (Li, [0057]-[0059] – the region of interest display processing component 148 may receive (or retrieve) a composited frame (or an identifier or memory location at the frame buffer 134 associated with the composited frame) from the region of interest determining component 144. The region of interest display processing component 148 may also receive (or retrieve) coordinates associated with a region of interest of the composited frame. For example, the region of interest may correspond to the layer of interest identified by the region of interest determining component 144. In some such examples, the region of interest display processing component 148 may perform the one or more display processing techniques on the composited frame at the region bounded by the received coordinates). Thus, it would have been obvious, in view of Zhou, to configure Li’s apparatus as claimed by detecting and performing the frame rotation animation process by the GPU. The motivation is to avoid a mismatch of mapping data between the DPU and the GPU, and improve the visual representation.
Claims 17-28, 29, and 30 claim a method, an apparatus, and a computer-readable medium based on the apparatus of claims 1-16; therefore, they are rejected under a similar rationale.
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 30 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claimed “computer-readable medium” can be a wave carrier embodying the signals.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHU K NGUYEN whose telephone number is (571)272-7645. The examiner can normally be reached M-F 8-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F. Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHU K NGUYEN/Primary Examiner, Art Unit 2616
Prosecution Insights