CTNF 18/896,490 CTNF 99370 Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Objections Claims 5 and 12 are objected to because of the following informalities: Line 1, “circuitry configured to” should read “circuitry is configured to”. Claim Rejections - 35 USC § 102 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-15 AIA Claim (s) 1-5, 8-9, 16-17, and 20 are rejected under 35 U.S.C. 102( a)(1 ) as being anticipated by Walker et al. (US Patent Pub. No. 2008/0292216 A1) . Regarding claim 1 , Walker teaches a device comprising: an electronic display configured to display an image frame based on processed image data (Fig. 2A, Para. 40, “Referring to FIG. 2A, there is shown an image processing application 250 comprising a mobile communication device 251 with a display screen 253”); and image processing circuitry configured to: process a first tile, of a plurality of tiles of the image frame, of input image data based on changing a first respective portion of the input image data to generate a first respective portion of the processed image data (Fig. 5, Para. 57, “in step 503 an image data tile may be received by the processor 205 from the image source 201. In step 505, the data tile may be processed utilizing the ISP block 209”); and process a second tile, of the plurality of tiles, of the input image data based on changing a second respective portion of the input image data to generate a second respective portion of the processed image data based on a sub-portion of the first respective portion of the input image data (Fig. 4, reprinted below, shows tiles being processed as well as sub-portions of neighboring tiles being used to process the image data of a second tile, PNG media_image1.png 389 715 media_image1.png Greyscale ). Regarding claim 2 , Walker teaches all of the elements of claim 1, as stated above, as well as wherein the image processing circuitry is configured to determine locations of the plurality of tiles relative to the image frame, wherein the plurality of tiles sub-divide the image frame for processing (Fig. 4, reprinted above). Regarding claim 3 , Walker teaches all of the elements of claim 1, as stated above, as well as wherein the image processing circuitry is configured to: fetch the first respective portion of the input image data corresponding to the first tile of the plurality of tiles; and fetch the second respective portion of the input image data corresponding to the second tile of the plurality of tiles (Fig. 4, reprinted above). Regarding claim 4 , Walker teaches all of the elements of claim 3, as stated above, as well as wherein respective portions of the input image data corresponding to the plurality of tiles are fetched from memory only once (Para. 52, “The data from the image source 401 may be received in tile format, as opposed to line-by-line. In this regard, the tiles or blocks of pixels may be addressed individually, as opposed to lines of pixels, which may greatly increase speed and reduce memory requirements of the image processing.”, there is no indication that the tiled image data is fetched from memory more than once ). Regarding claim 5 , Walker teaches all of the elements of claim 3, as stated above, as well as wherein the image processing circuitry configured to store the sub-portion of the first respective portion of the input image data via one or more buffers for use in processing the second tile (Para. 54, “In this manner. the memory requirements in processing the data may be reduced since less than a single tile may require storage in a buffer before the data may be processed by a processor enabled to handle data in the appropriate tile size.”, a tile may be stored in a buffer, with the sub-portion (overlap) being included ). Regarding claim 8 , Walker teaches all of the elements of claim 1, as stated above, as well as wherein the second tile is adjacent the first tile (Fig. 4, reprinted above). Regarding claim 9 , Walker teaches all of the elements of claim 1, as stated above, as well as wherein respective portions of the processed image data of the plurality of tiles do not overlap (Para. 53, “Each tile may overlap with neighboring tiles by a variable number of pixels, up to 64 for example, to reduce edge effects in the processing of adjacent tiles.”, tiles may overlap by a variable amount to reduce edge effects, indicating that tiles may also not overlap ). Claim 16 corresponds to claims 1 and 3 and is rejected under the same analysis. Regarding claim 17 , Walker teaches all of the elements of claim 16, as stated above, as well as wherein the operations comprise determining locations or sizes and the locations of the plurality of tiles relative to the image frame (Fig. 4, reprinted above). Regarding claim 20 , Walker teaches all of the elements of claim 16, as stated above, as well as wherein the second respective portion of the input image data is greater than the second respective portion of the processed image data, wherein the second respective portion of the input image data comprises neighbor data associated with pixel locations within the first tile, wherein the sub-portion of the first respective portion of the input image data comprises the neighbor data (Fig. 4, reprinted above, shows the overlapping data from the first tile being used in the processing of the second tile ) . Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Walker . Regarding claim 18 , Walker teaches wherein the sizes are determined based on the effects of a distortion associated with processing a single tile of the plurality of tiles. Walker does not explicitly disclose wherein the size and location are determined based on a latency requirement. However, the goal of their system is to increase efficiency and image processing speed. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Walker to include wherein the locations or the sizes and the locations are determined based on a latency requirement associated with processing a single tile of the plurality of tiles. Walker teaches that the size of the tiles is variable, and that tile size may be increased when there are few distortions to increase efficiency and image processing speed (Para. 56). They also disclose that the location of the tiles is known (Fig. 4). One of ordinary skill in the art would have recognized that optimizing for a higher speed, as disclosed by Walker, directly dictates the time required to process image data (latency). It would have been a matter of routine optimization and design choice to select specific sizes and locations of the tiles such that the processing time for a single tile satisfies a predetermined latency requirement . 07-21-aia AIA Claim (s) 6-7, 10-11, and 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Walker in view of Wang et al. (US Patent Pub. No. 2017/0372452 A1) . Regarding claim 6 , Walker teaches all of the elements of claim 1, as stated above. Walker does not explicitly disclose wherein the image processing circuitry comprises first memory-to-memory scale and rotate (MSR) circuitry, and wherein processing the first tile comprises scaling, rotating, or scaling and rotating the first respective portion of the input image data corresponding to the first tile via the first MSR circuitry. Wang teaches wherein the image processing circuitry comprises first memory-to-memory scale and rotate (MSR) circuitry, and wherein processing the first tile comprises scaling, rotating, or scaling and rotating the first respective portion of the input image data corresponding to the first tile via the first MSR circuitry (Para. 32, “Display processor 14 may pre-fetch or fetch tiles of an image for processing. Example processing that may be performed by display processor 14 may include up-sampling, down-sampling, scaling, rotation, and other pixel processing.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Walker to incorporate the teachings of Wang to include memory-to-memory scale and rotate (MSR) circuitry wherein processing the first tile comprises scaling, rotating, or scaling and rotating the first respective portion of the input image data corresponding to the first tile via the first MSR circuitry. Walker discloses a method for processing images by dividing them into tiles and performing multiple non-limiting processing techniques to efficiently reduce the memory needed to perform such techniques. Wang discloses the ability to section images into tiles and to perform scaling and rotation on these tiles. One of ordinary skill in the art would have recognized that implementing the rotation and scale processing of Wang as one of the image processing techniques of Walker would have been a predictable variation using known techniques. Regarding claim 7 , Walker as modified in view of Wang teaches all of the elements of claim 6, as stated above, as well as wherein the image processing circuitry comprises second MSR circuitry, and wherein processing the second tile comprises scaling, rotating, or scaling and rotating the second respective portion of the input image data corresponding to the second tile via the second MSR circuitry in parallel with the first MSR circuitry scaling, rotating, or scaling and rotating a third respective portion of the input image data corresponding to a third tile of the plurality of tiles (Para. 31, “However, it should be understood that image rotation engine 28 may be separate from display processor 14 (e.g., an application specific integrated circuit (ASIC)) or may be implemented in another processing device, such as video codec 7, DSP 11, GPU 12, or any other processor.”; Para. 40, “GPU 12 may, in some instances, be built with a highly-parallel structure that provides more efficient processing of complex graphic-related operations than CPU 6. For example, GPU 12 may include a plurality of processing elements, such as shader units, that are configured to operate on multiple vertices or pixels in a parallel manner.”; Para. 41, “Software application 18 may invoke GPU driver 22, to issue one or more commands to GPU 12 for rendering one or more graphics primitives into displayable graphics images… The primitive definitions may also include primitive type information (e.g., triangle, rectangle, triangle fan, triangle strip, etc.), scaling information, rotation information, and the like.”, Parallel processing is a well-known method to maximize hardware utilization. Although Wang does not explicitly disclose utilizing second MSR circuitry in parallel with the first MSR circuitry to perform scaling/rotating, they do disclose that the rotation and scaling processing may be performed on a GPU, and that the GPU includes a plurality of parallel units capable of operating on pixels. It would have been obvious to one of ordinary skill in the art to utilize the highly-parallel GPU structure of Wang to process multiple of the already disclosed tiles concurrently, reducing frame processing latency. ). Claim 10 corresponds to claims 1, 3, and 6 and is rejected under the same analysis. Regarding claim 11 , Walker as modified in view of Wang teaches all of the elements of claim 10, as stated above, as well as wherein respective portions of the input image data corresponding to the plurality of tiles are fetched via direct memory access (DMA) only once ( Wang ; Para. 54, “DMA fetch unit 30 fetches pixels values of an input image from a memory”). Regarding claim 13 , Walker as modified in view of Wang teaches all of the elements of claim 10, as stated above, as well as wherein the first respective portion of the input image data comprises neighbor data of the input image data corresponding to pixel locations outside the first tile and along an edge of the first tile (Para. 58, “A current variable size tile may overlap at least one neighboring variable size tile. At least one neighboring data tile may include one or more of: above the current variable size tile, below the current variable size tile, left of the current variable size tile, and right of the current variable size tile.”), and wherein respective portions of the processed image data of the plurality of tiles do not overlap (Fig. 4, multiple respective portions of the plurality of tiles do not overlap ). Claim 14 corresponds to claim 7 and is rejected under the same analysis. Claim 15 corresponds to claim 6 and is rejected under the same analysis . 07-21-aia AIA Claim (s) 12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Walker as modified in view of Wang, further in view of Cheung et al. (US Patent Pub. No. 2016/0165238 A1) . Regarding claim 12 , Walker as modified in view of Wang does not explicitly disclose wherein the image processing circuitry configured to store the sub-portion of the first respective portion of the input image data via one or more neighbor buffers. However, Walker discloses buffering data as well as considering the pixels of neighboring tiles. Cheung teaches wherein the image processing circuitry configured to store the sub-portion of the first respective portion of the input image data via one or more neighbor buffers (Fig. 3, Para. 27, “Accordingly, as illustrated by FIG. 3, the pixel data for Tile 3 is loaded into the tile buffer 142 and the bottom CTB row 306 of Tile 0 (as the top-adjacent tile to Tile 3) is in the top neighbor buffer 136”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Walker and Wang to incorporate the teachings of Cheung to include wherein the image processing circuitry configured to store the sub-portion of the first respective portion of the input image data via one or more neighbor buffers. Walker teaches a method for processing images by dividing them into tiles. They also consider neighboring tiles pixels when performing this processing, however they do not explicitly disclose storing these pixels in a neighbor buffer. Cheung teaches a system for dividing an image into tiles for processing, where the neighboring pixels are stored in a neighbor buffer for superior edge filtering. One of ordinary skill in the art would have understood that implementing the overlapped pixels of Walker as the neighbor buffer disclosed by Cheung would have been a straightforward combination to improve edge filtering of tiles. Claim 19 corresponds to claim 12 and is rejected under the same analysis. Conclusion Pertinent Prior Art: US 2019/0340491 A1, “SCALABLE NEURAL NETWORK PROCESSING ENGINE”, Norden et al., published 2019, Common applicant, similar MSR architecture. Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID A WAMBST whose telephone number is (703)756-1750. The examiner can normally be reached M-F 9-6:30 EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /DAVID ALEXANDER WAMBST/Examiner, Art Unit 2663 /GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698 Application/Control Number: 18/896,490 Page 2 Art Unit: 2663 Application/Control Number: 18/896,490 Page 3 Art Unit: 2663 Application/Control Number: 18/896,490 Page 4 Art Unit: 2663 Application/Control Number: 18/896,490 Page 5 Art Unit: 2663 Application/Control Number: 18/896,490 Page 6 Art Unit: 2663 Application/Control Number: 18/896,490 Page 7 Art Unit: 2663 Application/Control Number: 18/896,490 Page 8 Art Unit: 2663 Application/Control Number: 18/896,490 Page 9 Art Unit: 2663 Application/Control Number: 18/896,490 Page 10 Art Unit: 2663 Application/Control Number: 18/896,490 Page 11 Art Unit: 2663
Prosecution Insights