DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The office action is in response to Applicant’s amendment filed 07/28/2026 which has been entered and made of record. Claims 1 and 15 have been amended. No claim has been newly added. Claims 1-20 are pending in the application.
Response to Arguments
Applicant’s arguments, filed 07/28/2026, on Page 8, First paragraph, with respect to double patenting rejection(s) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of double patenting rejection is made as fully explained below.
Applicant’s arguments, filed 07/28/2026, on Page 9, First and second paragraphs, with respect to the rejection(s) under 35 U.S.C. 103 have been fully considered, however they are not persuasive.
Applicant argues Fainstain, Arvo, and Luo do not teach the newly amended limitation of "the first subsample geometry pattern representing a surrounding pattern of the first pixel” and “the first blending weight vector being obtained based on a mapping relationship of different blending weight vectors associated with different subsample geometry patterns" as discussed during the interview.
Examiner respectfully disagrees. First, during the attorney interview, applicant suggests further defining “geometry pattern” to overcome edge pattern and further defining mapping between geometry pattern and weight function. The amendment does not include any distinction between geometry pattern and edge pattern.
Second, Arvo teaches the newly added claim language of "the first subsample geometry pattern representing a surrounding pattern of the first pixel”. Arvo’s edge mask represents a surrounding pattern of pixel. An edge mask can be thought of as a pattern of pixels that highlights the boundaries or “surrounding” changes between neighboring pixels in an image. It’s not just a single pixel, but a local neighborhood pattern that detects where pixel intensity changes sharply.
Third, examiner agrees Fainstain, Arvo, and Luo do not teach the newly added limitation of “the first blending weight vector being obtained based on a mapping relationship of different blending weight vectors associated with different subsample geometry patterns". However, Liao teaches a table mapping in Figure 13, edge pattern is used as an index to the table to locate an edge function indicating location of subpixels, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of edge function with Luo’s blending weight vectors with edge mask.
Conclusions: The rejections set in the previous Office Action are shown to have been proper, and the claims are rejected below. New citations and parenthetical remarks can be considered new grounds of rejection and such new grounds of rejection are necessitated by the Applicant's amendments to the claims. Therefore, the present Office Action is made final.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-10 and 12-19 of U.S. Patent No. 12536612. Although the claims at issue are not identical, they are not patentably distinct from each other.
It is noted that the instant application is a later-filed continuation of US Patent No. 12536612. It is also noted that both the instant application and 12536612 were filed by the same inventive entity and by a common assignee/owner. Claims 1 of 12536612 recites all the limitations of Claims 1 of the instant application, while also recite further limitations.
The claim 1 of the instant application recites “the first subsample geometry pattern including a plurality of indicators that respectively indicate comparison results of pairs in the first subsamples; the first subsample geometry pattern representing a surrounding pattern of the first pixel”; claim 1 of 12536612 recites “the first subsample geometry pattern being generated based on differences between values of subsamples in the surrounding region of the first pixel” as shown below, the claim language of “plurality of indicators that respectively indicate comparison results of pairs in the first subsamples” is obvious over the claim language of “differences between values of subsamples” as the comparison results of pairs is same as the difference between subsamples. The claim language of “representing a surrounding pattern of the first pixel” is obvious over the claim language of “the first subsample geometry pattern …… in the surrounding region of the first pixel” as the geometry patten is defined in the surrounding region of the pixel, it is a surrounding pattern. Therefore 12536612 anticipates the claims of the instant application. The claims of the instant application therefore are not patentably distinct from the earlier patent claims and as such are unpatentable for nonstatutory double patenting.
Claim mapping between current application and US Patent No. 12536612
Current
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
12536612
1
2
3
4
5
6
7
8
9
10
1
12
13
14
15
16
17
18
19
15
Current Application
12536612
Claim 1
Claim 1
A method for image rendering, the method comprising: obtaining, by a graphics processing unit (GPU), a first multi-sample anti-aliasing (MSAA) intermediate buffer from a rendering of a first image, the first MSAA intermediate buffer comprising a plurality of subsamples for each pixel of the first image;
determining, for a first pixel of the first image, a first subsample geometry pattern based on first subsamples in a surrounding region of the first pixel, the first subsample geometry pattern including a plurality of indicators that respectively indicate comparison results of pairs in the first subsamples;
the first subsample geometry pattern representing a surrounding pattern of the first pixel;
obtaining, based on the first subsample geometry pattern,
a first blending weight vector for a first location within the first pixel,
the first location being among a plurality of locations that define output pixel locations within the first pixel for upscaling the first pixel to a plurality of output pixels in an upscaled image of higher resolution than the first image, the first blending weight vector including first blending weights respectively for first candidate subsamples associated with the first pixel,
the first blending weight vector being obtained based on a mapping relationship of different blending weight vectors associated with different subsample geometry patterns;
and generating the upscaled image corresponding to the first image, a value for a first output pixel of the upscaled image at the first location within the first pixel of the first image being calculated as a weighted sum of the first candidate subsamples according to the first blending weights.
A method for image rendering in an electronic device, comprising: obtaining, by a graphics processing unit (GPU), a first multi-sample anti-aliasing (MSAA) intermediate buffer from a rendering of a first image, the first MSAA intermediate buffer comprising a plurality of subsamples for each pixel of the first image;
determining, for a first pixel of the first image, a first subsample geometry pattern based on first subsamples in a surrounding region of the first pixel, the first subsample geometry pattern being generated based on differences between values of subsamples in the surrounding region of the first pixel;
obtaining, based on a lookup table,
a first blending weight vector for a first location within the first pixel that is associated with the first subsample pattern in the lookup table,
the first location being among a plurality of locations that define output pixel locations within the first pixel for upscaling the first pixel to a plurality of output pixels in an upscaled image of higher resolution than the first image, the first blending weight vector including first blending weights respectively for first candidate subsamples associated with the first pixel,
the lookup table comprising a set of blending weight vectors that are associated with respective subsample geometry patterns;
and generating the upscaled image corresponding to the first image, a value for a first output pixel of the upscaled image at the first location within the first pixel of the first image being calculated as a weighted sum of the first candidate subsamples according to the first blending weights.
Claim 2
Claim 2
wherein the rendering of the first image generates for a pixel of the first image, a plurality of subsamples that have a rotated grid supersampling (RGSS) pattern.
wherein the rendering of the first image generates for a pixel of the first image, a plurality of subsamples that have a rotated grid supersampling (RGSS) pattern.
Claim 11
Claim 1
wherein the obtaining the first blending weight vector further comprises: determining an index to a lookup table according to the first subsample geometry pattern, the lookup table comprising a set of blending weight vectors that are indexed according to subsample geometry patterns.
…… the lookup table comprising a set of blending weight vectors that are associated with respective subsample geometry patterns; ……
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4-11, 15 and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fainstain et al. (IDS US 20170206626 A1), hereinafter as Fainstain, in view of Arvo et al. (IDS US 20130293565 A1), hereinafter as Arvo, further in view of IDS NPL Luo et al. (“The #-Filter Anti-Aliasing Based on Sub-Pixel Continuous Edges”), hereinafter as Luo, and Liao et al. (US 20060170703 A1), hereinafter as Liao.
Regarding claim 1, Fainstain teaches A method for image rendering (Fainstain paragraph [0040] “This shader unit receives an image, texture, or other input pixel data, and the shader unit performs a hybrid anti-aliasing resolve operation on the image to produce an anti-aliased image as output.”), the method comprising:
obtaining, by a graphics processing unit (GPU), a first multi-sample anti-aliasing (MSAA) intermediate buffer from a rendering of a first image (Fainstain paragraph [0028] “The processor is configured to store the plurality of sub-pixel sampling coordinates in the memory. Then, the processor utilizes the plurality of sub-pixel sampling coordinates to sample sub-pixel locations within each pixel of the image being rendered by the plurality of execution units.” And paragraph [0035] “GPU 130 expands dimensions of the first image to create a second image, filters the second image with a post-processing anti-aliasing filter to create a third image, and then performs averaging of the third image to create a fourth image, wherein the fourth image is a result of the anti-aliasing resolve operation.”), the first MSAA intermediate buffer comprising a plurality of subsamples for each pixel of the first image (Fainstain Figure 10, paragraph [0033] “GPU 130 includes at least sub-pixel coordinate table 135 and compute units 145A-N which are representative of any number and type of compute units that are used for graphics or general-purpose processing. Sub-pixel coordinate table 135 is a programmable table which stores the coordinates of sub-pixel sampling locations within the pixels of an image being rendered by GPU 130. It is noted that the term “sub-pixel sampling locations” is defined as the multiple locations for sampling the value of a parameter (e.g., color, depth, stencil, transparency) within a given pixel of an image being rendered. The term “sub-pixel sampling locations” indicates that there will be multiple sampling locations within each pixel of the image being rendered.” And paragraph [0063] “image 1005 with dimensions width (W) by height (H) utilizes multi-sampling (i.e., sub-pixel sampling) based on a 2×2 ordered grid (OG) as shown in diagram 1010.”);
Fainstain is not relied on for the below claim language determining, for a first pixel of the first image, a first subsample geometry pattern based on first subsamples in a surrounding region of the first pixel, the first subsample geometry pattern including a plurality of indicators that respectively indicate comparison results of pairs in the first subsamples, the first subsample geometry pattern representing a surrounding pattern of the first pixel; obtaining, based on the first subsample geometry pattern, a first blending weight vector for a first location within the first pixel, the first location being among a plurality of locations that define output pixel locations within the first pixel for upscaling the first pixel to a plurality of output pixels in an upscaled image of higher resolution than the first image, the first blending weight vector including first blending weights respectively for first candidate subsamples associated with the first pixel, the first blending weight vector being obtained based on a mapping relationship of different blending weight vectors associated with different subsample geometry patterns; and generating the upscaled image corresponding to the first image, a value for a first output pixel of the upscaled image at the first location within the first pixel of the first image being calculated as a weighted sum of the first candidate subsamples according to the first blending weights.
Arvo teaches determining, for a first pixel of the first image, a first subsample geometry pattern based on first subsamples in a surrounding region of the first pixel (Arvo teaches an edge mask as the subsample geometry pattern, edge is a type of geometry primitives, paragraph [0055] “Assume also that each pixel is sampled using a four sample MSAA scheme. After detecting edge pixels, coding unit 60 may generate an edge mask for the block that identifies the locations of the edge pixels. For example, coding unit 60 may generate a 16 bit edge mask that identifies the locations of edge pixels in the 4.times.4 block (e.g., 1 bit per pixel, where 0 represents a non-edge pixel and 1 represents an edge pixel, or vice versa).”), the first subsample geometry pattern including a plurality of indicators that respectively indicate comparison results of pairs in the first subsamples (Arvo teaches comparison between subsamples to decide edge pixel, in order to generate edge mask, paragraph [0054] “Coding unit 60 may detect an edge pixel by identifying samples associated with a pixel that have differing values (e.g., at least one color, brightness, and/or depth value that is not equal to the other samples associated with the other samples of the pixel).”, paragraph [0044] “a "pixel" may refer to a single fragment that is visible in an image. A "sample" may refer to a single value that contributes to a pixel. For example, a sample may include a color value (e.g., a single RGBA value) that contributes to one pixel in the final image…… in a four sample MSAA mode, if all four samples are not equal, the corresponding pixel may be classified as an edge pixel.”), the first subsample geometry pattern representing a surrounding pattern of the first pixel (Arvo teaches the edge mask generated based on the surrounding features of the pixel, paragraph [0080] “GPU 36 may generate an edge mask to indicate which pixels are edge pixels. In the example shown in FIGS. 3A-3C, the edge mask may be 0100 (e.g., in raster order, with the 1 indicating 122 B as the edge pixel).”).
Fainstain and Arvo are in the same field of endeavor, namely image processing, especially in the field of antialiasing. Arvo teaches a method of deciding edge pixels for multi-sample anti-aliasing to reduce the amount of data between GPU and memory (Arvo paragraph [0048] “The bandwidth between local GPU memory 38 and system memory may be further strained when GPU 36 uses multi-sample anti-aliasing (MSAA) techniques due to the additional samples associated with MSAA.” And paragraph [0050] “Aspects of this disclosure generally relate to compressing graphics data, which may reduce the amount of data that needs to be transferred between GPU memory 38 and system memory).”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Arvo with the method of Fainstain to achieve a better antialiasing result with MSAA.
Fainstain in view of Arvo are not relied on for the below claim language obtaining, based on the first subsample geometry pattern, a first blending weight vector for a first location within the first pixel, the first location being among a plurality of locations that define output pixel locations within the first pixel for upscaling the first pixel to a plurality of output pixels in an upscaled image of higher resolution than the first image, the first blending weight vector including first blending weights respectively for first candidate subsamples associated with the first pixel; and generating the upscaled image corresponding to the first image, a value for a first output pixel of the upscaled image at the first location within the first pixel of the first image being calculated as a weighted sum of the first candidate subsamples according to the first blending weights.
Luo teaches obtaining, based on the first subsample geometry pattern, a first blending weight vector for a first location within the first pixel, the first location being among a plurality of locations that define output pixel locations within the first pixel (Luo teaches determining an edge mask for each subsample of MSAA, and further teaches combining it in equation 7 with the blending weight and color information of the subsample, Page 7, Figure 6. Arvo teaches using a geometry edge pattern mask, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Luo with Arvo)
PNG
media_image1.png
283
620
media_image1.png
Greyscale
for upscaling the first pixel to a plurality of output pixels in an upscaled image of higher resolution than the first image (Luo does not explicitly teach higher resolution of output image, Fainstain teaches the resulting image with higher resolution, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Luo with Fainstain with Arvo to have a better antialiasing result for high resolution image. Fainstain paragraph [0064] “the averaging filter can change the resolution of image 1025 to a different resolution than the original image 1005…… with the resultant image having a larger resolution than the starting image.”), the first blending weight vector including first blending weights respectively for first candidate subsamples associated with the first pixel (Luo teaches in question 7, a kernel as blending weight vector, an edge mask for subsamples to get candidate subsamples. Page 4, first paragraph, “The weight of #-filter is an array of size n, and only 12 sub-pixels are used in the four samples. In order to improve the calculation performance, we currently use the kernel, as shown in Figure 5(middle), but the kernel can be expanded according to rendering requirements……. Moreover, the common filtering methods can be used as weights, such as low pass filtering, high pass filtering, directional filtering, Laplacian filtering, and Gaussian filtering”); …… and generating the upscaled image corresponding to the first image, a value for a first output pixel of the upscaled image at the first location within the first pixel of the first image being calculated as a weighted sum of the first candidate subsamples according to the first blending weights (Luo Page 4, first paragraph “the #-filter is represented by an array wi with the weights, and the color c of each pixel is calculated by adjacent sub-pixels ci, as in (7).
c
=
∑
i
=
0
n
e
w
i
c
i
”).
Fainstain, Arvo and Luo are in the same field of endeavor, namely image processing. Luo teaches a method of using edge detection filter for anti-aliasing based on sub-pixel to solve geometry edges aliasing and the flickering problem (Luo Page 1, abstract “It can solve the geometry edges aliasing and the flicker problem in deferred shading.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Luo with the method of Fainstain in view of Arvo to improve the result of MSAA.
Fainstain in view of Arvo and Luo is not relied on for the below claim language ……the first blending weight vector being obtained based on a mapping relationship of different blending weight vectors associated with different subsample geometry patterns……
Liao teaches ……the first blending weight vector being obtained based on a mapping relationship of different blending weight vectors associated with different subsample geometry patterns…… (Liao teaches using a look up table as the mapping between geometry patten and weight factors in Figure 13, edge pattern is used as an index to the table to locate an edge function indicating location of subpixels, Luo teaches blending weight vectors with edge mask, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of lookup table indexed by edge pattern with the blending weights of Luo, paragraph [0059] “Reference is now made to the second row of the table, where the compression type value 1310 of "01" indicates that one edge intersects multiple pixels within the tile. In this case, the data field value 1320 is "11xxxx", where the "xxxx" is an edge pattern number and the "11" is an unutilized portion of the compression code. The edge pattern number is used to obtain an edge function from a lookup table. The edge function will be used to determine where each sub-pixel is located relative to the edge in each of the partially covered pixels.”),
Fainstain, Arvo, Luo and Liao are in the same field of endeavor, namely image processing, especially in the field of antialiasing. Liao teaches using a lookup table with edge pattern number to reduce memory traffic and improve efficiency (Liao paragraph [0038] “the reduction of data transmitted between the GPU and the frame buffer using the process described herein greatly improves the efficiency of the programmable GPU processor.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liao with the method of Fainstain in view of Arvo and Luo to improve efficiency.
Regarding claim 4, Fainstain in view of Arvo, Luo and Liao teach The method of claim 1, and further teach wherein the first subsamples in the surrounding region of the first pixel comprises a plurality of subsamples in the first pixel (Fainstain Figure 5, “10 samples per pixel”, paragraph [0051] “Diagram 515 illustrates the sub-pixel sampling locations for a single pixel.”).
Regarding claim 5, Fainstain in view of Arvo, Luo and Liao teach The method of claim 4, and further teach wherein the first subsamples in the surrounding region of the first pixel comprises one or more subsamples in one or more adjacent pixels of the first pixel (Luo Figure 5 and 6, a Kernel with 12 bits covering four pixels, Page 6, last paragraph, “The two shading conditions of one sub-pixel covered by the fragment in different sample ways are shown in Figure 5. Compared with the anti-aliasing shading of the single pixel, the shading result is closer to the practical geometry condition from the neighboring sub-pixel (see Figure 5(a4,b4)). For example, the common sub-pixel anti-aliasing only focuses on the local pixels rather than the whole edge pixels. It will lose the detail of the edges. Meanwhile, more pixels can build more accurate sub-pixel edges, but it will increase performance. Usually, four neighborhood samples are enough to balance performance and effect. “).
Fainstain, Arvo, Luo and Liao are in the same field of endeavor, namely image processing. Luo teaches a method of using edge detection filter for anti-aliasing based on sub-pixel to solve geometry edges aliasing and the flickering problem (Luo Page 1, abstract “It can solve the geometry edges aliasing and the flicker problem in deferred shading.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Luo with the method of Fainstain, Arvo and Liao to improve the result of MSAA.
Regarding claim 6, Fainstain in view of Arvo, Luo and Liao teach The method of claim 1, and further teach wherein the first subsample geometry pattern is a subsample difference pattern (Arvo teaches an edge mask as the subsample difference pattern, paragraph [0077-0080] “FIGS. 3A-3C are conceptual diagrams illustrating encoding an example 2.times.2 block of pixels 120. In the example shown in FIGS. 3A-3C each pixel 122A-122D has four associated samples (shown as boxes in pixels 122), which represents a 4.times. MSAA scheme…… GPU 36 may generate an edge mask to indicate which pixels are edge pixels. In the example shown in FIGS. 3A-3C, the edge mask may be 0100 (e.g., in raster order, with the 1 indicating 122B as the edge pixel)”), and the determining the first subsample geometry pattern further comprises: determining a difference between two subsamples in the first subsamples (Arvo paragraph [0054] “Coding unit 60 may detect an edge pixel by identifying samples associated with a pixel that have differing values (e.g., at least one color, brightness, and/or depth value that is not equal to the other samples associated with the other samples of the pixel).”); and determining a bit in the first subsample geometry pattern based on the difference (Arvo paragraph [0055] “Assume also that each pixel is sampled using a four sample MSAA scheme. After detecting edge pixels, coding unit 60 may generate a 16 bit edge mask that identifies the locations of edge pixels in the 4.times.4 block (e.g., 1 bit per pixel, where 0 represents a non-edge pixel and 1 represents an edge pixel, or vice versa).”).
Fainstain, Arvo, Luo and Liao are in the same field of endeavor, namely image processing, especially in the field of antialiasing. Arvo teaches a method of deciding edge pixels for multi-sample anti-aliasing to reduce the amount of data between GPU and memory (Arvo paragraph [0048] “The bandwidth between local GPU memory 38 and system memory may be further strained when GPU 36 uses multi-sample anti-aliasing (MSAA) techniques due to the additional samples associated with MSAA.” And paragraph [0050] “Aspects of this disclosure generally relate to compressing graphics data, which may reduce the amount of data that needs to be transferred between GPU memory 38 and system memory).”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Arvo with the method of Fainstain in view of Luo and Liao to achieve a better antialiasing result with MSAA.
Regarding claim 7, Fainstain in view of Arvo, Luo and Liao teach The method of claim 6, and further teach wherein the difference is at least one of a color difference, a luminance difference, and a depth difference (Arvo paragraph [0054] “Coding unit 60 may detect an edge pixel by identifying samples associated with a pixel that have differing values (e.g., at least one color, brightness, and/or depth value that is not equal to the other samples associated with the other samples of the pixel).”).
Fainstain, Arvo, Luo and Liao are in the same field of endeavor, namely image processing, especially in the field of antialiasing. Arvo teaches a method of deciding edge pixels for multi-sample anti-aliasing to reduce the amount of data between GPU and memory (Arvo paragraph [0048] “The bandwidth between local GPU memory 38 and system memory may be further strained when GPU 36 uses multi-sample anti-aliasing (MSAA) techniques due to the additional samples associated with MSAA.” And paragraph [0050] “Aspects of this disclosure generally relate to compressing graphics data, which may reduce the amount of data that needs to be transferred between GPU memory 38 and system memory).”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Arvo with the method of Fainstain in view of Luo and Liao to achieve a better antialiasing result with MSAA.
Regarding claim 8, Fainstain in view of Arvo, Luo and Liao teach The method of claim 1, and further teach wherein the first subsamples that are used to generate the first subsample geometry pattern are of a same set of subsamples as the first candidate subsamples (Luo teaches all 4 subsamples of a pixel covering by triangle, they are the same set of the candidate subsamples. Page 4, Figure 2(b), “Coverage information: The G channel of the RT2 stores coverage, which is the sample mask of each sub-pixel with MSAA, as shown in Figure 2. ….. all covered by one fragment (see Figure 2b).”).
Fainstain, Arvo, Luo and Liao are in the same field of endeavor, namely image processing. Luo teaches a method of using edge detection filter for anti-aliasing based on sub-pixel to solve geometry edges aliasing and the flickering problem (Luo Page 1, abstract “It can solve the geometry edges aliasing and the flicker problem in deferred shading.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Luo with the method of Fainstain, Arvo and Liao to improve the result of MSAA.
Regarding claim 9, Fainstain in view of Arvo, Luo and Liao teach The method of claim 1, and further teach wherein the first subsamples that are used to generate the first subsample geometry pattern include the first candidate subsamples (Luo teaches one subsamples of a pixel covering by triangle, forming a single candidate subsample among all 4 subsamples. Page 4, Figure 2(b), “Coverage information: The G channel of the RT2 stores coverage, which is the sample mask of each sub-pixel with MSAA, as shown in Figure 2. ….. one sub-pixel covered by one fragment (see Figure 2d)”).
Fainstain, Arvo, Luo and Liao are in the same field of endeavor, namely image processing. Luo teaches a method of using edge detection filter for anti-aliasing based on sub-pixel to solve geometry edges aliasing and the flickering problem (Luo Page 1, abstract “It can solve the geometry edges aliasing and the flicker problem in deferred shading.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Luo with the method of Fainstain, Arvo and Liao to improve the result of MSAA.
Regarding claim 10, Fainstain in view of Arvo, Luo and Liao teach The method of claim 1, and further teach wherein the first candidate subsamples include the first subsamples that are used to generate the first subsample geometry pattern (Luo teaches in equation 7 a weighted sum of 12 candidate subsamples among 4 pixels, in the case all 12 subsamples are defined as edge, they all contribute to the weighted sum final result. Page 7, Figure 6).
Fainstain, Arvo, Luo and Liao are in the same field of endeavor, namely image processing. Luo teaches a method of using edge detection filter for anti-aliasing based on sub-pixel to solve geometry edges aliasing and the flickering problem (Luo Page 1, abstract “It can solve the geometry edges aliasing and the flicker problem in deferred shading.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Luo with the method of Fainstain, Arvo and Liao to improve the result of MSAA.
Regarding claim 11, Fainstain in view of Arvo, Luo and Liao teach The method of claim 1, wherein the obtaining the first blending weight vector further comprises: and further teach determining an index to a lookup table according to the first subsample geometry pattern, the lookup table comprising a set of blending weight vectors that are indexed according to subsample geometry patterns (Liao teaches a look up table in Figure 13, edge pattern is used as an index to the table to locate an edge function indicating location of subpixels, Luo teaches blending weight vectors with edge mask, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of lookup table indexed by edge pattern with the blending weights of Luo, paragraph [0059] “Reference is now made to the second row of the table, where the compression type value 1310 of "01" indicates that one edge intersects multiple pixels within the tile. In this case, the data field value 1320 is "11xxxx", where the "xxxx" is an edge pattern number and the "11" is an unutilized portion of the compression code. The edge pattern number is used to obtain an edge function from a lookup table. The edge function will be used to determine where each sub-pixel is located relative to the edge in each of the partially covered pixels.”, paragraph [0065] “ If this is the case, then the edge pattern number is read in block 1554, the corresponding edge function is looked up in a table based on the edge pattern number 1556, and sub-pixels in partially covered pixels are checked to determine which side of the edge they are on 1558.”).
Fainstain, Arvo, Luo and Liao are in the same field of endeavor, namely image processing, especially in the field of antialiasing. Liao teaches using a lookup table with edge pattern number to reduce memory traffic and improve efficiency (Liao paragraph [0038] “the reduction of data transmitted between the GPU and the frame buffer using the process described herein greatly improves the efficiency of the programmable GPU processor.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Liao with the method of Fainstain in view of Arvo and Luo to improve efficiency.
Regarding claim 15, it recites similar limitations of claim 1 but in an image processing apparatus form. The rationale of claim 1 rejection is applied to reject claim 15. In addition, Fainstain teaches An image processing apparatus, comprising processing circuitry configured to (Fainstain paragraph [0031] “Referring now to FIG. 1, a block diagram of one embodiment of a computing system 100 is shown”).
Regarding claim 18, Fainstain in view of Arvo, Luo and Liao teach The image processing apparatus of claim 15, and further teach wherein the first subsamples in the surrounding region of the first pixel comprise a plurality of subsamples in the first pixel and one or more subsamples in one or more adjacent pixels of the first pixel (Luo Figure 5 and 6, a Kernel with 12 bits covering four pixels, Page 6, last paragraph, “The two shading conditions of one sub-pixel covered by the fragment in different sample ways are shown in Figure 5. Compared with the anti-aliasing shading of the single pixel, the shading result is closer to the practical geometry condition from the neighboring sub-pixel (see Figure 5(a4,b4)). For example, the common sub-pixel anti-aliasing only focuses on the local pixels rather than the whole edge pixels. It will lose the detail of the edges. Meanwhile, more pixels can build more accurate sub-pixel edges, but it will increase performance. Usually, four neighborhood samples are enough to balance performance and effect. “, Fainstain Figure 5, “10 samples per pixel”, paragraph [0051] “Diagram 515 illustrates the sub-pixel sampling locations for a single pixel.”).
Fainstain, Arvo, Luo and Liao are in the same field of endeavor, namely image processing. Luo teaches a method of using edge detection filter for anti-aliasing based on sub-pixel to solve geometry edges aliasing and the flickering problem (Luo Page 1, abstract “It can solve the geometry edges aliasing and the flicker problem in deferred shading.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Luo with the method of Fainstain, Arvo and Liao to improve the result of MSAA.
Regarding claim 19, claim 19 has similar limitations as claim 6, therefore it is rejected under the same rationale as claim 6.
Regarding claim 20, claim 20 has similar limitations as claim 11, therefore it is rejected under the same rationale as claim 11.
Claim(s) 2-3 and 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fainstain et al. (IDS US 20170206626 A1), hereinafter as Fainstain, in view of Arvo et al. (IDS US 20130293565 A1), hereinafter as Arvo, further in view of IDS NPL Luo et al. (“The #-Filter Anti-Aliasing Based on Sub-Pixel Continuous Edges”), hereinafter as Luo, Liao et al. (US 20060170703 A1), hereinafter as Liao and IDS RGSS (https://blog.demofox.org/2015/04/23/4-rook-antialiasing-rgss/), hereinafter as RGSS.
Regarding claim 2, Fainstain in view of Arvo, Luo and Liao teach The method of claim 1, but are not relied on for the below claim language wherein the rendering of the first image generates for a pixel of the first image, a plurality of subsamples that have a rotated grid supersampling (RGSS) pattern. RGSS teaches wherein the rendering of the first image generates for a pixel of the first image, a plurality of subsamples that have a rotated grid supersampling (RGSS) pattern (RGSS, Page 1, last paragraph, “4-Rook anti aliasing takes 4 samples per pixel, and does so in the pattern below with the specified blend weights. This pattern is also sometimes called rotated grid supersampling (RGSS) and is also a subset of “N-Rook” supersampling where your sample points within a pixel don’t share a vertical or horizontal line with any other sample point. The N-Rook sample patterns are good at breaking up horizontal or vertical aliasing”).
Fainstain, Arvo, Luo, Liao and RGSS are in the same field of endeavor, namely image processing, especially in the field of antialiasing. RGSS teaches 4 samples per pixel in the rotated grid supersampling pattern to achieve a better antialiasing result (RGSS pages 1-2, “The N-Rook sample patterns are good at breaking up horizontal or vertical aliasing. Interestingly, 4-Rook AA looks less blurry than quincunx so is higher quality.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of RGSS with the method of Fainstain in view of Arvo, Luo and Liao to achieve a better antialiasing result.
Regarding claim 3, Fainstain in view of Arvo, Luo, Liao and RGSS teach The method of claim 2, and further teach wherein the rendering of the first image generates four subsamples of 4x RGSS pattern for the pixel of the first image (RGSS page 2, first drawing, showing 4 subsamples in RGSS pattern).
Fainstain, Arvo, Luo, Liao and RGSS are in the same field of endeavor, namely image processing, especially in the field of antialiasing. RGSS teaches 4 samples per pixel in the rotated grid supersampling pattern to achieve a better antialiasing result (RGSS pages 1-2, “The N-Rook sample patterns are good at breaking up horizontal or vertical aliasing. Interestingly, 4-Rook AA looks less blurry than quincunx so is higher quality.”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of RGSS with the method of Fainstain in view of Arvo, Luo and Liao to achieve a better antialiasing result.
Regarding claim 16, claim 16 has similar limitations as claim 2, therefore it is rejected under the same rationale as claim 2.
Regarding claim 17, claim 17 has similar limitations as claim 3, therefore it is rejected under the same rationale as claim 3.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fainstain et al. (IDS US 20170206626 A1), hereinafter as Fainstain, in view of Arvo et al. (IDS US 20130293565 A1), hereinafter as Arvo, further in view of IDS NPL Luo et al. (“The #-Filter Anti-Aliasing Based on Sub-Pixel Continuous Edges”), hereinafter as Luo, Liao et al. (US 20060170703 A1), hereinafter as Liao, and Janis et al. (IDS US 20230206394 A1), hereinafter as Janis.
Regarding claim 12, Fainstain in view of Arvo, Luo and Liao teach The method of claim 11, but are not relied on for the below claim language further comprising: pre-training the set of blending weight vectors for the subsample geometry patterns by using at least a reference image of a resolution that is equal to or higher than the upscaled image. Janis teaches further comprising: pre-training the set of blending weight vectors for the subsample geometry patterns by using at least a reference image of a resolution that is equal to or higher than the upscaled image (Janis paragraph [0074] “In at least one embodiment, this upscaled image 210 can be provided as input to a neural network 212 to determine one or more blending factors or blending weights. In at least one embodiment, neural network 212 also receives as input a prior high resolution image in this sequence that is warped and provided to neural network 212 along with this upscaled image 210.”).
Fainstain, Arvo, Luo, Liao and Janis are in the same field of endeavor, namely image processing. Janis teaches a method of deciding blending weight based on neural network to improve image resolution (Janis paragraph [0045] “an upscaling process, such as a deep learning-based super sampling or super-resolution process, can be used to increase a resolution of one or more images, such as images or video frames in a sequence or video stream”). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Janis with the method of Fainstain in view of Arvo, Luo and Liao to improve the performance of MSAA.
Allowable Subject Matter
Claims 13-14 objected to as being dependent upon a rejected base claim, but would be allowable if overcame the double patenting rejection and rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 13, the closest prior art of Janis teaches using neural network to infer multiple reconstruction kernel parameters for up sampling and filtering input image (Janis paragraph [0049-0056] “a blending process can be described using these example quantities: g: Gating factor predicted by neural network; b: Jitter aware blending factor predicted by neural network; p: Parameters that form a reconstruction kernel predicted by a neural network, such as in a case of an anisotropic gaussian kernel, where these define a covariance matrix; j: Current input jitter vector, denoting sampling position within an input pixel; c.sub.u: Current input sample color at input pixel coordinate u (integer coordinates); h: Warped preceding frame output color; K(x, p): Value of a reconstruction kernel parametrized by p, at offset x. Kernel is centered at a respective output pixel.”). However, Janis fails to teach the combined limitation below as whole obtaining a second MSAA intermediate buffer from a rendering of a second image having a same camera setting as the reference image, the second image having a same resolution as the first image; collecting votes of candidate subsamples for a subsample geometry pattern based on the second MSAA intermediate buffer and the reference image; and determining blending weights in a blending weight vector associated with the subsample geometry pattern according to the votes of the candidate subsamples.
Furthermore, no prior art of record either alone or in combination teaches the limitation above as whole. Therefore, claim 13 is considered to be allowable.
Claim 14 contains allowable subject matter because they depend on claim 13 that contains allowable subject matter.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAOMING WEI whose telephone number is (571)272-3831. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
/XIAOMING WEI/Examiner, Art Unit 2611
Prosecution Insights