DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
1. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 3/13/26 has been entered.
Response to Amendment
2. This action is in response to the amendment filed on 3/13/2026. Claims 1-3, 7-12, and 17-18 have been amended. Claim 8 presented new limitations that were not properly indicated as amended. Claims 1-20 remain rejected in the application.
Response to Arguments
3. Applicant’s arguments with respect to claim 1, and similarly claims 17 and 18, with respect to the rejection under 35 U.S.C. 103 regarding that the prior art does not teach the limitation(s): "based on comparing: an interpolation feature describing complexity of image content of a first pixel block in the first image; and a complexity threshold ..." and "performing the first interpolation on the first pixel block if the interpolation feature does not exceed the complexity threshold; or performing the second interpolation on the first pixel block if the interpolation feature exceeds the complexity threshold" have been considered but are moot because of the new ground of rejection. In addition, applicant’s argument with respect to claim 2, that the prior art does not teach the limitation(s): "determining the complexity threshold based on whether the first pixel block is in a target region of the first image" has also been considered but is moot because of the new ground of rejection. Claims 1-2, 17 and 18 are now disclosed by Lin, Kwak, and Taoka. Applicant’s argument with respect to claim 8, that the prior art does not teach the limitation(s): "associating a trapezoidal target region with a first genre of content; and associating an elliptical target region with a second genre of content" has been considered but is moot because of the new ground of rejection. Claim 8 is now disclosed by Lin Kwak, Taoka, Na Tae Young, and Hongyu Bian.
4. Regarding arguments to claims 3-7, 9-16 and 19-20, they are dependent on independent claims 1 and 18 respectively. Applicant does not argue anything other than independent claim 1, and similarly claims 17 and 18, and dependent claims 2 and 8. The limitations in those claims, in conjunction with their combination, has previously been established and explained.
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1-2, 9-13, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US-9123138-B2, hereinafter "Lin") in view of Kwak, (WO-2017/217656-A1), and further in view of Taoka, (US-2019/0205019-A1).
7. As per claim 1, Lin discloses: An image processing method executed in a computer device, the method comprising:
acquiring a first image having a first resolution; (Lin, col. 4, lines 64-67, “An input image is upscaled by a scale factor to generate an upscaled input image (block 402). For example, input image “It” may be processed by an upscale module 308 to increase a resolution of the input image ...” and col. 3, lines 33-34, “The computing device 102 is illustrated as including an image processing module 104.”)
[[based on comparing:]]
an interpolation feature describing [[complexity]] of image content of a first pixel block in the first image; and (Lin, col. 9, lines 8-18, “Additionally, variations to the techniques may be made to address different color spaces and color channels. For example, the upscale techniques employed by the image upscale system 110 may be applied to a luminance channel “Y,” and the result may be combined with different algorithms applied to the “Cb” and “Cr” color channels. … In another example, the upscale techniques may be applied to each color channel (e.g., color space) separately, e.g., may be applied independently to red, green, and blue color channels.” and col. 6, lines 35-38, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602).”)
[[a complexity threshold, selecting between]] a first interpolation and a second interpolation for upscaling the first pixel block; (Lin, col. 6, lines 35-38, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602).” and col. 9, lines 19-23, “In a further example, the upscale techniques may be applied to one or multiple color channels jointly. This may be performed by searching for the nearest patches in each of the color spaces and applying upscaling consistently for different color spaces using the patch search results.” and col. 5, lines 1-3, “A variety of different techniques may be employed to perform this upscaling, an example of which is Bicubic interpolation.” and col. 8, lines 23-28, ”Accordingly, the image upscale system 110 may be configured to adaptively limit a number of iterations performed by the factor upscale module 210, and then use an adjustment upscale module 216 (e.g., analytical interpolation) to resize the output ...” and col. 4, lines 6-11, “Accordingly, in this example the image upscale system 110 may employ one or more image upscale modules 114 that are configured to preserve image edges and texture details, an example of which is described as follows and shown in a corresponding figure as well as at the image 904 also shown in FIG. 9.”; Examiner’s note: As disclosed by Lin, different interpolation methods can be employed and the number of iterations performed limited, which are each different interpolation methods.)
obtaining an interpolated pixel block by:
performing the first interpolation on the first pixel block if the interpolation feature [[does not exceed the complexity threshold;]] or (See Lin, col. 5, lines 1-3; col. 8, lines 23-28; col. 2, line 59-col. 3, line 3; col. 9, lines 8-13; and col. 9, lines 30-38 below.)
performing the second interpolation on the first pixel block if the interpolation feature [[exceeds the complexity threshold;]] and (Lin, col. 5, lines 1-3, “A variety of different techniques may be employed to perform this upscaling, an example of which is Bicubic interpolation.” and col. 8, lines 23-28, ”Accordingly, the image upscale system 110 may be configured to adaptively limit a number of iterations performed by the factor upscale module 210, and then use an adjustment upscale module 216 (e.g., analytical interpolation) to resize the output ...” and col. 2, line 59-col. 3, line 3, “In a further example, algorithm parameters may be adapted with respect to algorithm iterations, which may be performed to increase efficiency of computing device resource utilization and speed of performance. For instance, algorithm parameters may be adapted to enforce a minimum and/or maximum number to iterations, cease iterations for image sizes over a threshold amount, set sampling step sizes for patches, employ techniques based on color channels (which may include independence and joint processing techniques), and so on as further described in the Iteration-Adaptive Parameter Update Section.” and col. 9, lines 8-13, “Additionally, variations to the techniques may be made to address different color spaces and color channels. For example, the upscale techniques employed by the image upscale system 110 may be applied to a luminance channel “Y,” and the result may be combined with different algorithms applied to the “Cb” and “Cr” color channels.” and col. 9, lines 30-38, “Yet further, at each iteration performed by the factor upscale module 210, a detail-preserving upscaling algorithm may be employed to upscale its input by a factor of “s.” If the original image is noisy, for instance, an image denoising algorithm may be applied before the upscaling iterations to suppress noise boosting. Additionally, in the initial denoising step, the search region may be adapted by setting its radius proportional to the input denoising parameter to speed-up the process for images with relatively small amounts of noise.”)
outputting a second image with a second resolution based on the interpolated pixel block, the second resolution being greater than the first resolution, wherein the first interpolation and the second interpolation comprise up-sampling of the first pixel block, and computational resource consumption of the second interpolation is greater than computational resource consumption of the first interpolation. (Lin col. 8, lines 48-53, “Further, the image upscale system 110 may be configured to cease performance of the iterations if the image size would exceed a threshold. For example, an expression “w×h>M” may be employed where “w” and “h” are the width and height of the input image for current iteration and “M” is the threshold.” and col. 8, lines 23-33, “Accordingly, the image upscale system 110 may be configured to adaptively limit a number of iterations performed by the factor upscale module 210, and then use an adjustment upscale module 216 (e.g., analytical interpolation) to resize the output of the factor upscale module 210 to the target image size of the upscaled image data 112. In such a way, efficiency of the image upscale system 110 may be increased (e.g., in terms of computational resources and time spent performing the processing) with little compromise on the final image quality. This may be performed in a variety of ways.” and col. 9, lines 39-42, “In a further example, variations may be made for setting algorithm parameters. For example, a scale factor “s” may be set differently over iterations, e.g. increasing “s” over iterations in the range of 1:1 to 1:5.”)
8. Lin doesn't explicitly disclose but Kwak discloses: based on comparing: (See Kwak, page 4-5, [26] below.)
[[an interpolation feature describing]] complexity [[of image content of a first pixel block in the first image; and]] (See Kwak, page 4-5, [26] below.)
a complexity threshold, [[selecting between a first interpolation and a second interpolation for upscaling the first pixel block;]] (See Kwak, page 4-5, [26] below.)
[[obtaining an interpolated pixel block by:]]
[[performing the first interpolation on the first pixel block if the interpolation feature]] does not exceed the complexity threshold; [[or]] (See Kwak, page 4-5, [26] below.)
[[performing the second interpolation on the first pixel block if the interpolation feature]] exceeds the complexity threshold; [[and]] (Kwak, page 4-5, [26], “In some embodiments, the first compression frame generation step may include: an image complexity calculation step for calculating the complexity of an image for a plurality of sub-regions constituting the image area to be processed; an image complexity determination step for determining whether the complexity of the sub-regions is below a preset standard; and a complexity standard image processing step for performing a first image processing on the sub-regions when the complexity of the sub-regions is below the preset standard.”; Examiner’s note: Kwak discloses a process that based on a calculated complexity for a plurality of sub-regions in an image and a complexity threshold (mentioned as “preset standard”), a comparison is made to determine if the complexity is above or below the “preset standard” threshold.)
9. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Lin to include comparing the complexity of image content with a complexity threshold that does or does not exceed that threshold, of Kwak. The motivation for this modification could have been to determine which image regions that are better suited for various image processing methods. For instance, a “simple” image region may require less processing than a “complex” image region. In this case, choosing a proper interpolation method from a threshold value could ensure both a quality image while maintaining performance. It would help prevent processing waste for image regions that don’t require an intensive interpolation process.
10. Lin in view of Kwak doesn't explicitly disclose but Taoka discloses: [[a complexity threshold,]] selecting between [[a first interpolation and a second interpolation for upscaling the first pixel block;]] (Taoka, [0065], “The selector 36 may extract a reference area including peripheral pixels of the interpolation object position from the image and evaluate complexity of the reference area. The selector 36 may select a directional interpolation pixel generated by directional interpolation when the complexity of the reference area is low. On the other hand, when the complexity of the reference area is high, the selector 36 may select an arithmetic interpolation pixel generated by arithmetic interpolation. The evaluation of the complexity will be described later.” and [0010], “The selector may determine a complexity level of the reference area, and select either the directional interpolation pixel or the arithmetic interpolation pixel as the interpolation pixel for the interpolation object position based on the complexity level of the reference area, and the complexity level of the reference area may be determined according to differences of pixel values between adjacent pixels in a horizontal direction in the image and differences of pixel values between adjacent pixels in a vertical direction in the image.”)
11. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of Lin in view of Kwak to include selecting between a first interpolation and a second interpolation for upscaling the first pixel block, of Taoka. The motivation for this modification could have been to determine which image regions that are better suited for various image processing methods. For instance, a “simple” image region may require less processing than a “complex” image region. In this case, choosing a proper interpolation method from a threshold value could ensure both a quality image while maintaining performance. It would help prevent processing waste for image regions that don’t require an intensive interpolation process.
12. As per claim 2, Lin in view of Kwak, and further in view of Taoka discloses: The method according to claim 1, further comprising determining the complexity threshold based on whether the first pixel block is in a target region of the first image. (Kwak, page 41, [447], “Specifically, the step of setting the processing target image area divides the compression target frame into a plurality of image blocks according to a pre-set standard, and if the compression target frame is not an independently encoded frame, some of the image blocks among the plurality of image blocks of the compression target frame may be set as the processing target image area.” and Kwak, page 35, [380], “Meanwhile, as illustrated in FIG. 23, the first frame conversion unit (10500) includes a complexity determination unit (10510) that calculates the complexity of detailed regions constituting an image block or a processing target image region and determines whether the complexity of the image block or detailed region is below or less than a preset standard; and a first image processing unit (10520) that performs first image processing on the image block or detailed region when the complexity of the image block or detailed region is below the preset standard.” and Lin col. 6, lines 35-38, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602).”; Examiner’s note: Kwak discloses steps that determine which images block may (or may not) be set as the target image region when a condition is met (and thus, these blocks would be within the target region). For image blocks set as the target image region, a complexity calculation is conducted on these regions.)
13. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Lin in view of Taoka to include determining the complexity threshold based on whether the first pixel block is in a target region of the first images, of Kwak. The motivation for this modification could have been to filter and ensure that pixel blocks are within a target region to be processed. This would save computational resources to not process pixel blocks that are not within a target region.
14. As per claim 9, Lin in view of Kwak, and further in view of Taoka discloses: The method according to claim 1, wherein the selecting is based on whether a position of the first pixel block is within a target region that is a partial region of the first image. (Kwak, page 41, [447], “Specifically, the step of setting the processing target image area divides the compression target frame into a plurality of image blocks according to a pre-set standard, and if the compression target frame is not an independently encoded frame, some of the image blocks among the plurality of image blocks of the compression target frame may be set as the processing target image area.” and Kwak, page 35, [380], “Meanwhile, as illustrated in FIG. 23, the first frame conversion unit (10500) includes a complexity determination unit (10510) that calculates the complexity of detailed regions constituting an image block or a processing target image region and determines whether the complexity of the image block or detailed region is below or less than a preset standard; and a first image processing unit (10520) that performs first image processing on the image block or detailed region when the complexity of the image block or detailed region is below the preset standard.” and Taoka, [0065], “The selector 36 may extract a reference area including peripheral pixels of the interpolation object position from the image and evaluate complexity of the reference area. The selector 36 may select a directional interpolation pixel generated by directional interpolation when the complexity of the reference area is low. On the other hand, when the complexity of the reference area is high, the selector 36 may select an arithmetic interpolation pixel generated by arithmetic interpolation. The evaluation of the complexity will be described later.” and Lin, col. 6, line 62-65, “A threshold is then applied using the computed image distances to determine whether to use the identified patch or the patch at the location in the image to predict a patch for use in generating the upscaled image (block 606).” and Lin, col. 6, lines 35-45, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602). ... This similarity may be expressed as relative “image distances” between the patches and used as part of a determination as to which of the patches are to be used as part of the upscale operation.” and Lin, col. 6, lines 49-53, “The content-adaptive patch finding module 402, for instance, may estimate “p2” as the position “p” in the local neighborhood of “p1” (which is the center location), such that “∥U(q1)−B(p)∥” is minimized. An image distance may then be computed between the patch in the upscaled input image and the identified patch (block 604).”; As disclosed by Kwak, image blocks and target image area are partial regions of the image.)
The motivation for this modification is the same as claim 2.
15. As per claim 10, Lin in view of Kwak, and further in view of Taoka discloses: The method according to claim 1, complexity threshold is based on: (Kwak, page 4-5, [26], “In some embodiments, the first compression frame generation step may include: an image complexity calculation step for calculating the complexity of an image for a plurality of sub-regions constituting the image area to be processed; an image complexity determination step for determining whether the complexity of the sub-regions is below a preset standard; and a complexity standard image processing step for performing a first image processing on the sub-regions when the complexity of the sub-regions is below the preset standard.”; Examiner’s note: Kwak discloses a process that based on a calculated complexity for a plurality of sub-regions in an image and a complexity threshold (mentioned as “preset standard”), a comparison is made to determine if the complexity is above or below the “preset standard” threshold.)
image content of the first image and position information about the first pixel block in the first image. (Lin, col. 9, lines 8-13, “Additionally, variations to the techniques may be made to address different color spaces and color channels. For example, the upscale techniques employed by the image upscale system 110 may be applied to a luminance channel “Y,” and the result may be combined with different algorithms applied to the “Cb” and “Cr” color channels.” and col. 6, lines 35-38, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602).” and col. 9, lines 30-38, “Yet further, at each iteration performed by the factor upscale module 210, a detail-preserving upscaling algorithm may be employed to upscale its input by a factor of “s.” If the original image is noisy, for instance, an image denoising algorithm may be applied before the upscaling iterations to suppress noise boosting. Additionally, in the initial denoising step, the search region may be adapted by setting its radius proportional to the input denoising parameter to speed-up the process for images with relatively small amounts of noise.”)
16. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Lin in view of Taoka to include that the complexity threshold is based on image content of the first image and position information about the first pixel block in the first image, of Kwak. The motivation for this modification could have been to account for the type of the image content and its position to determine what process is necessary for the region. Depending on if the content is complex or not will potentially determine the intensity of an interpolation process and only in those regions where the content is considered complex. This helps to optimize the interpolation process so that it does not use unnecessary computational resources for image regions that are not considered complex.
17. As per claim 11, Lin in view of Kwak, and further in view of Taoka discloses: The method according to claim 1, wherein the selecting comprises: (Taoka, [0065], “The selector 36 may extract a reference area including peripheral pixels of the interpolation object position from the image and evaluate complexity of the reference area. The selector 36 may select a directional interpolation pixel generated by directional interpolation when the complexity of the reference area is low. On the other hand, when the complexity of the reference area is high, the selector 36 may select an arithmetic interpolation pixel generated by arithmetic interpolation. The evaluation of the complexity will be described later.” and [0010], “The selector may determine a complexity level of the reference area, and select either the directional interpolation pixel or the arithmetic interpolation pixel as the interpolation pixel for the interpolation object position based on the complexity level of the reference area, and the complexity level of the reference area may be determined according to differences of pixel values between adjacent pixels in a horizontal direction in the image and differences of pixel values between adjacent pixels in a vertical direction in the image.”)
determining an image body region in the first image first image; and (Lin, col. 6, lines 35-45, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602). ... This similarity may be expressed as relative “image distances” between the patches and used as part of a determination as to which of the patches are to be used as part of the upscale operation.”)
selecting either: (Taoka, [0065], “The selector 36 may extract a reference area including peripheral pixels of the interpolation object position from the image and evaluate complexity of the reference area. The selector 36 may select a directional interpolation pixel generated by directional interpolation when the complexity of the reference area is low. On the other hand, when the complexity of the reference area is high, the selector 36 may select an arithmetic interpolation pixel generated by arithmetic interpolation. The evaluation of the complexity will be described later.” and [0010], “The selector may determine a complexity level of the reference area, and select either the directional interpolation pixel or the arithmetic interpolation pixel as the interpolation pixel for the interpolation object position based on the complexity level of the reference area, and the complexity level of the reference area may be determined according to differences of pixel values between adjacent pixels in a horizontal direction in the image and differences of pixel values between adjacent pixels in a vertical direction in the image.”)
a third target threshold if a position of the first pixel block is within the image body region; or (Lin, col. 6, lines 35-45, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602). ... This similarity may be expressed as relative “image distances” between the patches and used as part of a determination as to which of the patches are to be used as part of the upscale operation.” and col. 6, line 62-65, “A threshold is then applied using the computed image distances to determine whether to use the identified patch or the patch at the location in the image to predict a patch for use in generating the upscaled image (block 606).” and col. 6, lines 35-45, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602). ... This similarity may be expressed as relative “image distances” between the patches and used as part of a determination as to which of the patches are to be used as part of the upscale operation.” and col. 6, lines 49-53, “The content-adaptive patch finding module 402, for instance, may estimate “p2” as the position “p” in the local neighborhood of “p1” (which is the center location), such that “∥U(q1)−B(p)∥” is minimized. An image distance may then be computed between the patch in the upscaled input image and the identified patch (block 604).”)
a fourth target threshold if the position of the first pixel block is outside the image body region, wherein the third target threshold is less than the fourth target threshold. (Lin, col. 6, lines 35-45, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602). ... This similarity may be expressed as relative “image distances” between the patches and used as part of a determination as to which of the patches are to be used as part of the upscale operation.” and col. 6, line 62-65, “A threshold is then applied using the computed image distances to determine whether to use the identified patch or the patch at the location in the image to predict a patch for use in generating the upscaled image (block 606).” and col. 6, lines 35-45, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602). ... This similarity may be expressed as relative “image distances” between the patches and used as part of a determination as to which of the patches are to be used as part of the upscale operation.” and col. 6, lines 49-53, “The content-adaptive patch finding module 402, for instance, may estimate “p2” as the position “p” in the local neighborhood of “p1” (which is the center location), such that “∥U(q1)−B(p)∥” is minimized. An image distance may then be computed between the patch in the upscaled input image and the identified patch (block 604).” and col. 7, lines 14-17, “In this way, preference may be given to the “in place” patch. For instance, the threshold may be used such that an “in place” patch is used unless a patch is found such that a ratio of the image distances is closer than the threshold.”)
The motivation for this modification is the same as claims 1, 2, and 10.
18. As per claim 12, Lin in view of Kwak, and further in view of Taoka discloses: The method according to claim 11, wherein the determining an image body region comprises:
invoking a first image recognition model to identify a target object in the first image, and determining a display region of the target object as the image body region in the first image; or (Lin, col. 6, lines 35-48, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602). For example, the content-adaptive patch finding module 502 of the factor upscale module 210 may be employed as part of a search for a most-similar patch in the local neighborhood 314 in smoothed image “B” 306 to patch from the upscaled image “U” 310. This similarity may be expressed as relative “image distances” between the patches and used as part of a determination as to which of the patches are to be used as part of the upscale operation. The search may be performed to give preference to a patch at an “in place” location as opposed to patches at other locations to reduce likelihood of blurred edges and texture.” and col. 7, lines 53-56, “Weights are then assigned to the identified patch (i.e., the most similar patch) and the patch at the location in the image (i.e., the in-place patch) based on respective content metrics (block 808).”)
invoking a second image recognition model to determine an image type of the first image, and determining a corresponding image body region according to the image type. (Lin, col. 7, lines 40-43, “The content metric may describe consistency of gradient orientations to distinguish informative local patches (such as edges and corners) versus uniform or noisy patches.” and col. 9, lines 32-38, “If the original image is noisy, for instance, an image denoising algorithm may be applied before the upscaling iterations to suppress noise boosting. Additionally, in the initial denoising step, the search region may be adapted by setting its radius proportional to the input denoising parameter to speed-up the process for images with relatively small amounts of noise.”)
19. As per claim 13, Lin in view of Kwak, and further in view of Taoka discloses: The method according to claim 1, wherein the performing the first interpolation comprises:
performing the first interpolation on the first pixel block according to third pixel blocks to obtain the interpolated pixel block, wherein the third pixel blocks comprise adjacent pixel blocks located around the first pixel block; or (Lin, col. 8, lines 23-28, “Accordingly, the image upscale system 110 may be configured to adaptively limit a number of iterations performed by the factor upscale module 210, and then use an adjustment upscale module 216 (e.g., analytical interpolation) to resize the output ...” and, col. 6, lines 35-38, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602).” and col. 5, line 62-col. 6, line 2, “Portions of patches “It+1(q1)” that are overlapping may then be averaged to arrive at a result for the image upscale iteration “It+1” 316. This process may then be repeated to predict patches for each location of the upscaled image data. Further, as previously described in relation to FIG. 2, this technique may be repeated for a plurality of different iterations such that the image upscale iteration “It+1” is used as the input image “It” 302 for the next iteration.”)
the performing the second interpolation comprises:
performing the second interpolation on the first pixel block according to fourth pixel blocks to obtain the interpolated pixel block, wherein the fourth pixel blocks comprise adjacent pixel blocks located around the first pixel block, and wherein a number of the fourth pixel blocks is greater than a number of the third pixel blocks. (Lin, col. 8, lines 23-28, “Accordingly, the image upscale system 110 may be configured to adaptively limit a number of iterations performed by the factor upscale module 210, and then use an adjustment upscale module 216 (e.g., analytical interpolation) to resize the output ...” and, col. 6, lines 35-38, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602).” and col. 5, lines 25-30, “For instance, the blurred and smoothed images may be processed by a prediction module 312 to make a prediction to upscale the input image “It” 302. The underlying approach may be based on patches, which may include predefined areas of an image, e.g., five by five pixels although other examples are also contemplated.” and col. 9, lines 39-42, “In a further example, variations may be made for setting algorithm parameters. For example, a scale factor “s” may be set differently over iterations, e.g. increasing “s” over iterations in the range of 1:1 to 1:5.”)
20. Claim 17, which is similar in scope to claim 1, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 1.
21. Claims 3-5 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US-9123138-B2, hereinafter "Lin") in view of Kwak, (WO-2017/217656-A1), further in view of Taoka, (US-2019/0205019-A1), and further in view of Hoshii et al. (US-2002/0015162-A1, hereinafter "Hoshii").
22. As per claim 3, Lin in view of Kwak, and further in view of Taoka discloses: The method according to claim 1, wherein color information about the first image comprises a luminance factor; and
the method comprises determining the interpolation feature by: (Lin, col. 9, lines 8-13, “Additionally, variations to the techniques may be made to address different color spaces and color channels. For example, the upscale techniques employed by the image upscale system 110 may be applied to a luminance channel “Y,” and the result may be combined with different algorithms applied to the “Cb” and “Cr” color channels.” and col. 5, lines 1-3, “A variety of different techniques may be employed to perform this upscaling, an example of which is Bicubic interpolation.”)
23. Lin in view of Kwak, and further in view of Taoka doesn't explicitly disclose but Hoshii discloses: calculating a direction feature of the first pixel block according to luminance factors of a plurality of second pixel blocks, wherein the direction feature describes a luminance difference between the first pixel block and the plurality of second pixel blocks. (Hoshii, page 13, ¶ [0142], “At the same time, the edge angle and line thickness of the source pixels are decided, based on the luminance values of given pixels that fall within the zone comprising 5×5 pixels.” and page 11, ¶ [0142], “It is assumed that the pattern of the pixels shown in FIG. 11B matches one of the prepared edge patterns. Because of matching with an edge pattern, the pattern shown in FIG. 11B shows a characteristic that the difference between the luminance values of the pixels positioned on the upper side in the j direction of the figure and those of the pixels positioned on the lower side in the j direction of the figure tends to be great, but the luminance difference in the horizontal direction on paper is little. Thus, this pattern is regarded as a horizontally parallel edge. To this edge pattern, pixel interpolation processing is then executed, according to a predetermined rule.”)
24. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Lin in view of Kwak, and further in view of Taoka to include calculating a direction feature of the first pixel block according to luminance factors of the plurality of second pixel blocks, wherein the direction feature describes a luminance difference between the first pixel block and the plurality of second pixel blocks and determine the direction feature as the interpolation feature of Hoshii. The motivation for this modification could have been to improve an interpolation technique where accounting for the direction of the luminance difference between two pixel blocks, the interpolation can account for image features, such as edges and produce higher quality results.
25. As per claim 4, Lin in view of Kwak, and further in view of Taoka, and further in view of Hoshii discloses: The method according to claim 3, wherein the calculating a direction feature of the first pixel block according to luminance factors of the plurality of second pixel blocks comprises:
determining luminance differences between the first pixel block and the plurality of second pixel blocks in a first direction and a second direction; and (Hoshii, page 13, ¶ [0142], “At the same time, the edge angle and line thickness of the source pixels are decided, based on the luminance values of given pixels that fall within the zone comprising 5×5 pixels.” and page 15, ¶ [0200], “When the 16 grid points surrounding the point Puv to be interpolated are given their tone values, the luminance of the point Puv is determined, subject to the influence of these tone values on the point Puv.”)
encapsulating the luminance differences into two-dimensional floating-point data to determine a luminance feature of the first pixel block; and (Hoshii, page 15, ¶ [0201], “The 16 grid points act on the point Puv to be interpolated with the degree of their influence depending on their distance from the point Puv, which is expressed as described above. The degree of influence of the tone data of all grid points on the point Puv in the X and Y directions in the aggregate can be expressed in a general equation as below ...”)
determining a sum of a first direction component and a second direction component of the luminance feature as the direction feature of the first pixel block, wherein the first direction and the second direction are perpendicular to each other in the first image. (Hoshii, page 12, ¶ [0159], “For the calculation to obtain the tone values of pixels b, c, and f of interpolations, source pixels to be used differ, according to the luminance values of given pixels that fall within the zone comprising 5×5 pixels. If the pattern is a right-angled edge, source L, M, R pixels are used to generate. If the pattern is not a right-angled edge, source G, H, N, and S pixels are used to generate pixels b, c, and f of interpolations.” and page 12, ¶ [0158], “In the present embodiment of the invention, interpolation processing is executed for a horizontal edge, according to the above rules, with luminance change in the direction perpendicular to the edge being taken into consideration.” and page 12, ¶ [0150],”In the present embodiment of the invention, interpolation is executed such that change of luminance in the direction perpendicular to the above horizontally lengthening edge is also reflected in the interpolation. If luminance changes in the direction perpendicular to the horizontally lengthening edge, the tone values of the source Q, R, and S pixels are assigned to Pn so that the appearance of edge change will be reflected in the tone values of the g, h, and i pixels of interpolations.”)
26. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 3 of Lin in view of Kwak, and further in view of Taoka to include determining the luminance differences between the first pixel block and the plurality of second pixel blocks in a first direction and a second direction, encapsulating the luminance differences into two-dimensional floating-point data, and determining a sum of a first direction component and a second direction component of the luminance feature wherein the first direction and the second direction are perpendicular to each other of Hoshii. The motivation for this modification could have been to improve an interpolation technique where accounting for the direction of the luminance difference between two pixel blocks, the interpolation can account for image features, such as edges and produce higher quality results. Encapsulating the luminance difference also provides a distribution of values for the interpolation method to consider. By also accounting for luminance directions that are perpendicular to each other, this will help account for right angle gradients or structure in an image during interpolation.
27. As per claim 5, Lin in view of Kwak, and further in view of Taoka, and further in view of Hoshii discloses: The method according to claim 4, wherein the determining luminance differences comprises:
determining a first luminance difference of the first pixel block in the first direction according to a difference in luminance factor between: (Hoshii, page 13, ¶ [0142], “At the same time, the edge angle and line thickness of the source pixels are decided, based on the luminance values of given pixels that fall within the zone comprising 5×5 pixels.” and page 15, ¶ [0200], “When the 16 grid points surrounding the point Puv to be interpolated are given their tone values, the luminance of the point Puv is determined, subject to the influence of these tone values on the point Puv.”)
a second pixel block at a front side of the first pixel block; and
a second pixel block at a rear side of the first pixel block in the first direction in the plurality of second pixel blocks; and (Hoshii, page 15, ¶ [0200], “When the 16 grid points surrounding the point Puv to be interpolated are given their tone values, the luminance of the point Puv is determined, subject to the influence of these tone values on the point Puv. ... When taking notice of the X axis direction, we express the distance of the above 16 grid points from the point Puv as follows: the distance to the outer left grid points is x1; the distance to the inner left grid points is x2; the distance to the inner right grid points is x3; and the distance to the outer right grid points is x4. We use function f (x) to express the degree of influence of the tone of the gird points on the luminance of the point Puv, according to the above distance.”)
determining a second luminance difference of the first pixel block in the second direction according to a difference in luminance factor between: (Hoshii, page 13, ¶ [0142], “At the same time, the edge angle and line thickness of the source pixels are decided, based on the luminance values of given pixels that fall within the zone comprising 5×5 pixels.” and page 15, ¶ [0200], “When the 16 grid points surrounding the point Puv to be interpolated are given their tone values, the luminance of the point Puv is determined, subject to the influence of these tone values on the point Puv.”)
a second pixel block at a front side of the first pixel block; and
a second pixel block at a rear side of the first pixel block in the second direction in the plurality of second pixel blocks; and (Hoshii, page 15, ¶ [0200], “When the 16 grid points surrounding the point Puv to be interpolated are given their tone values, the luminance of the point Puv is determined, subject to the influence of these tone values on the point Puv. ... When taking notice of the Y axis direction, we express the distance of the above 16 grid points from the point Puv as follows: the distance to the top grid points is y1, the distance to the inner grid points above the point Puv is y2, the distance to the inner grid points below the point Puv is y3, and the distance to the bottom grid points is x4. Similarly, we use function f (y) to express the degree of influence of the tone of the gird points on the luminance of the point Puv, according to the above distance.”)
the encapsulating the luminance differences comprises encapsulating the first luminance difference and the second luminance difference into the two-dimensional floating-point data to determine the luminance feature of the first pixel block. (Hoshii, page 15, ¶ [0200], “We use function f (x) to express the degree of influence of the tone of the gird points on the luminance of the point Puv, according to the above distance. ... Similarly, we use function f (y) to express the degree of influence of the tone of the gird points on the luminance of the point Puv, according to the above distance.”)
28. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 4 of Lin in view of Kwak, and further in view of Taoka to include determining a first luminance difference of the first pixel block in the first direction according to a difference in luminance factor between a second pixel block at a front side and rear side of the first pixel block, determining a second luminance difference of the first pixel block in the second direction according to a difference in luminance factor between a second pixel block at a front side and rear side of the first pixel block, and encapsulate the luminance differences into the two-dimensional floating-point data value of Hoshii. The motivation for this modification could have been to improve an interpolation technique where accounting for the direction of the luminance difference between two pixel blocks, the interpolation can account for image features, such as edges and produce higher quality results. Encapsulating the luminance difference also provides a distribution of values for the interpolation method to consider.
29. Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US-9123138-B2, hereinafter "Lin") in view of Kwak, (WO-2017/217656-A1), further in view of Taoka, (US-2019/0205019-A1), and further in view of Tang (US-10339632-B2).
30. As per claim 6, Lin in view of Kwak, and further in view of Taoka discloses: The method according to claim 1, further comprising:
dividing the first image into at least two pixel blocks according to a division rule, the first pixel block being any pixel block of the at least two pixel blocks; and
wherein the outputting further comprises: (Lin, col. 6, lines 35-38, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602).” and col. 5, lines 25-30, “For instance, the blurred and smoothed images may be processed by a prediction module 312 to make a prediction to upscale the input image “It” 302. The underlying approach may be based on patches, which may include predefined areas of an image, e.g., five by five pixels although other examples are also contemplated.”)
31. Lin in view of Kwak, and further in view of Taoka doesn't explicitly disclose but Tang discloses: concatenating the interpolated pixel block and the second image according to a combination rule that comprises an inverse ordering rule to the division rule. (Tang, col. 14, lines 36-53, “The determining module 410 is configured to determine a high-frequency region of the color-block image. The first converting module 420 is configured to convert a part of the color-block image within the high-frequency region into a first image using a first interpolation algorithm. The first image includes first simulation pixels arranged in an array, and each photosensitive pixel 212 corresponds to one first simulation pixel. The second converting module 430 is configured to convert a part of the color-block image beyond the high-frequency region into a second image using a second interpolation algorithm. The second image includes second simulation pixels arranged in an array, and each photosensitive pixel 212 corresponds to one second simulation pixel. A complexity of the second interpolation algorithm is less than that of the first interpolation algorithm. The merging module 440 is configured to merge the first image and the second image into a simulation image corresponding to the color-block image.”)
32. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Lin in view of Kwak, and further in view of Taoka to include concatenating the interpolated pixel block and the second image according to a combination rule that comprises an inverse ordering rule to the division rule of Tang. The motivation for this modification could have been used to assist the interpolation process so that the process by more efficiently recombine and up-scale an image based on the pixel block’s complexity.
33. Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US-9123138-B2, hereinafter "Lin") in view of Kwak, (WO-2017/217656-A1), further in view of Taoka, (US-2019/0205019-A1), further in view of Na Tae Young et al. (KR-20190083216-A, hereinafter "Na Tae Young"), and further in view of Hongyu Bian et al. (CN-101246552-A, hereinafter "Hongyu Bian").
34. As per claim 7, Lin in view of Kwak, and further in view of Taoka discloses: The method according to claim 1, further comprising:
determining the complexity threshold based on whether the first pixel block is within a target region of the first image, (Kwak, page 41, [447], “Specifically, the step of setting the processing target image area divides the compression target frame into a plurality of image blocks according to a pre-set standard, and if the compression target frame is not an independently encoded frame, some of the image blocks among the plurality of image blocks of the compression target frame may be set as the processing target image area.” and Kwak, page 35, [380], “Meanwhile, as illustrated in FIG. 23, the first frame conversion unit (10500) includes a complexity determination unit (10510) that calculates the complexity of detailed regions constituting an image block or a processing target image region and determines whether the complexity of the image block or detailed region is below or less than a preset standard; and a first image processing unit (10520) that performs first image processing on the image block or detailed region when the complexity of the image block or detailed region is below the preset standard.” and Lin col. 6, lines 35-38, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602).”; Examiner’s note: Kwak discloses steps that determine which images block may (or may not) be set as the target image region when a condition is met (and thus, these blocks would be within the target region). For image blocks set as the target image region, a complexity calculation is conducted on these regions.) [[wherein different shapes of target regions are associated with different genres of content.]]
35. Lin in view of Kwak, and further in view of Taoka doesn't explicitly disclose but Na Tae Young discloses: [[determining the complexity threshold based on whether the first pixel block is within a target region of the first image,]] wherein [[different shapes of]] target regions are associated with different genres of content. (Na Tae Young, page 5, [0008], “The present embodiment provides a content providing device characterized by comprising: a processing unit that divides a video frame into a plurality of regions based on an object recognition result within the video frame and outputs an object image corresponding to each divided region; a learning unit that receives the object image for each divided region, applies it to an artificial neural network model, and calculates genre information for each divided region based on the application result; and a control unit that generates metadata for quality adjustment of each divided region based on the genre information calculated using the learning unit and outputs a quality adjustment signal including the metadata.” and page 5, [0007], “In addition, the present embodiment aims to maximize the sense of immersion and realism in the video for the viewer by dividing the video frame, which is subject to image quality improvement, into multiple regions based on the result of object recognition, and applying different output characteristics to each divided region according to the genre information identified for each divided region.”)
36. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Lin in view of Kwak, and further in view of Taoka to include using target regions that are associated with different genres of content, of Na Tae Young. The motivation for this modification could have been to customize the image complexity process depending on the genre of content presented. Thus, if it is determined that the content has little motion between frames, a simpler interpolation process with be used for the content and vice-versa for high-action content. Thus, the process will adjust automatically depending on the content.
37. Lin in view of Kwak, further in view of Taoka, and further in view of Na Tae Young doesn't explicitly disclose but Hongyu Bian discloses: [[determining the complexity threshold based on whether the first pixel block is within a target region of the first image, wherein]] different shapes of [[target regions are associated with different genres of content.]] (Hongyu Bian, page 3, [0004], “The purpose of this invention is to provide a method for recognizing elliptical and rectangular targets in images based on image processing technology, which can quickly and effectively identify rectangular or square targets and elliptical or circular targets in images.”)
38. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Lin in view of Kwak, further in view of Taoka, and further in view of Na Tae Young to include different shapes of target regions are associated with different genres of content, of Hongyu Bian. The motivation for this modification could have been to customize the processing on the type of content further to focus on specific areas where the content is important. Thus, if it is detected that a specific shape for a particular genre is being used, the processing can focus on the region of that shape and save computational power for not processing the entire image.
39. As per claim 8, Lin in view of Kwak, further in view of Taoka, further in view of Na Tae Young, and further in view of Hongyu Bian discloses: The method according to claim 7, further comprising:
associating a trapezoidal target region with a first genre of content; and (See Hongyu Bian, page 3, [0004] and Na Tae Young, page 5, [0008] below.)
associating an elliptical target region with a second genre of content. (Hongyu Bian, page 3, [0004], “The purpose of this invention is to provide a method for recognizing elliptical and rectangular targets in images based on image processing technology, which can quickly and effectively identify rectangular or square targets and elliptical or circular targets in images.” and Na Tae Young, page 5, [0008], “The present embodiment provides a content providing device characterized by comprising: a processing unit that divides a video frame into a plurality of regions based on an object recognition result within the video frame and outputs an object image corresponding to each divided region; a learning unit that receives the object image for each divided region, applies it to an artificial neural network model, and calculates genre information for each divided region based on the application result; and a control unit that generates metadata for quality adjustment of each divided region based on the genre information calculated using the learning unit and outputs a quality adjustment signal including the metadata.”; Examiner’s note: The rectangular target region disclosed by Hongyu Bian falls within the definition of a trapezoid as it has four sides and at least one pair of parallel sides.)
The motivation for this modification is the same as claim 7.
40. Claims 14-16 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US-9123138-B2, hereinafter "Lin") in view of Kwak, (WO-2017/217656-A1), further in view of Taoka, (US-2019/0205019-A1), and further in view of Colenbrander (US-11731043-B2).
41. As per claim 14, Lin in view of Kwak, and further in view of Taoka discloses: The method of claim 1, wherein the computer device comprises … (see rejection of claim 1 above).
42. Lin in view of Kwak, and further in view of Taoka doesn't explicitly disclose but Colenbrander discloses: a game device, wherein the first resolution comprises an output resolution of a game engine, and the second resolution comprises a display resolution of the game device, and wherein acquiring the first image comprises acquiring the first image from the game engine. (Colenbrander, col. 5, lines 13-18, “By way of example without limitation, the client device 122 can be a game console ... capable of streaming gameplay of a video game from a cloud gaming provider as in the present disclosure.” and col. 1, line 65-col. 2, line 1, “In some implementations, a method is provided, including the following operations: executing a video game by a cloud game machine, the execution of the video game includes rendering gameplay video ...” and col. 4, lines 19-30, “There are a few settings which can be adjusted such as the bitrate of the audio or video encoder, frame rate and video resolution. However, these knobs are suboptimal because the streaming software receives completed audio or video frames to encode. While the server can adjust resolution, it does so by scaling the already rendered image ... These adjustments ... process video and audio that has already been rendered.” and col. 9, lines 25-29, “In current cloud gaming systems, the streaming logic/server receives the completed video frames from the video game, and can only scale the video frames in dimensions and in color depth after the fact.”)
43. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 1 of Lin in view of Kwak, and further in view of Taoka to include a game device, wherein the first resolution comprises an output resolution of a game engine, and the second resolution comprises a display resolution of the game device, and wherein acquiring the first image comprises acquiring the first image from the game engine of Colenbrander. The motivation for this modification could have been utilize interpolation techniques with a game engine and game console device in order to produce higher quality output and resolutions, especially in relation to the resources available to render and display the video game.
44. As per claim 15, Lin in view of Kwak, and further in view of Taoka, and further in view of Colenbrander discloses: The method according to claim 14, further comprising determining the first resolution based on attribute information about the game device, wherein the attribute information about the game device comprises at least one of the following:
a computing power of the game device, a load condition of the game device, a temperature of the game device, or a model feature of the game device. (Colenbrander, col. 7, lines 19-33, “By way of example without limitation, the game stream quality falling below a given threshold can be defined by various measures, such as packet loss exceeding a predefined threshold, bandwidth falling below a predefined threshold, latency exceeding a predefined threshold, etc. The resulting reduction in game quality setting can include various measures described in further detail herein, such as reducing resolution or framerate, reducing texture quality, reducing detail settings, reducing the scope of the virtual environment that is rendered, etc. Similar to the above, if the game stream quality rises above a given threshold, then no action may be taken in some implementations, whereas in other implementations, then at method operation 210, the video game is optionally instructed to increase a game quality setting.” and col. 17, lines 59-65, “An advantage of using a distributed game engine is that it is possible to take advantage of elastic computing, wherein computing resources can be scaled up or down depending upon needs. For example, in a large multiplayer game executed traditionally on a single hardware server, after for example about 100 players, hardware resources become limited, so that more players cannot be added.” and col. 18, lines 5-10, “Thus, a cloud game engine can have functionality distributed to different processing entities. It will be appreciated that different functions can be executed in different frameworks. For example, some functions (e.g. social) might be easier to run in a container, whereas graphics might be better run using a VM connected to a GPU.” and col. 10, lines 7-10, “In some implementations, detection of switching from one device to another can trigger adjustments performed by the video game, such as the above-mentioned changes in video resolution, frame rate, or aspect ratio.”)
45. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 14 of Lin in view of Kwak, and further in view of Taoka to include the that the attribute information about the game device comprises at least one of the following: a computing power of the game device, a load condition of the game device, a temperature of the game device, or a model feature of the game device of Colenbrander. The motivation for this modification could have been provide information about how a video game is running so that the video game engine or game device can dynamically adjust to produce the highest quality game experience while still within available computing resources.
46. As per claim 16, Lin in view of Kwak, and further in view of Taoka, and further in view of Colenbrander discloses: The method according to claim 15, wherein the determining the first resolution comprises:
determining the first resolution as A1 by B1 if the attribute information about the game device satisfies a target condition; or
determining the first resolution as A2 by B2 if the attribute information about the game device does not satisfy the target condition, wherein A1 is greater than A2 and/or B1 is greater than B2, and the target condition comprises at least one of the following: (Colenbrander, col. 2, lines 43-60, “In some implementations, a method is provided, including the following method operations: executing a video game by a cloud game machine, the execution of the video game includes rendering gameplay video ... responsive to detecting a change in the connection quality between the streaming server and the client device, then adjusting the rendering of the gameplay video by the cloud game machine, wherein adjusting the rendering of the gameplay video by the cloud game machine includes adjusting a frame rate or a resolution for the gameplay video.” and col. 9, lines 36-48, “However, rather than scaling such a video frame after it has already been generated, the video game itself can be instructed to (or caused to) render the video frame at a different resolution, which enables the video game to optimize the rendering for the chosen resolution, e.g. so that text quality is optimized. In this way, the resolution can be dynamically changed to any arbitrary resolution without penalty from scaling after the fact. For example, when stream quality or network conditions worsen, then the video game 104 can be caused to decrease the video resolution; and when stream quality or network conditions improve, then the video game 104 can be caused to increase the video resolution.”)
the computing power of the game device being greater than a target power threshold, the load condition of the game device being less than a target load threshold, the temperature of the game device being less than a target temperature threshold, or the model feature of the game device exceeding a target model feature. (Colenbrander, col. 7, lines 19-33, “By way of example without limitation, the game stream quality falling below a given threshold can be defined by various measures, such as packet loss exceeding a predefined threshold, bandwidth falling below a predefined threshold, latency exceeding a predefined threshold, etc. The resulting reduction in game quality setting can include various measures described in further detail herein, such as reducing resolution or framerate, reducing texture quality, reducing detail settings, reducing the scope of the virtual environment that is rendered, etc. Similar to the above, if the game stream quality rises above a given threshold, then no action may be taken in some implementations, whereas in other implementations, then at method operation 210, the video game is optionally instructed to increase a game quality setting.” and col. 17, lines 59-65, “An advantage of using a distributed game engine is that it is possible to take advantage of elastic computing, wherein computing resources can be scaled up or down depending upon needs. For example, in a large multiplayer game executed traditionally on a single hardware server, after for example about 100 players, hardware resources become limited, so that more players cannot be added.” and col. 18, lines 5-10, “Thus, a cloud game engine can have functionality distributed to different processing entities. It will be appreciated that different functions can be executed in different frameworks. For example, some functions (e.g. social) might be easier to run in a container, whereas graphics might be better run using a VM connected to a GPU.” and col. 10, lines 7-10, “In some implementations, detection of switching from one device to another can trigger adjustments performed by the video game, such as the above-mentioned changes in video resolution, frame rate, or aspect ratio.”)
47. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the method of claim 15 of Lin in view of Kwak, and further in view of Taoka to include determining the first resolution as A1 by B1 if the attribute information about the game device satisfies a target condition or determining the first resolution as A2 by B2 if the attribute information about the game device does not satisfy the target condition, wherein the computing power of the game device being greater than a target power threshold, the load condition of the game device being less than a target load threshold, the temperature of the game device being less than a target temperature threshold, or the model feature of the game device exceeding a target model feature of Colenbrander. The motivation for this modification could have been increase the quality of the video game output by determining which screen resolution is best suited for gameplay and also depending on the resources or conditions with the game device.
48. Claim 18 is similar in scope to claim 1 except for additional limitations that Lin in view of Kwak, and further in view of Taoka, and further in view of Colenbrander discloses: One or more non-transitory computer-readable media storing instructions that, when executed, cause: (Lin, col. 11, lines 8-17, ““Computer-readable storage media” may refer to media and/or devices that enable persistent and/or non-transitory storage of information ... The computer-readable storage media includes ... a method or technology suitable for storage of information such as computer readable instructions …”)
determining a first resolution and a second resolution, the first resolution being an output resolution of a game engine, and the second resolution being a display resolution of a game device; (Colenbrander, col. 5, lines 13-18, “By way of example without limitation, the client device 122 can be a game console ... capable of streaming gameplay of a video game from a cloud gaming provider as in the present disclosure.” and col. 1, line 65-col. 2, line 1, “In some implementations, a method is provided, including the following operations: executing a video game by a cloud game machine, the execution of the video game includes rendering gameplay video ...” and col. 4, lines 19-30, “There are a few settings which can be adjusted such as the bitrate of the audio or video encoder, frame rate and video resolution. However, these knobs are suboptimal because the streaming software receives completed audio or video frames to encode. While the server can adjust resolution, it does so by scaling the already rendered image ... These adjustments ... process video and audio that has already been rendered.” and col. 9, lines 25-29, “In current cloud gaming systems, the streaming logic/server receives the completed video frames from the video game, and can only scale the video frames in dimensions and in color depth after the fact.”)
49. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the non-transitory computer-readable media of Lin in view of Kwak, and further in view of Taoka to include determining a first resolution and a second resolution, the first resolution being an output resolution of a game engine, and the second resolution being a display resolution of a game device of Colenbrander. The motivation for this modification could have been utilize interpolation techniques with a game engine and game console device in order to produce higher quality output and resolutions, especially in relation to the resources available to render and display the video game.
50. As per claim 19, Lin in view of Kwak, and further in view of Taoka, and further in view of Colenbrander discloses: The one or more non-transitory computer-readable media of claim 18, wherein the instructions, when executed, cause determining the interpolation feature by:
calculating the interpolation feature according to a plurality of second pixel blocks, the plurality of second pixel blocks comprising adjacent pixel blocks located around the first pixel block. (Lin col. 6, lines 35-38, “As before, the factor upscale module 210 may be used to identify which of a plurality of patches in a neighborhood of a location in an image is most similar to a patch in an upscaled input image (block 602).”)
51. Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US-9123138-B2, hereinafter "Lin") in view of Kwak, (WO-2017/217656-A1), further in view of Taoka, (US-2019/0205019-A1), further in view of Colenbrander (US-11731043-B2), and further in view of Hoshii et al. (US-2002/0015162-A1, hereinafter "Hoshii").
52. Claim 20, which is similar in scope to dependent claims 3, 19 and independent claim 18 is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 3.
Conclusion
53. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW CLOTHIER whose telephone number is (571)272-4667. The examiner can normally be reached Mon-Fri 8:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW CLOTHIER/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights