DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 28th January, 2026 has been entered.
Response to Amendment
This action is in response to the amendment filed on 28th January, 2026. Claims 1, 12, and 19 have been amended. Claim 17 has been cancelled. Claims 1-16 and 18-19 remain rejected in the application.
Response to Arguments
Applicant's arguments with respect to Claims 1, 12, and 19 filed on 28th January, 2026, with respect to the rejection under 35 U.S.C. § 103, regarding that the prior art does not teach the limitation(s): "generate a first temporary upscaling image of an image upscaled in software using an algorithm of the first hardware scaler that converts the image into an image with a preconfigured high-definition resolution, generate a second temporary upscaling image of the image upscaled in software using an algorithm of the second hardware scaler that converts the image into an image of a lower quality than the first temporary upscaling image" has been fully considered, but are moot because of new grounds for rejection. It has now been taught by the combination of Zhai and Damkat.
Regarding arguments to Claims 2-11, 13-16, and 18, they directly/indirectly depend on independent Claims 1, 12, and 19 respectively. Applicant does not argue anything other than independent Claims 1, 12, and 19. The limitations in those claims, in conjunction with combination, was previously established as explained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 9-13, 16, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhai et al. (US 20140072029 A1, previously cited), hereinafter referenced as Zhai, in view of Damkat (US 20110081094 A1).
Regarding Claim 1, Zhai discloses an electronic device (Zhai, [0016]: teaches a video/decoder system 100 <read on electronic device>) comprising:
a display (Zhai, [0027]: teaches a display device 320);
a plurality of hardware scalers including a first hardware scaler and a second hardware scaler (Zhai, [0027]: teaches post-processor 312 including multiple scalers, such as hardware scaler 316 <read on second hardware scaler> and software scaler 318 <read on first hardware scaler> as shown in FIG. 3; [0046]: teaches two or more software and hardware scalers being implemented; Note: it should be noted that the scalers being used are being interpreted as hardware scalers that process software-based upscaling code), wherein
PNG
media_image1.png
223
566
media_image1.png
Greyscale
memory including one or more storage media, storing instructions (Zhai, [0047]: teaches a non-transitory computer-readable storage medium <read on memory>, such as storage device <read on storage media>, that stores instruction sets); and
at least one processor including processing circuitry (Zhai, [0047]: teaches the non-transitory computer-readable storage medium storing sets of instructions that are executable by a processor in a system <read on processing circuitry>) wherein
the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:generate a first temporary upscaling image of an image upscaled in software using an algorithm of the first hardware scaler that converts the image into an image with a preconfigured high-definition resolution (Zhai, [0027]: teaches post-processor 312 including software scaler 318 <read on first hardware scaler> that upscales image data to fit the display area of the display device using the proper resolution <read on converting image into image with preconfigured HD resolution>; [0030]: teaches the software scaler including an adjustable scaling algorithm <read on software using algorithm>; [0038]: teaches a controller reviewing each available scaler that meets a budget requirement and selects "the one that meets the characteristic budgets and that has the highest output image quality <read on first temporary image>"; Note: it should be noted that the scalers and their respective images prior to selection are by default temporary images <read on first temporary image>),
generate a second temporary upscaling image of the image upscaled in software using an algorithm of the second hardware scaler that converts the image into an image of a lower quality than the first temporary upscaling image (Zhai, [0027]: teaches post-processor 312 including hardware scaler 316 <read on second hardware scaler> that upscales image data to fit the display area of the display device; [0005]: teaches hardware video scalers using fixed algorithms <read on using algorithm>; [0038]: teaches a controller reviewing each available scaler that meets a budget requirement and selects "the one that meets the characteristic budgets and that has the highest output image quality <read on first temporary image>"; Note: it should be noted that the scalers and their respective images prior to selection are by default temporary images <read on second temporary image>; additionally, it is being interpreted that there are differing image qualities as one temporary image has the highest quality, where at least one of the other temporary images have lower image quality than the selected highest output image quality),
[[identify a difference value of a pixel between the first temporary upscaling image and the second temporary upscaling image,]]
select one of the first hardware scaler or the second hardware scaler [[based on the identified difference value and a preconfigured threshold value]] (Zhai, [0028]: teaches controller 404 selecting "a scaler to perform a scaling operation and multiple scalers 406 <read on second hardware scaler>, 408 <read on first hardware scaler> to scale the input video data"),
upscale the image by using the selected one hardware scaler (Zhai, [0035]: teaches upscaling received video data based on controller 208 determining which scaler to use for a portion of the video data, where "such portion may include a sequence of video frames, a single frame, or a part of a frame" as shown in FIG. 5), and
PNG
media_image2.png
319
341
media_image2.png
Greyscale
control the display to display the upscaled image (Zhai, [0027]: teaches video decoder 302 including "a post-processor 312 that prepares the video data for display," where "the post-processor 312 may additionally include multiple scalers, such as a hardware scaler 316 and a software scaler 318, and a controller 314 to manage the scalers"; [0027]: further teaches scaling image data to fit the display area of the display device 320 <read on control display to display upscaled image>).
However, Zhai does not expressly disclose
identify a difference value of a pixel between the first temporary upscaling image and the second temporary upscaling image, and
select one of the first hardware scaler or the second hardware scaler based on the identified difference value and a preconfigured threshold value.
Damkat discloses
identify a difference value of a pixel between the first temporary upscaling image and the second temporary upscaling image (Damkat, [0115]: teaches generating a reciprocal average squared pixel value difference <read on difference value of pixel> for a set of pixels when comparing images <read on first and second temporary images>), and
select one of the first hardware scaler or the second hardware scaler based on the identified difference value and a preconfigured threshold value (Damkat, [0115]: teaches generating a reciprocal average squared pixel value difference <read on difference value of pixel> for a set of pixels when comparing images, where a distance criterion <read on preconfigured threshold value> is used).
Damkat is analogous art with respect to Zhai because they are from the same field of endeavor, namely upscaling video frames using scalers. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to compare sets of pixels that are present in each frame of a sequence of video frames as taught by Damkat into the teaching of Zhai. The suggestion for doing so would allow the system to perform resolution comparisons, thereby enabling selective upscaling of image portions, resulting in a more dynamic upscaling process that yields improved results. Therefore, it would have been obvious to combine Damkat with Zhai.
Regarding Claim 12, it recites the limitations that are similar in scope to Claim 1, but in a control method. As shown in the rejection, the combination of Zhai and Damkat discloses the limitations of Claim 1. Additionally, Zhai discloses a control method of an electronic device, the control method (Zhai, [0035]: teaches a method <read on control method> for selecting between multiple available scalers in a system; [0016]: teaches a video/decoder system 100 <read on electronic device>) comprising:…
Thus, Claim 12 is met by Zhai according to the mapping presented in the rejection of Claim 1, given the electronic device corresponds to a control method.
Regarding Claim 19, it recites the limitations that are similar in scope to Claim 1, but in a non-transitory computer readable medium. As shown in the rejection, the combination of Zhai and Damkat discloses the limitations of Claim 1. Additionally, Zhai discloses a non-transitory computer readable medium which, when implemented by an electronic device related to a control method, causes the electronic device to implement operations (Zhai, [0047]: teaches a system using a non-transitory computer-readable storage medium that stores a set of instructions <read on operations>; [0035]: teaches a method <read on control method> for selecting between multiple available scalers in a system; [0016]: teaches a video/decoder system 100 <read on electronic device>) comprising:…
Thus, Claim 19 is met by Zhai according to the mapping presented in the rejection of Claim 1, given the electronic device corresponds to a non-transitory computer readable medium.
Regarding Claims 2 and 13, the combination of Zhai and Damkat discloses the electronic device and the control method of Claims 1 and 12 respectively. Additionally, Zhai further discloses wherein the instructions, when executed by the at least one processor individually or collectively, further cause the electronic device to:
divide the first temporary upscaling image and the second temporary upscaling image into preconfigured areas (Zhai, [0003]: teaches general coding systems in the art dividing frames into blocks <read on preconfigured areas> of pixels; [0028]: teaches controller 404 assigning scalers for a sequence of frames <read on first and second temporary upscaling images> for each portion of a frame), and
[[identify a difference value between a pixel included in the divided area of the first temporary upscaling image and a corresponding pixel included in the divided area of the second temporary upscaling image.]]
However, Zhai does not expressly disclose
identify a difference value between a pixel included in the divided area of the first temporary upscaling image and a corresponding pixel included in the divided area of the second temporary upscaling image.
Damkat discloses
identify a difference value between a pixel included in the divided area of the first temporary upscaling image and a corresponding pixel included in the divided area of the second temporary upscaling image (Damkat, [0115]: teaches generating a reciprocal average squared pixel value difference <read on difference value of pixel> for a set of pixels <read on corresponding pixel> when comparing images <read on first and second temporary images>, where a distance criterion is used to determine a match metric <read on corresponding pixels included in both divided areas>; Note: it should be noted that the match metric is being interpreted as an identified difference value present in both frames).
Damkat is analogous art with respect to Zhai because they are from the same field of endeavor, namely upscaling video frames using scalers. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to compare sets of pixels that are present in each frame of a sequence of video frames as taught by Damkat into the teaching of Zhai. The suggestion for doing so would allow the system to perform resolution comparisons, thereby enabling selective upscaling of image portions, resulting in a more dynamic upscaling process that yields improved results. Therefore, it would have been obvious to combine Damkat with Zhai.
Regarding Claim 9, the combination of Zhai and Damkat discloses the electronic device of Claim 1. Additionally, Zhai further discloses wherein the instructions, when executed by the at least one processor individually or collectively, further cause the electronic device to:
upscale the image to a preconfigured high-definition resolution image by using the first scaler in case that the first scaler is selected (Zhai, [0027]: teaches post-processor 312 including multiple scalers to switch between using controller 314, such as selecting software scaler 318 <read on selecting first scaler>, where the image data is upscaled to the proper resolution of the display area of the display device 320 <read on upscale image to preconfigured HD resolution image>),
deliver the upscaled image and resolution information of the upscaled image to the display (Zhai, [0027]: teaches controller 314 receiving feedback that indicates output size or resolution display parameters <read on deliver upscaled image and resolution information of upscaled image to display>; Note: it should be noted that although not expressly stated, it is common in the art for contemporary displays to briefly display the current resolution output on screen whenever resolution modifications are applied), and
control the display to display the delivered upscaled image based on the upscaled resolution information (Zhai, [0027]: teaches scaling image data to fit the display area of the display device 320 <read on control display to display upscaled image>, where controller 314 receives feedback that indicates output size and resolution display parameters <read on upscaled resolution information>).
Regarding Claim 10, the combination of Zhai and Damkat discloses the electronic device of Claim 1. Additionally, Zhai further discloses wherein the instructions, when executed by the at least one processor individually or collectively, further cause the electronic device to:
deliver the image and resolution information of the image to be upscaled to the display in case that the second scaler is selected (Zhai, [0021]: teaches pre-processor 206 including multiple scalers to switch between using controller 208; [0022]: teaches controller 208 selecting the appropriate scaler 210 <read on select second scaler> or 212; [0023]: teaches controller 208 setting the desired output resolution <read on deliver resolution information of image to be upscaled> for hardware scaler 210),
upscale the image by using the second scaler based on the resolution information of the image to be upscaled (Zhai, [0021]: teaches using scalers to upscale video for display, where the scaler is either a hardware scaler 210 <read on second scaler> or software scaler 212), and
control the display to display the upscaled image (Zhai, [0027]: teaches video decoder 302 including "a post-processor 312 that prepares the video data for display," where "the post-processor 312 may additionally include multiple scalers, such as a hardware scaler 316 and a software scaler 318, and a controller 314 to manage the scalers"; [0027]: further teaches scaling image data to fit the display area of the display device 320 <read on control display to display upscaled image>).
Regarding Claim 11, the combination of Zhai and Damkat discloses the electronic device of Claim 1. Additionally, Zhai further discloses wherein
the first hardware scaler is included in the at least one processor (Zhai, [0027]: teaches video decoder 302 including post-processor 312, which further includes multiple scalers, such as hardware scaler 316 and software scaler 318 <read on first hardware scaler>), and wherein
the second hardware scaler is included in a display driver integrated circuit (IC) (DDI) that drives the display (Zhai, [0029]: teaches hardware scaler 406 <read on second hardware scaler> being implemented as an application specific integrated circuit (ASIC) <read on IC DDI> or image signal processor (ISP)).
Regarding Claim 16, the combination of Zhai and Damkat discloses the control method of Claim 12. Additionally, Zhai further discloses wherein
the second hardware scaler upscales the image into an image of a lower quality than the image upscales by the first hardware scaler (Zhai, [0030]: teaches the software scaler <read on first scaler> having "certain additional flexibilities that may result in a higher quality output than the restricted hardware scalers 406 <read on second scaler>"; [0046]: teaches two or more software and hardware scalers being implemented; Note: it should be noted that the software scaler having "additional flexibilities" is being interpreted as it being able to generate higher quality images than hardware scalers).
Claims 3-4 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhai et al. (US 20140072029 A1, previously cited), hereinafter referenced as Zhai, in view of Damkat (US 20110081094 A1) as applied to Claims 2 and 13 above respectively, and further in view of Milanfar et al. (US 20180253826 A1, previously cited), hereinafter referenced as Milanfar.
Regarding Claims 3 and 14, the combination of Zhai and Damkat discloses the electronic device and the control method of Claims 2 and 13 respectively. Additionally, Zhai further discloses wherein the instructions, when executed by the at least one processor individually or collectively, further cause the electronic device to:
[[identify difference values in all the divided areas,]]
[[identify an average value of the identified difference values,]]
select the first hardware scaler [[in case that the identified average value is greater than or equal to a preconfigured first threshold value]] (Zhai, [0028]: teaches a multiple scaler unit 400 including a controller 404 that selects an appropriate scaler to perform a scaling operation, such as upscaling input video data using software scaler 408 <read on select first hardware scaler>), and
select the second hardware scaler [[in case that the identified average value is less than the preconfigured first threshold value]] (Zhai, [0028]: teaches a multiple scaler unit 400 including a controller 404 that selects an appropriate scaler to perform a scaling operation, such as upscaling input video data using hardware scaler 406 <read on select second hardware scaler>).
However, Zhai does not expressly disclose
identify difference values in all the divided areas,
identify an average value of the identified difference values,
select the first hardware scaler in case that the identified average value is greater than or equal to a preconfigured first threshold value, and
select the second hardware scaler in case that the identified average value is less than the preconfigured first threshold value.
Damkat discloses
identify difference values in all the divided areas (Damkat, [0115]: teaches generating a reciprocal average squared pixel value difference <read on difference value of pixels> for a set of pixels when comparing images),
identify an average value of the identified difference values (Damkat, [0115]: teaches generating a reciprocal average <read on average value> squared pixel value difference <read on difference value of pixels> for a set of pixels when comparing images),
[[select the first hardware scaler in case that the identified average value is greater than or equal to a preconfigured first threshold value, and]]
[[select the second hardware scaler in case that the identified average value is less than the preconfigured first threshold value.]]
Damkat is analogous art with respect to Zhai because they are from the same field of endeavor, namely upscaling video frames using scalers. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to compare sets of pixels that are present in each frame of a sequence of video frames as taught by Damkat into the teaching of Zhai. The suggestion for doing so would allow the system to perform resolution comparisons, thereby enabling selective upscaling of image portions, resulting in a more dynamic upscaling process that yields improved results. Therefore, it would have been obvious to combine Damkat with Zhai.
However, the combination of Zhai and Damkat does not expressly disclose
select the first hardware scaler in case that the identified average value is greater than or equal to a preconfigured first threshold value, and
select the second hardware scaler in case that the identified average value is less than the preconfigured first threshold value.
Milanfar discloses
select the first hardware scaler in case that the identified average value is greater than or equal to a preconfigured first threshold value (Milanfar, [0072]: teaches the upscaling runtime system determining that the amount of change between the upscaled low resolution image and the aggregated filtered patches that used average filters <read on identified average value> for a given area/patch satisfies a threshold amount <read on identified average value being greater than or equal to preconfigured first threshold value>), and
select the second hardware scaler in case that the identified average value is less than the preconfigured first threshold value (Milanfar, [0072]: teaches the upscaling runtime system determining that the amount of change between the upscaled low resolution image and the aggregated filtered patches that used average filters for a given area/patch not satisfying a threshold amount <read on identified average value being less than or equal to preconfigured first threshold value>).
Milanfar is analogous art with respect to Zhai, in view of Damkat because they are from the same field of endeavor, namely upscaling video image frames. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to divide pixels of the low resolution image into buckets for filtering and aggregation as taught by Milanfar into the teaching of Zhai, in view of Damkat. The suggestion for doing so would allow the system to determine a weighted average for an optimized upscaled image output, thereby yielding predictable results. Therefore, it would have been obvious to combine Milanfar with Zhai, in view of Damkat.
Regarding Claims 4 and 15, the combination of Zhai and Damkat discloses the electronic device and the control method of Claims 2 and 13 respectively. Additionally, Zhai further discloses wherein the instructions, when executed by the at least one processor individually or collectively, further cause the electronic device to:
[[identify difference values in all the divided areas,]]
[[identify a preconfigured number of difference values in descending order among the identified difference values,]]
[[identify a sum of the identified preconfigured number of difference values,]]
select the first hardware scaler [[in case that the identified sum is greater than or equal to a preconfigured second threshold value]] (Zhai, [0028]: teaches a multiple scaler unit 400 including a controller 404 that selects an appropriate scaler to perform a scaling operation, such as upscaling input video data using software scaler 408 <read on select first hardware scaler>), and
select the second hardware scaler [[in case that the identified sum is less than the preconfigured second threshold value]] (Zhai, [0028]: teaches a multiple scaler unit 400 including a controller 404 that selects an appropriate scaler to perform a scaling operation, such as upscaling input video data using hardware scaler 406 <read on select second hardware scaler>).
However, Zhai does not expressly disclose
identify difference values in all the divided areas,
identify a preconfigured number of difference values in descending order among the identified difference values,
identify a sum of the identified preconfigured number of difference values,
select the first hardware scaler in case that the identified sum is greater than or equal to a preconfigured second threshold value, and
select the second hardware scaler in case that the identified sum is less than the preconfigured second threshold value.
Damkat discloses
identify difference values in all the divided areas (Damkat, [0115]: teaches generating a reciprocal average squared pixel value difference <read on difference value of pixels> for a set of pixels when comparing images),
identify a preconfigured number of difference values in descending order among the identified difference values (Damkat, [0112]: teaches a first and second sample pixel sets <read on descending order of identified preconfigured number of difference values>, which corresponds to a comparison pixel set),
identify a sum of the identified preconfigured number of difference values (Damkat, [0115]: teaches generating a reciprocal average <read on sum> squared pixel value difference <read on difference value of pixels> for a set of pixels when comparing images),
[[select the first hardware scaler in case that the identified sum is greater than or equal to a preconfigured second threshold value, and]]
[[select the second hardware scaler in case that the identified sum is less than the preconfigured second threshold value.]]
Damkat is analogous art with respect to Zhai because they are from the same field of endeavor, namely upscaling video frames using scalers. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to compare sets of pixels that are present in each frame of a sequence of video frames as taught by Damkat into the teaching of Zhai. The suggestion for doing so would allow the system to perform resolution comparisons, thereby enabling selective upscaling of image portions, resulting in a more dynamic upscaling process that yields improved results. Therefore, it would have been obvious to combine Damkat with Zhai.
However, the combination of Zhai and Damkat does not expressly disclose
select the first hardware scaler in case that the identified sum is greater than or equal to a preconfigured second threshold value (Milanfar, [0072]: teaches the upscaling runtime system determining that the amount of change between the upscaled low resolution image and the aggregated filtered patches that used average filters <read on identified sum value> for a given area/patch satisfies a threshold amount <read on identified sum value being greater than or equal to preconfigured second threshold value>; Note: it should be noted that the preconfigured first and second threshold values are being interpreted as similar terms), and
select the second hardware scaler in case that the identified sum is less than the preconfigured second threshold value (Milanfar, [0072]: teaches the upscaling runtime system determining that the amount of change between the upscaled low resolution image and the aggregated filtered patches that used average filters for a given area/patch not satisfying a threshold amount <read on identified sum value being less than or equal to preconfigured second threshold value>).
Milanfar is analogous art with respect to Zhai, in view of Damkat because they are from the same field of endeavor, namely upscaling video image frames. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to divide pixels of the low resolution image into buckets for filtering and aggregation as taught by Milanfar into the teaching of Zhai, in view of Damkat. The suggestion for doing so would allow the system to determine a weighted average for an optimized upscaled image output, thereby yielding predictable results. Therefore, it would have been obvious to combine Milanfar with Zhai, in view of Damkat.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Zhai et al. (US 20140072029 A1, previously cited), hereinafter referenced as Zhai, in view of Damkat (US 20110081094 A1) as applied to Claim 2 above respectively, and further in view of Motilla et al. (US 20230325977 A1, previously cited), hereinafter referenced as Motilla.
Regarding Claim 5, the combination of Zhai and Damkat discloses the electronic device of Claim 2. Additionally, Zhai further discloses wherein the instructions, when executed by the at least one processor individually or collectively, further cause the electronic device to:
[[identify difference values in all the divided areas,]]
select the first hardware scaler [[in case that a largest difference value among the identified difference values is greater than or equal to a preconfigured third threshold value]] (Zhai, [0028]: teaches a multiple scaler unit 400 including a controller 404 that selects an appropriate scaler to perform a scaling operation, such as upscaling input video data using software scaler 408 <read on select first hardware scaler>), and
select the second hardware scaler [[in case that the largest difference value is less than the preconfigured third threshold value]] (Zhai, [0028]: teaches a multiple scaler unit 400 including a controller 404 that selects an appropriate scaler to perform a scaling operation, such as upscaling input video data using hardware scaler 406 <read on select second hardware scaler>).
However, Zhai does not expressly disclose
identify difference values in all the divided areas,
select the first hardware scaler in case that a largest difference value among the identified difference values is greater than or equal to a preconfigured third threshold value, and
select the second hardware scaler in case that the largest difference value is less than the preconfigured third threshold value.
Damkat discloses
identify difference values in all the divided areas (Damkat, [0115]: teaches generating a reciprocal average squared pixel value difference <read on difference value of pixels> for a set of pixels when comparing images),
[[select the first hardware scaler in case that a largest difference value among the identified difference values is greater than or equal to a preconfigured third threshold value, and]]
[[select the second hardware scaler in case that the largest difference value is less than the preconfigured third threshold value.]]
Damkat is analogous art with respect to Zhai because they are from the same field of endeavor, namely upscaling video frames using scalers. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to compare sets of pixels that are present in each frame of a sequence of video frames as taught by Damkat into the teaching of Zhai. The suggestion for doing so would allow the system to perform resolution comparisons, thereby enabling selective upscaling of image portions, resulting in a more dynamic upscaling process that yields improved results. Therefore, it would have been obvious to combine Damkat with Zhai.
However, the combination of Zhai and Damkat does not expressly disclose
select the first hardware scaler in case that a largest difference value among the identified difference values is greater than or equal to a preconfigured third threshold value, and
select the second hardware scaler in case that the largest difference value is less than the preconfigured third threshold value.
Motilla discloses
select the first hardware scaler in case that a largest difference value among the identified difference values is greater than or equal to a preconfigured third threshold value (Motilla, [0124]: teaches selecting "the tiles with the highest detail score <read on largest difference value among identified difference values being greater than or equal to preconfigured third threshold value> as candidates for performing upscaling to (e.g. super resolution), unless they are in a list of tiles for which it is not necessary to perform super resolution on (e.g. index value = 0, or tile already upscaled and stored in cache)"; [0125]: teaches "selecting may comprise selecting one or more image portions having a luminance value above a predetermined threshold <read on preconfigured third threshold value>"), and
select the second hardware scaler in case that the largest difference value is less than the preconfigured third threshold value (Motilla, [0124]: teaches selecting "the tiles with the highest detail score as candidates for performing upscaling to (e.g. super resolution), unless they are in a list of tiles for which it is not necessary to perform super resolution on (e.g. index value = 0, or tile already upscaled and stored in cache) <read on largest difference value among identified difference values being less than preconfigured third threshold value>"; [0125]: teaches "selecting may comprise selecting one or more image portions having a luminance value above a predetermined threshold <read on preconfigured third threshold value>").
Motilla is analogous art with respect to Zhai, in view of Damkat because they are from the same field of endeavor, namely upscaling image frames. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a cache transfer mechanism to temporarily store generated frames for future comparisons as taught by Motilla into the teaching of Zhai, in view of Damkat. The suggestion for doing so would allow the system to determine how similar each frame is during the encoding stage, thereby offering the system context on which scaler to switch to for improved optimization and performance. Therefore, it would have been obvious to combine Motilla with Zhai, in view of Damkat.
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Zhai et al. (US 20140072029 A1, previously cited), hereinafter referenced as Zhai, in view of Damkat (US 20110081094 A1) as applied to Claim 1 above respectively, and further in view of Subramaniam et al. (US 20240111840 A1, previously cited), hereinafter referenced as Subramaniam.
Regarding Claim 6, the combination of Zhai and Damkat discloses the electronic device of Claim 1. The combination of Zhai and Damkat does not expressly disclose the limitations of Claim 6; however, Subramaniam discloses wherein the at least one processor comprises at least one of
a central processing unit (Subramaniam, [0032]: teaches an electronic device 300 that includes nodes 302-306, which "are separate computational resources that include hardware for performing computational, control, memory access, and/or other operations," where "node 302 is a central processing unit (CPU) or CPU core and nodes 304-306 are GPUs or GPU cores—and thus electronic device 300 includes a mixture of a CPU and GPUs" as shown in FIG. 3),
PNG
media_image3.png
529
380
media_image3.png
Greyscale
a neural network processing unit (Subramaniam, [0032]: teaches the electronic device including nodes 302-306, which "are separate computational resources that include hardware for performing computational, control, memory access, and/or other operations," where "at least one of nodes 302-306 is or includes a neural network accelerator <read on neural network processing unit>, i.e., a functional block that is arranged to dynamically process neural network data and/or neural network internal elements to improve the performance of processing instances of input data through a neural network"), and
a graphics processing unit (Subramaniam, [0032]: teaches the electronic device including nodes 302-306, which "are separate computational resources that include hardware for performing computational, control, memory access, and/or other operations," where "nodes 302-306 are graphics processing units (GPUs) or GPU cores, each having a local GPU memory (i.e., memory 308)"), and wherein
the instructions, when executed by the at least one processor individually or collectively, further cause the electronic device to generate the first temporary upscaling image and the second temporary upscaling image by using the graphics processing unit in case that a usage amount of the graphics processing unit is less than a preconfigured value (Subramaniam, [0046]: teaches the convolutional neural network including "a number of convolutional layers, addition layers, rectified linear unit or 1×1 layers, and/or other layers used for image upscaling," where "the instances of input data that are processed in the neural network are lower resolution digital images <read on generate first and second temporary upscaling images> and the overall result of the neural network are higher resolution digital images"; [0047]: teaches the processing circuitry including GPU cores that perform operations for processing input data; [0074]: teaches the processing circuitry choosing a tiling scheme to upscale digital images, where if there is enough performance overhead <read on usage amount being less than preconfigured value>, then the processing circuitry forgoes selecting a tiling scheme; Note: it should be noted that it is common in the art for GPU hardware to be used for neural networks; in addition, it is being interpreted that the GPU will upscale the image in a single pass without selecting a tiling scheme through a neural network).
Subramaniam is analogous art with respect to Zhai, in view of Damkat because they are from the same field of endeavor, namely upscaling image frames. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a convolutional neural network that selects an appropriate tiling scheme for image upscaling as taught by Subramaniam into the teaching of Zhai, in view of Damkat. The suggestion for doing so would allow the selection process to be further automated, thereby allowing the system to more accurately determine when and where to apply scalers and when to switch said scalers. Therefore, it would have been obvious to combine Subramaniam with Zhai, in view of Damkat.
Regarding Claim 7, the combination of Zhai, Damkat, and Subramaniam discloses the electronic device of Claim 6. The combination of Zhai and Damkat does not expressly disclose the limitations of Claim 7; however, Subramaniam discloses wherein the instructions, when executed by the at least one processor individually or collectively, further cause the electronic device to
generate the first temporary upscaling image and the second temporary upscaling image by using the neural network processing unit in case that the usage amount of the graphics processing unit exceeds the preconfigured value (Subramaniam, [0046]: teaches the convolutional neural network including "a number of convolutional layers, addition layers, rectified linear unit or 1×1 layers, and/or other layers used for image upscaling," where "the instances of input data that are processed in the neural network are lower resolution digital images <read on generate first and second temporary upscaling images> and the overall result of the neural network are higher resolution digital images"; [0074]: teaches the neural network choosing a tiling scheme to upscale digital images, where if there is enough performance overhead, then the neural network forgoes selecting a tiling scheme; Note: it should be noted that if there isn't enough performance overhead, then the neural network selects a tiling scheme to aid in upscaling the images, where the lack of performance overhead is being interpreted as the GPU exceeding the preconfigured value).
Subramaniam is analogous art with respect to Zhai, in view of Damkat because they are from the same field of endeavor, namely upscaling image frames. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a convolutional neural network that selects an appropriate tiling scheme for image upscaling as taught by Subramaniam into the teaching of Zhai, in view of Damkat. The suggestion for doing so would allow the selection process to be further automated, thereby allowing the system to more accurately determine when and where to apply scalers and when to switch said scalers. Therefore, it would have been obvious to combine Subramaniam with Zhai, in view of Damkat.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Zhai et al. (US 20140072029 A1, previously cited), hereinafter referenced as Zhai, in view of Damkat (US 20110081094 A1), and further in view of Subramaniam et al. (US 20240111840 A1, previously cited), hereinafter referenced as Subramaniam as applied to Claim 7 above respectively, and further in view of Liktor et al. (US 20230146259 A1, previously cited), hereinafter referenced as Liktor.
Regarding Claim 8, the combination of Zhai, Damkat, and Subramaniam discloses the electronic device of Claim 7. The combination of Zhai, Damkat, and Subramaniam does not expressly disclose the limitations of Claim 8; however, Liktor discloses wherein the instructions, when executed by the at least one processor individually or collectively, further cause the electronic device to
generate the first temporary upscaling image and the second temporary upscaling image by using the central processing unit in case that the central processing unit includes a data processing acceleration function (Liktor, [0456]: teaches hardware <read on CPU> that includes a matrix accelerator (e.g., tensor accelerator 2723) <read on data processing acceleration function> using matrix extensions (XMX) to upscale images <read on generate first and second temporary upscaling images>; Note: it should be noted that Intel incorporates the matrix accelerator in CPU hardware).
Liktor is analogous art with respect to the combination of Zhai, Damkat, and Subramaniam because they are from the same field of endeavor, namely upscaling image frames. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a matrix accelerator that utilizes matrix extensions for image upscaling as taught by Liktor into the combined teaching of Zhai, Damkat, and Subramaniam. The suggestion for doing so would allow the system to perform image upscaling on local hardware, thereby yielding predictable and desirable results. Therefore, it would have been obvious to combine Liktor with the combination of Zhai, Damkat, and Subramaniam.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Zhai et al. (US 20140072029 A1, previously cited), hereinafter referenced as Zhai, in view of Damkat (US 20110081094 A1) as applied to Claim 16 above respectively, and further in view of Kim et al. (US 20200126186 A1, previously cited), hereinafter referenced as Kim.
Regarding Claim 18, the combination of Zhai and Damkat discloses the control method of Claim 16. The combination of Zhai and Damkat does not expressly disclose the limitations of Claim 18; however, Kim discloses wherein the second scaler is
a bilinear scaler (Kim, [0184]: teaches scaler 630 and bypass scaler 660 increasing the resolution of an input image using a bilinear scaler <read on second scaler>), and wherein
the bilinear scaler converts an image into Full High Definition (FHD) image (Kim, [0169]: teaches obtaining DNN setting information for increasing the resolution of a second image by about two times, from 1280x720 to 1920x1080 <read on convert image to FHD image>).
Kim is analogous art with respect to Zhai, in view of Damkat because they are from the same field of endeavor, namely image upscaling hardware. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement a bilinear scaler to increase image resolutions using a deep neural network as taught by Kim into the teaching of Zhai, in view of Damkat. The suggestion for doing so would allow dynamic upscale resolutions on image frames that are dependent on a content update frequency. Therefore, it would have been obvious to combine Kim with Zhai, in view of Damkat.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lee et al. (US 20210125380 A1) discloses an AI encoder-decoder model for down-scaling and upscaling images;
Moon et al. (US 20140355892 A1) discloses searching for areas matching each other between a plurality of input images;
Ndiour et al. (US 20140010478 A1) discloses a pixel-adaptive interpolation algorithm for image upscaling;
Park et al. (US 20210295468 A1) discloses an AI upscaler and down-scaler that generates AI encoding data;
Reunanen (US 20230016641 A1) discloses processing a series of pathology images; and
Wang et al. (US 20240153033 A1) discloses an automatic content-dependent image processing algorithm selection.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KARL TRUONG whose telephone number is (703)756-5915. The examiner can normally be reached 10:30 AM - 7:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.D.T./Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614