DETAILED ACTION
Response to Amendment
Claims 1-2, 5, 7-10, 12, 14, and 16-20 were previously pending. Applicant’s amendment filed July 22, 2026, has been entered in full. Claims 1and 16 are amended. Claims 10 is cancelled. No new claims are added. Accordingly, claims 1-2, 5, 7-9, 12, 14, and 16-20 are now pending.
Response to Arguments
Applicant argues that amendments to the claims have overcome the previous rejections under 35 U.S.C. § 112 (Remarks filed July 22, 2026, hereinafter Remarks: Page 6). Examiner agrees. The previous rejections under 35 U.S.C. § 112 are withdrawn.
Applicant traverses the previous rejections under 35 U.S.C. § 102 and § 103, arguing that the previously cited Zhang reference does not disclose all elements of the amended claims (Remarks: Pages 6-8). Examiner respectfully disagrees.
Applicant notes an amendment to claim 1 to explicitly recite “the second image upscaling process being different from the first image upscaling process” (Remarks: Page 6). Applicant at least implicitly argues that Examiner’s previous broadest reasonable interpretation (BRI) of “first” and “second” should be withdrawn in view of this amendment and Examiner agrees – the previous claim interpretation is withdrawn. Applicant argues that this added limitation is supported in the specification based on Fig. 3 and pars. [0094]-[0106] of the specification (Remarks: Pages 6-7). Examiner notes that using different upscaling processes on different image portions entered into a cache, as is contemplated in the specification, does not necessarily mean that a second group of image portions retrieved from the cache was generated using an upscaling process different from the one applied to a first group of image portions in a current image. Nevertheless, given that the cache includes multiple image portions processed with multiple different upscaling processes, the preponderance of the evidence suggests that one of ordinary skill in the art would have recognized the possibility that a second group of image portions retrieved from the cache would have a different upscaling process that what is applied to a first group of image portions, such that the amended claims have adequate written description under 35 U.S.C. § 112(a).
Applicant further argues that Zhang does not teach the different upscaling process recited in the amended claims (Remarks: Pages 7-8). Examiner respectfully disagrees. Zhang describes at least three different upscaling processes that may be used to upscale each individual block of a given frame: motion compensation prediction, bicubic upsampling, and super resolution (SR) (e.g., [0055] including table; [0031]). The blocks that have been upscaled in different ways for a current frame are then saved for processing the next frame and may be adaptively transferred into that next frame. Note that while Zhang’s explanation only considers two adjacent frames for simplicity ([0031]), Zhang does contemplate transferring to further frames, specifically stating that its techniques may “produce an enhanced third frame (not shown) from the enhanced second video frame” ([0028]). At least in such a case, where blocks are being transferred from the enhanced second video frame to the enhanced third frame, the blocks from the second frame (i.e., the image portions retrieved from a cache) are not all necessarily generated using the computationally-intensive super resolution. Some also may be generated using motion compensation prediction or bicubic upsampling (e.g., [0055] including table), such that a second group of blocks from the second frame (i.e., a second group of one or more image portions retrieved from a cache) has been upscaled using a different upscaling process than a first/SR upscaling process, as required by the claimed invention.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on July 30, 2026, is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 16 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Claim 16 states that “one or more of the plurality of image portions are received from the server device.” Claim 16 depends from claim 1, which already requires “receiving, from a server device and by a client device, and image … wherein the image comprises a plurality of image portions.” Thus, claim 1 already requires that one or more of the plurality of image portions (which are comprised in the image) are received from the server device and claim 16 does not further limit its subject matter.
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-2, 5, 7, 14, and 16-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by ‘Zhang’ (US 2017/0339431 A1; previously cited).
Regarding claim 1, Zhang discloses a computer-implemented (e.g., Figure 5) method (e.g., Figs. 1-2), the method comprising:
receiving, from a server device and by a client device (e.g., Fig. 1, compressed bit stream 110 is received; The device from which it is received is a server device and the device that receives it is the client device), an image which is one of a plurality of images forming an image stream (e.g., Fig. 1, bitstream 110 encodes a plurality of frames 130, including a first frame 131 and a second frame 132), wherein the image comprises a plurality of image portions (e.g., [0025], [0056], Fig. 1, each frame image is divided into plural blocks, each block being an image portion);
receiving, from the server device and by the client device, metadata relating to processing of the image (e.g., [0024]-[0025], Fig. 1, syntax elements 135), wherein the metadata indicates, for each of the plurality of image portions, that upscaling is foregone for the image portion or one of a plurality of available image upscaling processes to apply to the image portion (e.g., [0055] including Table 1, the syntax elements for each block indicate what type of block it is, which in turn indicates whether upscaling is foregone [for Inter zero residual blocks] or which of a plurality of available upscaling processes to apply to the block [SR algorithm is applied to Anchored-type blocks, while bicubic upsampling is applied to Intra-type blocks, for example]);
determining a first group of one or more image portions from the plurality of image portions to apply a first image upscaling process to from the plurality of available image upscaling processes based on the metadata for the first group of one or more image portions (e.g., [0055] including Table 1, determination based on metadata/syntax elements of which blocks are Anchored-type blocks, all of which will have an SR algorithm applied as a first image upscaling process);
selecting the first group of one or more image portions based on the determination (e.g., [0055] including Table 1, Anchored blocks are selected for SR algorithm);
applying the first image upscaling process to the first group of one or more image portions (e.g., [0055] including Table 1, SR algorithm is applied to the Anchored blocks);
retrieving, from a cache, a second group of one or more image portions (e.g., [0025]-[0027], Fig. 1, second blocks that were previously upscaled for a prior frame are retrieved for insertion into a current frame; The plain meaning of a cache is a place where data is stored for later use and this plain meaning is not inconsistent with the specification; Wherever Zhang stores upscaled blocks from a prior frame so that they can be retrieved and inserted into a later frame falls within the BRI of a cache; See, e.g., [0062], Fig. 5, memory 506) to which a second image upscaling process of the plurality of available image upscaling processes was previously applied, the second image upscaling process being different from the first image upscaling process (e.g., [0031], [0055] (including table), blocks in an image may be upscaled using at least three different processes: super resolution (SR), bicubic upsampling, or motion compensation prediction; Note from [0028], that blocks may be transferred from second frame to a third frame – i.e., the previous frame may include blocks upscaled using any of the available upscaling processes, not just SR; As noted above, the “first image upscaling process” is SR and is applied to anchored blocks; Any of the other block types transferred to the current frame have been upscaled using a different upscaling process); and
generating an upscaled version of the image based at least in part on combining the first group of one or more image portions to which the first image upscaling process was applied and the second group of one or more image portions (e.g., [0027], Fig. 1, SR upscaling is applied to a portion of frame 170, while other portions are transferred from a previous frame – i.e., the first and second groups are combined to generate an upscaled version of the image).
Regarding claim 2, Zhang discloses the computer-implemented method of claim 1, wherein the first image upscaling process comprises a neural network based super resolution model (e.g., [0026], [0034], [0004], computationally-intensive super-resolution (SR) algorithm may be neural network based).
Regarding claim 5, Zhang discloses the computer-implemented method of claim 1, wherein the metadata comprises a tile map comprising the plurality of image portions (e.g., [0024]-[0025], Fig. 1, syntax element metadata maps locations of the block image portions), the tile map indicating which image portions to be selected (e.g., [0055] including table 1, syntax elements for each block indicate what type of block it is, which indicates which blocks are to be selected for SR algorithm upscaling).
Regarding claim 7, Zhang discloses the computer-implemented method of claim 1, wherein additional metadata is sent from the server device for each image of the plurality of images which form the image stream (e.g., [0024]-[0025], Fig. 1, syntax element metadata is received for each of plural images 130).
Regarding claim 14, Zhang discloses the computer-implemented method of claim 1, further comprising
selecting a third group of one or more image portions (e.g., [0055] including Table 1, Intra and/or Inter-predicted blocks with zero motion vector are selected as a third group of blocks); and
applying a third image upscaling process to the third group of one or more image portions (e.g., [0055] including Table 1, bicubic upsampling is applied to the “third” blocks), wherein the third image upscaling process is less computationally demanding than the first image upscaling process (e.g., [0034]).
Regarding claim 16, Zhang discloses the computer-implemented method of claim 1, wherein one or more of the plurality of image portions are received from the server device (See, e.g., mapping in claim 1 and Fig. 1, encoded bitstream 110 that includes the image portions is received from a server).
Regarding claim 17, Zhang discloses the method of claim 1, wherein a resolution of the image subsequent to the applying of the first image upscaling process to the first group of one or more image portions is 3840 x 2160 pixels or above (e.g., [0030], “3840x2160 (4K)”).
Regarding claim 18, Zhang discloses the computer-implemented method of claim 1, further comprising:
determining a fourth group of one or more image portions of the image to be used in a subsequent image in the image stream comprising at least the image and the subsequent image (e.g., [0028], further transfer processing into third frame, etc.), the method comprising:
calculating, for each image portion of the plurality of image portions, an average pixel intensity difference between the image and the subsequent image (e.g., [0040], “mean absolute magnitude of the residual block”; Note that the residual block indicates intensity difference between subsequent image block/portion and prediction from current image block/portion);
if the average pixel intensity difference of one or more of the plurality of image portions is below a predetermined threshold (e.g., [0040], less than
η
), adding the one or more image portions to the fourth group of one or more image portions and storing a location of the fourth group of one or more image portions (e.g., [0040], “If the magnitude of the residual is below the residual thresholdr [sic]
η
, a transfer is performed”; Note from [0019] that transferring a block means replicating/copying it from a first frame to a second frame); and
using the fourth group of one or more image portions in the subsequent image (e.g., [0027], Fig. 1).
Regarding claim 19, Zhang discloses a client computing device (e.g., Fig. 5) comprising one or more processors (Fig. 5, processor 502) that are associated with a memory (Fig. 5, memory 506), the one or more processors configured with executable instructions which, when executed, cause the client computing device to carry out the computer-implemented method of claim 1 (e.g., [0059]-[0060], [0064]-[0065]).
Regarding claim 20, Zhang discloses a system (e.g., Fig. 5) comprising,
a memory (Fig. 5, memory 506); and
one or more processors configured to perform the method of claim 1 (e.g., [0058]-[0060], [0064]-[0065]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang and further in view of ‘Ozer’ (“Beginners’ Guide to Adaptive Bitrate Streaming,” 10 May 2021; previously cited).
Regarding claim 8, Zhang teaches the computer-implemented method of claim 5.
Zhang teaches (e.g., [0024], Fig. 1, bitstream 110 of image data and syntax elements) video streaming, but does not explicitly teach receiving a manifest file from the server device.
However, Ozer does teach techniques for video streaming that include receiving, from a server device, a manifest file (e.g., page 5, master playlist) comprising information relating to the interpretation of metadata at the client device (e.g., page 5, includes information relating to interpretation of specific media metadata at the client device, such as metadata pertaining to specific rungs on the encoding ladder), wherein the manifest file comprises one or more of:
i) a number of columns in a tile grid;
ii) a number of rows in a tile grid;
iii) a library each index of the plurality of indices and their related image upscaling process of the plurality of available image upscaling processes;
iv) an indication whether the server device supports embedding the tile map for each image in the plurality of images forming the image stream;
v) an indication of availability of an additional stream of data comprising full resolution data (e.g., Page 5, Fig. 4, top rung of encoding ladder is an indication of availability of an additional stream of data comprising full (i.e., highest) resolution data); and/or
vi) instructions for compositing the tiles in the tile grid to generate the image.
Ozer teaches that manifest files allow clients to access adaptive bitrate (ABR) streaming (e.g., page 4), which advantageously delivers optimum viewing experience for a range of client devices and connection speeds (e.g., page 2, 1st paragraph).
Ozer also demonstrates that manifest files can be used to communicate information about video coding from a server to a client (e.g., page 6). Yang requires sharing such information between client and server, such as a library matching each of different indices to a corresponding image upscaling process (e.g., [0353]-[0354]; [0363], correspondence between identifier and matching upscaling DNN settings needs to be provided to the client so that it can select appropriate upscaling/second DNN settings).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the method of Zhang as applied above with the manifest file of Ozer in order to improve the method with the reasonable expectation that this would result in a method that allowed clients to access ABR streaming, thereby advantageously allowing delivery of optimum viewing experiences for a range of client devices and connection speeds, and/or allowed clients to receive necessary information about video coding. This technique for improving the method of Zhang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Ozer.
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Ozer to obtain the invention as specified in claim 8.
Claim(s) 9 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of ‘Wang’ (US 2024/0153033 A1; previously cited).
Regarding claim 9, Zhang teaches the method of claim 1.
Zhang selects groups of image portions to subject to computationally-expensive super resolution or computationally-inexpensive upsampling depending on various metadata (e.g., [0055]). Nevertheless, Zhang does not explicitly teach determining a third group of portions based on the metadata further comprising metadata indicative of image portions containing one or more salient regions of the image, wherein a salient region of the image comprises one or more image portions which have a saliency value above a predetermined threshold.
However, Wang does teach techniques for selecting groups of image portions to subject to computationally-expensive super resolution or computationally-inexpensive upsampling depending on various metadata (e.g., Fig. 1), including determining a third group of portions (i.e., salient image blocks that warrant using a neural-network-based super resolution algorithm – [0031]) based on the metadata further comprising metadata indicative of image portions containing one or more salient regions of the image (e.g., [0031], Fig. 1, object detection results metadata are communicated), wherein a salient region of the image comprises one or more image portions which have a saliency value above a predetermined threshold (e.g., [0031], object recognition is applied “to find objects in the image that are sufficiently important to warrant using a NN algorithm on a section with an important object”).
The purpose of Zhang’s FAST technique is to accelerate video upscaling by applying super resolution (SR) to only a subset of video data (e.g., [0019]). Wang’s teachings indicate that video SR can be further accelerated by applying computation-intensive, neural-network-based SR only to important, salient regions (e.g., [0039], [0026], [0029]).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the method of Zhang with the saliency-based portion selection of Wang in order to improve the method with the reasonable expectation that this would result in a method that could further accelerate video upscaling. This technique for improving the method of Zhang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Wang.
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Wang to obtain the invention as specified in claim 9.
Regarding claim 12, Zhang teaches the method of claim 1.
Zhang selects groups of image portions to subject to computationally-expensive super resolution or computationally-inexpensive upsampling depending on various metadata (e.g., [0055]). While Zhang recognizes that high-performance upscaling may be too computationally-expensive for some client devices (e.g., [0004]), Zhang generally tunes its thresholds to maximize image quality (e.g., [0041], maximizing PSNR).
Zhang does not explicitly teach applying a local calibration test on the client device to determine a calibration score to determine an upper limit on a number of image portions to which the first image upscaling process can be applied.
However, Wang does teach applying a local calibration test on the client device to determine a calibration score to determine an upper limit on a number of image portions to which the first image upscaling process can be applied (e.g., [0038], application developer, device manufacturer, etc., can adjust threshold to make an appropriate trade-off between performance gain and computation cost; e.g., [0037], the threshold controls and determines an upper limit on the number of image portions to which the first/NN image upscaling process can be applied – i.e., the lower the threshold, the fewer the portions to which first/NN upscaling will be applied, and vice versa; For at least this reason, the threshold can be seen as a calibration score and the process of setting it can be seen as a calibration test).
Wang’s techniques advantageously consider the tradeoff between image quality and computational expense and allow an adjustment of the number of regions of interest in accordance with a given client device’s capability. Considering this tradeoff is advantageous because user experience is affected by more than just image quality. For example, even if given very high-quality video, a user’s experience may be relatively poorer if that video freezes and buffers due to a client device’s computational capability being exceeded.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the method of Zhang with the calibration test of Wang in order to improve the method with the reasonable expectation that this would result in a method that advantageously considered the tradeoff between image quality and computational expense. This technique for improving the method of Zhang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Wang.
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Wang to obtain the invention as specified in claim 12.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEOFFREY E SUMMERS whose telephone number is (571)272-9915. The examiner can normally be reached Monday-Friday, 7:00 AM to 3:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GEOFFREY E SUMMERS/Examiner, Art Unit 2669
Prosecution Insights