DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claim
Currently, claims 1, 3-10, and 12-22 are pending.
Information Disclosure Statement
The information disclosure statements (IDS) were submitted on 09/08/2026. The submission are in compliance with the provisions of 37 CFR § 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments / Amendments
Applicant’s arguments have been fully considered, but they are not persuasive, see discussion below.
Rejections under 35 U.S.C. § 103:
The applicant argued that the combination of Kotra, Yuan, and Yea fails to disclose down-sampling a video unit of a video using a down-sampling method, where the down-sampling method includes discrete cosine transform interpolation filter (DCTIF) and the video unit is down-sampled using DCTIF, and where the down-sampling is performed prior to a super resolution (SR) process and prior to a conversion between the video comprising the video unit and a bitstream of the video
As to the above argument, Kotra discloses down-sampling a video unit of a video using a down-sampling method for a given picture is downsampled by a factor of 2x using the inbuilt Reference Picture Resampling (RPR) mechanism of VVC (Versatile Video Coding) Test Model (VTM11) ((Abstract, prior to encoding); and Bristol Visual Information - Deep Video Compression (BVI DVC) dataset [4] is used as the training dataset. BVI DVC dataset consists of a total of 200 sequences in 4 different spatial resolutions. Each sample in the training data is basically a tuple of (decoded, original (2x resolution)). The training data uses the BVI_DVC data downsampled by 2x using VTM downsampling filter and then encoded using VTM. Moreover, the QPs used to generate the training data were QP points 20, 25, 30 and 35 (Section 2.1).
In addition to Koreta, YUAN teaches discrete cosine transform interpolation filter (DCTIF) and the video unit is down-sampled using DCTIF for half-precision sub-pixel sample interpolation for the motion compensation enhancement performing direct transform on the integer pixel samples to cause the integer pixel samples to be into the DCT domain and down-sampling is performed on the processed block to obtain the at least one second matching block after the processed block is obtained ([0066],; [0075], [0080]).
Furthermore, YEA teaches performing a super resolution (SR) process and prior to a conversion between the video comprising the video unit and a bitstream of the video with a source frame is first down-scaled (or down-sampled) by a down-scaler (also referred to as a down-sampler) (1311) in a non-normative way and thus encoded at the low spatial resolution (e.g., 2 k x 2 k) and is encoded using an encoding tool (1312) . The filtered down-sampled picture is up-scaled (or up-sampled) by an up-scaler (1314) to generate an up-scaled (or up-sampled) picture having the high spatial resolution (e.g., 4 k x 2 k) and is used as a reference picture having the high spatial resolution (e.g., 4 k x 2 k) ([0146], FIG. 13).
It should be further noted that Applicant has not presented any specific arguments with regards to the rejections of the dependent claims.
Accordingly, Examiner maintains the rejection with regards to above arguments.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-10 & 12-22 are rejected under 35 U.S.C. 103 as being unpatentable over Kotra et al. (Neural Network-based Super Resolution Status, 20-28 April 2021, hereinafter Kotra) in view of YUAN et al. (US 20240098271 A1 hereinafter YUAN) and YEA et al. (US 20220201307 A1 hereinafter YEA).
Regarding Claim 1, Kotra discloses a method of processing video data, comprising:
down-sampling a video unit of a video using a down-sampling method (Abstract, prior to encoding, a given picture is downsampled by a factor of 2x using the inbuilt RPR mechanism of VTM11); and
performing the conversion based on the video unit as down-sampled (Section 2.1 BVI DVC dataset [4] is used as the training dataset. BVI DVC dataset consists of a total of 200 sequences in 4 different spatial resolutions. Each sample in the training data is basically a tuple of (decoded, original (2x resolution)). The training data uses the BVI_DVC data downsampled by 2x using VTM downsampling filter and then encoded using VTM. Moreover, the QPs used to generate the training data were QP points 20, 25, 30 and 35).
Kotra does not explicitly disclose wherein the down-sampling method includes discrete cosine transform interpolation filter (DCTIF) and the video unit is down-sampled using DCTIF, and wherein the down-sampling method is performed prior to a super resolution (SR) process and prior to a conversion between the video comprising the video unit and a bitstream of the video
YUAN teaches wherein the down-sampling method includes discrete cosine transform interpolation filter (DCTIF) and the video unit is down-sampled using DCTIF ([0066], using a discrete cosine transform interpolation filter (DCTIF) in video coding standards for half-precision sub-pixel sample interpolation for the motion compensation enhancement performing direct transform on the integer pixel samples to cause the integer pixel samples to be into the DCT domain; [0075], [0080], down-sampling is performed on the processed block to obtain the at least one second matching block after the processed block is obtained).
Therefore, it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of discrete cosine transform interpolation filter (DCTIF) as taught by YUAN ([0066]) into the encoding & decoding system of Kotra in order to ensure the same decoding quality, it can save the code rate, so as to improve the coding and coding efficiency and the coding efficiency is improved by eliminating video time redundancy, which is mainly applied to motion compensation and motion estimation portion of inter-frame prediction (YUAN, [0003]).
Kotra & Yuan do not explicitly disclose wherein the down-sampling method is performed prior to a super resolution (SR) process and prior to a conversion between the video comprising the video unit and a bitstream of the video
YEA teaches wherein the down-sampling method is performed prior to a super resolution (SR) process and prior to a conversion between the video comprising the video unit and a bitstream of the video ([0146], FIG. 13, a source frame is first down-scaled (or down-sampled) by a down-scaler (also referred to as a down-sampler) (1311) in a non-normative way and thus encoded at the low spatial resolution (e.g., 2 k x 2 k) and is encoded using an encoding tool (1312) . The filtered down-sampled picture is up-scaled (or up-sampled) by an up-scaler (1314) to generate an up-scaled (or up-sampled) picture having the high spatial resolution (e.g., 4 k x 2 k) and is used as a reference picture having the high spatial resolution (e.g., 4 k x 2 k)).
Therefore, it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of down-sampling method is performed prior to a super resolution (SR) process and prior to a conversion as taught by YEA ([0146]) into the encoding & decoding system of Kotra in order to provide video coding and decoding that reduces redundancy in the input video signal, through compression. The processing circuitry for video encoding down samples the first block to obtain a down-sampled block having the low spatial resolution and down-samples the reference block (YEA, [0017]).
Regarding Claim 5, Kotra in view of YUAN & YEA discloses the method of claim 1, YEA teaches wherein the down-sampling method is signaled in the bitstream from an encoder side to a decoder side ([0146], FIG. 13, a source frame is first down-scaled (or down-sampled) by a down-scaler (also referred to as a down-sampler) (1311)). The same reason or rational of obviousness motivation applied as used above in claim 1.
Regarding Claim 6, Kotra in view of YUAN & YEA discloses the method of claim 5, YEA teaches wherein the bitstream includes an index indicating the down-sampling method to use (([0146], FIG. 13, a source frame is first down-scaled (or down-sampled) by a down-scaler (also referred to as a down-sampler) (1311)). The same reason or rational of obviousness motivation applied as used above in claim 1.
Regarding Claim 7, Kotra in view of YUAN & YEA discloses the method of claim 5, YUAN & YEA teaches wherein the bitstream includes one or more coefficients of the down-sampling method, or wherein the bitstream includes information used to derive the one or more coefficients of the down-sampling method ([0066], using a discrete cosine transform interpolation filter (DCTIF) in video coding standards for half-precision sub-pixel sample interpolation for the motion compensation enhancement performing direct transform on the integer pixel samples to cause the integer pixel samples to be into the DCT domain; [0075], [0080], down-sampling is performed on the processed block to obtain the at least one second matching block after the processed block is obtained).
Regarding Claim 8, Kotra in view of YUAN & YEA discloses the method of claim 5, YEA teaches wherein the down-sampling method is included in one of a sequence header, a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, a slice header, a coding tree unit (CTU), a coding tree block (CTB), and a rectangular region of the video unit ([0151] Given an output frame dimension of W×H (corresponding to the high spatial resolution) and the up-sampling ratio d/8, the encoder and the decoder can calculate the low-resolution coded frame dimension as w×H where a reduced (or down-sampled) width w is (8 W+d/2)/d. An input to the linear upscaling process performed by the up-scaler (e.g., (1314) or (1324)) is a reduced frame of the low spatial resolution w×H and an output from the linear upscaling process performed by the up-scaler (e.g., (1314) or (1324)) is a frame with the high spatial resolution W×H as specified in a frame header (also referred to as a picture header)). The same reason or rational of obviousness motivation applied as used above in claim 5.
Regarding Claim 9, Kotra in view of YUAN & YEA discloses the method of claim 5, YUAN & YEA teaches wherein different down-sampling methods are signaled for different color components of the video unit which are down-sampled using different down-sampling methods ([ 0151] Given an output frame dimension of W×H (corresponding to the high spatial resolution) and the up-sampling ratio d/8, the encoder and the decoder can calculate the low-resolution coded frame dimension as w×H where a reduced (or down-sampled) width w is (8 W+d/2)/d. An input to the linear upscaling process performed by the up-scaler (e.g., (1314) or (1324)) is a reduced frame of the low spatial resolution w×H and an output from the linear upscaling process performed by the up-scaler (e.g., (1314) or (1324)) is a frame with the high spatial resolution W×H as specified in a frame header (also referred to as a picture header)). The same reason or rational of obviousness motivation applied as used above in claim 5.
Regarding Claim 10, Kotra in view of YUAN & YEA discloses the method of claim 1, YEA teaches wherein the down-sampling method is determined by a decoder side and communicated to an encoder side using an inter-active application ([0147] At the decoder side, a decoding tool (1322) is used to decode the filtered down-sampled picture that is received by the decoder.) The same reason or rational of obviousness motivation applied as used above in claim 1.
Regarding Claim 12, Kotra in view of YUAN & YEA discloses the method of claim 1, YEA discloses wherein the down-sampling ratio of the down-sampling method is two for all frames in one sequence; or wherein the down-sampling ratio of the down-sampling method is two for all CTUs of one frame; or wherein the down-sampling ratio of the down-sampling method is two for a first frame and is four for a next frame ([0150] When the super resolution coding mode is applied only in one direction (e.g., the horizontal direction or the vertical direction), the scaling-ratio can be refer to an up-sampling ratio or a corresponding down-sampling ratio. An up-sampling ratio can have nine possible values given as d/8). The same reason or rational of obviousness motivation applied as used above in claim 1.
Regarding Claim 13, Kotra in view of YUAN & YEA discloses the method of claim 1, YEA discloses wherein a combination of different down-sampling ratios for different types of the video unit is used, and wherein the down-sampling ratio is two for a frame and the down-sampling ratio is four for a coding tree unit (CTU) within the frame ([0150] When the super resolution coding mode is applied only in one direction (e.g., the horizontal direction or the vertical direction), the scaling-ratio can be refer to an up-sampling ratio or a corresponding down-sampling ratio. An up-sampling ratio can have nine possible values given as d/8). The same reason or rational of obviousness motivation applied as used above in claim 1.
Regarding Claim 14, Kotra in view of YUAN & YEA discloses the method of claim 1, Kotra discloses wherein the down-sampling comprises applying a first down-sampling ratio to the video unit to generate a first down-sampling result, applying a second down-sampling ratio to the video unit to generate a second down-sampling result, calculating a quality metric when each of the first down-sampling result and the second down-sampling result are up-sampled using a same up-sampling method, and determining the first down-sampling ratio or the second down-sampling ratio based on a comparison of the quality metric of the first down-sampling result and the second down-sampling result (Abstract, applying a Neural-Network based super-resolution used as upsampling filter in the context of VVC RPR and prior to encoding, a given picture is downsampled by a factor of 2x using the inbuilt RPR mechanism of VTM11. PSNR of the coded frame is computed by calculating the MSE between the original picture and the upsampled version of the decoded picture. The upsampled picture is generated by the Neural Network- based upsampling filter instead of the existing VTM up-sampling filter).
Regarding Claim 15, Kotra in view of YUAN & YEA discloses the method of claim 14, wherein the quality metric is Peak Signal to Noise Ratio (PSNR) or Multi-Scale Structural Similarity (MS-SSIM) (Abstract, applying a Neural-Network based super-resolution used as upsampling filter in the context of VVC RPR and prior to encoding, a given picture is downsampled by a factor of 2x using the inbuilt RPR mechanism of VTM11. PSNR of the coded frame is computed by calculating the MSE between the original picture and the upsampled version of the decoded picture. The upsampled picture is generated by the Neural Network- based upsampling filter instead of the existing VTM up-sampling filter).
Regarding Claim 16, Kotra in view of YUAN & YEA discloses the method of claim 14, Kotra discloses wherein the determination of the first down-sampling ratio and the second down-sampling ratio is performed at an encoder (Abstract, applying a Neural-Network based super-resolution used as upsampling filter in the context of VVC RPR and prior to encoding, a given picture is downsampled by a factor of 2x using the inbuilt RPR mechanism of VTM11. PSNR of the coded frame is computed by calculating the MSE between the original picture and the upsampled version of the decoded picture. The upsampled picture is generated by the Neural Network- based upsampling filter instead of the existing VTM up-sampling filter).
Regarding Claims 17 & 18, Analogous rejection as the rejection of Claim 1 applies.
Kotra discloses BVI DVC dataset [4] is used as the training dataset. BVI DVC dataset consists of a total of 200 sequences in 4 different spatial resolutions. Each sample in the training data is basically a tuple of (decoded, original (2x resolution)). The training data uses the BVI_DVC data downsampled by 2x using VTM downsampling filter and then encoded using VTM. Moreover, the QPs used to generate the training data were QP points 20, 25, 30 and 35 (Section 2.1).
Regarding Claim 19, Apparatus claim 19 of using the corresponding method claimed in claim 1, and the rejections of which are incorporated herein for the same reasons as used above.
Regarding Claim 20, Computer-readable claim 20 of using the corresponding method claimed in claim 1, and the rejections of which are incorporated herein for the same reasons as used above.
Regarding Claims 21& 22, Analogous rejection as the rejection of Claim 12 & 13 applies.
Claims 3-4, are rejected under 35 U.S.C. 103 as being unpatentable over Kotra et al. (Neural Network-based Super Resolution Status, 20-28 April 2021, hereinafter Kotra) in view of YUAN, YEA & Tuzel et al. (US 20170256033, hereinafter Tuzel).
Regarding Claim 3, Kotra in view of YUAN & YEA discloses the method of claim 1 but does not explicitly disclose wherein the down-sampling method includes a bilinear interpolation method, and the video unit is down-sampled using bilinear interpolation.
Tuzel teaches wherein the down-sampling method includes a bilinear interpolation method, and the video unit is down-sampled using bilinear interpolation ([0003] nearest neighbor interpolation, bilinear interpolation and bicubic interpolation. Interpolation based image super-resolution produces smoothed images where details are of the image are lost or have inadequate quality. To obtain sharp high-resolution images, some methods used image sharpening filters such as bilateral filtering after the interpolation; [0004] machine learning techniques to learn the parameters of the image super-resolution methods with local constraints are enforced as priors based on image statistics and exemplar patches. Global constraints are typically not available for the generic SR problem, which limits the plausible upsampling factor)
Therefore, it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of down-sampling method includes a bilinear interpolation as taught by Tuzel ([0003]) into the encoding & decoding system of Kotra in order to provide increased an accuracy of upsampling, while preserving an efficiency of a feed-forward processing by jointly modeling and learning global and local constraints using a deep architecture and to enable training a fully connected network, an interpolation, and a convolution in a concurrent manner to reduce an error between upsampled set of images and corresponding set of high-resolution images (Tuzel, [0027]).
Regarding Claim 4, Kotra in view of YUAN & YEA discloses the method of claim 1 but does not explicitly disclose wherein the down-sampling method includes a bicubic interpolation method, and the video unit is down-sampled using bicubic interpolation.
Tuzel teaches wherein the down-sampling method includes a bicubic interpolation method, and the video unit is down-sampled using bicubic interpolation ([0003] nearest neighbor interpolation, bilinear interpolation and bicubic interpolation. Interpolation based image super-resolution produces smoothed images where details are of the image are lost or have inadequate quality. To obtain sharp high-resolution images, some methods used image sharpening filters such as bilateral filtering after the interpolation).
Therefore, it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of down-sampling method includes a bilinear interpolation as taught by Tuzel ([0003]) into the encoding & decoding system of Kotra in order to provide increased an accuracy of upsampling, while preserving an efficiency of a feed-forward processing by jointly modeling and learning global and local constraints using a deep architecture and to enable training a fully connected network, an interpolation, and a convolution in a concurrent manner to reduce an error between upsampled set of images and corresponding set of high-resolution images (Tuzel, [0027]).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Samuel D Fereja whose telephone number is (469)295-9243. The examiner can normally be reached 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DAVID CZEKAJ can be reached at (571) 272-7327. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAMUEL D FEREJA/Primary Examiner, Art Unit 2487
Prosecution Insights