DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 9-20 are rejected under 35 U.S.C. 103 as being unpatentable over Deshpande et al., hereinafter referred to as Deshpande (US 2024/0205462 A1) in view of Hannuksela et al., hereinafter referred to as Hannuksela (US 2023/0112309 A1).
As per claim 1, Deshpande discloses a method for video processing, comprising:
performing a conversion between a video and a bitstream of the video (Deshpande: Para. [0159] discloses “generate a compliant bitstream [claimed performing a conversion between a video and a bitstream of the video] representing the video data”), wherein at least one neural-network post-processing filter (NNPF) is applied on at least one video unit associated with the video (Deshpande: Para. [0113] discloses “specifies a neural network that may be used as a post-processing filter [claimed at least one neural-network post-processing filter (NNPF) is applied]” and Para. [0146] discloses “post-processing filtering for the current picture [claimed on at least one video unit associated with the video]”), and the bitstream comprises a first indication indicating whether quality information (Deshpande: Para. [0125] discloses “nnpfc_formatting_and_purpose_flag [claimed first indication] equal to 1 specifies that syntax elements related to the filter purpose … are present [claimed indicating whether quality information […] is present in the bitstream]”).
However, Deshpande does not explicitly disclose “… quality information of the at least one video unit associated with the applying of the at least one NNPF …”.
Further, Hannuksela is in the same field of endeavor and teaches “quality information of the at least one video unit associated with the applying of the at least one NNPF” (Hannuksela: Para. [0405] discloses “specifies the expected gain [claimed quality information] that can be achieved if the neural network signalled in filter_sei_message()is applied [claimed of the at least one video unit associated with the applying of the at least one NNPF]”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, and having the teachings of Deshpande and Hannuksela before him or her, to modify the encoding decoding neural network post-filter system of Deshpande to include the quality information feature as described in Hannuksela. The motivation for doing so would have been to improve decoder side application to decide whether or not to apply the neural network based on the expected gain and available resources.
As per claim 2, Deshpande-Hannuksela disclose the method of claim 1, wherein the quality information comprises at least one of the following: one or more metric types for measuring quality of the at least one video unit, one or more quality changes of the at least one video unit that correspond to the one or more metric types, each of the one or more quality changes being caused by the applying of the at least one NNPF and measured based on the corresponding metric type, or one or more quality values of the at least one video unit after the at least one NNPF being applied, the one or more quality values corresponding to the one or more metric types, and each of the one or more quality values being measured based on the corresponding metric type (Hannuksela: Para. [0405] discloses “gain is expressed with respect to one or more predefined metrics” and Hannuksela: Para. [0407] discloses “gain may be expressed as expected PSNR improvement” [claimed one or more metric types for measuring quality of the at least one video unit]”).
As per claim 3, Deshpande-Hannuksela disclose the method of claim 1, wherein the first indication indicates that the quality information is present in the bitstream (Deshpande: Para. [0125] discloses “nnpfc_formatting_and_purpose_flag equal to 1 specifies that syntax elements related to the filter purpose … are present [claimed first indication indicates that the quality information is present in the bitstream]”), and at least one indication indicating the quality information is present in the bitstream (Hannuksela: Para. [0409] discloses “the gain according to any example above may be contained in one or more syntax elements [claimed at least one indication indicating the quality information is present in the bitstream] within a purpose_sei_message()”).
As per claim 4, Deshpande-Hannuksela disclose the method of claim 1, wherein the at least one video unit comprises one of the following:
a whole video sequence of the video,
a single picture of the video, or
a plurality of pictures of the video (Deshpande: Para. [0146] discloses “for the current picture [claimed a single picture of the video]”).
As per claim 5, Deshpande-Hannuksela disclose the method of claim 2, wherein the one or more metric types comprise at least one of the following: peak signal-to-noise ratio (PSNR), structural similarity (SSIM), multi scale structural similarity (MS-SSIM), or
video multi-method assessment fusion (VMAF), or wherein one of the one or more quality changes is determined based on one of the following: a difference between quality of the at least one video unit after the at least one NNPF being applied and quality of the at least one video unit without the at least one NNPF being applied, or a difference between quality of the at least one video unit without the at least one NNPF being applied and quality of the at least one video unit after the at least one NNPF being applied (Hannuksela: Para. [0407] discloses “PSNR improvement [claimed peak signal-to-noise ratio (PSNR)], or MS-SSIM improvement [claimed multi scale structural similarity (MS-SSIM)].”.).
As per claim 9, Deshpande-Hannuksela disclose the method of claim 1, wherein the first indication indicates that the quality information is not present in the bitstream (Deshpande: Para. [0125] discloses “nnpfc_formatting_and_purpose_flag equal to 0 specifies that no syntax elements related to the filter purpose … are present [claimed first indication indicates that the quality information is not present in the bitstream]”), and at least one indication indicating the quality information is not present in the bitstream (Deshpande: Para. [0125] discloses “nnpfc_formatting_and_purpose_flag equal to 0 specifies that no syntax elements related to the filter purpose … are present [claimed at least one indication indicating the quality information is not present in the bitstream]”; [therefore, when the formatting and purpose flag is 0, the syntax elements indicating the gain and quality information are entirely omitted from the bitstream]).
As per claim 10, Deshpande-Hannuksela disclose the method of claim 1, wherein at least one indication indicating the quality information is present in the bitstream independently from the first indication (Hannuksela: Para. [0404] discloses “sei_container {purpose_sei_message(); filter_sei_message(); gain_sei_message();}” and Para. [0405] discloses “contain also gain_sei_message() … which is an SEI message that specifies the expected gain [claimed at least one indication indicating the quality information is present in the bitstream independently from the first indication]”; [therefore, the gain SEI message is a distinct, independent message from the purpose SEI message where the primary formatting flag resides]).
As per claim 11, Deshpande-Hannuksela disclose the method of claim 2, wherein the at least one indication is comprised in a video message unit in the bitstream (Deshpande: Para. [0112] discloses “a Neural-network post-filter characteristics SEI message [claimed video message unit in the bitstream]”).
As per claim 12, Deshpande-Hannuksela disclose the method of claim 11, wherein the video message unit is a supplemental enhancement information (SEI) message (Deshpande: Para. [0112] discloses “a Neural-network post-filter characteristics SEI message [claimed supplemental enhancement information (SEI) message]”).
As per claim 13, Deshpande-Hannuksela disclose the method of claim 12, wherein the SEI message is a neural-network post-filter activation (NNPFA) SEI message (Deshpande: Para. [0142] discloses “Neural-network post-filter activation SEI message [claimed neural-network post-filter activation (NNPFA) SEI message]”).
As per claim 14, Deshpande-Hannuksela disclose the method of claim 1, wherein the bitstream further comprises at least one first syntax element indicating a type of the at least one video unit, and one of candidates for the type is a first region type different from a picture, a slice and a coding tree unit, or
wherein a syntax element nnpfa_picture_enabling_flag[i] in an NNPFA SEI message in the bitstream indicates a usage of NNPF at a picture level, or
wherein a syntax element nnpfa_picture_index[i] in an NNPFA SEI message in the bitstream indicates a usage of NNPF at a picture level, or
wherein a syntax element nnpfa_slice_enabling_flag[i] in an NNPFA SEI message in the bitstream indicates a usage of NNPF at a slice level, and only one NNPF is indicated by a syntax element nnpfa_id in the NNPFA SEI message, or
wherein a syntax element nnpfa_slice_index[i] in an NNPFA SEI message in the bitstream indicates a usage of NNPF at a slice level, and one or more NNPFs are indicated by a syntax element nnpfa_id[i] in the NNPFA SEI message, or
wherein a syntax element nnpfa_ctu_enabling_flag[i] in an NNPFA SEI message in the bitstream indicates a usage of NNPF at a coding tree unit (CTU) level, and only one NNPF is indicated by a syntax element nnpfa_id in the NNPFA SEI message, or
wherein a syntax element nnpfa_ctu_index[i] in an NNPFA SEI message in the bitstream indicates a usage of NNPF at a CTU level, and one or more NNPFs are indicated by a syntax element nnpfa_id[i] in the NNPFA SEI message (Deshpande: Para. [0037] discloses “The use of specified post-processing filters for specific pictures is indicated with neural-network post-filter activation SEI messages. [claimed a syntax element nnpfa_picture_enabling_flag[i] in an NNPFA SEI message in the bitstream indicates a usage of NNPF at a picture level]”.).
As per claim 15, Deshpande-Hannuksela disclose the method of claim 1, wherein a syntax element indicating whether NNPF is used in current region is required to be smaller than an eighth value, the eight value being determined based on the number of NNPFs, or
wherein the bitstream comprises a syntax element nnpfa_picture_enabling_flag indicating whether NNPF is used at a picture level, or
wherein the bitstream comprises a syntax element nnpfa_picture_index indicating whether NNPF is used at a picture level, or
wherein a syntax element nnpfa_slice_enabling_flag[i] in the bitstream equal to a first value indicates that NNPF is used for a sub-region of the i-th slice of a current picture, and i is an integer, or
wherein the bitstream comprises a syntax element nnpfa_slice_index[i] indicating whether NNPF is used at a slice level (Deshpande: Para. [0037] discloses “The use of specified post-processing filters for specific pictures is indicated with neural-network post-filter activation SEI messages. [claimed a syntax element nnpfa_picture_enabling_flag[i] in an NNPFA SEI message in the bitstream indicates a usage of NNPF at a picture level]”.).
As per claim 16, Deshpande-Hannuksela disclose the method of claim 1, wherein the conversion includes encoding the video into the bitstream (Deshpande: Para. [0159] discloses “Video encoder 106 may include any device configured to receive video data and generate a compliant bitstream representing the video data [claimed encoding the video into the bitstream]”).
As per claim 17, Deshpande-Hannuksela disclose the method of claim 1, wherein the conversion includes decoding the video from the bitstream (Deshpande: Para. [0193] discloses “Video decoder 124 may include any device configured to receive a bitstream … and reproduce video data therefrom [claimed decoding the video from the bitstream]”).
As per claims 18-20, the claim(s) recites analogous limitations to claim(s) * above, and is/are therefore rejected on the same premise.
Claims 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Deshpande in view of Hannuksela in further view of He et al., hereinafter referred to as He (US 2022/0321917 A1).
As per claim 6, Deshpande-Hannuksela disclose the method of claim 2, wherein the one or more metric types comprise a plurality of metric types,
However, Deshpande-Hannuksela do not explicitly disclose the one or more quality changes comprise a plurality of quality changes.
Furthermore, He is in the same field of endeavor and teaches the one or more quality changes comprise a plurality of quality changes (He: Paras. [0125]-[0126] disclose “pqm_type[i] indicates the i-th quality metric type … Table 7 … 0 PSNR 1 wPSNR 2 WS-PSNR 3 PSNRsequence [plurality of metric types]” and He: Para. [0127] discloses “pqm_value[i] specifies the value of the i-th quality metric [claimed plurality of quality changes]”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, and having the teachings of Deshpande-Hannuksela, and He before him or her, to modify the encoding decoding neural network post-filter system of Deshpande-Hannuksela to include the quality changes feature as described in He. The motivation for doing so would have been to improve coding efficiency by providing multiple options simultaneously to better evaluate the picture quality and coding performance across different dimensions.
As per claim 7, Deshpande-Hannuksela disclose the method of claim 6, wherein the bitstream comprises a second indication indicating the number of the plurality of metric types (He: Para. [0124] discloses “pqm_cnt_minus1 plus 1 [claimed second indication] specifies the number of luma component quality metrics [claimed indicating the number of the plurality of metric types]”).
As per claim 8, Deshpande-Hannuksela disclose the method of claim 7, wherein at least one indication indicating the quality information follows the second indication in the bitstream (He: Paras. [0122]-[0123] (Table 6) disclose “pqm_cnt_minus1 u(8) for( i = 0; i <= pqm_cnt_minus1; i++ ) { pqm_type[i] u(8) pqm_value[i] u(16) } [claimed at least one indication indicating the quality information follows the second indication in the bitstream]”; [therefore, the loop syntax structure containing the metric types and values physically follows the count indication syntax element in the bitstream]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure and can be viewed in the list of references.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEET DHILLON whose telephone number is (571)270-5647. The examiner can normally be reached M-F: 5am-1:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath V. Perungavoor can be reached at 571-272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PEET DHILLON/Primary Examiner
Art Unit: 2488
Date: 09-14-2026
Prosecution Insights