DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 23 July 2026 have been fully considered but they are not persuasive.
On pages 20 – 21, applicant argues that Karczewicz does not teach “forming a weighted sum of the filter results of the first set of classes according to the classification values” as claimed because Karczewicz teaches obtaining weights that are either fixed or transmitted in the bitstream and not determined by classification. While applicant’s arguments are understood, examiner respectfully disagrees. Examiner relies on Karczewicz in maintaining the rejection.
Under MPEP 2123, a reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. At present, the teachings of Karczwicz reasonably suggest to a person of ordinary skill in the art “forming a weighted sum of the filter results of the first set of classes according to the classification values” as claimed.
Karczewicz first teaches that the system forms a weighted sum of filter results of the first set of classes according to classification values. See, e.g. pars. 207 – 211: describing that the system forms a weighted sum of filtering results. The weights used in the weighted sum are based on filter usage. See, Karczewicz, e.g. par. 211: describing that weights used in determining the weighted sum are based on filter usage information. Specifically, each filter is associated with a weight, the weight being applied based on whether the associated filter is used. See, Karczewicz, e.g. par. 207 – 212: describing that each weight is indexed, the indexing being identical to the filter index [Nf], a weight being used based on whether the filter associated with each weight is used. Karczewicz further teaches that each filter is associated with a classification. See, e.g. Fig. 9, and pars. 113 – 150: depicting and describing that the system determines a classification for each sample and filters the sample using a filter associated with each classification. Because each filter is associated with a classification and because each filter has an associated weight, Karczewicz reasonably suggests to a person of ordinary skill in the art that each weight is associated with a classification. Karczewicz, therefore, teaches “forming a weighted sum of the filter results of the first set of classes according to the classification values” as claimed. The rejection, therefore, is maintained.
Examiner Remarks
Claims are interpreted in the alternative only.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 6, 8, 40 – 49, 52, 54 – 56, 58, 116, 117, and 119 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ye et al. (US 2010/0158103) (hereinafter Ye) in view of Karczewicz et al. (US 2022/0201292)(hereinafter Karczewicz) in view of Ma et al. (WO 2022/072659) (hereinafter Ma).
Regarding claims 1, 58, 116, 117, and 119, Ye teaches an apparatus for decoding a video from a bitstream, the decoding apparatus configured to perform a decoding method, an apparatus for encoding a video into a bitstream, the encoding apparatus configured to perform an encoding method, the method for decoding a video bitstream, the method for encoding bitstream, and a method for storing a video, wherein the decoding method and the encoding method comprise:
reconstruct, based on the bitstream, the video using block-based predictive decoding, transform-based residual decoding and a prediction loop into which an in-loop filter tool is serially connected (e.g. Fig. 3 and pars. 65 – 67: depicting and describing that the system reconstructs, from encoded video, the video using predictive decoding [element 54], transform-based residual decoding [elements 56 and 58], and an in-loop filter serially connected to the prediction loop [e.g. Fig. 3, depicting a prediction loop [element 54] being serially connected to in loop filter [element 67]),
wherein the second in-loop filter is configured to subject pre-reconstructed samples of a current picture to an adaptive in-loop filtering, ALF (e.g. par. 73: describing that the second filter is an adaptive loop filter),
wherein the second in-loop filter is configured to perform, based on the bitstream, a mode switching one or more first modes of performing the adaptive in-loop filtering, and optionally, a second mode of bypassing the second in-loop filter (e.g. par. 73: describing that the system determines whether or not to apply adaptive loop filtering to a block of the bitstream).
Ye does not explicitly teach:
wherein the in-loop filter tool comprises a serial connection of a first in-loop filter and a second in-loop filter,
wherein the one or more first modes involve the second in-loop filter assigning a classification to pre-reconstructed samples of the current picture and filtering the pre-reconstructed samples with a filter transfer function which is adapted to the classification,
wherein the classification of the one or more first modes is a soft-classification,
wherein each of the first modes uses a convolutional neural network, CNN,
wherein the one or more first modes involve the second in-loop filter assigning a classification to pre-reconstructed samples of the current picture and filtering the pre-reconstructed samples with a filter transfer function which is adapted to the classification, the classification of the one or more first modes is a soft-classification,
wherein the second in-loop filter is configured to perform the soft classification for first pre-reconstructed samples by:
assigning, for each first pre-reconstructed sample, a classification value to each of a first set of classes, with each of which an associated FIR filter is associated, and
performing the adaptive in-loop filtering, in case of using the soft classification for the assigning the classification, by, at each first pre- reconstructed sample, applying, for each class of the first set of classes, the associated FIR filter associated with the respective class to the pre-reconstructed samples to obtain a filter result, and forming a weighted sum of the filter results of the first set of classes according to the classification values.
Karczewicz, however, teaches an encoding apparatus, a decoding apparatus, an encoding method, a decoding method, and a non-transitory digital storage medium:
wherein the in-loop filter tool comprises a serial connection of a first in-loop filter and a second in-loop filter (e.g. Fig. 17 and pars. 258 – 259: depicting and describing that the loop filter includes a serial connection of a first loop filter [deblocking filter and SAO filter] and a second loop filter [adaptive loop filtering [alf]),
wherein the one or more first modes involve the second in-loop filter assigning a classification to pre-reconstructed samples of the current picture and filtering the pre-reconstructed samples with a filter transfer function which is adapted to the classification (e.g. Fig. 9, and pars. 113 – 150: depicting and describing that in applying adaptive loop filtering, the system determines a classification for each sample and filters the sample using a filter associated with each classification),
wherein the classification of the one or more first modes is a soft-classification (e.g. par. 153: describing that the classification step classifies a sample into multiple classes, wherein classifying a sample into multiple classes is the equivalent of the soft-classification),
wherein the one or more first modes involve the second in-loop filter assigning a classification to pre-reconstructed samples of the current picture and filtering the pre-reconstructed samples with a filter transfer function which is adapted to the classification, the classification of the one or more first modes is a soft-classification (Fig. 9, and pars. 113 – 150: depicting and describing that for each sample [R(x,y)], the system classifies the sample into a first set of classes, each of the first set of classes having an associated first filter [First Stage], the system then applying the associated filters for each class to the sample to generate intermediate filtered samples, wherein the associated filters are FIR filters, and wherein classifying a sample into a plurality of classes in a first set of classes is the equivalent of the first classification),
wherein the second in-loop filter is configured to perform the soft classification for first pre-reconstructed samples by:
assigning, for each first pre-reconstructed sample, a classification value to each of a first set of classes, with each of which an associated FIR filter is associated (Fig. 9, and pars. 113 – 150: depicting and describing that for each sample [R(x,y)], the system classifies the sample into a first set of classes, each of the first set of classes having an associated first filter [First Stage], the system then applying the associated filters for each class to the sample to generate intermediate filtered samples, wherein the associated filters are FIR filters, and wherein the intermediate filtered samples are the equivalent of the first filtered version), and
performing the adaptive in-loop filtering, in case of using the soft classification for the assigning the classification, by, at each first pre- reconstructed sample, applying, for each class of the first set of classes, the associated FIR filter associated with the respective class to the pre-reconstructed samples to obtain a filter result, and forming a weighted sum of the filter results of the first set of classes according to the classification values (e.g. pars. 207 – 211: describing that the system determines a weight for each class the sample is classified into and performs a weighted sum of the intermediate filtered samples to obtain a filtered sample of the sample).
Ma, however, teaches an encoding apparatus, a decoding apparatus, an encoding method, a decoding method, and a non-transitory digital storage medium:
wherein each of the first modes uses a CNN (e.g. Fig. 18 and par. 143: depicting and describing that the adaptive loop filter uses a CNN).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Ye by adding the teachings of Karczewicz in order for the in-loop filter to include a serial connection between a first in-loop filter and a second in-loop filter, for the one or more first modes to involve the second in-loop filter assigning a classification to pre-reconstructed samples of the current picture and filtering the pre-reconstructed samples with a filter transfer function which is adapted to the classification, and for the classification to be a soft classification, and by adding the teachings of Ma in order for each of the first modes to use a CNN. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves overall quality of decoded video data by accounting for local features of video data better than can be done by a single stage ALF (Karczewicz, e.g. par. 36: describing a desire to improve the overall quality of decoded video data) and because the modification improves coding efficiency by applying neural network technique to video coding (Ma, e.g. par. 8: describing a desire to improve coding efficiency by applying neural network techniques to video coding).
Turning to claim 6, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye does not explicitly teach:
wherein the classification of the one or more first modes is CNN based.
Ma, however, teaches an apparatus for decoding a video from a bitstream:
wherein the classification of the one or more first modes is CNN based (e.g. Fig. 18 and par. 143: depicting and describing that classification is performed using CNN).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Ye by adding the teachings of Ma in order for the classification of the one or more first modes is CNN based. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves coding efficiency by applying neural network technique to video coding (Ma, e.g. par. 8: describing a desire to improve coding efficiency by applying neural network techniques to video coding).
Regarding claim 8, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye does not explicitly teach:
wherein the second in-loop filter is configured to perform the adaptive in-loop filtering by use of FIR filters adapted in a sample- wise manner.
Karczewicz, however, teaches apparatus for decoding a video from a bitstream:
wherein the second in-loop filter is configured to perform the adaptive in-loop filtering by use of FIR filters adapted in a sample- wise manner (e.g. Figs. 9, 11, and 13, and pars. 114 – 152: depicting and describing that the system performs adaptive loop filtering by applying filters adapted in a sample-wise manner based on classification)
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Ye by adding the teachings of Karczewicz in order for the second in-loop filter to be configured to perform the adaptive in-loop filtering by use of FIR filters adapted in a sample-wise manner. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves overall quality of decoded video data by accounting for local features of video data better than can be done by a single stage ALF (Karczewicz, e.g. par. 36: describing a desire to improve the overall quality of decoded video data).
Regarding claim 40, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye further teaches:
configured to perform the mode switching by use of a syntax element in the bitstream (e.g. par. 76: describing that the system determines whether to apply or not apply ALF filtering to a block is based on filter flag sent in the bitstream, wherein the filter flag is the equivalent of the syntax element), and
configured to perform the mode switching by:
determining, within a predetermined picture area, a measure for prediction quality or prediction imperfection within the predetermined picture area (e.g. Fig. 5, element 507, and pars. 82 – 83: depicting and describing that the system determines whether there are non-zero residuals in the block, wherein whether there are non-zero residuals is the equivalent of the measure for prediction quality or prediction imperfection, and wherein the block is the equivalent of the predetermined picture area), and
checking whether the measure for prediction or prediction imperfection fulfills a further predetermined criterion, and if so, inferring that the syntax element, if same relates to the predetermined picture area, assumes a predetermined value not corresponding to any first mode, or any first mode exceeding a predetermined complexity, or has a decreased value domain which excludes the one or more first modes, or any first mode exceeding the predetermined complexity, and is decreased relative to a complete value domain the syntax element has outside the predetermined picture area, so that a bit rate for signaling at least one value in the decreased value domain, which does not correspond to any first mode, or any first mode exceeding the predetermined complexity, has a smaller bitrate consumption than compared to a corresponding value in the complete value domain (e.g. Fig. 5, elements 507, 508, and 509, and pars. 82 – 83: depicting and describing that when the system determines that a block does not contain any non-zero residual values, the system determines that ALF flag is 0, indicating that ALF filtering is not applied to the block, wherein whether a block contains any non-zero residual values is the equivalent of the measure for prediction quality or prediction imperfection fulfilling a predetermined criterion, and wherein determining a value of the ALF flag is zero when there are no non-zero residual values in a block is the equivalent of inferring the syntax element for the predetermined picture area is a value not corresponding to any first mode).
Turning to claim 41, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye further teaches:
configured to perform the mode switching based on measure for prediction quality or prediction imperfection within a predetermined picture area by disabling the one or more first modes, or any first mode exceeding a predetermined complexity, for the predetermined picture area if the measure for prediction quality or prediction imperfection fulfills a further predetermined criterion (e.g. Fig. 5, element 507, and pars. 82 – 83: depicting and describing that the system determines whether to apply [ALFflag = 1] or skip [ALFflag = 0] ALF filtering for a block based on whether the block has any non-zero residuals, wherein non-zero residuals is the equivalent of the measure for prediction quality or prediction imperfection, and wherein the block is the equivalent of the predetermined picture area, and wherein skipping ALF filtering is the equivalent of disabling the one or more first modes).
Regarding claim 42, Ye, Karczewicz, and Ma teach all of the limitations of claims 1 and 40, as discussed above. Ye further teaches:
wherein the measure for prediction quality or prediction imperfection includes one or more of the prediction residual being zero within the predetermined picture area, the areal fraction in which the prediction residual is zero, a number of coded non-zero transform coefficients, an energy of coded transform coefficients (e.g. Fig. 5, element 507, and pars. 82 – 83: depicting and describing that the system determines whether there are non-zero prediction residuals in the block).
Turning to claim 43, Ye, Karczewicz, and Ma teach all of the limitations of claims 1 and 40, as discussed above. Ye further teaches:
wherein the predetermined picture area is a coding tree root block, coding block, or slice (e.g. Fig. 5, element 507, and pars. 82 – 83: depicting and describing that the predetermined picture area is a coding block).
Regarding claim 44, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye further teaches:
configured to perform the mode switching by use of a syntax element in the bitstream (e.g. pars. 34 and 76: describing that the system determines whether to apply or not apply ALF filtering to a block is based on filter flag sent in the bitstream, wherein the filter flag is the equivalent of the syntax element), and
configured to perform the mode switching by:
determining a prediction type or inter-prediction hierarchy level of a picture (e.g. Fig. 4 and par. 81: depicting and describing that the system determines whether a block is an intra coded block or an inter-coded block, wherein determining whether a block is intra-coded or inter-coded is the equivalent of determining a prediction type), and
checking whether the prediction type or inter-prediction hierarchy level fulfils an even further predetermined criterion, and if so, inferring that the syntax element, if same relates to the picture, assumes a predetermined value not corresponding to any first mode, or any first mode exceeding the predetermined complexity, or has a decreased value domain which excludes the one or more first modes, or any first mode exceeding a predetermined complexity, and is decreased relative to a complete value domain the syntax element has outside the picture, so that a bit rate for signaling at least one value in the decreased value domain, which does not correspond to any first mode, or any first mode exceeding the predetermined complexity, has a smaller bitrate consumption than compared to a corresponding value in the complete value domain (e.g. Figs. 4 and 9, and pars. 81, and 96 - 97: depicting and describing that when the block is inter-coded, the system determines to not apply the ALF filtering to the block, wherein not applying ALF filtering when a block is inter-coded is the equivalent of inferring the syntax element does not correspond to the first mode) .
Turning to claim 45, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye further teaches:
configured to perform the mode switching based on prediction type or inter-prediction hierarchy level of a picture by disabling the one or more first modes, or any first mode exceeding a predetermined complexity for the picture if the measure for prediction quality or prediction imperfection fulfills an even further predetermined criterion (e.g. Figs. 4 and 9, and pars. 81 and 96 – 97: depicting and describing that the system determines whether to apply ALF filtering based on whether a block is inter-coded or intra-coded).
Regarding claim 46, Ye, Karczewicz, and Ma teach all of the limitations of claims 1 and 44, as discussed above. Ye further teaches:
wherein the prediction type indicates whether the picture is inter-predicted based on reference pictures preceding and succeeding the picture in presentation time order, with the even further predetermined criterion being fulfilled if this is the case, and/or the inter- prediction hierarchy level of a picture indicates a temporal hierarchy level of the picture in a GOP, with the even further predetermined criterion being fulfilled if the hierarchy level exceeds same threshold (e.g. Figs. 4 and 9, and pars. 81 and 96 – 97: depicting and describing that the system determines does not apply ALF filtering when the block is inter-coded, wherein it is known to those of ordinary skill in the art that an inter-coded block is a block that uses reference pictures preceding in time order, succeeding in time order, or both preceding and succeeding in time order).
Turning to claim 47, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye further teaches:
configured to perform the mode switching by use of a syntax element in the bitstream (e.g. pars. 34 and 76: describing that the system determines whether to apply or not apply ALF filtering to a block is based on filter flag sent in the bitstream, wherein the filter flag is the equivalent of the syntax element), and
configured to perform the mode switching by
checking whether a predetermined picture portion has at least one reference picture which succeeds a picture of the predetermined picture portion in presentation time order (e.g. Figs. 4 and 9, and pars. 81, and 96 - 97: depicting and describing that the system determines whether a block is inter-coded, wherein it is known to those of ordinary skill in the art that an inter-coded block uses reference pictures preceding in time order, succeeding in time order, or both preceding and succeeding in time order), and
if so, inferring that the syntax element, if same relates to the predetermined picture portion, assumes a predetermined value not corresponding to any first mode, or any first mode exceeding the predetermined complexity, or has a decreased value domain which excludes the one or more first modes, or any first mode exceeding the predetermined complexity, and is decreased relative to a complete value domain the syntax element has outside the predetermined picture portion, so that a bit rate for signaling at least one value in the decreased value domain, which does not correspond to any first mode, or any first mode exceeding the predetermined complexity, has a smaller bitrate consumption than compared to a corresponding value in the complete value domain (e.g. Figs. 4 and 9, and pars. 81, and 96 - 97: depicting and describing that when the block is inter-coded, the system determines to not apply the ALF filtering to the block, wherein not applying ALF filtering when a block is inter-coded is the equivalent of inferring the syntax element does not correspond to the first mode).
Regarding claim 48, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye further teaches:
configured to perform the mode switching in dependence on whether for a predetermined picture portion at least one reference picture succeeds a picture of the predetermined picture portion in presentation time order by disabling the one or more first modes, or any first mode exceeding a predetermined complexity for the predetermined picture portion if this is the case (e.g. Figs. 4 and 9, and pars. 81 and 96 – 97: depicting and describing that the system determines does not apply ALF filtering when the block is inter-coded, wherein it is known to those of ordinary skill in the art that an inter-coded block is a block that uses reference pictures preceding in time order, succeeding in time order, or both preceding and succeeding in time order).
Turning to claim 49, Ye, Karczewicz, and Ma teach all of the limitations of claims 1 and 47, as discussed above. Ye further teaches:
wherein the predetermined picture portion is a slice or a whole picture (e.g. par. 64: describing that syntax information defines how to apply filtering for blocks within a slice).
Regarding claim 52, Ye, Karczewicz, and Ma teach all of the limitations of claims 1 and 47, as discussed above. Ye further teaches:
wherein the predetermined picture portion is a slice, a whole picture or a CTU or a CU (e.g. par. 64: describing that syntax information defines how to apply filtering for blocks within a slice).
Turning to claim 54, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye does not explicitly teach:
wherein the second in-loop filter is configured to perform the soft classification for first pre-reconstructed samples by:
assigning, for each first pre-reconstructed sample, a classification value to each of a first set of classes, with each of which an associated first FIR filter and a second filter is associated, and
performing the adaptive in-loop filtering, in case of using the soft classification for the assigning the classification, by
applying, for each class of the first set of classes, the associated first FIR filter associated with the respective class onto the pre-reconstructed samples to obtain a first filtered version,
weighting, for each class of the first set of classes, the first filtered version at each sample position with the classification value assigned to the respective class for the first pre- reconstructed sample at the respective sample position to obtain an second filtered version,
applying, for each class of the first set of classes, the associated second FIR filter associated with the respective class onto the second filtered version to obtain a third filtered version,
subjecting, for each second pre-filtered version, the third filtered version obtained for the classes of the first set, to a summation,
wherein for each class of the first set of classes coefficients, and/or a size and/or shape of a kernel of the second FIR filter are conveyed in the bitstream.
Karczewicz, however, teaches an apparatus for decoding a video from a bitstream:
wherein the second in-loop filter is configured to perform the soft classification for first pre-reconstructed samples by:
assigning, for each first pre-reconstructed sample, a classification value to each of a first set of classes, with each of which an associated first FIR filter and a second filter is associated (Karczewicz, Fig. 9 and pars. 113 – 150: depicting and describing that for each sample [R(x,y)], the system classifies the sample into a first set of classes, each of the first set of classes having an associated first filter and second filter [associated filter for each classification in first stage [element 400] and second stage [element 410]], wherein the associated filters are FIR filters), and
performing the adaptive in-loop filtering, in case of using the soft classification for the assigning the classification, by
applying, for each class of the first set of classes, the associated first FIR filter associated with the respective class onto the pre-reconstructed samples to obtain a first filtered version (Fig. 9, and pars. 113 – 150: depicting and describing that for each sample [R(x,y)], the system classifies the sample into a first set of classes, each of the first set of classes having an associated first filter [First Stage], the system then applying the associated filters for each class to the sample to generate intermediate filtered samples, wherein the associated filters are FIR filters, and wherein the intermediate filtered samples are the equivalent of the first filtered version),
weighting, for each class of the first set of classes, the first filtered version at each sample position with the classification value assigned to the respective class for the first pre- reconstructed sample at the respective sample position to obtain an second filtered version (e.g. par. 207 – 211: describing that the system determines a weight for each intermediate filtered sample) ,
applying, for each class of the first set of classes, the associated second FIR filter associated with the respective class onto the second filtered version to obtain a third filtered version (e.g. Fig. 9 and pars. 113 – 150: depicting and describing that the system applies a second filter [second stage] to the intermediate filtered samples, a signaled filter associated with each of the classifications),
subjecting, for each second pre-filtered version, the third filtered version obtained for the classes of the first set, to a summation (e.g. par. 207 – 211: describing that the system performs a weighted sum of all filtered samples to obtain a filtered sample),
wherein for each class of the first set of classes coefficients, and/or a size and/or shape of a kernel of the second FIR filter are conveyed in the bitstream (e.g. par. 149: describing that the second filter set is signaled in the bitstream, wherein the second filter set is a second FIR filter).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Ye by adding the teachings of Karczewicz in order to perform the soft classification by assigning a classification value to each of a first set of classes, each class associated with a first FIR filter and a second filter, and perform the adaptive loop filtering using soft classification by applying, for each class of the first set of classes, the associated FIR filter to obtain a first filtered version, weighting the first filtered version based on the classification value to obtain a second filtered version, applying the second FIR filter to the second filtered version to obtain a third filtered version, and subjecting the third filtered version to a summation. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves overall quality of decoded video data by accounting for local features of video data better than can be done by a single stage ALF (Karczewicz, e.g. par. 36: describing a desire to improve the overall quality of decoded video data).
Regarding claim 55, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye does not explicitly teach:
wherein the second in-loop filter is configured to switch, based on the bitstream, between 1) performing the soft classification for first pre-reconstructed samples in a first manner by assigning, for each first pre-reconstructed sample, a classification value to each of a first set of classes, with each of which an associated FIR filter is associated, and performing the adaptive in-loop filtering, in case of using the soft classification for the assigning the classification, by applying, at each first pre- reconstructed sample, for each class of the first set of classes, the associated FIR filter associated with the respective class to the pre-reconstructed samples to obtain a filter result, and forming a weighted sum of the filter results of the first set of classes according to the classification values; and 2) performing the soft classification for first pre-reconstructed samples in a second manner by assigning, for each first pre-reconstructed sample, a classification value to each of a first set of classes, with each of which an associated first FIR filter and a second filter is associated, and performing the adaptive in-loop filtering, in case of using the soft classification for the assigning the classification, by
applying, for each class of the first set of classes, the associated first FIR filter associated with the respective class onto the pre-reconstructed samples to obtain a first filtered version,
weighting, for each class of the first set of classes, the first filtered version at each sample position with the classification value assigned to the respective class for the first pre- reconstructed sample at the respective sample position to obtain a second filtered version,
applying, for each class of the first set of classes, the associated second FIR filter associated with the respective class onto the second filter version to obtain a third filtered version,
subjecting, for each second pre-filtered version, the third filtered version obtained for the classes of the first set, to a summation,
wherein for each class of the first set of classes coefficients, and/or a size and/or shape of a kernel of the second FIR filter are conveyed in the bitstream.
Karczewicz, however, teaches an apparatus for decoding a video from a bitstream:
wherein the second in-loop filter is configured to switch, based on the bitstream, between 1) performing the soft classification for first pre-reconstructed samples in a first manner by assigning, for each first pre-reconstructed sample, a classification value to each of a first set of classes, with each of which an associated FIR filter is associated, and performing the adaptive in-loop filtering, in case of using the soft classification for the assigning the classification, by applying, at each first pre- reconstructed sample, for each class of the first set of classes, the associated FIR filter associated with the respective class to the pre-reconstructed samples to obtain a filter result, and forming a weighted sum of the filter results of the first set of classes according to the classification values; and 2) performing the soft classification for first pre-reconstructed samples in a second manner by assigning, for each first pre-reconstructed sample, a classification value to each of a first set of classes, with each of which an associated first FIR filter and a second filter is associated, and performing the adaptive in-loop filtering, in case of using the soft classification for the assigning the classification (e.g. Fig. 13 and pars. 207 – 212: depicting and describing that the system switches between 1) classifying, for each sample [R(x,y)] into a plurality of a first set of classes, each of the first set of classes having an associated filter, the system performing filtering of each sample by applying the associated filters based on the class and obtaining a weighted sum to determine a filtered sample, and 2) classifying each sample into a first set of classifications, each classification associated with a first filter and a second filter, the system performing the filtering based on applying the first filter and second filter based on the identified classes, the switching determined by filter usage information, the filter usage information being information in the bitstream [see, e.g. par. 209: describing that the method of adaptive loop filtering performed is determined by filter usage information signaled in the bitstream]), by
applying, for each class of the first set of classes, the associated first FIR filter associated with the respective class onto the pre-reconstructed samples to obtain a first filtered version (Fig. 9, and pars. 113 – 150: depicting and describing that for each sample [R(x,y)], the system classifies the sample into a first set of classes, each of the first set of classes having an associated first filter [First Stage], the system then applying the associated filters for each class to the sample to generate intermediate filtered samples, wherein the associated filters are FIR filters, and wherein the intermediate filtered samples are the equivalent of the first filtered version),
weighting, for each class of the first set of classes, the first filtered version at each sample position with the classification value assigned to the respective class for the first pre- reconstructed sample at the respective sample position to obtain an second filtered version (e.g. par. 207 – 211: describing that the system determines a weight for each intermediate filtered sample),
applying, for each class of the first set of classes, the associated second FIR filter associated with the respective class onto the second filter version to obtain a third filtered version (e.g. Fig. 9 and pars. 113 – 150: depicting and describing that the system applies a second filter [second stage] to the intermediate filtered samples, a signaled filter associated with each of the classifications) ,
subjecting, for each second pre-filtered version, the third filtered version obtained for the classes of the first set, to a summation (e.g. par. 207 – 211: describing that the system performs a weighted sum of all filtered samples to obtain a filtered sample),
wherein for each class of the first set of classes coefficients, and/or a size and/or shape of a kernel of the second FIR filter are conveyed in the bitstream (e.g. par. 149: describing that the second filter set is signaled in the bitstream, wherein the second filter set is a second FIR filter).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Ye by adding the teachings of Karczewicz in order to switch between performing the soft classification in a first manner and a second manner based on the bitstream by applying the first FIR filter associated with each class to obtain a first filtered version, weighting, for each class, the first filtered version to obtain a second filtered version, applying, for each class, the second associated filter to obtain a third filtered version, and subjecting the third filtered version to obtain a summation. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves overall quality of decoded video data by accounting for local features of video data better than can be done by a single stage ALF (Karczewicz, e.g. par. 36: describing a desire to improve the overall quality of decoded video data).
Turning to claim 56, Ye, Karczewicz, and Ma teach all of the limitations of claims 1 and 55, as discussed above. Ye does not explicitly teach:
configured to perform the switching between performing the performing the soft classification for the first pre-reconstructed samples in the first or second manner in units of one or more of: coding tree root blocks into which the current picture is pre-subdivided in rows and columns of coding tree root blocks, and from which onwards the picture is subdivided into coding blocks by recursive multi-tree partitioning of the coding tree root blocks, coding blocks into which the current picture is subdivided by pre-subdividing the current picture into coding tree root blocks in rows and columns of coding tree root blocks, and subdividing the picture further from the coding tree root blocks onwards by recursive multi-tree partitioning of the coding tree root blocks, and at least one of slices of the current picture; pictures of the video, a sequence of Pictures of the video, the video.
Karczewicz, however, teaches an apparatus for decoding a video from a bitstream:
configured to perform the switching between performing the performing the soft classification for first pre-reconstructed samples in the first or second manner in units of one or more of: coding tree root blocks into which the current picture is pre-subdivided in rows and columns of coding tree root blocks, and from which onwards the picture is subdivided into coding blocks by recursive multi-tree partitioning of the coding tree root blocks, coding blocks into which the current picture is subdivided by pre-subdividing the current picture into coding tree root blocks in rows and columns of coding tree root blocks, and subdividing the picture further from the coding tree root blocks onwards by recursive multi-tree partitioning of the coding tree root blocks, and slices of the current picture; pictures of the video, a sequence of Pictures of the video, the video (e.g. pars. 209: describing that the filter usage information is signaled in units of coding tree block, coding block, slice, picture of the video [PPS] or sequence of pictures of the video [SPS]).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Ye by adding the teachings of Karczewicz in order to perform the switching between the performing of the classification in the first manner and the second manner in units of one or more of coding tree root blocks into which the current picture is pre-subdivided, coding blocks, slices, pictures of the video, sequence of pictures of the video or the video. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves overall quality of decoded video data by accounting for local features of video data better than can be done by a single stage ALF (Karczewicz, e.g. par. 36: describing a desire to improve the overall quality of decoded video data).
Claim(s) 39 and 57 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ye et al. (US 2010/0158103) (hereinafter Ye) in view of Karczewicz et al. (US 2022/0201292)(hereinafter Karczewicz) in view of Ma et al. (WO 2022/072659) (hereinafter Ma) as applied to claim 1 above, and further in view of Xiu et al. (WO 2020/061082) (hereinafter Xiu).
Regarding claim 39, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye does not explicitly teach:
configured to perform the mode switching based on an estimation of a measure of complexity incurred by the second in-loop filter or the one or more first modes of the second in-loop filter within a predetermined video or picture section so far by disabling the one or more first modes, or any first mode exceeding a predetermined complexity for the predetermined video or picture section if the estimation fulfills a predetermined criterion.
Xiu, however, teaches an apparatus for decoding video from a bitstream:
configured to perform the mode switching based on an estimation of a measure of complexity incurred by the second in-loop filter or the one or more first modes of the second in-loop filter within a predetermined video or picture section so far by disabling the one or more first modes, or any first mode exceeding a predetermined complexity for the predetermined video or picture section if the estimation fulfills a predetermined criterion (e.g. pars. 113 and Tables 6: describing that the system disables filtering of blocks above a certain size because the number of multiplications per sample incurred by the filtering blocks above the determined size is greater than the number of multiplications per sample incurred without filtering the block, wherein the number of multiplications per sample is the equivalent of the measure of complexity, and wherein the number of multiplications per sample incurred filtering the block being greater than the number of multiplications incurred without filtering the block is the equivalent of the estimation of a measure for the complexity incurred by the second in-loop filter fulfilling the predetermined criterion).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Ye by adding the teachings of Xie in order to perform the mode switching based on an estimation of a measure of complexity incurred by the second in-loop filter or the one or more first modes of the second in-loop filter within a predetermined video or picture section so far by disabling the one or more first modes, or any first mode exceeding a predetermined complexity for the predetermined video or picture section of the estimation fulfills a predetermined criterion. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves coding efficiency.
Turning to claim 57, Ye, Karczewicz, and Ma teach all of the limitations of claim 1, as discussed above. Ye does not explicitly teach:
configured to, based on an estimation of a measure for multiplications per sample incurred by the second in-loop filter for the current picture so far, switch to the second mode of bypassing the second in-loop filter by disabling the soft classification if the estimation fulfills a predetermined criterion.
Xiu, however, teaches an apparatus for decoding a video from a bitstream:
configured to, based on an estimation of a measure for multiplications per sample incurred by the second in-loop filter for the current picture so far, switch to the second mode of bypassing the second in-loop filter by disabling the soft classification if the estimation fulfills a predetermined criterion (e.g. pars. 113 and Tables 6: describing that the system disables filtering of blocks above a certain size because the number of multiplications per sample incurred by the filtering blocks above the determined size is greater than the number of multiplications per sample incurred without filtering the block, wherein the number of multiplications per sample incurred filtering the block being greater than the number of multiplications incurred without filtering the block is the equivalent of the estimation of a measure for multiplications per sample incurred by the second in-loop filter fulfilling the predetermined criterion).
It therefore would have been obvious to one of ordinary skill in the art to modify the teachings of Ye by adding the teachings of Xie in order to perform the mode switching based on an estimation of a measure for multiplications per sample incurred by the second in-loop filter for the current picture by disabling the soft classification if the estimation fulfills a predetermined criterion. One of ordinary skill in the art would have been motivated to make such a modification because the modification improves coding efficiency.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHANIKA M BRUMFIELD whose telephone number is (571)270-3700. The examiner can normally be reached M-F 8:30 - 5 PM AWS.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Czekaj can be reached at 571-272-7327. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
SHANIKA M. BRUMFIELD
Examiner
Art Unit 2487
/SHANIKA M BRUMFIELD/Examiner, Art Unit 2487
/Dave Czekaj/Supervisory Patent Examiner, Art Unit 2487
Prosecution Insights