Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Karczewicz(US 2022/0201292) in view of Li (US 2023/0007246).
For claims 1, 19 and 20, Karczewicz discloses a method for processing video data, comprising ([0110] Video encoder 200 and video decoder 300 may be configured to perform filter coefficient signaling):
determining to employ a plurality of adaptive loop filter (ALF) parameter sets ([0110] . . . In VVC version 1, ALF coefficients are signaled in ALF adaptation parameter sets (APS))
for a single picture, a single slice, or a single tile ([0150] . . . the fixed filter set F(f, i) and classifier C(f, i) may be chosen based on coding information, such as picture/CTU/CU/PU/TU size); and
performing a conversion between a visual media data and a bitstream based on ALF parameter sets ([0153) . . . video encoder 200 and video decoder 300 apply first stage ALF 540 to reconstructed sample R(x,y) of a reconstructed block.).
While Karczewicz teaches “based on ALF parameter sets,” it does not expressly teach “based on the plurality of ALF parameter sets.”
Li teaches performing a conversion between a visual media data and a bitstream based on the plurality of ALF parameter sets ([0154] In the JVET-N0242 implementation, the clipping parameters k(i, j) are specified for each ALF filter, one clipping value is signaled per filter coefficient. It means that up to 12 clipping values can be signaled in the bitstream per Luma filter and up to 6 clipping values for the Chroma filter.).
It would be obvious to a person with ordinary skill in the art to combine the filtering teachings of Karczewicz with the plurality of filter teachings of Li for the predictable benefit of complying with video compression standard JVET-N0242 implementation.
For claim 2, Karczewicz discloses wherein in an ALF, luma filters are trained for multiple alternatives ([0098] FIG. 5 shows an example of merging 25 luma classes into 7 merged classes (0 to 6)), and
wherein an ALF parameter set from the plurality of ALF parameter sets contains at least one of following parameters: a number of the multiple alternatives; an input source for each tap of the ALF; and classifier information; a non-linear clipping control; a non-linear clipping parameter; a number of merged classes; a map of merged classes; or coefficients of each filter of the ALF ([0085] The non-linear function is defined with a clipping functions as: [0086] In VVC version 1, as shown in equation (4), the clipping parameter b.sub.i for a coefficient c.sub.i, is determined by a clipping index d.sub.i. BD is the internal bit depth.).
For claim 3, Karczewicz discloses wherein in an ALF, chroma filters are trained for multiple alternatives ([0112] This disclosure describes techniques that may further improve the performance of ALF. For example, in VVC, when filtering a sample, only one classifier and one filter can be applied. However, this disclosure described techniques for applying multiple classifiers and filters to capture more local feature), and
wherein an ALF parameter set from the plurality of ALF parameter sets contains at least one of following parameters: a number of the multiple alternatives; an input source for each tap of the ALF; a non-linear clipping control; a non-linear clipping parameter; or coefficients of each filter of the ALF ([0104] . . . the filter shapes have 7 taps and 5 taps for luma and chroma components, respectively. [0110] . . . . ALF coefficients are signaled in ALF adaptation parameter sets (APS). One APS may contain one set of luma filters with up to 25 filters, up to 8 chroma filters and up to 8 cross-component ALF (CC-ALF) filters.).
For claim 4, Karczewicz discloses wherein the ALF parameter set from the plurality of ALF parameter sets contains a parameter of a cross component ALF (CC-ALF), wherein the parameter of the CC-ALF comprises a number of filters contained in the CC-ALF and/or coefficients of the CC-ALF ([0110] One APS may contain one set of luma filters with up to 25 filters, up to 8 chroma filters and up to 8 cross-component ALF (CC-ALF) filter).
For claim 5, Karczewicz discloses wherein the plurality of ALF parameter sets are included in the bitstream independently ([0033] adaptive loop filtering (ALF). Parameters for these filtering operations may either be determined by a video encoder and explicitly signaled in the encoded video bitstream or may be implicitly determined by a video decoder without needing the parameters to be explicitly signaled in the encoded video bitstream.).
For claim 6, Karczewicz discloses wherein a parameter in a first ALF parameter set from the plurality of ALF parameter sets is reused or inherited from a second ALF parameter set from the plurality of ALF parameter sets, wherein the first ALF parameter set is different from the second ALF parameter set, and wherein the parameter comprises at least one of: filter coefficients; class merging results; a clipping parameter; alternative information; or classifier information ([0114] In the example of FIG. 9, video decoder 300 applies first stage ALF 400 to reconstructed sample R(x,y) to determine intermediate filtered signal R′. Video decoder 300 then applies second stage ALF 410 to R′ to determine a value for filtered sample 430.).
For claim 7, Karczewicz discloses wherein an ALF processing unit selects a parameter from the plurality of ALF parameters sets and wherein the selected parameter is included in one or more available adaptation parameter sets (APSs) of the bitstream; and wherein an index referring to the selected parameter is included in a picture level, a slice level, a coding tree unit (CTU), or a coding unit (CU) level of the bitstream.
For claim 8, Karczewicz discloses wherein an ALF processing unit selects a best parameter from a first set of the plurality of ALF parameter sets that is trained for or targeting an original picture; or wherein the ALF processing unit selects the best parameter from a second set of the plurality of ALF parameter sets that is trained for or targeting a picture filtered by a motion compensated temporal filtering (MCTF) filter ([0099] For each class, the filter index (from 0 to 6 in this example) is signaled in an ALF_APS.).
For claim 9, Karczewicz discloses wherein information of the plurality of ALF parameter sets is specified in an adaptation parameter set (APS) in the bitstream; wherein ALF parameters in different parameters sets from the plurality of ALF parameter sets are included in a same adaptation parameter set (APS) in the bitstream; or wherein ALF parameters in different parameters sets from the plurality of ALF parameter sets are included in a different adaptation parameter sets (APSs) in the bitstream ([0099] e.g. signaled in an ALF_APS.).
For claim 10, while Karczewicz does not, Li teaches further comprising storing or maintaining multiple reconstructions for a current picture, a current slice, or a current tile, wherein a first possible final reconstruction is an ALF filtered picture using parameters trained for or targeting an original picture; wherein a second possible final reconstruction is an ALF filtered picture using parameters trained for or targeting a picture filtered by a motion compensated temporal filtering (MCTF) filter; and wherein at least one of the multiple reconstructions is used as a reference picture for other pictures, and at least one of following is true: wherein a first one of the multiple reconstructions is used as a forward reference picture; wherein a second one of the multiple reconstructions is used as a backward reference picture; wherein a third one of the multiple reconstructions is used as a long-term reference picture; or wherein a fourth one of the multiple reconstructions is used as a display picture for a current picture ([0160] The selected clipping values are coded in the “alf_data” syntax element by using a Golomb encoding scheme corresponding to the index of the clipping value in the above Table 5. This encoding scheme is the same as the encoding scheme for the filter index. TABLE-US-00012 TABLE 1-2 Network Information for NN-based Video Coding Tool Testing in Training Stage Network Information in Training Stage Mandatory GPU Type GPU: Tesla-V100-SXM2-32 GB Framework: PyTorch v1.6 Number of GPUs per Task 2 Epoch: 90 Batch size: 64 Training time: 60 h/model Training data information: DIV2K, BVI-DVC Training configurations for VTM-11.0 + new MCTF, QP generating compressed {17, 22, 27, 32, 37, 42} training data (if different to VTM CTC): Loss function: L1, L2 Optional Number of iterations Patch size 128 × 128 Learning rate: 1e−4 Optimizer: ADAM Preprocessing: Other information). It would be obvious to a person with ordinary skill in the art to combine the filtering teachings of Karczewicz with the plurality of filter teachings of Li for the predictable benefit of complying with video compression standard JVET-N0242 implementation.
For claim 11, while Karczewicz does not, Li teaches wherein at least two reconstructed pictures are generated by two ALF parameters sets from the plurality of ALF parameters sets; wherein a first reconstructed picture generated by a first ALF parameter set is used as a reference picture for other pictures; wherein a second reconstructed picture generated by a second ALF parameter set is output for display; and wherein use of the first reconstructed picture as the reference picture for other pictures or the second reconstructed picture being set as the output for display is indicated in the bitstream ([0079] The encoder 500 further includes an intra prediction component 508 and a motion estimation/compensation (ME/MC) component 510 configured to receive input video. The intra prediction component 508 is configured to perform intra prediction, while the ME/MC component 510 is configured to utilize reference pictures obtained from a reference picture buffer 512 to perform inter prediction. Residual blocks from inter prediction or intra prediction are fed into a transform component 514 and a quantization component 516 to generate quantized residual transform coefficients, which are fed into an entropy coding component 518. The entropy coding component 518 entropy codes the prediction results and the quantized transform coefficients and transmits the same toward a video decoder (not shown). Quantization components output from the quantization component 516 may be fed into an inverse quantization component 520, an inverse transform component 522, and a reconstruction (REC) component 524. The REC component 524 is able to output images to the DF 502, the SAO 504, and the ALF 506 for filtering prior to those images being stored in the reference picture buffer 512.). It would be obvious to a person with ordinary skill in the art to combine the filtering teachings of Karczewicz with the plurality of filter teachings of Li for the predictable benefit of complying with video compression standard JVET-N0242 implementation.
For claim 12, while Karczewicz does not, Li teaches wherein a video coding unit takes an ALF filtered reconstruction as a reference picture, and wherein at least one of following is true: wherein the video coding unit takes the ALF filtered reconstruction with a learning target of an original picture as a forward reference picture; wherein the video coding unit takes the ALF filtered reconstruction with a learning target of a picture filtered by a motion compensated temporal filtering (MCTF) filter as a forward reference picture; wherein the video coding unit takes the ALF filtered reconstruction with the learning target of the original picture as a backward reference picture; or wherein the video coding unit takes the ALF filtered reconstruction with the learning target of the picture filtered by the MCTF filter as a backward reference picture ([0139] Filter parameters signaling is discussed. In the JEM, GALF filter parameters are signalled for the first CTU, i.e., after the slice header and before the SAO parameters of the first CTU. Up to 25 sets of luma filter coefficients could be signalled. To reduce bits overhead, filter coefficients of different classification can be merged. Also, the GALF coefficients of reference pictures are stored and allowed to be reused as GALF coefficients of a current picture. The current picture may choose to use GALF coefficients stored for the reference pictures and bypass the GALF coefficients signalling. In this case, only an index to one of the reference pictures is signalled, and the stored GALF coefficients of the indicated reference picture are inherited for the current picture.). It would be obvious to a person with ordinary skill in the art to combine the filtering teachings of Karczewicz with the plurality of filter teachings of Li for the predictable benefit of complying with video compression standard JVET-N0242 implementation.
For claim 13, while Karczewicz does not, Li teaches wherein a first syntax element is included in the bitstream to indicate whether the multiple ALF filtered reconstruction is used; and wherein the first syntax element is signaled in a sequence parameter set (SPS), picture parameter set (PPS), picture header, slice header, adaptation parameter set (APS), coding tree unit (CTU), or coding unit (CU). ([0139] Filter parameters signaling is discussed. In the JEM, GALF filter parameters are signalled for the first CTU, i.e., after the slice header and before the SAO parameters of the first CTU). It would be obvious to a person with ordinary skill in the art to combine the filtering teachings of Karczewicz with the plurality of filter teachings of Li for the predictable benefit of complying with video compression standard JVET-N0242 implementation.
For claim 14, while Karczewicz does not, Li teaches wherein multiple different ALF reconstructions are accessed during a prediction loop stage; or wherein the multiple different ALF reconstructions are accessed during a loop filter stage ([0079] e.g. the ALF 506 for filtering prior to those images being stored in the reference picture buffer 512.). It would be obvious to a person with ordinary skill in the art to combine the filtering teachings of Karczewicz with the plurality of filter teachings of Li for the predictable benefit of complying with video compression standard JVET-N0242 implementation.
For claim 15, Karczewicz discloses wherein the method is applied to luma independently ([0099] A luma filter set contains 25 filters).
For claim 16, Karczewicz discloses wherein the method is applied to luma and chroma jointly ([0104] Video encoder 200 and video decoder 300 may be configured to perform techniques for line buffer reduction. As shown in FIGS. 2A and 2B, in the vertical direction, the filter shapes have 7 taps and 5 taps for luma and chroma components, respectively. As a result, in VVC Test Model 2.0 (VTM-2.0), when decoding a row of CTUs, due to the delay of the deblocking filter and SAO filter, 7 luma lines and 4 chroma lines of the upper CTU row must be stored in a line buffer for ALF.).
For claim 17, Karczewicz discloses wherein the conversion includes encoding the visual media data into the bitstream ([0035] The video decoder may determine the first set of filters, as well as which classifiers map to which filters, based on syntax included in the bitstream.).
For claim 18, Karczewicz discloses wherein the conversion includes decoding the visual media data from the bitstream ([0035] The video decoder may determine the first set of filters, as well as which classifiers map to which filters, based on syntax included in the bitstream.).).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hu; Nan et al. US 20230010869 A1 SIGNALED ADAPTIVE LOOP FILTER WITH MULTIPLE CLASSIFIERS IN VIDEO CODING
Li; Yue et al. US 20220286695 A1 Neural Network-Based In-Loop Filter With Residual Scaling For Video Coding
LIM; Sung Chang et al. US 20220295058 A1 IMAGE ENCODING/DECODING METHOD AND DEVICE EMPLOYING IN-LOOP FILTERING
LIM; Sung Chang et al. US 20220248006 A1 ADAPTIVE IN-LOOP FILTERING METHOD AND APPARATUS
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NEIL MIKESKA whose telephone number is (571)272-3917. The examiner can normally be reached M-F: 6a - 2p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jay Patel can be reached at (571) 272-2988. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NEIL R MIKESKA/Primary Examiner, Art Unit 2485
Prosecution Insights