DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
2. The information disclosure statement (IDS) was submitted on 05/28/2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
The applied references have no common inventorship with the instant application.
3. Claims 1-16, are rejected under 35 U.S.C. 103 as being obvious over Tong Shao et al., (hereinafter Shao) “EE1-related: Further complexity reduction on the joint EE1-0 (LOP.2) unified filter”; 32nd Meeting, Hannover, DE, 13-20 Oct. 2023; JVET-AF0071-v3 in view of Liqiang Wang et al., (hereinafter Wang); “EE1-1.2: neural network based in-loop filter with a single model”; 26th Meeting, Tencent, 20-29 April 2022; Doc. JVET-Z0091-v4.
Re Claim 1. Shao discloses, a method of coding video data (a video coding method generating the head block coded blocks, in Fig.1 and Fig.3), the method comprising:
receiving video data indicative of a picture (receiving at RecEXTY, the YUV components of a picture, per Figs.1 and 2 or the Y, Cb, Cr, components at Sec.2 and Sec.3);
reconstructing a block of the video data to generate a reconstructed block (the reconstructed blocks at the head block brought as inputs of the luma channel RecEXTY and of the chroma channels UV at RecEXTUV at the Neural Network (NN) in Figs.1 and 3); and
performing a neural network (NN)-based filter process on the reconstructed block to generate a filtered block (performing a convolutional NN filtering process on the head blocks in Figs.1 and 3), wherein the NN-based filter process comprises applying an NN-based filter comprising a constrained head block (the inputs to NN represented by the convolutional blocks CONV from the head block at different levels of constraint comprising of CONV_3x3 or CONV_1x1 convolutional strides in Figs.1 and 3),
the constrained head block having a plurality of input channels and being configured to independently extract, for each of the plurality of input channels, a respective number of output channels (the constrained head block is comprised of a plurality of input channels independently extracting the input convoluted CONV channels and generating a corresponding number of NN output channels of each Back-Bone Block, CY and CUV or NY and NUV respectively, through the NN convolutional layers at the NN output of a corresponding number of channels CY x4 and CUV x2, in Figs.1 and 3), and
fuse outputs associated with of each of the plurality of input channels into a single output (fusing the plurality of input channels as described for the back bone block C1, comprising two adjacent 1x1 convolutional layers, fused at CONVY 1x1, at Fig.3 and Sec.1, fusing adjacent 1x1 convolutions, Abstract and Sec.2),
wherein a sum of the respective number of output channels is less than or equal to a number of fused output channels of the single output (would have been obvious to the skilled in the art to deduct that the number of a NN backbone output extracted from the independent constrained channels would have been smaller than the resulted number of summed channels after fusing the two convolutional 1x1, blocks of the respective layers, e.g., at the single output of the fused layers per Fig.3).
However, the analogous art to Wang, teaches about the rationale behind the number of input/output channels at an NN filter architecture as claimed,
wherein a sum of the respective number of output channels is less than or equal to a number of fused output channels of the single output (defining the number of channels firstly going up before the activation layer of the NN, then going down after the activation layer, at Sec.2.1 and Fig.2).
In consideration to the common coding method of performing NN filtering of reconstructed video blocks disclosed at a pre/post-processing architecture disclosed in Shao, the ordinary skilled in the art would have found obvious before the effective filing date of invention, and find the incentive to verify specific functional elements deemed to reduce the NN complexity suggested in Shao (at Fig.2) thus seeking explicit methods of such application as those found in Wang, teaching the change in the number of channels (Sec.2.1), by which such combination would be predictable.
Re Claim 2. Shao and Wang disclose, the method of claim 1, wherein the NN-based filter further comprises:
Shao teaches about, a transition block having a first number of output features for luma and a second number of output features for chroma, wherein the first number is larger than the second number (according to Fig.1 the CONV+PRELU+CONV2+PRELU blocks represent the transition blocks where the last transition PRELU block comprises the Y- luma channel and the two UV -channels, where by the representation of the YUV format into the Y, Cb, Cr format, where based on the pixel-block representation of 4:2:2, would have been obvious that the luma 4x4 pixel format is larger than any of the Cb or Cr channels of 4:2, i.e., 4x2 formats);
a plurality of luma backbone blocks configured to process the output features for luma (the plurality of luma channels are configured as the independently processed luma backbone of the NN block channels Back Bone, CY, per NY, in Fig.1); and
a plurality of chroma backbone blocks configured to process the output features for chroma (the plurality of chroma Cb, Cr, channels are configured and independently processed at the chroma backbone of the NN block channels Back Bone, CUV, per NUV, in Fig.1).
Re Claim 3. Shao and Wang disclose, the method of claim 2,
Shao teaches about, wherein the first number equals 24 and the second number equals 8, the first number equals 16 and the second number equals 8, or the first number equals 12 and the second number equals 4 (according to the Headblock of parameter selection table in Figs.1 and 3).
Re Claim 4. Shao and Wang disclose, the method of claim 1,
Shao teaches about, wherein NN-based filter further comprises a transition block including one or more convolutions and an activation block, the activation block being after all of the one or more convolutions of the transition block (the transition block of layers comprising the separable convolutions, sepCONV comprising the convolutions sepConv 1x3 and sepConv 3x1 fused at CONV 1x1 Cy being aggregated at PRELU for the Y luma NN core and similarly for the CONV 1x1 Cuv and aggregated PRELU of the chroma NN core per Fig.1 and 3, Sec.1 and Sec.2).
Re Claim 5. Shao and Wang disclose, the method of claim 1,
Shao teaches about, wherein d6 >= sum (d1, d2, d3,d4,d4, d5), where d1 through d5 are numbers of the respective output channels and d6 is the number of fused output channels of the single output (the d1…d5 are the output reconstructed channels as inputs to the convolutional NN, and the d6 is the sum representing the input channels at the single output of the convolution layer CONV 1x1, d6 in Figs.1 and 3).
Re Claim 6. Shao and Wang disclose, the method of claim5,
Shao teaches about, wherein the plurality of input channels comprise an indication of reconstructions samples, prediction samples, a boundary strength map, a second quantization parameter, and a first quantization parameter, and reconstruction samples, and wherein d1= 4, d2 = 2, d3 = 1, d4=1, d5=1, and d6 = 16 (as being defined at the lookup Headblock table in Figs.1, or 3, as parameters defined for d1…d5 extracted channels at the head block and the d6 would represent the number of fused output channels of the single output CONV, 1x1, d6 layer).
Re Claim 7. Shao and Wang disclose, the method of claim 5,
Shao teaches that, wherein the plurality of input channels comprise an indication of reconstructions samples, prediction samples, a boundary strength map, a second quantization parameter, and a first quantization parameter, and wherein d1 = 12, d2=8, d3=4, d4=2, d5=2, and d6 = 32 (as being defined at the lookup Headblock table in Figs.1, or 3, as parameters defined for d1…d5 extracted channels at the head block and the d6 would represent the number of fused output channels of the single output CONV, 1x1, d6 layer).
Re Claim 8. Shao and Wang disclose, the method of claim1,
Shao teaches about, wherein the NN-based filter further comprises a transition block combined with a fusion block, wherein the fusion block comprises a 1x1 convolution block having a first number of channels (d6) (the CONV 1x1, d6 in Figs. 1 and 3), wherein the transition block comprises a first separable convolution having a second number of channels (C1), a second separable convolution having a third number of channels (C2), and a third convolution having a fourth number of channels (C3), and wherein d6 = C1 = C2 = C3 (and finding obvious representations for the convolutional layers as C1 to the CONV 1x1, d6; the C2 for the CONV 2, 3x3, Cy+Cuv; and the C3 for the back bone block viewed as decomposition parameters in Fig.3).
Re Claim 9. This claim represents the device for coding video data, implementing each and every limitation of the coding process and comprising a memory an processors (Wang: a memory at Abstract, Sec.1, Sec.4, and a process Sec.1 or 2 Fig.1) of the method claim 1, hence it is rejected over the same mapped evidence mutatis mutandis.
Re Claim 10. This claim represents the device for coding video data, implementing each and every limitation of the coding process and comprising a memory an processors (Wang: a memory at Abstract, Sec.1, Sec.4, and a process Sec.1 or 2 Fig.1) of the method claim 2, hence it is rejected over the same mapped evidence mutatis mutandis.
Re Claim 11. This claim represents the device for coding video data, implementing each and every limitation of the coding process and comprising a memory an processors (Wang: a memory at Abstract, Sec.1, Sec.4, and a process Sec.1 or 2 Fig.1) of the method claim 3, hence it is rejected over the same mapped evidence mutatis mutandis.
Re Claim 12. This claim represents the device for coding video data, implementing each and every limitation of the coding process and comprising a memory an processors (Wang: a memory at Abstract, Sec.1, Sec.4, and a process Sec.1 or 2 Fig.1) of the method claim 14 hence it is rejected over the same mapped evidence mutatis mutandis.
Re Claim 13. This claim represents the device for coding video data, implementing each and every limitation of the coding process and comprising a memory an processors (Wang: a memory at Abstract, Sec.1, Sec.4, and a process Sec.1 or 2 Fig.1) of the method claim 5, hence it is rejected over the same mapped evidence mutatis mutandis.
Re Claim 14. This claim represents the device for coding video data, implementing each and every limitation of the coding process and comprising a memory an processors (Wang: a memory at Abstract, Sec.1, Sec.4, and a process Sec.1 or 2 Fig.1) of the method claim 6, hence it is rejected over the same mapped evidence mutatis mutandis.
Re Claim 15. This claim represents the device for coding video data, implementing each and every limitation of the coding process and comprising a memory an processors (Wang: a memory at Abstract, Sec.1, Sec.4, and a process Sec.1 or 2 Fig.1) of the method claim 7, hence it is rejected over the same mapped evidence mutatis mutandis.
Re Claim 16. This claim represents the device for coding video data, implementing each and every limitation of the coding process and comprising a memory an processors (Wang: a memory at Abstract, Sec.1, Sec.4, and a process Sec.1 or 2 Fig.1) of the method claim 8, hence it is rejected over the same mapped evidence mutatis mutandis.
4. Claim 17, is rejected under 35 U.S.C. 103 as being obvious over Shao and Wang in view of Zhao Wang et al., (hereinafter Zhao) (US 12,363,350).
Re Claim 17. Shao and Wang disclose, the device of claim 9, but do not expressly teach the presence of a display,
Zhao teaches the, wherein the device is configured to decode video data and wherein the device further comprises a display configured to display the picture (a display, Background).
The ordinary skilled in the art would have found obvious to combine the teachings of Shao and Wang performing video coding methods which inherently comprise a visual device, i.e., the display expressly taught in Zhao, by which the combination would have been found predictable.
5. Claims 18-20, are rejected under 35 U.S.C. 103 as being obvious over Zhao Wang et al., (hereinafter Zhao) (US 12,363,350) in view of Shao and Wang.
Re Claim 18. Zhao device for encoding video data (encoder at Detailed Description), the device comprising:
one or more memories configured to store a picture of video data (memory as a video data storage, Fig.1 and Description); and
one or more processors in communication with the one or more memories, the one or more processors (processors as part of the CPU at Detailed Description, Fig.13)configured to:
determine video data indicative of the picture (a video sequence 100 from the camera, Fig.1);
reconstruct a block of the video data to generate a reconstructed block (at the video encoding process, reconstructing a picture, Detailed Description); and Shao and Wang teach about, perform a neural network (NN)-based filter process on the reconstructed block to generate a filtered block, wherein the NN-based filter process comprises applying an NN-based filter comprising a constrained head block, the constrained head block having a plurality of input channels and being configured to independently extract, for each of the plurality of input channels, a respective number of output channels, and fuse outputs associated with each of the plurality of input channels into a single output, wherein a sum of the respective number of output channels is less than or equal to a number of fused output channels of the single output (these limitations find a similar evidentiary support at the mapped claim 1, applied mutatis mutandis).
The ordinary skilled in the art would have identified the NN based filtering the reconstructed video blocks disclosed at a pre/post-processing architecture at Fig.13, or Fig.14, in Zhao for representing the hardware structure supporting the coding in-loop filtering process similarly disclosed in Shao, and found obvious before the effective filing date of invention, and find the incentive to verify specific functional elements deemed to reduce the NN complexity suggested in Shao (at Fig.2) thus seeking explicit methods of such application as those found in Wang, teaching the change in the number of channels (Sec.2.1), by which such combination would be predictable.
Re Claim 19. Zhao in view of Shao and Wang disclose, the device of claim 18, wherein the NN-based filter further comprises:
Shao teaches about, a transition block having a first number of output features for luma and a second number of output features for chroma, wherein the first number is larger than the second number; a plurality of luma backbone blocks configured to process the output features for luma; and a plurality of chroma backbone blocks configured to process the output features for chroma (where each and every limitation of claim 2, are similarly applied mutatis mutandis).
Re Claim 20. Zhao in view of Shao and Wang disclose, the device of claim 18, Shao teaches about, wherein the NN-based filter further comprises a transition block including one or more convolutions and an activation block, the activation block being after all of the one or more convolutions of the transition block (the transition block of layers comprising the separable convolutions, sepCONV comprising the convolutions sepConv 1x3 and sepConv 3x1 fused at CONV 1x1 Cy being aggregated at PRELU for the Y luma NN core and similarly for the CONV 1x1 Cuv and aggregated PRELU of the chroma NN core per Fig.1 and 3, Sec.1 and Sec.2).
Conclusion
6. The prior art made of record and not relied upon, is considered pertinent to applicant's disclosure as cited below;
Shingala et al., AHG11: Complexity Reduction on Neural-Network
Loop Filter, Joint Video Experts Team (JVET) of ITU-T SG 16 WP3 and ISO/IEC JTC
1/SC 29 27th Meeting, by teleconference, 13-22 July 2022, Document: JVET-AA0080-v2.
Junru Li et al.,: “AGH11:Complexity Reduction of NN-based loop-filters”, JVET-AF0206-v2; 32nd Meeting, Hannover, DE, 13-20 Oct. 2023.
Tong Shao et al., (hereinafter Shao); “A Low Complexity Convolutional Neural Network with Fused CP Decomposition for In-Loop Filtering in Video Coding”, Dolby Laboratories, Inc., Sunnyvale, CA 94085, USA; DOI: 10.1109/DCC55655.2023.00032 © 2023 IEEE, or
US 2025/0119592.
See PTO-892 form. Applicant is required under 37 C.F.R. 1.111(c) to consider these references when responding to this action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DRAMOS KALAPODAS whose telephone number is (571)272-4622. The examiner can normally be reached on Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Czekaj can be reached on 571-272-7327. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DRAMOS KALAPODAS/Primary Examiner, Art Unit 2487
Prosecution Insights