Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5, and 10-18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wan, US 2021/0368211 A1.
Regarding claim 1, Wan discloses: a method of decoding video data, the method comprising:
decoding data for at least a portion of a picture of video data;
deriving, from the decoded data for the at least portion of the picture, a set of pixels for the at least portion of the picture and two or more sets of supplementary data (See auxiliary information 703 and 707 in figure 7. These sets are respective first and second sets of supplementary data, and as disclosed in [0038], this data can comprise block division information and/or QP information.), the two or more sets of supplementary data being separate from the pixels (See figure 7; auxiliary information is distinct from YUV pixel information.);
combining the two or more sets of supplementary data for the at least portion of the picture into a single set of supplementary data (See “input fusion unit 708 in figure 7. As disclosed in [0127], “That is, the first auxiliary information 703 and the second auxiliary information 707 may flexibly select the fusion stage in the concatenate processing structure, which is not specifically limited in the implementations of the present disclosure.”); and
executing a neural network filter, using the at least portion of the picture and the single set of supplementary data as inputs to the neural network filter, to filter the pixels for the at least portion of the picture (See step 304 in figure 3.).
Regarding claim 2, Wan discloses: the method of claim 1, wherein each of the sets of supplementary data corresponds to a respective two-dimensional plane (See [0085]), and wherein combining the two or more sets of supplementary data comprises concatenating each of the two-dimensional planes into a three-dimensional volume (See splitting-merging-splitting-merging-splitting processing structure, as in figure 7, and description in [0111]. Splitting and merging processes are concatenating.).
Regarding claim 3, Wan discloses: the method of claim 1, wherein each of the sets of supplementary data corresponds to a respective two-dimensional plane (See [0085], Specifically, the CU dividing information is converted into a Coding Unit Map (CU map), which is represented by a two-dimensional matrix.), and wherein combining the two or more sets of supplementary data comprises combining each of the two-dimensional planes into a single two-dimensional plane (See Input fusion unit 708, as disclosed in [0122], “input fusion unit 708 is configured to fuse the two new colour components and then input the same to the joint processing unit 709”).
Regarding claim 4, Wan discloses: the method of claim 3, wherein combining each of the two-dimensional planes comprises combining co-located values for each pixel sample position from each of the two-dimensional planes (See [0121], “The input fusion unit 708 is configured to fuse the three new colour components with the second auxiliary information 707, and then input the same to the joint processing unit 709.).
Regarding claim 5, Wan discloses: the method of claim 4, wherein combining the co-located values comprises calculating Planecomb(i,j) = function(Planea (i,j), Planeb(i,j), ... , Planen(i,j)), wherein the two or more sets of supplementary data comprise Planea, Planeb, . .. Planen (See [0121], disclosing fusion of Y, U, and V color components, and their respective auxiliary information for input to joint processing unit 706.).
Regarding claim 10, Wan discloses: the method of claim 3, further comprising performing one or more spatial dimension adjustments to the two or more two-dimensional input planes prior to combining the two-dimensional input planes (See [0123], where Wan discloses upsampling a U colour component and/or a V colour component to bring them into consistent resolution with the Y component, prior to fusion at joint processing unit 709.).
Regarding claim 11, Wan discloses: the method of claim 10, wherein the spatial dimension adjustments include one or more of upsampling (See “upsampling” of U and V components, disclosed in [0123]), downsampling, or padding.
Regarding claim 12, Wan discloses: the method of claim 1, further comprising performing a feature extraction process on one or more of the two or more sets of supplementary data prior to combining the two or more sets of supplementary data (See [0113], describing with respect to figure 5 a first layer of the convolutional kernel which performs feature extraction on the a.).
Regarding claim 13, Wan discloses: the method of claim 1, wherein the supplementary data includes one or more of coding unit (CU) partition information, prediction unit (PU) partition information, transform unit (TU) partition information, deblocking filter information, boundary strength (BS) information, long or short deblocking filter information, strong or weak deblocking filter information, quantization parameters (QPs) (See [0102], disclosing first auxiliary information may be one or more of block dividing information, quantization parameter information), intra-prediction information, inter-prediction information, distance between the picture and a reference picture for the picture, or motion information of coded blocks of the picture.
Regarding claim 14, Wan discloses: the method of claim 1, further comprising encoding the at least portion of the picture prior to decoding the at least portion of the picture (See [0038], “In an implementation of the present disclosure, the convolutional neural network filter 201 may be directly deployed at the encoding end and the decoding end after filtering network training, so there is no need to transmit any filter-related parameters.).
Device claims 15-18 are directed to a device implementing a method that corresponds, respectively, to the method claimed in claims 1-3, 12, and 13, respectively. Therefore claims 15-18 are rejected for the same reasons of anticipation as given above, respectively, for claims 1-3, 12, and 13.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6, 19, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wan in view of Galpin, US 2020/0213587
Regarding claim 6, Wan discloses: the method of claim 4, wherein combining the co-located values comprises calculating a weighted sum of co-located values of the supplementary data.
Galpin discloses in [0069] using a weighted average of the input of several CNNs having different QPs. A weighted average of co-located QPs for a current CU is then used to filter the image.
It would have been obvious to one having ordinary skill in the art before the applicant’s effective filing date to incorporate the feature disclosed in Galpin of using a weighted average of inputs from several CNNs to filter a current coding unit, in order to improve the accuracy of post-processing by aggregating the training results of several different convolutional neural networks. See Galpin [0069].
Regarding claim 19, Wan does not disclose explicitly: the device of claim 15, further comprising a display configured to display the picture.
However, Galpin discloses a display for use with a neural network-based post-processing filter. See figure 10 and [0083].
It would have been obvious to one having ordinary skill in the art before the time of the applicant’s effective filing date to incorporate a display device, as disclosed in Galpin. Doing so would have required only routine skill in the art, as Wan is directed to a video encoder/decoder for output to display devices. The combination would have merely entailed combining known prior art elements to achieve predictable results.
Regarding claim 20, Wan does not disclose: the device of claim 15, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box
However, Galpin discloses implementing a neural network postprocessing filter with “personal computers, laptop computers, smartphones, tablet computers, digital multimedia set top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers.”
It would have been obvious to one having ordinary skill in the art before the time of the applicant’s effective filing date to incorporate a display device, as disclosed in Galpin. Doing so would have required only routine skill in the art, as Wan is directed to a video encoder/decoder for output to display devices. The combination would have merely entailed combining known prior art elements to achieve predictable results. See MPEP 2143.I.A.
Claims 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Wan, in view Chen, US 2022/010960 A1
Regarding claim 8, Wan discloses the limitations of claim 3, upon which claim 8 depends. Wan does not disclose: the method of claim 3, further comprising performing one or more operations on values of the single two-dimensional plane (See [0087]-[0088], which discloses a clipping function applied to sample values of an image (2-D array) input to an adaptive loop filter.)
It would have been obvious to one having ordinary skill in the art before the time of the applicant’s effective filing date to incorporate a clipping function applied to the input image of the loop filter, as disclosed in Chen, in order to make ALF more efficient by reducing the impact of neighbor sample values that are too different with the current sample value. See Chen [0087]
Regarding claim 9, the combination of Wan in view of Chen discloses the limitations of claim 8, up: the method of claim 8, wherein the one or more operations include one or more of a clipping operation, a minimum operation, or a maximum operation (See [0087]-[0088] in Chen.).
Allowable Subject Matter
Claim 7 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 7, the method of claim 4, wherein combining the co-located values comprises calculating Planecomb (i,j) =A1* (QPSlice(i,j) - QPBase(i,j))/A2 + B1 *QPBase(i,j)/B2 +C1* B2(i,j)/C2) .
The closest prior art, Wan, discloses an in-loop filter for post processing video that uses a convolutional neural network to concatenate (fusing and splitting) auxiliary data such as QP data and/or coding unit boundary data, with color sample data. Wan discloses simply adding these co-located value (See [0101] of colour components and auxiliary data), but does not disclose combining them in the manner claimed in claim 7.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLE M LOTFI whose telephone number is (571)272-8762. The examiner can normally be reached 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Pendleton can be reached at 571-272-7527. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KYLE M LOTFI/ Examiner, Art Unit 2425