DETAILED ACTION
Claims 39-57 are pending for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant's claim under US PRO 63/330035 filed on 4/12/2022.
Response to Arguments
Applicant's arguments filed 5/11/2026 have been fully considered but they are not persuasive. Applicant argues on pages 10-11 that Wang does not sufficiently disclose prongs (i)-(iii) in an NN/ML model found in claim 39. Examiner respectfully disagrees.
Examiner points to Wang (citations include Page 2, Fig.2 and Sec 2.1) that illustrates and teaches a NN filter. The document states the network inputs at least a reconstructed image and a prediction image. The disclosed prediction image (pred_yuv) allows for the generated image to an inter- or an intra-predicted image, representing the predicted portions of an image through a reference/compensated block. With respect to the input to the ML Model, the images mentioned in the first paragraph of 2.1 (Network Architecture) are being fed into the NN (ML model). Meeting the conditions in (i)-(iii), where only one condition needs to be met, and being an input for the NN (ML Model). The Examiner disagrees and maintains that the Wang reference teaches the portions of claims 39 and 54, as indicated below.
Applicant’s arguments, see Arguments on pages 12 and 13, filed 5/11/2026, with respect to the 35 USC 103 rejection of claim 56 have been fully considered and are persuasive. The 35 USC 103 rejection of claim 56 has been withdrawn and is now indicated as being allowed.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 39, 50, 54 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al (AHG11: Neural Network based in-loop filter with constrained storage and low complexity), in view of Zhang et al, US 2022/0335269 A1.
Regarding Claim 39, Wang discloses a method for generating an encoded video or a decoded video, the method comprising:
obtaining values of reconstructed samples (Wang Fig.2, page 2 – reconstructed image [rec_yuv]);
obtaining input information comprising: i) prediction mode information indicating that a filtered sample block is an intra-predicted block, an inter-predicted block that is uni-predicted, or an inter-predicted block that is bi-predicted, ii) motion vector information indicating a number of motion vectors used for prediction, and/or iii) information about skipped samples (Wang Fig.2, page 2 – prediction image [pred_yuv]);
providing the values of reconstructed samples and the input information to a machine learning (ML) model, thereby generating at least one ML output data (Wang Fig.2, Sec 2.1 page 2 – Proposed NN filter shown in Fig.2 and – see inputs and output_yuv).
Even though Wang teaches a NN based in-loop filter, Wang does not explicitly disclose based at least on said at least one ML output data, generating the encoded video or the decoded video.
Zhang teaches based at least on said at least one ML output data, generating the encoded video or the decoded video (Zhang [0042] – The difference between the weights of the finetuned neural network and the weights of the neural network before finetuning is referred to as the weight-update. This weight-update needs to be encoded, provided to the decoder side together with the encoded video data, and used at the decoder side for updating the neural network filter. The updated neural network filter is then used as part of the video decoding process).
Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to modify Wang to have at least one ML output data, generating the encoded video or the decoded video, as taught by Li. One would be motivated as the means encoded/decoded video data would what is being generated with the input information and the ML model.
With regard to claim 50, the claim limitations are essentially the same as claim 39 but in a different embodiment. Therefore, the rational used to reject claim 39 is applied to claim 50.
With regard to claim 54, the claim limitations are essentially the same as claim 39 but in a different embodiment. Therefore, the rational used to reject claim 39 is applied to claim 54.
Claim(s) 45 and 46 are rejected under 35 U.S.C. 103 as being unpatentable over Wang and Zhang, in view of Li et al US 2022/0101095 A1.
Regarding Claim 45, Wang and Zhang teach the method of claim 39, as outlined above.
However, Wang does not explicitly disclose the information about filtered samples comprises values of deblocked samples.
Li teaches the information about filtered samples comprises values of deblocked samples (Li [0146] – One or more convolutional neural network (CNN) filter models are trained as an in-loop filter or post-processing method for reducing the distortion incurred during compression. The interaction between the CNN filtering and the non-deep learning-based filtering method denoted by NDLF, controlling of our CNN filtering method, CNN filter models will be discussed in this invention. In one example, the NDLF may include one or more of Deblocking filter, SAO, ALF, CC-ALF, LMCS, bilateral filter, transform-domain filtering method, etc. al; [0060] – FIG. 5 shows an example of encoder block diagram of VVC, which contains three in-loop filtering blocks: deblocking filter (DF), sample adaptive offset (SAO) and ALF).
Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to modify Wang to have the information about filtered samples comprises values of deblocked samples, as taught by Li. One would be motivated as the deblocking samples assist quality by assisting edges in the blocks.
Regarding Claim 46, Wang and Zhang teach the method of claim 39, as outlined above.
However, Wang does not explicitly disclose the information about skipped samples indicates whether samples belong to a block that did not go through a process processing residual samples, and the process comprises inverse quantization and inverse transformation
Li teaches the information about skipped samples indicates whether samples belong to a block that did not go through a process processing residual samples, and the process comprises inverse quantization and inverse transformation (Li [0138]-[0143] – The current CNN-based loop filtering has the following problems: 4. CNN-based loop filters in prior-arts are utilized on all of reconstructed frames, causing the frames coded later to be overly filtered a. For example, in the Random Access (RA) configuration, blocks in frames within high temporal layers may choose skip mode with a large probability, which means that the reconstruction of current frame is copied from the previous reconstruction frames. Since the previous frames are filtered using the CNN-based loop filter, applying the CNN-based loop filter on the current frame is equivalent to applying the CNN-filter twice on same content).
Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to modify Wang to have information about skipped samples indicates whether samples belong to a block that did not go through a process processing residual samples, and the process comprises inverse quantization and inverse transformation, as taught by Li. One would be motivated as the skipped samples reduce processing power.
Allowable Subject Matter
Claim 51-53, 55-57 are allowed. The closest prior arts are the Wang, Zhang, and Li references cited above.
Neither Wang, Zhang, nor Li nor other relevant art or combination of relevant art, teaches a method and an apparatus for generating an encoded video or a decoded video, the method comprising: obtaining machine learning (ML) input data, wherein the ML input data comprises: i) values of reconstructed samples; ii) values of predicted samples; iii) block boundary strength (BBS) information indicating strength of a filtering applied to a boundary of samples; and iv) quantization parameters (QPs); providing the ML input data to a ML model, thereby generating ML output data; and generating, based at least on the ML output data, the encoded video or the decoded video, wherein the ML input data does not include partition information indicating how a luma picture is partitioned into coding tree units (CTUs) and how luma CTUs are partitioned into coding units (CUs), the ML model comprises a first computational model (CM), and the first CM comprises a first convolution layer (CL) and a first parametric rectified linear unit (PReLU) coupled to the first CL.
Further, neither Wang, Zhang, nor Li nor other relevant art or combination of relevant art, teaches an apparatus comprising: memory; and processing circuitry, wherein the apparatus is configured to: obtain values of reconstructed samples; obtain quantization parameters, QPs; provide the reconstructed sample values and the quantization parameters to a machine learning, ML, model, thereby generating ML output data; generate, based at least on the ML output data, first output sample values; provide the first output sample values to a group of two or more attention residual blocks connected in series, thereby generating second output sample values; and generate the encoded video or the decoded video based on the second output sample values, wherein the group of attention residual blocks comprises a first attention residual block disposed at one end of the series of attention residual blocks, and the first attention residual block is configured to receive input data consisting of the first output sample values and the QPs.
Further, neither Wang, Zhang, nor Li nor other relevant art or combination of relevant art, teaches an apparatus comprising: memory; and processing circuitry, wherein the apparatus is configured to: obtain machine learning (ML) input data, wherein the ML input data comprises: i) values of luma components of reconstructed samples; ii) values of chroma components of reconstructed samples; iii) values of luma components of predicted samples; iv) values of chroma components of predicted samples; v) first block boundary strength (BBS) information indicating strength of a filtering applied to a boundary of luma components of samples; vi) second BBS information indicating strength of a filtering applied to a boundary of chroma components of samples; and (vii) quantization parameters (QPs); provide the ML input data to a ML model, thereby generating ML output data; and generate, based at least on the ML output data, the encoded video or the decoded video.
Claims 40-44 and 47-49 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMIR SHAHNAMI whose telephone number is (571)270-0707. The examiner can normally be reached Monday - Friday 8:00 am to 4:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Joseph Ustaris can be reached at 571-272-7383. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMIR SHAHNAMI/ Primary Examiner, Art Unit 2483
Prosecution Insights