DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 1/06/2026 and 5/04/2026 were filed after the mailing date of the Non-Final Rejection on 12/04/2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-12 and 14-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over HANNUKSELAet al (EP 3633990 A1) in view of YANG (US 20190124348 A1).
Regarding claim 1, FIG. 5-9 of HANNUKSELA disclose a method for video processing [e.g. FIG. video encoding/decoding], comprising: determining, for a conversion between a current video block [e.g. FIG. 5; an input image block] of a video and a bitstream of the video [e.g. FIG. 5; bitstream], a distortion value [e.g. distortion between the reconstructed image block and the input image block] of the current video block based on a set of distortion metrics [e.g. FIG. 5; 510], the set of distortion metrics comprising at least one of: a first distortion metric determined according to a first machine learning model [e.g. FIG. 5-6; performing the rate-distortion optimization for both humans and machines by minimizing also the distortion or error of the neural network], a second distortion metric determined according to a second machine learning model [e.g. FIG. 5-6], and performing the conversion based on the distortion value [e.g. FIG. 5-6]. HANNUKSELA further disclose in accordance with a determination that the first machine learning model is applied for at least one of a plurality of candidate modes of the current video block [e.g. HANNUKSELA; FIG. 5-6 and 9; A different set of parameters is trained for inter frames within the CNN].
It is noted that HANNUKSELA differs to the present invention in that HANNUKSELA fails to explicitly disclose a third distortion metric and in accordance with a determination that the first machine learning model is applied for at least one of a plurality of candidate modes of the current video, determining the first distortion metric as the distortion value; and in accordance with a determination that the first machine learning model is not applied for at least one of the plurality of candidate modes of the current video block, determining the third distortion metric as the distortion value.
However, YANG teaches the well-known concept of a method for video processing comprising a first distortion metric determined according to a first machine learning model [e.g. Figs. 10-11; ML1, performing the machine learning based prediction for distortion optimization processing], a second distortion metric determined according to a second machine learning model [e.g. ML2, MLn], or a third distortion metric determined without using the first and second machine learning models [e.g. ML3…]; and in accordance with a determination that the first machine learning model is applied for at least one of a plurality of candidate modes of the current video, determining the first distortion metric as the distortion value [See Figs. 10-11, para. 86, 95-97, e.g. using ML]; and in accordance with a determination that the first machine learning model is not applied for at least one of the plurality of candidate modes of the current video block, determining the third distortion metric as the distortion value [See Figs. 10-11, para. 86, 95-97; e.g. using non-ML].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the video coding system disclosed by HANNUKSELA to exploit the well-known a machine learning based prediction technique taught by YANG as above, in order to provide an enhanced prediction block by applying a machine learning technique to the prediction block [See YANG; abstract and [0005-0008]].
Regarding claim 2, wherein the plurality of candidate modes are not partitioning modes [e.g. YANG: FIG. 3-5, 10-11].
Regarding claim 3, HANNUKSELA and YANG further disclose in accordance with a determination that the second machine learning model is applied for at least one of a plurality of candidate modes of the current video block, determining a minimum one of the second and third distortion metrics as the distortion value; and in accordance with a determination that the second machine learning model is not applied for at least one of the plurality of candidate modes of the current video block, determining the distortion value based on the first and third distortion metrics.
Regarding claim 4, HANNUKSELA and YANG further disclose in accordance with a determination that the first machine learning model is applied to the plurality of candidate modes, determining the first distortion metric as the distortion value; and in accordance with a determination that the first machine learning model is not applied to the plurality of candidate modes, determining the third distortion metric as the distortion value [e.g. HANNUKSELA; FIG. 5-6 and 9; A different set of parameters is trained for inter frames within the CNN; YANG: FIG. 3-5; ML for inter or intra mode].
Regarding claim 5, HANNUKSELA and YANG further disclose determining the distortion value based on a combination of the set of distortion metrics, wherein the combination of the set of distortion metrics is determined based on at least one of: coding statistics of the current video block [e.g. YANG: YANG: FIG. 5-6; the type of the machine learning can be selected by the MLBE block 120 according to the coding information MLS_Info, and the parameter set], a first usage of the first machine learning model, a second usage of the second machine learning model, or a priority order of the set of distortion metrics.
Regarding claim 6, HANNUKSELA and YANG further disclose determining the priority order based on at least one of the following: a coding mode of the current video block [e.g. YANG: YANG: FIG. 3-5; ML2 for intra and ML3 is for inter mode], or coding statistics of the current video block.
Regarding claim 7, HANNUKSELA and YANG further disclose the coding statistics comprises at least one of: a prediction mode of the current video block [e.g. HANNUKSELA; FIG. 5-6 and 9; A different set of parameters is trained for inter frames within the CNN; YANG: FIG. 3-5; ML for inter or intra mode], a type of the prediction mode, a quantization parameter (QP) of the current video block, a temporal layer of the current video block, or a slice type of the current video block.
Regarding claim 8, HANNUKSELA and YANG further disclose a fourth distortion metric comprises a minimum one of the second and third distortion metrics [e.g. HANNUKSELA; FIG. 5-6 and 9; rate-distortion optimization process; YANG: FIG. 3-5; ML1, 2, 3, 4,5…], a priority of the fourth distortion metric is higher than a priority of the first distortion metric, or wherein a priority of the first distortion metric is higher than a priority of the third distortion metric [e.g. YANG: FIG. 5-6].
Regarding claim 9, HANNUKSELA and YANG further disclose a plurality of candidate modes of the current video block are partitioning modes [e.g. HANNUKSELA; FIG. 6; YANG: FIG. 3-5; the MLBE block 120 may select an optimum machine learning technique with reference to a variety of information, such as a prediction mode, a feature of a motion vector, a partition form of an image], and the combination of the set of distortion metric comprises the first, the second and the third distortion metric [e.g. HANNUKSELA; FIG. 5-6 and 9; YANG: FIG. 3-5].
Regarding claim 10, HANNUKSELA and YANG further disclose the first machine learning model comprises one of the following: a deblocking filter , a sample adaptive offset (SAO), or an adaptive loop filer (ALF) [e.g. HANNUKSELA; FIG. 4 and 6; deblocking, sample adaptive offset (SAO), and/or adaptive loop filtering (ALF); YANG: FIG. 3-5; [0047]; at least one of a deblocking filter, a sample adaptive offset (SAO) filter, or an adaptive loop filter (ALF) to a reconstruction block or a reconstruction picture].
Regarding claim 11, HANNUKSELA and YANG further disclose the second machine learning model comprises a convolutional neural network (CNN) model [e.g. HANNUKSELA; FIG. 5-6; some neural network architectures only units in adjacent layers are connected; convolutional layers].
Regarding claim 12, HANNUKSELA and YANG further disclose the first machine learning model is the same with the second machine learning model [e.g. HANNUKSELA; FIG. 5-6; YANG: FIG. 3-5; ML for inter prediction mode or for motion vector].
Regarding claim 14, HANNUKSELA and YANG further disclose a rate-distortion optimization (RDO) process [e.g. HANNUKSELA: rate distortion optimization (RDO); YANG: enhanced prediction block using a rate-distortion optimization (RDO) value] on the current video block, wherein performing the RDO process on the current video block comprises: performing the RDO process on a plurality of candidate modes based on a rate-distortion cost [e.g. HANNUKSELA; FIG. 5-6; YANG: FIG. 3-5], wherein the rate-distortion cost is determined based on a sum of the distortion value and a weighted rate of the current video block, wherein a number of the plurality of candidate modes comprises one of: 1, 2, 3 or 4 [e.g. YANG: FIG. 3-5; ML1, ML2,…MLn].
Regarding claim 15, HANNUKSELA and YANG further disclose the set of distortion metrics further comprises at least one of the following: a minimum one of the third distortion metric and a fifth distortion metric determined according to one of a plurality of machine learning models [e.g. HANNUKSELA; FIG. 5-6; YANG: FIG. 3-5], the plurality of machine learning models comprising the first and second machine learning models, a sixth distortion metric determined according to a predefined or selected model of the plurality of machine learning models, a scaled metric of the third distortion metric, or a weighted sum of the first, second, third, fifth or sixth metric.
Regarding claim 16, HANNUKSELA and YANG further disclose the conversion includes encoding the current video block into the bitstream [e.g. HANNUKSELA: FIG. 4 and 6-7; encoder].
Regarding claim 17, HANNUKSELA and YANG further disclose the conversion includes decoding the current video block from the bitstream [e.g. HANNUKSELA: FIG. 4 and 6-7; decoder].
Regarding claim 18, this is an apparatus that includes same limitation as in claim 1 above, the rejection of which are incorporated herein.
Regarding claim 19, this is a non-transitory computer-readable storage medium that includes same limitation as in claim 1 above, the rejection of which are incorporated herein.
Regarding claim 20, this is a method that includes same limitation as in claim 1 and 18 above, the rejection of which are incorporated herein.
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over HANNUKSELAet al (EP 3633990 A1) in view of YANG (US 20190124348 A1) and KIM et al (US 20220182618 A1).
Regarding claim 13, HANNUKSELA and YANG further disclose a concept of index of the selected candidates [e.g. HANNUKSELA: index to select a motion vector predictor from the candidate list] or index of machine learning [e.g. YANG: ML1, ML2,…MLn, n is an integer], but HANNUKSELA and YANG fail to explicitly disclose the detail of the index of ML.
However, KIM teaches the well-known concept of for a first index of the first machine learning model is the same with a second index of the second machine learning model [e.g. FIG. 20-23; a filter using artificial neural network models trained differently depending on a classification index; perform learning so that the output of the artificial neural network becomes identical to the classification index stored in the database of the artificial neural network].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the video coding system disclosed by HANNUKSELA to exploit the well-known a machine learning based prediction technique taught by YANG and the well-known image encoding/decoding technique taught by KIM as above, in order to provide an enhanced prediction block by applying a machine learning technique to the prediction block [See YANG; abstract and [0005-0008]] and reduced distortion attributable to image encoding [See KIM; [0009]].
Response to Arguments
Based the on amendments made to claim 20, the 102 rejection is withdrawn.
Applicant's arguments filed 3/04/2026 have been fully considered but they are not persuasive.
Applicant argues with respect to claims 1 and 18-20 that Yang doesn’t not disclose “in accordance with a determination that the first machine learning model is applied for at least one of a plurality of candidate modes of the current video, determining the first distortion metric as the distortion value; and in accordance with a determination that the first machine learning model is not applied for at least one of the plurality of candidate modes of the current video block, determining the third distortion metric as the distortion value”. The examiner respectfully disagrees. Yang discloses in accordance with a determination that the first machine learning model is applied for at least one of a plurality of candidate modes of the current video, determining the first distortion metric as the distortion value [See Figs. 10-11, para. 86, 95-97, e.g. using ML]; and in accordance with a determination that the first machine learning model is not applied for at least one of the plurality of candidate modes of the current video block, determining the third distortion metric as the distortion value [See Figs. 10-11, para. 86, 95-97; e.g. using non-ML]. Furthermore, Yang disclose that each block has a rate-distortion optimization (RDO) value that indicates its coding efficiency. The system uses the RDO value to make a selection that provides a better compression efficiency [See Yang para 86].
Based on at least the foregoing remarks, the 103 rejection claims 1-20 are maintained.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Joseph G Ustaris whose telephone number is (571)272-7383. The examiner can normally be reached 9-5pm M-Th.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Colleen A Fauz can be reached at 571-272-1667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSEPH G USTARIS/Supervisory Patent Examiner, Art Unit 2483
Prosecution Insights