Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of Group 3 in the reply filed on 1/20/2026 is acknowledged.
Claim Interpretation
Claim 20 relates to a non-transitory computer-readable medium storing a bitstream formed by certain instructions executed by a processor. The process is analogous to storing a bitstream generated by a particular coding method. In both cases the limitations relate only to a bitstream which is stored on a computer readable medium after all coding functions have been performed. Thus the bitstream no longer has any functional relationship with a processor or other machine, but instead relates to e.g. a video stored on a hard drive. The purpose of the stored bitstream is only to convey meaning to a human viewer of the coded bitstream and is thus non-functional descriptive material. Non-functional descriptive material used only to convey messages or meaning is given little patentable weight (See MPEP 2111.05). For the purposes of examination, the examiner will interpret the claim as relating only to a computer readable medium storing data.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6 and 16-29 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al (2020/0404253) in view of Chen et al (2021/0084315 hereafter referred to by second inventor Liao in order to avoid confusion).
In regard to claim 1 Chen discloses a video coding method for motion refinement in video (Chen note DMVR technique shown in Fig. 13), comprising:
determining, by one or more processors, an initial motion vector of a video block of a video frame form the video (Chen Fig. 13 and par. 160 note DMVR technique including determining initial bi-prediction motion vectors (MV)s);
determining, by the one or more processors, a matching target based on a combination of a first reference block from a first reference frame in the video and a second reference block from a second reference frame in the video (Chen par. 160 note determining matching costs based on distortion between reference blocks, note the initial cost associated with the distortion of reference blocks at the initial motion vector location as the ‘matching target’);
performing, by the one or more processors, a bilateral matching based motion refinement process at a block level to obtain a refined motion vector is obtained for the video block (Chen par. 160 note local bilateral matching search to determine a refined MV); and
refining, by the one or more processors, a motion vector for each sub-block in the video block using the refined motion vector of the video block as a starting point for the motion vector for the sub-block, wherein refining the motion vector at a sub-block level applies an affine motion model of the video block (Chen pars 220-221 note applying DVMR to bi-directional affine coded blocks by individually refining the CPMVs for the affine coded block then refining the sub-block MVs in the motion vector field based on the refined CPMVs).
It is noted that Chen does not disclose details regarding a weighted combination of reference blocks, or iterative matching during motion vector refinement. However, Liao discloses a DMVR technique in which bi-lateral matching is based on a weighted combination of reference blocks which and uses iterative updating of an initial motion vector to determine a refined motion vector (Liao par. 112 note DMVR is applied to bi-prediction with equal weights, further note Figs 6-7 and pars 115-117 DMVR performed using an iterative search including updating the MV with the search result providing the smallest SAD value in each iteration to determine a refined motion vector).
It is therefore considered obvious that one of ordinary skill in the art before the effective filing date of the invention would recognize the advantage of using equally weighted prediction and iterative searching as taught by Liao in the DMVR process of Chen in order to gain the advantaged of reduced complexity motion refinement as suggested by Liao (Liao par. 115).
In regard to claim 2 refer to the statements made in the rejection of claim 1 above. Chen and Liao disclose determining the matching target further comprises:
determining a first weight for the first reference block and a second weight for the second reference block respectively (Liao par. 112 note reference blocks must use equal weighting); and
determining a combination of the first reference block and the second reference block (Chen par. 160 note reference blocks are combined by a subtraction operation to generate a single distortion measure)
In regard to claim 3 refer to the statements made in the rejection of claim 2 above. Liao further discloses the first weight and the second weight are identical to corresponding weights derived at an encoder side for weighted bi-predictions; or
the first weight and the second weight have predetermined values (Liao par. 112 note the weights of reference blocks in DMVR must be equal weights, or a predetermine value of 0.5).
In regard to claim 4 refer to the statements made in the rejection of claim 1 above. Chen and Liao further disclose that performing the bi-lateral matching based motion refinement process further comprises:
using the initial motion vector to initialize an intermediate motion vector (Chen par. 160-161 note DMVR process initialized by an initial motion vector further note par. 221 a CMPV of an affine coded block may be used as an initial motion vector);
determining a motion refinement for the intermediate motion vector based on the matching target (Chen pars. 160 and 221 note determining a motion refinement for each CPMV using the DMVR process); and
updating the intermediate motion vector based on the motion refinement (Chen par. 221 note using the refined CPMVs).
In regard to claim 5 refer to the statements made in the rejection of claim 4 above. Liao further discloses performing the bilateral matching based motion refinement process further comprises:
determining whether a predetermined iteration-stop condition is satisfied (Liao Figs 5-7 and pars 112-117 note par 116 and step 616 of fig. 6, determining if an iteration limit is reached);
responsive to the predetermined iteration-stop condition being satisfied determining the intermediate motion vector to be the refined motion vector (Liao par. 116 and Fig. 6 step 616 note if the maximum iterations is reached, the ‘No’ path from step 616 is followed and the motion vector is selected as the refined integer motion vector); or
responsive to the predetermined iteration-stop condition not being satisfied, continuing to iteratively determine the motion refinement for the intermediate motion vector and update the intermediate motion vector based on the motion refinement until the predetermined iteration-stop condition is satisfied (Liao par. 116 and Fig. 6 step 616 note if the iteration stop condition is not satisfied the ‘Yes’ path from step 616 is followed and motion refinement continues)
In regard to claim 6 refer to the statements made in the rejection of claim 5 above. Chen and Liao further disclose that the motion refinement is determined through a calculation based derivation, a search based derivation or a combination of the calculation based derivation and the search based derivation (Chen par. 221 note switching from an affine model to a translational motion model for each CPMV and using refined CPMVs in an affine model to refine a sub-block as calculation based refinements, further note Liao pars 112-117 DMVR process for search based derivation used in combination with the calculation based derivations of Chen).
In regard to claim 16 refer to the statements made in the rejection of claim 6 above. Chen and Liao further disclose that the motion refinement is determined through the combination of the calculation based derivation and the search based derivation, and wherein performing the bilateral matching based motion refinement process comprises:
determining the motion refinement for the intermediate motion vector through the calculation based derivation based on the matching target and updating the intermediate motion vector based on the motion refinement determined through the calculation based derivation(Chen par. 221 note determining an intermediate motion vector by obtaining the CPMVs of the affine model, and converting them a translational motion model by in order to refine them individually);
determining the motion refinement for the intermediate motion vector again through the search based derivation based on the matching target (Chen par. 221 note translational motion based CPMVs used as an input vector into a DVMR process, Liao Fig. 6 and pars 112-117 note refining an initial motion vector using a search based derivation); and
updating the intermediate motion vector again based on the motion refinement through the search based derivation (Liao Fig. 6 and pars 112-117 note the initial motion vector is updated after each search iteration).
In regard to claim 17 refer to the statements made in the rejection of claim 1 above. Chen and Liao further disclose that the bilateral matching based motion refinement process is performed to obtained the refined motion vector when one of the following conditions is satisfied:
one of the first reference frame and the second reference frame is preceding to the video frame and another of the first reference frame and the second reference frame is after the video frame (Chen Fig. 13 note past and future reference frames, Liao Fig. 5 note past and future reference frames); or
both the first reference frame and the second reference frame are preceding to or after the video frame, and a temporal distance between the first reference frame and the second reference frame meets a predetermined threshold.
In regard to claim 21 refer to the statements made in the rejection of claim 16 above. Chen further discloses determining the motion refinement for the intermediate motion vector through the calculation based derivation based on the matching target comprises:
determining, based on the intermediate motion vector, a current prediction of the video block (Chen par. 221 note using CPMVs individually as initial motion vectors for DMVR, note par. 160 a prediction is obtained for the initial motion vector);
determining an assumed motion model between the current prediction and the matching target, wherein the assumed motion model is used for motion refinement calculation (Chen pars. 160 and 221 note refining each CPMV using a translational motion model); and
calculating the motion refinement for the intermediate motion vector based on the assumed motion model (Chen pars 160 and 221 note refined CPMVs determined using the translational based DMVR process).
In regard to claim 22 refer to the statements made in the rejection of claim 21 above. Liao further discloses that the predetermined iteration stop-condition is satisfied if the intermediate motion vector converges or a total number of iterations meets a predetermined threshold (Liao Fig. 6 and par. 116 note search iterations stop if the MVdiff between iterations is zero or if a maximum number of iterations is reached).
In regard to claim 23 refer to the statements made in the rejection of claim 21 above. Chen further discloses a total number of parameters of the assumed motion model is equal to a total number of parameters of the affine motion model (Chen par. 221 note refined CPMVs are used to determine a refined sub-block motion field in a using the same affine model as the initial affine prediction, further note pars 162-166 note affine motion models may use 4 or 6 parameters).
In regard to claim 24 refer to the statements made in the rejection of claim 21 above. Chen further discloses a total number of parameters of the assumed motion model is different form a total number of parameters of the affine motion model (Chen par. 221 note using a translational motion model to refine individual CMPVs using DMVR, note translation is a 2 parameter model as it consists of horizontal and vertical translation, further note pars. 162-166 the affine model from which the CPMVs are converted is either a 4 or 6 parameter model).
In regard to claim 25 refer to the statements made in the rejection of claim 16 above. Liao further discloses determining the motion refinement for the intermediate motion vector again through the search based derivation based on the matching target comprises:
generating a first modified motion vector based on the intermediate motion vector and a first motion vector change within a predetermined search range Liao Fig. 7 and par 117 note a first search iteration is performed based on a first search range of adjacent blocks around an intermediate motion vector, and a modified vector is selected corresponding to a minimum SAD value); and
determining whether to assign the first motion-vector change as the motion refinement based on a matching cost associated with the intermediate motion vector and a current matching cost associated with the first modified motion vector (Liao Fig. 7 and par. 117 note if the minimum SAD corresponds to the center search position the current motion vector is used without further change, if the center position does not have the minimum SAD value the current motion vector is changed to the modified vector with the new minimum SAD value).
In regard to claim 26 refer to the statements made in the rejection of claim 25 above. Liao further discloses that the current matching cost associated with the first modified motion vector is determined by:
determining a current prediction of the video block based on the first modified motion vector (Liao Fig. 7 note determining predictors for each location identified in the search area) ; and
determining the current matching cost associated with the first modified motion vector based on the matching target and the current prediction of the video block (Liao par. 117 not determining a SAD value for each predictor).
In regard to claim 27 refer to the statements made in the rejection of claim 25 above. Liao further discloses that responsive to the current matching cost associated with the first modified motion vector being less than the matching cost associated with the intermediate motion vector, deriving the motion refinement to be the first motion-vector change so that the intermediate motion vector is updated to be the first modified motion vector (Liao Fig. 7 and par. 117 note determining SAD values for a current, central motion vector and motion vectors associated with adjacent blocks, when the minimum SAD corresponds to a block that is not the central block pointed to by the current intermediate motion vector, then current vector is updated to the vector associated with the minimum SAD value).
In regard to claim 28 refer to the statements made in the rejection of claim 25 above. Liao further discloses:
responsive to the current matching cost associated with the first modified motion vector being equal to or greater than the matching cost associated with the intermediate motion vector, not assigning the first motion-vector change as the motion refinement (Liao Fig. 7 and par. 117 note if the SAD value associated with a first adjacent block, e.g. P1, is not less than the central block P0, then it is not selected as a new refined motion vector);
generating a second modified motion vector based on the intermediate motion vector and a second motion-vector change within the predetermined search range and determining whether to assign the second motion-vector change as the motion refinement based on the matching cost associated with the intermediate motion vector and another current matching cost associated with the second modified motion vector (Liao Fig. 7 and par. 117 note computing a SAD value for another adjacent block, e.g. P2, and determining whether the SAD is less than the SAD associated with the central block P0 and if so assigning P2 as the new current motion vector).
In regard to claim 29 refer to the statements made in the rejection of claim 25 above. Liao further discloses that the predetermined iteration-stop condition is satisfied if available motion-vector changes within the predetermined search range are detected and processed or a total number of iterations satisfies a predetermined threshold (Liao Figs. 6-7 and pars 116-117 note stop condition is satisfied if the maximum iterations are reached, or the selected motion vector does not change between iterations).
Claims 18-19 relate to a video coding apparatus including a memory and a processor that implement process steps corresponding to the method described in claims 1-2 above. Refer to the statements made in regard to claims 1-2 above for the rejection of claims 18-19 which will not be repeated here for brevity. In particular regard to claim 18 Chen further discloses a memory and a processor (Chen pars 81-83 and par. 110 note processor and memory used in encoder and decoder)
In regard to claim 20 Chen further discloses a non-transitory computer readable storage medium storing a bitstream formed by instructions executed by a processor (Chen Fig. 1 and par. 46 note storage device 112 comprising non-transitory media, further note par. 43 bitstream generated by encoding is stored on the media 112).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEREMIAH CHARLES HALLENBECK-HUBER whose telephone number is (571)272-5248. The examiner can normally be reached Monday to Friday from 9 A.M. to 5 P.M.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached at (571)272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEREMIAH C HALLENBECK-HUBER/Primary Examiner, Art Unit 2481
Prosecution Insights