DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is responsive to the correspondence filled on 09/03/2026.
Claims 36-55 are presented for examination.
IDS Considerations
The information disclosure statement (IDS) submitted on 09/03/2026 is/are being considered by the examiner as the submission is in compliance with the provisions of 37 CFR 1.97.
Response to Arguments
Applicant's arguments filed 09/03/2026 with respect to claims 36-55 have been considered but are moot in view of the new ground(s) of rejection.
Claim amendments is related to nearly every limitation recited in independent claims and is now rejected using a new reference Huang (U.S. Pub. No. 20200288156 A1) because of scope change. Examiner notes that few other limitations of dependent and independent claims are also taught by Huang. As such those limitations are remapped using Huang to reduce the number of reference required for rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 36-37, 40, 42-45, 48, 50-53 and 54 is/are rejected under 35 U.S.C. 103 as being unpatentable over Huang (U.S. Pub. No. 20200288156 A1), in view of Zhou (U.S. Pub. No. 20220232204 A1).
Examiner’s Note: Video encoding and decoding are done using same and opposite algorithm.
Regarding to claim 36, 44 and 52:
36. Huang teach a video decoding device, comprising: a processor configured to: (Huang Fig. 1 [0022] FIG. 1 is a block diagram illustrating an example video encoding and decoding system 100 that may perform the affine motion compensation techniques of this disclosure. The techniques of this disclosure are generally directed to coding (encoding and/or decoding) video data, including techniques for coding video data using affine motion compensation. In general, video data includes any data for processing a video. Thus, video data may include raw, uncoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data. [0036] Video encoder 200 and video decoder 300 each may be implemented as any of a variety of suitable encoder and/or decoder circuitry, such as one or more microprocessors, digital signal processors (DSPs). [0122] In this manner, video decoder 300 represents an example of a video decoding device including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to receive the block of video data to be decoded using a subblock affine motion compensation mode, determine a size of one or more subblocks of the block based on one or more of an inter prediction direction or affine motion parameters of the block, partition the block into the one or more subblocks based on the determined size, and decode the one or more subblocks using the subblock affine motion compensation mode.) partition a video block into a plurality of sub-blocks; (Huang [0126] Given an affine motion model for a block, different motion vectors can be derived for each pixel in the block. Therefore, motion compensation can be performed pixel-by-pixel. However, to reduce the complexity, subblock based motion compensation may be used, wherein video encoder 200 and video decoder 300 partitions the block into multiple subblocks (e.g., where each subblock has a smaller block size than the original block) and each subblock is associated with one motion vector for motion compensation. Video encoder 200 and video decoder 300 may derive the motion vector for each subblock using a representative coordinate of the subblock. Typically, the center position of the subblock is used. [0127] In one example, video encoder 200 and video decoder 300 may partition a block into non-overlapping subblocks. The block width is blkW, the block height is blkH, the subblock width is sbW, and the subblock height is sbH. There are blkH/sbH rows of subblocks and blkW/sbW subblocks in each row. For a six-parameter affine motion model, video encoder 200 and video decoder 300 may derive the motion vector for the subblock (referred to as subblock MV) at ith row (0<=<blkW/sbW) and the j.sub.th (0<=j<blkH/sbH) column as:)
determine whether to split the sub-block for motion compensation, (Huang [0151] In a second example, video encoder 200 and video decoder 300 may be configured to determine the subblock size based on the affine motion parameters. If the affine motion parameters satisfy a first criterion, video encoder 200 and video decoder 300 set the subblock size to a first subblock size (sbW1, sbH1). Otherwise, if the affine motion parameters satisfy a second criterion, video encoder 200 and video decoder 300 set the subblock size to a second subblock size (sbW, sbH). Otherwise, if the affine motion parameters satisfy neither the first or second criterion, video encoder 200 and video decoder 300 set the subblock size to a third subblock size (sbW2, sbH2). In this example, sbW1*sbH1<sbW*sbH<sbW2*sbH2.) wherein the determination is based on a motion vector comparison associated with the sub-block; (Huang [0132] In another example, video encoder 200 and video decoder 300 store a MV field on a 4×4 block basis. That is, for each 4×4 subblock of a block of video data, video encoder 200 may determine and signal a motion vector. Both video encoder 200 and video decoder 300 may then calculate the specific motion vectors for the subblocks smaller than 4×4 using the following techniques. For example, video encoder 200 and video decoder 300 may calculate the motion vector of smaller subblocks within a 4×4 block by the motion vector {right arrow over (v)}=(v.sub.x, v.sub.y) stored for the 4×4 block and the given affine motion parameters {a, b, c, d}. Video encoder 200 and video decoder 300 may calculate the given affine motion parameters {a, b, c, d} based on the control-point motion vectors of the current coding block, video encoder 200 and video decoder 300 may calculate the given affine motion parameters based on four 4×4 subblocks in an 8×8 area, or video encoder 200 and video decoder 300 may calculate the given affine motion parameters based on three selected subblock MVs in the current coding block) based on a determination to split the sub-block, split the sub-block into a plurality of units, (Huang [0156] In a fifth example, video encoder 200 and video decoder 300 may be configured to determine the subblock size for each 8×8 area of a block based on the inter prediction direction and the four subblock motion vectors in the 8×8 area (e.g., the four subblock motion vectors in the 4×4 motion field subblocks as described above with reference to FIG. 7). As shown in FIG. 8, the four subblock motion vectors of 8×8 area 800 are {right arrow over (v)}.sub.i=(v.sub.ix, v.sub.iy) i=0,1,2,3. Video encoder 200 and video decoder 300 may calculate the parameters {a, b, c, d} as:
PNG
media_image1.png
279
180
media_image1.png
Greyscale
Here the parameters governing the size decision, because [0157] Then, similar approaches as described in previous examples above could be applied. The new motion vector(s) that is(are) used for motion compensation for the subblock(s) can be derived using the calculated parameters {a, b, c, d} and {right arrow over (v)}.sub.i. For example, if video encoder 200 and video decoder 300 determine the subblock size to be 8×8, then the new subblock motion vector can be derived as ({right arrow over (v)}.sub.0+{right arrow over (v)}.sub.3)/2, or ({right arrow over (v)}.sub.1+{right arrow over (v)}.sub.2)/2,or ({right arrow over (v)}.sub.0+{right arrow over (v)}.sub.1+{right arrow over (v)}.sub.2+{right arrow over (v)}.sub.3)/4, i.e., the average of some of the {right arrow over (v)}.sub.i. Some rounding may also be applied. If video encoder 200 and video decoder 300 determine the subblock size to be 2×2, then the subblock motion vector at the (i,j)'s subblock is derived as {right arrow over (v)})
wherein each unit of the plurality of units is that are smaller than the sub-block (Huang [0084] regardless of how small the subblocks are for performing the subblock affine motion compensation mode, video encoder 200 signals motion vectors at a 4×4 level. Affine unit 225 may then calculate the specific motion vectors for each actual subblock (e.g., 2×2, 4×2, or 2×4) of the 4×4 motion field subblocks from the affine motion parameters and the motion vector in the 4×4 motion field subblock. Accordingly, since the motion vectors are signaled at a 4×4 level rather than a smaller level (e.g., 2×2, 4×2, or 2×4), signaling overhead is reduced and coding efficiency may be increased. [0154] In one example, (sbW, sbH)=(4, 4), (sbW1, sbH1)=(2, 2) and (sbW2, sbH2)=(8, 8).) and, wherein each of the plurality of units comprises a plurality of pixels; (Huang [0126] Given an affine motion model for a block, different motion vectors can be derived for each pixel in the block. Therefore, motion compensation can be performed pixel-by-pixel) perform motion compensation for the plurality of units of the sub-block, (Huang [0148] Accordingly, in view of the above description, video encoder 200 and video decoder 300 may be configured to code a block of video data using subblock affine motion compensation mode where the size of the one or more subblocks of the block is smaller than 4×4. In this example, video encoder 200 and video decoder 300 may be configured to receive a respective motion vector for each 4×4 motion field subblock of the block, and calculate a respective subblock motion vector for each of the one or more subblocks within the respective 4×4 motion field subblock based on the respective motion vector for the 4×4 motion field subblock and the affine motion parameters. [0149] In one example, the size of the one or more subblocks is 2×2, and video encoder 200 and video decoder 300 are configured to calculate four motion vectors for the one or more subblocks. In another example, the size of the one or more subblocks is 2×4 or 4×2, and video encoder 200 and video decoder 300 are configured to calculate two motion vectors for the one or more subblocks) wherein each unit of the plurality of units is motion compensated using a respective motion vector; (Huang [0137] This section describes techniques for the calculation of a motion vector for a 2×2 subblock. In the case 2×2 subblocks (e.g., blocks 710, 712, 714, and 716 in FIG. 7) are chosen for partitioning the current block 700, video encoder 200 and video decoder 300 may derive the four motion vectors of the four 2×2 subblocks 710, 712, 714, and 715 in 4×4 motion field subblock 702 as shown below. [0138] The MV of top-left 2×2 subblock 710 (v,.sub.x0, v.sub.y0) is:
PNG
media_image2.png
462
619
media_image2.png
Greyscale
Calculation of Motion Vector for 2×4 Subblock)
and decode the video block based at least on the plurality of motion compensated units. (Huang [0122] In this manner, video decoder 300 represents an example of a video decoding device including a memory configured to store video data, and one or more processing units implemented in circuitry and configured to receive the block of video data to be decoded using a subblock affine motion compensation mode, determine a size of one or more subblocks of the block based on one or more of an inter prediction direction or affine motion parameters of the block, partition the block into the one or more subblocks based on the determined size, and decode the one or more subblocks using the subblock affine motion compensation mode. [0169] Video decoder 300 may predict the current block. For example, affine unit 317 may decode the one or more subblocks using the subblock affine motion compensation mode (378) as described above, to calculate a prediction block for the current block. Video decoder 300 may then inverse scan the reproduced coefficients (380) for the subblocks to create blocks of quantized transform coefficients. Video decoder 300 may then inverse quantize and inverse transform the coefficients to produce residual blocks (382). Video decoder 300 may ultimately decode the current block by combining the prediction blocks and the residual blocks (384).)
Huang [0037] Versatile Video Coding (VVC). A recent draft of the VVC standard is described in Bross, et al. “Versatile Video Coding (Draft 4),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13.sup.th Meeting: Marrakech, Mass., 9-18 Jan. 2019, JVET-M1001-v5 (hereinafter “VVC Draft 4”). Huang does not teach definition of sub-block size.
However Zhou teach for a sub-block of the plurality of sub-blocks, (Zhou [0005] In the current VVC design (see JVET-P2001, “Versatile Video Coding (Draft 7)”), the sub-block size for the affine mode is fixed to 4×4, which creates the 4×4 bi-directional prediction for the worst-case memory bandwidth consumption of motion compensation. Please note dependent claim 37 sub-block is a 4x4 subblock – this is in line with the teaching of Zhou. This also confirms the interpretation of Huang [0137] 2×2 subblock considered as units is that are smaller than the sub-block claimed in independent claims)
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Huang, further incorporating Zhou in video/camera technology. One would be motivated to do so, to incorporate sub-block size definition for better clarification. Examiner is citing Zhou to show the evidence of common knowledge.
Regarding to claim 37, 45 and 53:
37. Huang teach the device of claim 36, wherein the sub-block is a 4x4 subblock or an 8x8 sub-block. (Huang [0151] In a second example, video encoder 200 and video decoder 300 may be configured to determine the subblock size based on the affine motion parameters. If the affine motion parameters satisfy a first criterion, video encoder 200 and video decoder 300 set the subblock size to a first subblock size (sbW1, sbH1). Otherwise, if the affine motion parameters satisfy a second criterion, video encoder 200 and video decoder 300 set the subblock size to a second subblock size (sbW, sbH). Otherwise, if the affine motion parameters satisfy neither the first or second criterion, video encoder 200 and video decoder 300 set the subblock size to a third subblock size (sbW2, sbH2). In this example, sbW1*sbH1<sbW*sbH<sbW2* sbH2.). [0154] In one example, (sbW, sbH)=(4, 4), (sbW1, sbH1)=(2, 2) and (sbW2, sbH2)=(8, 8).)
Regarding to claim 40, 48 and 54:
40. Huang teach the device of claim 36, wherein the processor being configured to perform motion compensation for the plurality of units of the sub-block comprises the processor being further configured to: determine a first unit motion vector associated with a first unit of the plurality of units; obtain a first motion compensated unit based on the first unit motion vector; determine a second unit motion vector associated with a second unit of the plurality of units; and obtain a second motion compensated unit based on the second unit motion vector. (Huang [0137] This section describes techniques for the calculation of a motion vector for a 2×2 subblock. In the case 2×2 subblocks (e.g., blocks 710, 712, 714, and 716 in FIG. 7) are chosen for partitioning the current block 700, video encoder 200 and video decoder 300 may derive the four motion vectors of the four 2×2 subblocks 710, 712, 714, and 715 in 4×4 motion field subblock 702 as shown below. [0138] The MV of top-left 2×2 subblock 710 (v,.sub.x0, v.sub.y0) is:
PNG
media_image2.png
462
619
media_image2.png
Greyscale
Calculation of Motion Vector for 2×4 Subblock)
Regarding to claim 42 and 50:
42. Huang teach the device of claim 36, wherein the determination of whether to split the sub-block for motion compensation is based on a maximum difference between a motion vector associated with the sub-block and one or more control point motion vectors (CPMVs) associated with the sub-block. (Huang [0156] In a fifth example, video encoder 200 and video decoder 300 may be configured to determine the subblock size for each 8×8 area of a block based on the inter prediction direction and the four subblock motion vectors in the 8×8 area (e.g., the four subblock motion vectors in the 4×4 motion field subblocks as described above with reference to FIG. 7). As shown in FIG. 8, the four subblock motion vectors of 8×8 area 800 are {right arrow over (v)}.sub.i=(v.sub.ix, v.sub.iy) i=0,1,2,3. Video encoder 200 and video decoder 300 may calculate the parameters {a, b, c, d} as:
PNG
media_image1.png
279
180
media_image1.png
Greyscale
Para 155’s threshold criteria on the parameters {a, b, c, d} are thresholds on these CPMV-difference-derived quantities, governing the split decision [0151]. [0155] In another example, the first and second criteria are designed as follows:
a>−1&&a*TH1<1
&&c*TH2>−1&&c<1
&&b*TH1>−1&&b<1
&&d>−1&&d*TH2<1
&&(a+b)>−1&&(a+b)*TH1<1
&&(c+d)>−1&&(c+d)*TH2<1,
where TH1 is equal to S−m and TH2 is equal to S−n, m is the selected subblock width and n is the selected subblock height. For the first criterion, TH1=S−sbW1, and TH2=S−sbH1. For the second criterion, TH1=S−sbW2, and TH2=S−sbH2. S is a predefined value (e.g., 8). The above equations may be modified by scaling each number to a certain precision)
Regarding to claim 43 and 51:
43. Huang teach the device of claim 36, wherein the determination of whether to split the sub-block for the motion compensation is based on a maximum difference between a motion vector associated with the sub- block and respective motion vectors associated with one or more samples of the sub-block. (Huang [0137] This section describes techniques for the calculation of a motion vector for a 2×2 subblock. In the case 2×2 subblocks (e.g., blocks 710, 712, 714, and 716 in FIG. 7) are chosen for partitioning the current block 700, video encoder 200 and video decoder 300 may derive the four motion vectors of the four 2×2 subblocks 710, 712, 714, and 715 in 4×4 motion field subblock 702 as shown below. [0138] The MV of top-left 2×2 subblock 710 (v,.sub.x0, v.sub.y0) is:
PNG
media_image2.png
462
619
media_image2.png
Greyscale
Calculation of Motion Vector for 2×4 Subblock. The difference between the sub-block MV and each unit MV exactly the affine parameters are the sub-bock-MV to unit MV difference, because Para 155’s threshold criteria on the parameters {a, b, c, d} are thresholds on these CPMV-difference-derived quantities, governing the split decision [0151]. [0155] In another example, the first and second criteria are designed as follows:
a>−1&&a*TH1<1
&&c*TH2>−1&&c<1
&&b*TH1>−1&&b<1
&&d>−1&&d*TH2<1
&&(a+b)>−1&&(a+b)*TH1<1
&&(c+d)>−1&&(c+d)*TH2<1,
where TH1 is equal to S−m and TH2 is equal to S−n, m is the selected subblock width and n is the selected subblock height. For the first criterion, TH1=S−sbW1, and TH2=S−sbH1. For the second criterion, TH1=S−sbW2, and TH2=S−sbH2. S is a predefined value (e.g., 8). The above equations may be modified by scaling each number to a certain precision)
Claims 38-39 and 46-47 is/are rejected under 35 U.S.C. 103 as being unpatentable over Huang (U.S. Pub. No. 20200288156 A1), in view of Zhou (U.S. Pub. No. 20220232204 A1), further in view of Liu (U.S. Pub. No. 20210235083 A1).
Regarding to claim 38 and 46:
38. Huang teach the device of claim 36, Huang do not explicitly teach wherein the sub-block is split horizontally into the plurality of units.
However Liu teach wherein the sub-block is split horizontally into the plurality of units. (Liu [0439] the video block is horizontally split into the plurality of sub-blocks each having a height not larger than the second threshold if the height of the video block is larger than the second threshold)
The motivation for combining Huang and Zhou as set forth in claim 36 is equally applicable to claim 38. It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Huang, further incorporating Zhou and Liu in video/camera technology. One would be motivated to do so, to incorporate the sub-block is split horizontally into the plurality of units. This functionality will improve efficiency with predictable results.
Regarding to claim 39 and 47:
39. Huang teach the device of claims 36, Huang do not explicitly teach wherein the sub-block is split vertically into the plurality of units.
However Liu teach wherein the sub-block is split vertically into the plurality of units. (Liu [0438] the video block is vertically split into the plurality of sub-blocks each having a width not larger than the first threshold if the width of the video block is larger than the first threshold; and)
Claims 41, 49 and 55 is/are rejected under 35 U.S.C. 103 as being unpatentable over Huang (U.S. Pub. No. 20200288156 A1), in view of Zhou (U.S. Pub. No. 20220232204 A1), further in view of Zhang897 (U.S. Pub. No. 20220224897 A1).
Regarding to claim 41, 49 and 55:
41. Huang teach the device of claim 40, Huang do not explicitly teach wherein the processor is further configured to perform prediction refinement with optical flow (PROF) for the first unit of the plurality of units, wherein the performance of the PROF for the first unit comprises the processor being further configured to: determine a difference between a motion vector associated with the sub-block and the first unit motion vector associated with the first unit of the plurality of units; and refine the first motion compensated unit of the plurality of motion compensated units based on the determined difference.
However Zhang897 teach wherein the processor is further configured to perform prediction refinement with optical flow (PROF) for the first unit of the plurality of units, wherein the performance of the PROF for the first unit comprises the processor being further configured to: determine a difference between a motion vector associated with the sub-block and the first unit motion vector associated with the first unit of the plurality of units; and refine the first motion compensated unit of the plurality of motion compensated units based on the determined difference. (Zhang897 FIG. 29 teach sub-block MV [0486] To achieve a finer granularity of motion compensation, this contribution proposes a method to refine the sub-block based affine motion compensated prediction with optical flow. After the sub-block based affine motion compensation is performed, luma prediction sample is refined by adding a difference derived by the optical flow equation. The proposed PROF (prediction refinement with optical flow) is described as following four steps. [0487] Step 1) The sub-block-based affine motion compensation is performed to generate sub-block prediction I (i,j). [0488] Step 2) The spatial gradients g.sub.x(i,j) and g.sub.y (i,j) of the sub-block prediction are calculated at each sample location using a 3-tap filter [−1, 0, 1].
g.sub.x(i,j)=I(i+1,j)−1(i−1,j)
g.sub.y(i,j)=I(i,j+1)−I(i,j−1)
[0489] The sub-block prediction is extended by one pixel on each side for the gradient calculation. To reduce the memory bandwidth and complexity, the pixels on the extended borders are copied from the nearest integer pixel position in the reference picture. Therefore, additional interpolation for padding region is avoided. [0490] Step 3) The luma prediction refinement (denoted ΔI) as is calculated by the optical flow equation.
PNG
media_image3.png
46
393
media_image3.png
Greyscale
where the delta MV (denoted as Δv(i,j)) is the difference between pixel MV computed for sample location (i,j), denoted by v(i,j), and the sub-block MV of the sub-block to which pixel (i,j) belongs, as shown in FIG. 29. This is indicated by the arrows in FIG. 29)
The motivation for combining Huang and Zhou as set forth in claim 36 is equally applicable to claim 41. It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Huang, further incorporating Zhou and Zhang897 in video/camera technology. One would be motivated to do so, to incorporate perform prediction refinement with optical flow (PROF) for the first unit of the plurality of units. This functionality will improve quality with predictable results.
Closely related prior art
Examiner notes teaching of Zhang102 (U.S. Pub. No. 20180070102 A1) is/are pertinent to the independent claim(s) because it teaches video coding sub-block, however is not used because dependent claims are covered by primary reference.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NASIM N NIRJHAR whose telephone number is (571) 272-3792. The examiner can normally be reached on Monday - Friday, 8 am to 5 pm ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William F Kraig can be reached on (571) 272-8660. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NASIM N NIRJHAR/Primary Examiner, Art Unit 2896
Prosecution Insights