DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is responsive to the correspondence filled on 3/12/25.
Claims 1-20 are presented for examination.
IDS Considerations
The information disclosure statement (IDS) submitted on 6/17/25 is/are being considered by the examiner as the submission is in compliance with the provisions of 37 CFR 1.97.
Claim Rejections - 35 USC § 102
The following is a quotation of 35 U.S.C. 102(a)(1)/(a)(2) which forms the basis for all obviousness rejections set forth in this Office action:
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 9-14 and 19-20 is/are rejected under 35 U.S.C. 102 (a)(1) as being unpatentable over Biatek (U.S. Pub. No. 20210092405 A1).
Regarding to claim 1, 11 and 19:
Examiner’s Note: Encoding/coding and decoding are done using same opposite algorithm.
1. Biatek teach a method of coding video data, the method comprising: (Biatek Fig. 3 and Fig. 4 [0016] FIG. 1 is a block diagram illustrating an example video encoding and decoding system) coding a syntax element indicating a prediction mode type for a current block of video data; (Biatek [0032] This disclosure describes techniques for MIP in video coding that may address such issues. As described herein, a video decoder may store a plurality of MIP matrices. A MIP mode syntax element that is signaled in a bitstream indicates a MIP mode index for a current block of video data. Additionally, a transpose flag that is signaled in the bitstream. The video decoder may determine the input vector based on neighboring samples for the current block. The transpose flag indicates whether the input vector is transposed. Furthermore, the video decoder may determine a prediction signal.) determining a prediction mode having the prediction mode type (Biatek [0182] Furthermore, the techniques of this disclosure may involve the following semantic changes to section 7.4.9.5 of VVC Draft 6: intra_mip_flag [x0][y0] equal to 1 specifies that the intra prediction type for luma samples is matrix-based intra prediction [MIP is prediction mode type]. intra_mip_flag[x0][y0] equal to 0 specifies that the intra prediction type for luma samples is not matrix based intra prediction. When intra_mip_flag[x0][y0] is not present, it is inferred to be equal to 0.) from a set of two more possible prediction modes (Biatek [0085] Signaling of MIP mode and prediction mode: A flag specifying whether the MIP mode is used is signaled in the bitstream by the encoder for each Coding Unit (CU). If an MIP mode is to be applied, a Most Probable Mode (MPM) flag is signaled to indicate whether prediction mode is one of the MIP MPM modes or not. In MIP, 3 modes are considered for MPM and MPM mode indices are context coded with truncated binarization. Non-MPM mode indices are coded as fixed length code (FLC). The derivation of the MPMs is harmonized with conventional intra prediction mode by performing mode mapping between conventional intra prediction mode and MIP intra prediction mode [two more possible prediction modes] based on predefined mapping tables which depend on block size (i.e., idx(W,H)∈{0,1,2}). The following are forward (convention mode to MIP mode) and inverse (MIP mode to conventional mode) mode mapping tables.) based on one or more of information (Biatek [0345] When pred_mode_flag is not present, it is inferred as follows: [0346] If cbWidth is equal to 4 and cbHeight is equal to 4, pred_mode_flag is inferred to be equal to 1. [0347] Otherwise, if modeType is equal to MODE_TYPE _INTRA, pred_mode_flag is inferred to be equal to 1. [0348] Otherwise, if modeType is equal to MODE_TYPE_INTER, pred_mode_flag is inferred to be equal to 0. [0349] Otherwise, pred_mode_flag is inferred to be equal to 1 when decoding an I slice, and equal to 0 when decoding a P or B slice, respectively. [0368] intra_mip_flag[x0][y0] equal to 1 specifies that the intra prediction type for luma samples is matrix-based intra prediction. intra_mip_flag[x0][y0] equal to 0 specifies that the intra prediction type for luma samples is not matrix-based intra prediction. [0369] When intra_mip_flag[x0][y0] is not present, it is inferred to be equal to 0. [0370] intra_mip_transposed[x0][y0] specifies whether the input vector for matrix-based intra prediction mode for luma samples is transposed or not. [0371] intra_mip_mode[x0][y0] specifies the matrix-based intra prediction mode for luma samples. The array indices x0, y0 specify the location (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. [0372] intra_luma_ref_idx[x0][y0] specifies the intra predic. [0180]
PNG
media_image1.png
141
445
media_image1.png
Greyscale
In the equation above, mode denotes the MIP mode index, W indicates a width of the block, H indicates a height of the block. With respect to the equation above, there may be a first plurality of MIP matrices for blocks with W=H=4; a second plurality of MIP matrices for blocks with W=H=8; and a third plurality of MIP matrices for blocks with a maximum of the width or height greater than 8 (i.e., max(W,H)>8). In accordance with a technique of this disclosure, when the width and height of the block are equal to 4 and mode is greater than or equal to 18, the video coder may use a transpose of the MIP matrix (in a set of MIP matrices for W=H=4) having a MIP mode index equal to mode minus 18. When the width and height of the block are equal to 8 and mode is greater than or equal to 10, the video coder may use a transpose of the MIP matrix (in a set of MIP matrices for W=H=8) having a MIP mode index equal to mode minus 10. The effect of this change is the increase of one MIP mode (due to being able to transpose the MIP matrix corresponding to MIP mode index 0), which a video coder can use for coding blocks without increase in storage requirements.) related to the current block or information related to neighboring blocks of the current block; (Biatek [0155] In the example of FIG. 4, intra-prediction unit 318 may include a MIP unit 319 that may use a MIP mode to generate the prediction block. In accordance with one or more techniques of this disclosure, entropy decoding unit 302 may obtain, from the bitstream, a MIP mode syntax element indicating a MIP mode index for the current block. Additionally, entropy decoding unit 302 may obtain a transpose flag from the bitstream. MIP unit 319 may store a plurality of MIP matrices. MIP unit 319 may determine an input vector based on neighboring samples for a current block of the video data. The transpose flag indicates whether the input vector is transposed. Additionally, MIP unit 319 may determine a prediction signal) and coding the current block using the prediction mode. (Biatek [0085] Signaling of MIP mode and prediction mode: A flag specifying whether the MIP mode is used is signaled in the bitstream by the encoder for each Coding Unit (CU). If an MIP mode is to be applied, a Most Probable Mode (MPM) flag is signaled to indicate whether prediction mode is one of the MIP MPM modes or not. In MIP, 3 modes are considered for MPM and MPM mode indices are context coded with truncated binarization. Non-MPM mode indices are coded as fixed length code (FLC). The derivation of the MPMs is harmonized with conventional intra prediction mode by performing mode mapping between conventional intra prediction mode and MIP intra prediction mode based on predefined mapping tables which depend on block size (i.e., idx(W,H)∈{0,1,2}). The following are forward (convention mode to MIP mode) and inverse (MIP mode to conventional mode) mode mapping tables.)
Regarding to claim 2, 12 and 20:
2. Biatek teach the method of claim 1, wherein the prediction mode type is a matrix intra prediction (MIP) mode. (Biatek [0032] This disclosure describes techniques for MIP in video coding that may address such issues. As described herein, a video decoder may store a plurality of MIP matrices. A MIP mode syntax element that is signaled in a bitstream indicates a MIP mode index for a current block of video data. Additionally, a transpose flag that is signaled in the bitstream. The video decoder may determine the input vector based on neighboring samples for the current block. The transpose flag indicates whether the input vector is transposed. Furthermore, the video decoder may determine a prediction signal. Biatek [0182] Furthermore, the techniques of this disclosure may involve the following semantic changes to section 7.4.9.5 of VVC Draft 6: intra_mip_flag [x0][y0] equal to 1 specifies that the intra prediction type for luma samples is matrix-based intra prediction [MIP is prediction mode type]. intra_mip_flag[x0][y0] equal to 0 specifies that the intra prediction type for luma samples is not matrix based intra prediction. When intra_mip_flag[x0][y0] is not present, it is inferred to be equal to 0.)
Regarding to claim 3 and 13:
3. Biatek teach the method of claim 1, wherein the information related to the current block is a block size. (Biatek [0079] Out of the boundary samples, four samples in the case of W=H=4 and eight samples in all other cases are extracted by averaging. Specifically, the input boundary vectors bdry.sup.top and bdry.sup.left are reduced to smaller boundary vectors bdry.sub.red.sup.top and bdry.sub.red.sup.left by averaging neighboring boundary samples according to a predefined rule that depends on block size. Then, the two reduced boundary vectors bdry.sub.red.sup.top and bdry.sub.red.sup.left are concatenated to a reduced boundary vector bdry.sub.red which is thus of size four for blocks of shape 4×4 and of size eight for blocks of all other shapes. In the following equation, mode refers to the MIP mode and this concatenation may be defined as follows:
PNG
media_image2.png
167
373
media_image2.png
Greyscale
[0085] Signaling of MIP mode and prediction mode: A flag specifying whether the MIP mode is used is signaled in the bitstream by the encoder for each Coding Unit (CU). If an MIP mode is to be applied, a Most Probable Mode (MPM) flag is signaled to indicate whether prediction mode is one of the MIP MPM modes or not. In MIP, 3 modes are considered for MPM and MPM mode indices are context coded with truncated binarization. Non-MPM mode indices are coded as fixed length code (FLC). The derivation of the MPMs is harmonized with conventional intra prediction mode by performing mode mapping between conventional intra prediction mode and MIP intra prediction mode [two more possible prediction modes] based on predefined mapping tables which depend on block size (i.e., idx(W,H)∈{0,1,2}). The following are forward (convention mode to MIP mode)
Regarding to claim 4 and 14:
4. Biatek teach the method of claim 1, wherein the information related to neighboring blocks of the current block is an availability of neighboring samples. (Biatek [0106] The neighboring samples of the current block include the top boundary pixel values and the left boundary pixel values. For instance, transposing the input vector may correspond to concatenating the top boundary pixel values and the left boundary pixel values instead of vice versa. Transposing the input vector may effectively double the possibilities of how the input vector may be combined with a set of MIP matrices. Because of the increased number of possibilities, video encoder 200 may be able to select a combination of MIP matrix and transposed/non-transposed input vector that may otherwise not be available without signaling of the transpose flag. Thus, coding efficiency may be increased without increasing storage requirements associated with more MIP matrices.)
Regarding to claim 9:
9. Biatek teach the method of claim 1, wherein coding the current block using the prediction mode comprises encoding the current block using the prediction mode. (Biatek [0026] FIG. 11 is a flowchart illustrating an example method for encoding a current block. [0266] FIG. 11 is a flowchart illustrating an example method for encoding a current block. The current block may be or include a current CU. Although described with respect to video encoder 200 (FIGS. 1 and 3), it should be understood that other devices may be configured to perform a method similar to that of FIG. 11. [0267] In this example, video encoder 200 initially predicts the current block (350). For example, video encoder 200 may form a prediction block for the current block. Video encoder 200 may then calculate a residual block for the current block (352). For instance, as part of predicting the current block, video encoder 200 may perform the techniques for MIP [prediction mode] described in this disclosure. To calculate the residual block, video encoder 200 may calculate a difference between the original, unencoded block and the prediction block for the current block)
Regarding to claim 10:
10. Biatek teach the method of claim 1, wherein coding the current block using the prediction mode comprises decoding the current block using the prediction mode. (Biatek [0027] FIG. 12 is a flowchart illustrating an example method for decoding a current block of video data. [0268] FIG. 12 is a flowchart illustrating an example method for decoding a current block of video data. The current block may be or include a current CU. Although described with respect to video decoder 300 (FIGS. 1 and 4), it should be understood that other devices may be configured to perform a method similar to that of FIG. 12. [0269] Video decoder 300 may receive entropy encoded data for the current block, such as entropy encoded prediction information and entropy encoded data for transform coefficients of a residual block corresponding to the current block (370). Video decoder 300 may entropy decode the entropy encoded data to determine prediction information for the current block and to reproduce transform coefficients of the residual block (372). Video decoder 300 may predict the current block (374), e.g., using an intra- or inter-prediction mode as indicated by the prediction information for the current block, to calculate a prediction block for the current block. For instance, as part of predicting the current block, video decoder 300 may perform the techniques for MIP [prediction mode] described in this disclosure)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Biatek (U.S. Pub. No. 20210092405 A1), in view of Wang (U.S. Pub. No. 20250280117 A1).
Regarding to claim 5 and 15:
5. Biatek teach the method of claim 4, Biatek do not explicitly teach wherein determining the prediction mode comprises: determining the prediction mode to be a first prediction mode based on the availability of neighboring samples being above a threshold; and determining the prediction mode to be a second prediction mode based on the availability of neighboring samples being below or equal to a threshold.
However Wang teach wherein determining the prediction mode comprises: determining the prediction mode to be a first prediction mode based on the availability of neighboring samples being above a threshold; and determining the prediction mode to be a second prediction mode based on the availability of neighboring samples being below or equal to a threshold. (Wang teach mode selection based on threshold weight between 0 and n. When weight is 0, it means neighboring samples are not available. Claimed limition is broad and obvious from [0209] It is assumed here that there are 64 possible weight derivation modes for GPM and 67 possible intra prediction modes for GPM, which may be found in the VVC standard. However, it is not limited to only 64 possible weight derivation modes for GPM, or which 64 weight derivation modes are available. On the other hand, we should know that the reason for selecting 64 kinds for GPM in VVC is also a trade-off between an improvement of prediction effect and an improvement of overhead in the bitstream. In the present solution, a fixed logic is no longer used to performing encoding in the weight derivation mode, so in theory various weight derivation modes are used for the present solution and they are used more flexibly. Likewise, it is not limited to only 67 intra prediction modes for GPM, or which 67 intra prediction modes are available. Theoretically, all possible intra prediction modes may be used for GPM. For example, if the intra angular prediction mode is made more detailed and more intra angular prediction modes are generated, then more intra angular prediction modes are used for GPM. For example, the MIP (matrix-based intra prediction) mode in VVC may also be used in this solution, but considering that MIP has multiple sub-modes that can be selected, MIP is not provided in the embodiments for ease of understanding. There are also some wide-angle modes that may also be used in this solution, which will not be described in this embodiments. [0256] In an eighteenth clause, according to the tenth clause, where determining the candidate prediction mode list of the j-th prediction mode based on the weights of the neighboring blocks with respect to the j-th prediction mode includes: [0257] in response to that the weights of the neighboring blocks with respect to the j-th prediction mode are greater than or equal to a preset threshold, obtaining prediction modes of the neighboring blocks; and [0258] determining the candidate prediction mode list of the j-th prediction mode based on the prediction modes of the neighboring blocks. [0259] In a nineteenth clause, according to the eighteenth clause, where in response to that a value of the weight is in a range from 0 to n, the preset threshold is n/2, and n is a positive number. [0260] In a twentieth clause, according to the tenth clause, where in response to that the value of the weight is a first value or a second value, determining the candidate prediction mode list of the j-th prediction mode based on the weights of the neighboring blocks with respect to the j-th prediction mode includes: [0261] in response to that the weights of the neighboring blocks with respect to the j-th prediction mode are equal to the first value, obtaining the prediction modes of the neighboring blocks, the first value being greater than the second value; [0262] determining the candidate prediction mode list of the j-th prediction mode based on the prediction modes of the neighboring blocks.)
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Biatek, further incorporating Wang in video/camera technology. One would be motivated to do so, to incorporate determining the prediction mode to be a first prediction mode based on the availability of neighboring samples being above a threshold. This functionality will improve efficiency with predictable results.
Claims 7-8 and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Biatek (U.S. Pub. No. 20210092405 A1), in view of Jhu (U.S. Pub. No. 20250330626 A1).
Regarding to claim 7 and 17:
7. Biatek teach the method of claim 1, Biatek do not explicitly teach further comprising: performing decoder side intra mode derivation process using a whole prediction block, wherein the whole prediction block is not down-sampled.
However Jhu teach further comprising: performing decoder side intra mode derivation process (Jhu [0042] FIGS. 15A, 15B, 15C and 15D are diagrams illustrating a process of decoder side intra mode derivation.) using a whole prediction block, (Jhu [0298] The following examples represent situations when the matrix A cannot be solved, where default prediction values may be assigned to the whole current block:) wherein the whole prediction block is not down-sampled. (Jhu [0057] FIG. 30 is a diagram illustrating non-down-sampled luma samples. [0252] Q and R can be linear combination of current and neighboring luma samples either in a down-sampled domain (for example, the Q and R being pre-operated luma samples obtained by weighted-average operation) or without any down-sampling process.)
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Biatek, further incorporating Jhu in video/camera technology. One would be motivated to do so, to incorporate performing decoder side intra mode derivation process using a whole prediction block, wherein the whole prediction block is not down-sampled. This functionality will improve quality with predictable results.
Regarding to claim 8 and 18:
8. Biatek teach the method of claim 7, further comprising: determining one or more of a low frequency non-separate transform (LFNST) transform set (Biatek [0442] lfnst_idx[x0][y0] specifies whether and which one of the two low frequency non-separable transform kernels in a selected transform set is used. lfnst_idx[x0][y0] equal to 0 specifies that the low frequency non-separable transform is not used. The array indices x0, y0 specify the location (x0, y0) of the top-left sample of the considered transform block relative to the top-left sample of the picture.) or a transpose flag based on the decoder side intramode derivation process. (Part of OR condition, rejection is not required)
Allowable subject matter
Regarding to claim 6 and 16:
Claims 6 and 16is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims because the limitations of these dependent claims are not obvious from the prior art search when all the limitations of independent and intervening claims are taken into account.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NASIM N NIRJHAR whose telephone number is (571) 272-3792. The examiner can normally be reached on Monday - Friday, 8 am to 5 pm ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William F Kraig can be reached on (571) 272-8660. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NASIM N NIRJHAR/Primary Examiner, Art Unit 2896