DETAILED ACTIONNotice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1-2,5,8-9,13,16,19-20 rejected under 35 U.S.C. 103 as being unpatentable over LI; Xiang et al. (US 20200099941 A1) in view of Chen; Ya et al. (US 20240214553 A1)
Regarding claim 1, Li teaches,
A method of video coding at a decoder, (¶120-126 and fig. 7, “video decoder (710)” as depicted in fig. 7) comprising:
decoding prediction information (¶122 and fig. 7, “entropy decoder (771)” configured to reconstruct “certain symbols that represent the syntax elements of which the coded picture is made up”) of a current block in a current picture ¶122-126, syntax elements of the coded picture indicating such as “the mode in which a block is coded”) that is a part of a coded video sequence, (¶120-122 and fig. 7, entropy decoder (771) receiving “coded video sequence” as depicted in fig. 7) the prediction information indicating a local illumination compensation (LIC) mode for the current block, (¶130-131 and fig. 7, “LIC flag is signaled for a CU to indicate whether an LIC mode is applied to the CU” as part of video coded sequence received by video decoder (771))
decoding the current block (¶126,133, and fig. 7, “reconstruction module (774)”, depicted in fig. 7, configured to “form a reconstructed block” based on residual and “prediction with LIC mode”) based on the estimated LIC parameters (¶130 and 133, “LIC flag” signaled to indicate an “LIC mode” applied to a coding unit) of the LIC mode. (¶133, “prediction with LIC mode” enabled for the current block)
But does not explicitly teach,
multiple reference lines of the current block being available for LIC parameters estimation of the LIC mode;
selecting one or more reference lines from the multiple reference lines of the current block;
estimating the LIC parameters of the LIC mode based on the one or more reference lines;
However, Chen teaches additionally,
multiple reference lines of the current block (¶113-116 and fig. 17, “multiple lines reference template of a spatial neighboring block” for generating the template of the current block which includes “two reference lines from the same side of only one spatial reference block” as depicted in fig. 17-18) being available for LIC parameters estimation of the LIC mode; (¶116, “template with multi reference line is applied in the spatial LIC parameters estimation”)
selecting one or more reference lines from the multiple reference lines of the current block; (¶113-116, “uses the reconstructed samples located in the nearest reference line”)
estimating the LIC parameters of the LIC mode (¶113-116 and fig. 17-18, “spatial LIC parameters estimation”) based on the one or more reference lines; (¶113-116 and fig. 17-18, “spatial LIC parameters estimation” which describes process deriving spatial LIC parameters where “the template for the proposed spatial LIC only uses the reconstructed samples located in the nearest reference line”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen which derives local illumination compensation using the nearest reference line. This technique discloses a process that can improve the estimation accuracy of the illumination variation.
Regarding claim 2, Li with Chen teaches the limitations of claim 1,
Chen teaches additionally,
prediction information includes a first syntax element (¶115 and 157, syntax element “lic_mrl_flag”) indicating that the multiple reference lines of the current block are available for the LIC parameters estimation of the LIC mode. (¶115, “flag lic_mrl_flag indicating whether multi reference lines are applied for composing the template” indicating “template with multi reference line is applied in the spatial LIC parameters estimation”)
Regarding claim 5, Li with Chen teaches the limitations of claim 1,
Chen teaches additionally,
one or more reference lines (¶113-114, “multi reference lines”) includes a first subset and a second subset of the multiple reference lines (¶113-114, “multi reference lines from only left/above side are applied for horizontal/vertical directional modes”) for a top reference template (¶113-114, “above side” spatial neighboring block of “the current block”) and a left reference template of the current block, (¶113-114, left “side” spatial neighboring block of “the current block”) respectively, the first subset of the multiple reference lines being different from the second subset of the multiple reference lines. (¶113-114, “left-boundary template is applied for horizontal directional modes; and above-boundary template is used for vertical directional modes”)
Regarding claim 8, Li with Chen teaches the limitations of claim 1,
Chen teaches additionally,
prediction information indicates the one or more selected reference lines. (¶183 and 114, “Select a spatial neighboring block used as the reference block for spatial LIC parameters estimation” based on “multiple lines reference template of a spatial neighboring block”)
Regarding claim 9, Li with Chen teaches the limitations of claim 1,
Chen teaches additionally,
calculating multiple sets of the LIC parameters based on the one or more reference lines; (¶112-114 and figs. 16-18, “deriving of spatial LIC parameters process with reference template comprising the left boundary of a left neighboring block for intra prediction and with reference template comprising the above boundary of an above neighboring block for intra prediction for intra prediction”) and
determining the LIC parameters of the LIC mode (¶109, “estimating spatial LIC parameters when the intra prediction mode belongs to horizontal/vertical direction,”) based on at least one of the multiple sets of the LIC parameters. (¶109 and 114, estimating spatial LIC parameters when the intra prediction mode belongs to horizontal/vertical direction based on “two or three templates” used together “to calculate the spatial LIC parameters” such as “left-boundary template is applied for horizontal directional modes; and above-boundary template is used for vertical directional modes”)
Regarding claim 13, Li with Chen teaches the limitations of claim 1,
Chen teaches additionally,
calculating a slope parameter (¶84 and 57, “LMSE-based derivation is employed to calculate the values of the scaling factor α”) of the LIC model (¶84 and 57, scaling factor α “used for the spatial LIC”) based on the one or more reference lines; (¶57,84, and 113, LIC based on a linear model where a “scaling factor α” calculated by minimizing the different between a reference template, that corresponds with “multi reference lines of a spatial reference block” used as a template, and template of the current block T) and
adjusting the slope parameter (¶57 and Equation 1, temporal prediction process which adds “offset β” to a product “scaling factor α” disclosed in the LIC mathematical model of equation 1) based on a slope adjustment value. (¶84 and 57, “offset β used for the spatial LIC”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen which derives local illumination compensation using multi reference lines as a template. This technique discloses a process that can improve the estimation accuracy of the illumination variation.
Regarding claim 16, Li with Chen teaches the limitations of claim 13,
Chen teaches additionally,
a fifth syntax element (¶176, “spatial_lic_flag”) indicating whether signaling the slope adjustment value is enabled. (¶176, “when the spatial LIC applies (spatial_lic_flag is true)”, it uses a linear model for spatial illumination changes, using a scaling factor α and “an offset β”)
Regarding claim 19, it is the apparatus claim of method claim 1.
Li teaches additionally,
An apparatus for video coding, (¶174, “functions of the video decoder (310)”) comprising:
processing circuitry configured to (¶174, “processing circuitry that performs functions of the video decoder (310)”)
Regarding claim 20, Li teaches,
A method of video coding at an encoder, (¶112-119 and fig. 6, “video encoder (603)” depicted in fig. 6) comprising:
generating prediction information of a current block in a current picture that is a part of a coded video sequence, (¶119 and fig. 6, “entropy encoder (625)” configured to format “selected prediction information” in the “video coded sequence” bitstream for the “encoded block” as depicted in fig. 6) the prediction information indicating a local illumination compensation (LIC) mode for the current block, (¶130-131 and fig. 6, “LIC flag is signaled for a CU to indicate whether an LIC mode is applied to the CU” as part of video coded sequence output by entropy encoder (625))
encoding the current block (¶133, “when a current block is coded”) based on the estimated LIC parameters of the LIC mode. (¶133, “prediction with LIC mode” is performed “according to a linear model for illumination changes” when a current block is coded)
But does not explicitly teach,
multiple reference lines of the current block being available for LIC parameters estimation of the LIC mode;
selecting one or more reference lines from the multiple reference lines of the current block;
estimating the LIC parameters of the LIC mode based on the one or more reference lines;
However, Chen teaches additionally,
multiple reference lines of the current block (¶113-116 and fig. 17, “multiple lines reference template of a spatial neighboring block” for generating the template of the current block which includes “two reference lines from the same side of only one spatial reference block” as depicted in fig. 17-18) being available for LIC parameters estimation of the LIC mode; (¶116, “template with multi reference line is applied in the spatial LIC parameters estimation”)
selecting one or more reference lines from the multiple reference lines of the current block; (¶113-116, “uses the reconstructed samples located in the nearest reference line”)
estimating the LIC parameters of the LIC mode (¶113-116 and fig. 17-18, “spatial LIC parameters estimation”) based on the one or more reference lines; (¶113-116 and fig. 17-18, “spatial LIC parameters estimation” which describes process deriving spatial LIC parameters where “the template for the proposed spatial LIC only uses the reconstructed samples located in the nearest reference line”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen which derives local illumination compensation using the nearest reference line. This technique discloses a process that can improve the estimation accuracy of the illumination variation.
Claim 3-4 rejected under 35 U.S.C. 103 as being unpatentable over LI; Xiang et al. (US 20200099941 A1) in view of Chen; Ya et al. (US 20240214553 A1) in view of XU; Liying (US 20220360800 A1)
Regarding claim 3, Li with Chen teaches the limitations of claim 1,
But does not explicitly teach the additional limitations of claim 3,
However, Xu teaches additionally,
prediction information includes a second syntax element (¶434 and fig. 17, “line number indication information”) indicating a number of the multiple reference lines available for the LIC parameters estimation of the LIC mode. (¶434, line number indication information is coded based on “the number of candidate reference lines corresponding to the multi-reference line prediction mode prior to predicting the current block”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen with the coding of Xu which includes a line number indication. This allows for multi-reference line prediction that can reduce the number of context models to further reduce coding complexity.
Regarding claim 4, Li with Chen teaches the limitations of claim 1,
But does not explicitly teach the additional limitations of claim 4,
However, Xu teaches additionally,
prediction information includes a third syntax element (¶435 and fig. 17, “indication information of reference lines”) indicating a number of the one or more reference lines selected for the LIC parameters estimation of the LIC mode. (¶435, indication information of reference lines “coded based on a target reference line used in predicting the current block based on the multi-reference line prediction mode”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen with the coding of Xu which includes a line number indication. This allows for multi-reference line prediction that can reduce the number of context models to further reduce coding complexity.
Claim 6-7,10 rejected under 35 U.S.C. 103 as being unpatentable over LI; Xiang et al. (US 20200099941 A1) in view of Chen; Ya et al. (US 20240214553 A1) in view of Racape; Fabien et al. (US 11956436 B2)
Regarding claim 6, Li with Chen teaches the limitations of claim 1,
Chen teaches additionally,
one or more reference lines (¶113-114, “multi reference lines”) includes a first subset and a second subset of the multiple reference lines (¶113-114, “multi reference lines from only left/above side are applied for horizontal/vertical directional modes”) for the current block, (¶113-114, “left/above side” spatial neighboring block of “the current block”) respectively, the first subset of the multiple reference lines being different from the second subset of the multiple reference lines. (¶113-114, “left-boundary template is applied for horizontal directional modes; and above-boundary template is used for vertical directional modes”)
But does not explicitly teach,
a luma component and a chroma component of the current block
However, Racape teaches additionally,
multiple reference lines for a luma component (6:45-60 and embodiments 2-3, “multi-reference prediction with variable weights is applied on the luma component” for each block) and a chroma component of the current block (6:45-60 and embodiments 2-3, prediction process “the chroma block” applied using “multi-reference samples with a fixed set of weights” for each block)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen with the coding of Xu with the multiple reference prediction of Racape which applied multi-reference prediction to luma components and chroma blocks. This process reduces coding runtime while keeping a large part of the gains.
Regarding claim 7, Li with Chen teaches the limitations of claim 1,
Chen teaches additionally,
one or more reference lines (¶113-114, “multi reference lines”) includes a subset of the multiple reference lines (¶113-114, “multi reference lines from only left/above side are applied for horizontal/vertical directional modes”) that is applied to the current block, (¶113-114, “left/above side” spatial neighboring block of “the current block”) and wherein the prediction information includes a fourth syntax (¶87, “spatial neighboring blocks for MVP list construction”) indicating the subset of the multiple reference lines. (¶86-87 and 113, MVP list construction considered as “reference block candidates for the spatial LIC” parameter estimation which corresponds with the “multiple lines reference template of a spatial neighboring block “)
But does not explicitly teach,
to at least two of one luma component and two chroma components
However, Racape teaches additionally,
multiple reference lines that is applied (6:45-60 and embodiments 2-3, “multi-reference samples” applied to luma component and chroma block “for each block”) to at least two of one luma component and two chroma components of the current block (6:45-60 and embodiments 2-3, “multi-reference prediction” with variable weights “applied on the luma component” and “the chroma block” which utilize “multi-reference samples with a fixed set of weights” for “each block”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen with the coding of Xu with the multiple reference prediction of Racape which applied multi-reference prediction to luma components and chroma blocks. This process reduces coding runtime while keeping a large part of the gains.
Regarding claim 10, Li with Chen teaches the limitations of claim 1,
Chen teaches additionally,
calculating includes (¶112, “deriving of spatial LIC parameters process“)
calculating a separate one of the multiple sets of the LIC parameters for each of the one or more reference lines, (¶112, deriving of spatial LIC parameters process “with reference template comprising the left boundary of a left neighboring block for intra prediction and with reference template comprising the above boundary of an above neighboring block for intra prediction”)
the determining (¶109, “estimating spatial LIC parameters when the intra prediction mode belongs to horizontal/vertical direction,”) includes
determining the LIC parameters of the LIC mode (¶109 and 114, estimating spatial LIC parameters when the intra prediction mode belongs to horizontal/vertical direction based on “two or three templates” used together “to calculate the spatial LIC parameters” such as “left-boundary template is applied for horizontal directional modes; and above-boundary template is used for vertical directional modes”)
But does not explicitly teach,
determining the parameters of the mode based on a weighted average of the multiple sets of the parameters.
However, Racape teaches additionally,
determining the parameters (6:45-60 and embodiments 2-3, “multi-reference prediction”) of the mode based on a weighted average of the multiple sets of the parameters. (6:45-60 and embodiments 2-3, multi-reference prediction “with variable weights is applied on the luma component” and “multi-reference samples with a fixed set of weights” for chroma blocks)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen with the coding of Xu with the multiple reference prediction of Racape which applied multi-reference prediction. This process reduces coding runtime while keeping a large part of the gains.
Claim 11 rejected under 35 U.S.C. 103 as being unpatentable over LI; Xiang et al. (US 20200099941 A1) in view of Chen; Ya et al. (US 20240214553 A1) in view of BORDES; Philippe et al. (US 20220159290 A1)
Regarding claim 11, Li with Chen teaches the limitations of claim 9,
But does not explicitly teach the additional limitations of claim 11,
However, Bordes teaches additionally,
splitting samples of the one or more reference lines (¶95,105 and fig. 7, “multi model LIC where the models are split”) into a plurality of groups of samples based on a threshold, (¶95 and 105, two different LIC “models are split by a threshold”) and
calculating the multiple sets of the LIC parameters based on the plurality of groups of samples, (¶95 and 105, “A first model and first LIC parameters are used for a first subset of the samples and a second model and second LIC parameters are used for the second subset of the samples” split by a threshold) and
the determining (¶168 and fig. 18, “determine if this reconstructed sample will be used to determine the LIC parameters for the first LIC model or for the second LIC model” t step 330 as disclosed in fig. 18) includes
selecting one of the multiple sets of the LIC parameters as the LIC parameters of the LIC mode. (¶168 and fig. 18, determine LIC parameters at step 330 where “LIC parameters for the first LIC model” are compute using LIC-MO parameters at step 332 when “reconstructed sample is < threshold” is true and “LIC parameters for the second LIC model” are compute using LIC-M1 parameters at step 333 when “reconstructed sample is < threshold” is false)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen with the threshold of Bordes which splits LIC parameters. This allows for splitting the number of samples in each model to be substantially the same, increasing the validity of the LIC models.
Claim 12 rejected under 35 U.S.C. 103 as being unpatentable over LI; Xiang et al. (US 20200099941 A1) in view of Chen; Ya et al. (US 20240214553 A1) in view of URBAN; Fabrice et al. (US 20220159277 A1)
Regarding claim 12, Li with Chen teaches the limitations of claim 9,
But does not explicitly teach the additional limitations of claim 12,
However, Urban teaches additionally,
calculating includes (¶61-72, “compute LIC parameters for the block”)
calculating the multiple sets of the LIC parameters (¶64-72, “iterative derivation of refined LIC parameters”) based on multiple rounds of refinement process on the LIC parameters, (¶64, iterative derivation of refined LIC parameters “achieved by successively increasing the number of neighboring reconstructed samples and the corresponding reference samples”) and
the determining (¶64, “deriving LIC parameters derived”) includes
determining the LIC parameters of a final round of the multiple rounds as the LIC parameter of the LIC model. (¶64, “deriving LIC parameters from partial LIC parameters derived from neighboring samples of a subblock, by deriving LIC parameters for each subblock independently” which improves the linear model)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen application of LIC parameters of Urban which refines the LIC parameters. This a process allows for improving the accuracy of the linear model.
Claim 14-15 rejected under 35 U.S.C. 103 as being unpatentable over LI; Xiang et al. (US 20200099941 A1) in view of Chen; Ya et al. (US 20240214553 A1) in view of LIU; Hongbin et al. (US 20210352309 A1)
Regarding claim 14, Li with Chen teaches the limitations of claim 13,
But does not explicitly teach the additional limitations of claim 14,
However, Liu teaches additionally,
slope adjustment value (¶471, LIC parameter comprising an “offset factor”) is selected from a predefined set of adjustment values. (¶471, “scaling factor and offset factor is predefined as a default value”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen with the LIC parameters of Liu which predefines an offset factor. This allows for a framework used to store sets of LIC parameters that are maintained according to decoding history.
Regarding claim 15, Li with Chen teaches the limitations of claim 13,
But does not explicitly teach the additional limitations of claim 15,
However, Liu teaches additionally,
slope adjustment value is signaled in the prediction information. (¶386, “determining, for a first video unit, a set of local illumination compensation (LIC) parameters” including “an offset factor” used in updating a local illumination compensation table)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen with the LIC parameters of Liu which predefines an offset factor. This allows for a framework used to store sets of LIC parameters that are maintained according to decoding history.
Claim 17-18 rejected under 35 U.S.C. 103 as being unpatentable over LI; Xiang et al. (US 20200099941 A1) in view of Chen; Ya et al. (US 20240214553 A1) in view of BANG; Gun et al. (US 20210289201 A1)
Regarding claim 17, Li with Chen teaches the limitations of claim 1,
But does not explicitly teach the additional limitations of claim 17,
However, Bang teaches additionally,
current block is coded in a sub-block mode, (¶339, luma block is partitioned into subblocks for prediction which processes the “local illumination compensation on a per-subblock basis”) and the estimating includes:
calculating the LIC parameters for each sub-block of the current block; (¶339, “calculating the local illumination compensation parameter may be determined adaptively on a per-subblock basis) and
adjusting the LIC parameters of each sub-block (¶339-343, “calculating the local illumination compensation parameter may be determined adaptively on a per-subblock basis”) based on at least one LIC parameter adjustment value. (¶339-343, reconstructing/ calculating the local illumination compensation parameter may be determined based on “the size of the current decoding block or decoding subblock, the coordinates of the decoding block or subblock, the size of the MV, the size of the difference MV, the index of the reference picture, the inter prediction mode, MV accuracy (or resolution), or the like”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen with the local illumination compensation of Bang which can be on a per-subblock basis. This provides an adaptive illumination compensation, which enhances image coding efficiency.
Regarding claim 18, Li with Chen teaches the limitations of claim 17,
Bang teaches additionally,
determining multiple adjustment values each for a separate sub-block; (¶340, “information adaptively determined on a per-subblock basis” by information on “size” of the subblock, “coordinates” of the subblock”) and
determining a final candidate value based on the multiple adjustment values. (¶339, “calculating the local illumination compensation parameter” on a per-subblock basis determined by “information on the size of the current decoding block or decoding subblock, the coordinates of the decoding block or subblock, the size of the MV, the size of the difference MV, the index of the reference picture, the inter prediction mode, MV accuracy (or resolution), or the like”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the local illumination compensation of Li with the spatial local illumination compensation of Chen with the local illumination compensation of Bang which can be on a per-subblock basis. This provides an adaptive illumination compensation, which enhances image coding efficiency.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIMMY S LEE whose telephone number is (571)270-7322. The examiner can normally be reached Monday thru Friday 10AM-8PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Joseph G. Ustaris can be reached at (571) 272-7383. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSEPH G USTARIS/Supervisory Patent Examiner, Art Unit 2483
/JIMMY S LEE/Examiner, Art Unit 2483
Prosecution Insights