DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is responsive to the correspondence filled on 3/7/25.
Claims 1-20 are presented for examination.
IDS Considerations
The information disclosure statement (IDS) submitted on 12/1/25 and 3/7/25 is/are being considered by the examiner as the submission is in compliance with the provisions of 37 CFR 1.97.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5, 8-10, 12, 15-17 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over MacInnis (U.S. Pub. No. 20100232497 A1), in view of Thirumalai (U.S. Pub. No. 20160309149 A1).
Regarding to claim 1, 8 and 15:
Examiner’s note: Encoding and decoding are done using same and opposite algorithm.
1. MacInnis teach an image decoding method, comprising: (MacInnis Fig. 1A [0013] FIG. 7 is a block diagram of an exemplary video decoder) obtaining a number of coded bits (MacInnis [0102] The VLD 710 extracts from the incoming bit stream a number of bits equal to the determined sample size times the number of samples per Unit. Once these bits have been extracted, they are de-multiplexed and processed by subsequent decoding steps; these operations can be pipelined.) of a previous coding unit and a bitstream of a current coding unit in a current frame, (MacInnis [0024] The input, predictor & quantizer 105 predicts and quantizes the pixel components, resulting in quantized residuals. In certain embodiments of the present invention, the input, predictor, & quantizer 105 can predict each pixel components from previously encoded and reconstructed pixel components in the same frame 102( ). The mapper and variable length coder 110 codes the quantized residuals, resulting in coded bits. [0084] In an exemplary embodiment, the encoder and decoder perform identical rate control (RC) decisions, which control the value of Quant, without the encoder transmitting any bits to indicate quantization control. The rate control depends on the activity, measured by the sizes of the samples, of the previous Group, as well as fullness of the virtual buffer model, and a measure of the strength of an edge, if any, in the preceding samples) wherein the current coding unit is a coding unit decoded at a current moment, and the previous coding unit is a coding unit decoded at a moment before the current moment; (MacInnis [0071] The predicted size is a function of the sizes of previously coded samples. In an exemplary embodiment, the predicted size is the average, with rounding, of the sizes of the samples of the same component of the previous two samples, i.e. of the previous Unit, given that the Unit size is 2. If the Unit size is 4, the predicted size is the average of the sizes of the four samples of the same component of the previous Unit. If the Unit size is 3, the predicted size is generated by the average of the sizes of the last two samples of the same component of the previous Unit, thereby avoiding division of 3.) and determining a number of lossless bits of the current coding unit (MacInnis [0067] The operation of the Mapper and Variable Length Coder 110 will now be described with reference to FIG. 5, in accordance with an embodiment of the present invention. The Mapper and Variable Length Coder 110 uses entropy coding wherein coded sample values are coded using their natural 2s complement binary values, and the number of bits used to code each value is determined dynamically by a combination of the recent history of coded values of the same component and a prefix value associated with each Unit 505 of samples. In certain embodiments, a Unit 505 comprises 2 samples 510 of one component type, i.e. Y, Cb or Cr, or Alpha, R, G or B. Entropy coding is lossless coding. [0075] Referring again to FIG. 3, the quantizer 323 quantizes the residuals E'', which in general includes the case of lossless coding, using a quantization parameter Quant. Quant takes on values ranging from 0, i.e. lossless, to the value that corresponds to the highest value of QuantDivisor[ ] (see below). With an exemplary set of values of QuantDivisor and QuantOffset shown below, the value of Quant ranges from 0 to 17.) based on the number of coded bits of the previous coding unit, (MacInnis [0084] In an exemplary embodiment, the encoder and decoder perform identical rate control (RC) decisions, which control the value of Quant, without the encoder transmitting any bits to indicate quantization control. The rate control depends on the activity, measured by the sizes of the samples, of the previous Group, as well as fullness of the virtual buffer model, and a measure of the strength of an edge, if any, in the preceding samples.) and determining an average number of lossless bits of the current coding unit based on the number of coded bits of the previous coding unit, (MacInnis [0071] The predicted size is a function of the sizes of previously coded samples. In an exemplary embodiment, the predicted size is the average, with rounding, of the sizes of the samples of the same component of the previous two samples, i.e. of the previous Unit, given that the Unit size is 2. If the Unit size is 4, the predicted size is the average of the sizes of the four samples of the same component of the previous Unit. If the Unit size is 3, the predicted size is generated by the average of the sizes of the last two samples of the same component of the previous Unit, thereby avoiding division of 3.) wherein the number of lossless bits of the current coding unit indicates an expected number of bits obtained by performing lossless coding on the current coding unit; (MacInnis [0090] The following pseudo-code describes control of the quantization parameter, Quant, where the virtual buffer size is 16 Kbits. In this pseudo-code, "MaxBitsPerGroup" represents the pre-determined data rate in bits per group. Offset[ ] is a set of values that adjust the "target [expected] activity level" according to the fullness of the buffer model, which is represented by "Buffer fullness", and which is compared to various threshold values represented by BufTh1, BufTh2, etc. TABLE-US-00009 // Set target number of bits per Group according to buffer fullness if(Buffer_fullness < BufTh1))
MacInnis do not explicitly teach the average number of lossless bits of the current coding unit indicates an expected number of bits obtained by performing lossless coding on the current frame; and the number of lossless bits of the current coding unit and the average number of lossless bits of the current coding unit are used to determine a first quantization parameter used for decoding the bitstream of the current coding unit.
However Thirumalai teach the average number of lossless bits of the current coding unit indicates an expected number of bits obtained by performing lossless coding on the current frame; (Thirumalai [0066] In one approach, the QP for the current block (denoted as currQP) may be derived or calculated using the following equation:
PNG
media_image1.png
23
387
media_image1.png
Greyscale
[0067] where prevQP is the QP associated with the previous block, diffBits represents the difference between the previousBlockBits and targetBits, QpAdj is the QP offset value (e.g., QP adjustment value) that is calculated based on the magnitude of diffBits, previousBlockBits represents the number of bits used to code the previous block, and targetBits represents a target number of bits allocated to code the current block. When previousBlockBits>targetBits, diffBits is positive, and the current block QP may be derived by adding the offset value QpAdj to the prevQP value. In other words, the QP value does not decrease in value from the prevQP value when diffBits is positive. When previousBlockBits≦targetBits, diffBits is negative or zero, and currQP does not increase from the prevQP value. It is noted that the offset value QpAdj may be calculated, for example, as a function of diffBits in such a way that QpAdj monotonically increases as the magnitude of diffBits increases. [0072] FIG. 4 illustrates an example region of interest 200 which may be a frame or portion thereof, such as, for example, a slice of the frame. The region 200 may include and three successive blocks 205, 210, and 215. In this example, block 205 corresponds to a flat region/portion of the region 200, block 210 corresponds to a transition region/portion of the region 200, and block 215 corresponds to a complex region/portion of the region 200. As shown, the content of block 205 is flat, smooth, or uniform. The content of block 215 is textured and exhibits patterns that are not uniform throughout. The block 210 includes a transition from uniform to non-uniform content. A complexity calculation made be performed for each of the blocks 205, 210, and 215. [0073] As described above under the section “QP Calculation”, technique(s) may be implemented to calculate the QP based on the prevBlockBits which considers the local variations in the visual information. That is, while coding highly textured regions, the QP tends to increase and while coding smooth regions, the QP tends to decrease. Described below are QP calculation technique(s) that utilize the visual information present in future blocks which may be known or predicted.) and the number of lossless bits of the current coding unit and the average number of lossless bits of the current coding unit are used to determine a first quantization parameter used for decoding the bitstream of the current coding unit. (Thirumalai [0057] The rate controller 120 determines a set of coding parameters, e.g., a QP. The QP may be adjusted by the rate controller 120 based on the buffer fullness of the rate buffer 150 and image activity of the video data (e.g., a transition from complex to flat regions or vice versa) in order to maximize picture quality for a target bitrate which ensures that the rate buffer 150 does not overflow or underflow. The rate controller 120 also selects a particular coding option (e.g., a particular mode) for each block of the video data in order to achieve the optimal rate-distortion performance. The rate controller 120 minimizes the distortion of the reconstructed images such that it satisfies the bit-rate constraint, i.e., the overall actual coding rate fits within the target bit rate. Thus, one purpose of the rate controller 120 is to determine a set of coding parameters, such as QP(s), coding mode(s), etc., to satisfy instantaneous and average constraints on rate while maximizing rate-distortion performance. Thirumalai [0066] In one approach, the QP for the current block (denoted as currQP) may be derived or calculated using the following equation:
PNG
media_image1.png
23
387
media_image1.png
Greyscale
)
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify MacInnis, further incorporating Thirumalai in video/camera technology. One would be motivated to do so, to incorporate the average number of lossless bits of the current coding unit indicates an expected number of bits obtained by performing lossless coding on the current frame. MacInnis teach decoder level circuitry and Thirumalai teach frame level details. The combined functionality teaching will improve efficiency with predictable results.
Regarding to claim 2, 9 and 16:
2. MacInnis teach the method according to claim 1, wherein the determining a number of lossless bits of the current coding unit based on the number of coded bits of the previous coding unit comprises: (MacInnis [0071] The predicted size is a function of the sizes of previously coded samples. In an exemplary embodiment, the predicted size is the average, with rounding, of the sizes of the samples of the same component of the previous two samples, i.e. of the previous Unit, given that the Unit size is 2. If the Unit size is 4, the predicted size is the average of the sizes of the four samples of the same component of the previous Unit. If the Unit size is 3, the predicted size is generated by the average of the sizes of the last two samples of the same component of the previous Unit, thereby avoiding division of 3.)
MacInnis do not explicitly teach determining the number of lossless bits of the current coding unit based on the number of coded bits of the previous coding unit and a second quantization parameter, wherein the previous coding unit is the coding unit decoded at the moment before the current moment, the second quantization parameter is used to decode a bitstream of the previous coding unit, and the number of lossless bits of the current coding unit is used to update a number of lossless bits corresponding to image content of the current coding unit.
However Thirumalai teach determining the number of lossless bits of the current coding unit based on the number of coded bits of the previous coding unit and a second quantization parameter, (Thirumalai [0066] In one approach, the QP for the current block (denoted as currQP) may be derived or calculated using the following equation: [0067] where prevQP is the QP associated with the previous block. Current and previous QP are firest and second QP) wherein the previous coding unit is the coding unit decoded at the moment before the current moment, the second quantization parameter is used to decode a bitstream of the previous coding unit, (Thirumalai [0066] In one approach, the QP for the current block (denoted as currQP) may be derived or calculated using the following equation:
PNG
media_image1.png
23
387
media_image1.png
Greyscale
[0067] where prevQP is the QP associated with the previous block, diffBits represents the difference between the previousBlockBits and targetBits, QpAdj is the QP offset value (e.g., QP adjustment value) that is calculated based on the magnitude of diffBits, previousBlockBits represents the number of bits used to code the previous block, and targetBits represents a target number of bits allocated to code the current block. When previousBlockBits>targetBits, diffBits is positive, and the current block QP may be derived by adding the offset value QpAdj to the prevQP value. In other words, the QP value does not decrease in value from the prevQP value when diffBits is positive. When previousBlockBits≦targetBits, diffBits is negative or zero, and currQP does not increase from the prevQP value. It is noted that the offset value QpAdj may be calculated, for example, as a function of diffBits in such a way that QpAdj monotonically increases as the magnitude of diffBits increases. [0072] FIG. 4 illustrates an example region of interest 200 which may be a frame or portion thereof, such as, for example, a slice of the frame. The region 200 may include and three successive blocks 205, 210, and 215. In this example, block 205 corresponds to a flat region/portion of the region 200, block 210 corresponds to a transition region/portion of the region 200, and block 215 corresponds to a complex region/portion of the region 200. As shown, the content of block 205 is flat, smooth, or uniform. The content of block 215 is textured and exhibits patterns that are not uniform throughout. The block 210 includes a transition from uniform to non-uniform content. A complexity calculation made be performed for each of the blocks 205, 210, and 215. [0073] As described above under the section “QP Calculation”, technique(s) may be implemented to calculate the QP based on the prevBlockBits which considers the local variations in the visual information. That is, while coding highly textured regions, the QP tends to increase and while coding smooth regions, the QP tends to decrease. Described below are QP calculation technique(s) that utilize the visual information present in future blocks which may be known or predicted.)
and the number of lossless bits of the current coding unit is used to update a number of lossless bits corresponding to image content of the current coding unit. (Thirumalai [0084] Each technique used to update the QP may be different depending the type of visual information that has to be coded. For example, when all of the N blocks that have to be coded are highly textured then an alternative technique may be chosen to update the QP aggressively relative to the “default technique” (e.g., the technique described above in connection with FIG. 3). Alternatively, when a subset of blocks out of N are textured (e.g., the complexity of the subset of blocks is greater than a threshold value) and the remaining blocks are not complex (e.g., the complexity of the remaining blocks is less than a threshold value) or a first set of N blocks are textured and a next set of N blocks are not complex, then the encoder 20 and/or decoder 30 may select an alternative technique that is not very aggressive compared to the default technique when incrementing the QP value while coding the textured regions (e.g., the first set of N blocks). Alternatively, while coding flat regions, the encoder 20 and/or decoder 30 may select an alternative technique that is conservative in decreasing the QP relative to the default technique.)
Regarding to claim 3, 10 and 17:
3. MacInnis teach the method according to claim 2, wherein the method further comprises: determining, based on the number of coded bits of the previous coding unit, a number of bits of data in a bitstream buffer obtained by decoding the previous coding unit, wherein the bitstream buffer is used to store a bitstream or a partial bitstream of one or more coding units. (MacInnis [0081] The value of Quant is determined via a rate control scheme, which in an exemplary embodiment is performed identically in both the encoder and decoder. The rate control scheme bases its decisions on a measure of the activity of the most recently coded predetermined number of pixel components and on the fullness of the buffer model. The value of Quant is updated once per coded predetermined number of pixel components. [0082] Referring now to FIG. 6, there is illustrated a block diagram of an exemplary buffer model in accordance with an embodiment of the present invention. The model comprises a bitstream source 605, virtual buffer model 610, rate controller 120, and a bitstream consumption module 615. The virtual buffer model 610 models the behavior of a rate buffer where the output bit rate is exactly the specified bit rate. The specified bit rate may be in units of bits per pixel or per group of pixels, or it may be in other units such as bits per unit of time, such as bits per second. The bitstream source 605 can comprise the output of the Mapper and Variable Length Coder 110. Bits enter the virtual buffer model 610 when they are created. For example, the number of bits used to code a Group is added to the model 610 when the Group is coded. Bits leave the virtual buffer model 610 according to a pre-determined schedule. For example, the schedule may consist of a constant rate in units of bits per group. The virtual buffer model 610 may be implemented as an accumulator, where one value is added and other value is subtracted for every Group. Alternatively, the schedule of removing bits from the virtual buffer model may be in units of bits per second. Alternatively, the times at which bits are added to or subtracted from the buffer model 610 may be finer or coarser than a Group, and may use a construct other than a Group, such as a sample, a macroblock, a slice or a picture. In order to model the behavior of a FIFO, the fullness of the virtual buffer model 610 may be clamped to 0 when subtracting a number of bits from the fullness would otherwise result in a negative value of fullness)
Regarding to claim 5, 12 and 19:
5. MacInnis teach the method according to claim 1, wherein the method further comprises: decoding the bitstream of the current coding unit based on the first quantization parameter, to obtain a reconstructed image of the current coding unit. (MacInnis [0105] Referring now to FIG. 8, there is illustrated a flow diagram describing video encoding in accordance with an embodiment of the present invention. At 805, the quantization step is initialized to zero. At 810, a unit of pixel components is received. At 815, the encoding includes quantization using the quantization step and encoding the quantized values. At 820, the fullness of the virtual buffer is measured. At 825, the quantization step is adjusted based on the fullness measured at 820. At 830, a determination is made whether decoding of the frame is finished. If encoding of the frame is finished, 805 is repeated. If encoding of the frame is not finished, 810 is repeated. [0106] Referring now to FIG. 9, there is illustrated a flow diagram describing video decoding in accordance with an embodiment of the present invention. At 905, the quantization step is initialized to zero. At 910, a coded unit is decoded and the virtual buffer is updated. At 915, the decoding includes dequantizing using the quantization step. At 920, the fullness of the virtual buffer is measured. At 925, the quantization step is adjusted based on the fullness measured at 920. At 930, a determination is made whether decoding of the frame is finished [reconstructed image]. If encoding of the frame is finished, 905 is repeated. If encoding of the frame is not finished, 910 is repeated)
Claims 6, 13 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over MacInnis (U.S. Pub. No. 20100232497 A1), in view of Thirumalai (U.S. Pub. No. 20160309149 A1), further in view of Jang (U.S. Pub. No. 20200082506 A1).
Regarding to claim 6, 13 and 20:
6. MacInnis teach the method according to claim 1, MacInnis do not explicitly teach wherein the current coding unit and the previous coding unit are non-adjacent coding units.
However Jang teach wherein the current coding unit and the previous coding unit are non-adjacent coding units. (Jang [0066] There may be none of a left coding unit, an above coding unit and an above left coding unit of the current coding unit. On the other hand, there may be a previous coding unit of the current coding unit in coding order [non-adjacent coding units]. Thus, the quantization parameters of neighboring coding units adjacent to the current coding unit and the previous coding unit may be used to generate the quantization parameter predictor. The quantization parameters are retrieved as the following order; 1) the quantization parameter of a left neighboring coding unit, 2) the quantization parameter of an above neighboring coding unit, 3) the quantization parameter of an above-left neighboring coding unit, and 4) the quantization parameter of the previous coding unit.)
The motivation for combining MacInnis and Thirumalai as set forth in claim 1 is equally applicable to claim 6. It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify MacInnis, further incorporating Thirumalai and Jang in video/camera technology. One would be motivated to do so to incorporate the current coding unit and the previous coding unit are non-adjacent coding units. The combined functionality teaching will improve quality with predictable results.
Allowable subject matter
Regarding to claim 4, 7, 11, 14 and 18:
Claims 4, 7, 11, 14 and 18 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims because the limitations of these dependent claims are not obvious from the prior art search when all the limitations of independent and intervening claims are taken into account.
Regarding to claim 4, 11 and 18:
4. MacInnis teach the method according to claim 3, wherein the method further comprises: determining the number of lossless bits of the current coding unit based on the image content of the current coding unit; determining a number of lossy bits of the current coding unit based on the number of bits of the data in the bitstream buffer obtained by decoding the previous coding unit; determining an information amount of the current coding unit based on the number of lossy bits of the current coding unit and the average number of lossless bits of the current coding unit, (Please the rejection of claim 1. These limitations are repetitions and/or obvious over claim 1)
Prior art do not teach following specific details of pipe-lining steps: wherein the information amount indicates complexity of content that is expressed by the current coding unit and that is in content expressed by the current frame; determining a target number of bits of the current coding unit based on the number of lossless bits of the current coding unit and the information amount of the current coding unit, wherein the target number of bits of the current coding unit indicates an expected number of bits obtained by performing lossy coding on the current coding unit when the content of the current coding unit is referred to; and determining the first quantization parameter based on the target number of bits of the current coding unit.
Regarding to claim 7 and 14:
Claims 7 and 14 are rejected because of dependency.
Closely related prior art
Examiner notes teaching of U.S. Pub. No. Wang (U.S. Pub. No. 20250337915 A1) is/are pertinent to the independent claim(s), however is not used because dependent claims are better covered by primary reference.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NASIM N NIRJHAR whose telephone number is (571) 272-3792. The examiner can normally be reached on Monday - Friday, 8 am to 5 pm ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William F Kraig can be reached on (571) 272-8660. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NASIM N NIRJHAR/Primary Examiner, Art Unit 2896
Prosecution Insights