DETAILED ACTION
1. The Office Action is in response to Application 18908185 filed on 11/07/2024. Claim 1-18, 20-21 are pending.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
3. The information disclosure statements (IDS) submitted on 11/11/2024, 07/16/2025, are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in Application No. 18908185 filed on 11/07/2024.
Priority # Filling Data Country
202210365196.9 2022-04-08 CN
Claim Rejections - 35 USC § 112
5. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
6. Claim 1 and its dependent claims 2-17, 21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
For claim 1, it recites “the enhancement layer”, in “and inputting the residual feature map and the correction information into a decoding network to obtain second reconstructed pixels of the enhancement layer”. However, it is not clear if the enhancement layer refer to the entire enhancement layer of the image; or the enhancement layer of the target region, since, it recites previously as: “A method for decoding a region enhancement layer” and “obtaining an enhancement layer bitstream of the target region” and “obtain a residual feature map of an enhancement layer of the target region” .
Thus the scope of the claim and its dependent claims 2-17, 21 are unclear.
7. Claim 18 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention for the similar reason as for claim 1.
8. Claim 20 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention for the similar reason as for claim 1.
9. Claim 16 and its dependent claim 17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
For claim 16, it recites “the reconstructed image” in “parsing the base layer bitstream to obtain the reconstructed image of a second base layer of the image”. However, there is no antecedence basis for this limitation.
Thus the scope of the claim and its dependent claim 17 are unclear.
Claim Rejections - 35 USC § 103
10. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
11. Claims 1-2, 18, 20 are rejected are rejected under 35 U.S.C. 103 as being unpatentable over HE et al. (CN 112702604) and in view of WANG et al. (CN 101702963).
Regarding claim 1, HE teaches a method for decoding a region enhancement layer (fig. 1, 120; page 16, … hierarchical video decoding device 120) and comprising:
obtain first reconstructed pixels of a base layer in a reconstructed image (fig. 1, the base layer decoder 121 obtained first reconstructed pixels of a base layer in a reconstructed image ; page 16, …obtain the reconstructed low quality video frame and reconstructed high quality video frame, then, from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame);
input the first reconstructed pixels into a correction network to obtain correction information of the target region (fig. 1, the first reconstructed pixels is inputted into a convolutional neural network; page 16, … the processing object of the convolutional neural network is the buffer video frame output by the basic layer decoding image buffer);
obtain an enhancement layer bitstream of the target region (fig. 1, the enhanced layer decoder 122 obtain enhancement layer bitstream of the target region);
decode the enhancement layer bitstream to obtain a residual feature map of an enhancement layer of the target region (fig. 1, the output from the inner layer video frame has residual feature map of an enhancement layer of the target region; page 16, … into the trained convolutional neural network for inner quality lifting, to obtain the inner layer video frame quality after lifting, and then sending it into the enhancement layer decoding image buffer… use the previous enhanced layer reconstruction video frame as the convolutional neural network reference information; after performing motion repair and compression repair by the convolutional neural network, obtaining the inner layer video frame; in which, motion repair and compression repair is interpreted as a residual feature map);
and input the residual feature map and the correction information into a decoding network to obtain second reconstructed pixels of the enhancement layer (fig. 1, the convolutional neural network is a decoding network and a second reconstructed pixels of the enhancement layer is the output of the convolutional neural network; from fig. 1, the two inputs to the convolutional neural works are the residual feature map and the correction information; page 16, …from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame; at the same time from the enhancement layer decoding processing unit 122 out of the previously reconstructed high quality video, together into the trained convolutional neural network for inner quality lifting… to generate a high-quality video frame after quality improvement; in which, the high-quality video frame is interpreted as the second reconstructed pixels of the enhancement layer ).
It is noticed that HE does not disclose explicitly of the target region is only a target region of a plurality of regions in a reconstructed image.
WANG discloses of the target region is only a target region of a plurality of regions in a reconstructed image (fig. 3, AB, CD are only a target region of a plurality of regions in a reconstructed image).
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to incorporate the technology that the target region is only a target region of a plurality of regions in a reconstructed image as a modification to the method for the benefit of that focus on some specific regions to reduce the calculation of motion search (page 11).
Regarding claim 18, HE teaches a decoder (fig. 1, 120; page 16, … hierarchical video decoding device 120) comprising:
a memory configured to store instructions (page 13, … at least one memory storing computer-executable instructions);
and one or more processors coupled to the memory and configured to execute the instructions to cause the decoder to (page 13, …at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the hierarchical video coding method and/or hierarchical video decoding method):
obtain first reconstructed pixels of a base layer in a reconstructed image (fig. 1, the base layer decoder 121 obtained first reconstructed pixels of a base layer in a reconstructed image ; page 16, …obtain the reconstructed low quality video frame and reconstructed high quality video frame, then, from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame);
input the first reconstructed pixels into a correction network to obtain correction information of the target region (fig. 1, the first reconstructed pixels is inputted into a convolutional neural network; page 16, … the processing object of the convolutional neural network is the buffer video frame output by the basic layer decoding image buffer);
obtain an enhancement layer bitstream of the target region (fig. 1, the enhanced layer decoder 122 obtain enhancement layer bitstream of the target region);
decode the enhancement layer bitstream to obtain a residual feature map of an enhancement layer of the target region (fig. 1, the output from the inner layer video frame has residual feature map of an enhancement layer of the target region; page 16, … into the trained convolutional neural network for inner quality lifting, to obtain the inner layer video frame quality after lifting, and then sending it into the enhancement layer decoding image buffer… use the previous enhanced layer reconstruction video frame as the convolutional neural network reference information; after performing motion repair and compression repair by the convolutional neural network, obtaining the inner layer video frame; in which, motion repair and compression repair is interpreted as a residual feature map);
and input the residual feature map and the correction information into a decoding network to obtain second reconstructed pixels of the enhancement layer (fig. 1, the convolutional neural network is a decoding network and a second reconstructed pixels of the enhancement layer is the output of the convolutional neural network; from fig. 1, the two inputs to the convolutional neural works are the residual feature map and the correction information; page 16, …from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame; at the same time from the enhancement layer decoding processing unit 122 out of the previously reconstructed high quality video, together into the trained convolutional neural network for inner quality lifting… to generate a high-quality video frame after quality improvement; in which, the high-quality video frame is interpreted as the second reconstructed pixels of the enhancement layer ).
It is noticed that HE does not disclose explicitly of the target region is only a target region of a plurality of regions in a reconstructed image.
WANG discloses of the target region is only a target region of a plurality of regions in a reconstructed image (fig. 3, AB, CD are only a target region of a plurality of regions in a reconstructed image).
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to incorporate the technology that the target region is only a target region of a plurality of regions in a reconstructed image as a modification to the decoder for the benefit of that focus on some specific regions to reduce the calculation of motion search (page 11).
Regarding claim 20, HE teaches a computer program product (fig. 9) comprising computer-executable instructions that are stored on a non-transitory computer-readable storage medium and that, when executed by one or more processors (page 13, …at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the hierarchical video coding method and/or hierarchical video decoding method), cause an apparatus (fig. 1, 120; page 16, … hierarchical video decoding device 120) to:
obtain first reconstructed pixels of a base layer in a reconstructed image (fig. 1, the base layer decoder 121 obtained first reconstructed pixels of a base layer in a reconstructed image ; page 16, …obtain the reconstructed low quality video frame and reconstructed high quality video frame, then, from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame);
input the first reconstructed pixels into a correction network to obtain correction information of the target region (fig. 1, the first reconstructed pixels is inputted into a convolutional neural network; page 16, … the processing object of the convolutional neural network is the buffer video frame output by the basic layer decoding image buffer);
obtain an enhancement layer bitstream of the target region (fig. 1, the enhanced layer decoder 122 obtain enhancement layer bitstream of the target region);
decode the enhancement layer bitstream to obtain a residual feature map of an enhancement layer of the target region (fig. 1, the output from the inner layer video frame has residual feature map of an enhancement layer of the target region; page 16, … into the trained convolutional neural network for inner quality lifting, to obtain the inner layer video frame quality after lifting, and then sending it into the enhancement layer decoding image buffer… use the previous enhanced layer reconstruction video frame as the convolutional neural network reference information; after performing motion repair and compression repair by the convolutional neural network, obtaining the inner layer video frame; in which, motion repair and compression repair is interpreted as a residual feature map);
and input the residual feature map and the correction information into a decoding network to obtain second reconstructed pixels of the enhancement layer (fig. 1, the convolutional neural network is a decoding network and a second reconstructed pixels of the enhancement layer is the output of the convolutional neural network; from fig. 1, the two inputs to the convolutional neural works are the residual feature map and the correction information; page 16, …from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame; at the same time from the enhancement layer decoding processing unit 122 out of the previously reconstructed high quality video, together into the trained convolutional neural network for inner quality lifting… to generate a high-quality video frame after quality improvement; in which, the high-quality video frame is interpreted as the second reconstructed pixels of the enhancement layer ).
It is noticed that HE does not disclose explicitly of the target region is only a target region of a plurality of regions in a reconstructed image.
WANG discloses of the target region is only a target region of a plurality of regions in a reconstructed image (fig. 3, AB, CD are only a target region of a plurality of regions in a reconstructed image).
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to incorporate the technology that the target region is only a target region of a plurality of regions in a reconstructed image as a modification to the computer program product for the benefit of that focus on some specific regions to reduce the calculation of motion search (page 11).
Regarding claim 2, the combination of HE and WANG teaches the limitations recited in claim 1 as discussed above. In addition, HE further discloses that the correction information comprises pixel values of the target region or feature values of the target region (page 3, … generating an inner layer video frame feature having motion repair information… feature having compression damage repair information).
12. Claim 16 is rejected are rejected under 35 U.S.C. 103 as being unpatentable over Claims 1, 6-8 are rejected are rejected under 35 U.S.C. 103 as being unpatentable over HE et al. (CN 112702604) and in view of WANG et al. (CN 101702963). and further in view of CUI et al. (CN 114586364).
Regarding claim 16, the combination of HE and WANG teaches the limitations recited in claim 1 as discussed above. In addition, HE further discloses that obtaining a base layer bitstream of an image to which the target region belongs (fig. 1, the base layer decoder 121 does it). WANG further discloses that wherein the target region is one of the at least one region (fig. 3/fig. 4).
The motivation of combination is the same as in claim 1’s rejection.
It is noticed that HE does not disclose explicitly of parsing the base layer bitstream to obtain the reconstructed image of a second base layer of the image; and determining at least one region to be enhanced based on the reconstructed image.
CUI discloses of parsing the base layer bitstream to obtain the reconstructed image of a second base layer of the image (fig. 15/fig. 16; page 34, the input picture can be divided into four sub-regions. The right upper sub-region may be encoded as two layers, i.e., the layer 1 and the layer 4, and the lower right sub-region may be encoded as two layers, i.e., the layer 3 and the layer 5. In this case, layer 4 can perform motion compensation prediction reference layer 1, and layer 5 can perform motion compensation with reference to layer 3; in which, since layer 5 (second base layer) used layer 3 (first base layer) as reference, it is reconstructed by parsing the first base layer bitstream));
and determining at least one region to be enhanced based on the reconstructed image (fig. 11; page 31, The enhanced CSPS layer may refer to reconstructed pixels and motion vectors of the base layer corresponding to the same region).
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to incorporate the technology that parsing the base layer bitstream to obtain the reconstructed image of a second base layer of the image; and determining at least one region to be enhanced based on the reconstructed image as a modification to the system for the benefit of that enhance the quality of a region (page 31).
13. Claim 17 is rejected are rejected under 35 U.S.C. 103 as being unpatentable over Claims 1, 6-8 are rejected are rejected under 35 U.S.C. 103 as being unpatentable over HE et al. (CN 112702604) and in view of WANG et al. (CN 101702963) and further in view of CUI et al. (CN 114586364) and further in view of QI et al. (CN 110151133).
Regarding claim 16, the combination of HE, WANG and CUI teaches the limitations recited in claim 1 as discussed above. In addition, WANG further discloses that dividing the reconstructed image to obtain a plurality of regions (fig. 3/fig. 4).
The motivation of combination is the same as in claim 1’s rejection.
It is noticed that HE does not disclose explicitly of determining as the at least one region a first region that is of the regions and that has a variance greater than a first threshold; or determining as the at least one region a second region that is of the regions and that has a threshold proportion of pixels whose gradients are greater than a second threshold.
QI discloses of determining as the at least one region a first region that is of the regions and that has a variance greater than a first threshold (page 11, determining characteristic threshold value t, the image to be reconstructed is divided into target area Ω and the background area Ω b two parts; when the area of uai <t is Ω b, when the area of the uai ≥ t to Ω t); or determining as the at least one region a second region that is of the regions and that has a threshold proportion of pixels whose gradients are greater than a second threshold.
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to incorporate the technology that determining as the at least one region a first region that is of the regions and that has a variance greater than a first threshold; or determining as the at least one region a second region that is of the regions and that has a threshold proportion of pixels whose gradients are greater than a second threshold as a modification to the method for the benefit of that divide image according to different criteria (page 11).
Conclusion
14 The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See form 892.
15. Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZAIHAN JIANG whose telephone number is (571)272-1399. The examiner can normally be reached on flexible.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath Perungavoor can be reached on (571)272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-270-0655.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZAIHAN JIANG/Primary Examiner, Art Unit 2488