Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. The Office Action is in response to amendment filed on 04/13/2026.
Response to Amendment
3. The amendment filed on 04/13/2026, independent Claim 1, 18, 20 and claim 7 have been amended; Claim 19 is canceled; therefore, 1-18, 20-21 are pending.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in Application No. 18908185 filed on 11/07/2024.
Priority # Filling Data Country
202210365196.9 2022-04-08 CN
5. Response to Arguments
Applicant’s arguments filed on 04/13/2026, pages 9-15 have been fully considered.
Claim Rejections - 35 USC §112
The amendment filed on 04/13/2026 on independent claims 1, 18, 20 overcomes the 112(b) rejection in 04/13/2026.
New ground of rejection is found and presented in this office action
As the 112(b) rejection on claim 16, applicant argued that “claim 16 depends from claim 1, which recites obtaining first reconstructed pixels of a base layer of "only a target region
of a plurality of regions in a reconstructed image." Thus, "a reconstructed image" from claim 1
provides antecedent basis for "the reconstructed image" in claim 16, such that "the reconstructed
image" in claim 16 unambiguously refers to that antecedent”.
Examiner’s Response:
The “reconstructed image” in independent claim 1 is very general, it recites as:” obtaining first reconstructed pixels of a first base layer of only a target region of a plurality of regions in a reconstructed image”; however, the reconstructed image in claim 16 is very specific recited as: “parsing the base layer bitstream to obtain the reconstructed image of a second base layer of the image”; in which, the reconstructed image is an reconstructed image of a second base layer of the image; compared with these two limitations in claim 1 and 16, it can easily tell that the reconstructed image in claim 1 is for first base layer, and the second reconstructed image in claim 16 is of a reconstructed image of second base layer. Therefore, it is not justified to state that "a reconstructed image" from claim 1 provides antecedent basis for "the reconstructed image" in claim 16, since they refer to different things.
Therefore, the 112(b) rejection on claim 16 maintained.
Claim Rejections - 35 USC §103
Applicant’s arguments with respect to claim under 35 U.S.C. § 103 has been fully considered.
Basically, applicant argued that the prior arts (HE et al. (CN 112702604) and in view of WANG et al. (CN 101702963)) does not discloses the amended limitation of : “inputting the first reconstructed pixels into a correction network to obtain correction information of the target region” and “inputting the residual feature map and the correction information into a
decoding network to obtain second reconstructed pixels of the enhancement layer of the target region” in amended independent claims, since:
“Claim 1 requires a two-network pipeline: first, a correction network receives the first reconstructed pixels and produces correction information as its output; second, that correction
information, together with the residual feature map, is input into a separate, downstream decoding network to produce the second reconstructed pixels. These two steps are not merely sequential; they are structurally linked by the requirement that the output of the correction network serves as a discrete input to the decoding network. Claims 18 and 20 contain similar limitations".
“While He discloses a convolutional neural network that receives base layer video frames as input and produces a quality-improved inner layer video frame as output, He does not disclose a two-network pipeline in which a correction network produces correction information as a discrete intermediate output that is then passed as a separate input into a decoding network alongside the residual feature map”.
“He discloses a single convolutional neural network that simultaneously receives multiple inputs, i.e., the current and previous base layer video frames and the previously reconstructed enhancement layer video frame, and produces a single output, which is a quality-improved inner layer video frame. This single convolutional neural network, as aught by He, is not a correction network that feeds correction information into a separate decoding network, in the manner required by the claims. Instead, the Examiner's mapping requires that He's single convolutional neural network simultaneously serve as both the correction network and the decoding network in the two-network pipeline of claim 1. This is logically impossible. That is, a single network element cannot satisfy two structurally distinct and sequentially dependent claim elements. Claim 1 requires that correction information, which is the output of the correction network, exists as a discrete intermediate output that is then passed as a separate input into the decoding network alongside the residual feature map. In He, there is no such intermediate output; there is no discrete handoff between two distinct networks; and there is no downstream network that
receives both a residual feature map and correction information as concurrent discrete inputs. He's single convolutional neural network collapses the entire pipeline into a single operation with a single output, which is incompatible with the two-network structure required by claim 1”
“Additionally, Wang, Cui, and Qi do not supply the shortcomings of He with respect to this structural limitation. As such, the combination of He, Wang, Cui, and Qi fails to disclose all of the limitations set forth in independent claims 1, 18, and 20, and consequently does not render obvious claims 1-2, 16-18 and 20”.
Examiner’s Response:
After reviewing the claim limitations and the prior arts, examiner believe that the current prior arts (HE et al. (CN 112702604) and in view of WANG et al. (CN 101702963)) still teach the aforementioned limitation. Follows are reason:
The current claim language in independent claims are broad, it does not reflect what the applicant argued. For example, it does not recite that the correction network and “a decoding network” should be two different networks.
HE’s convolution network, as shown in fig. 4, has many components and different layers, also as suggested in page 20-21, as: “that the structure and various parameters of the convolutional neural network shown in FIG. 4 and described in the following description and various parameters are used for implementing example of the operation of the convolutional neural network and each module described above with reference to FIG. 2 to FIG. 3… FIG. 4 (a) shows the specific structure of the motion repairing module according to the exemplary embodiment of the present disclosure. As shown in FIG. 4 (a), the motion repair module may include a feature extraction module, multi-scale feature extraction operator, space attention set mechanism and conventional convolution module… As shown in FIG. 4 (a), the input of the characteristic extraction module is sent to the convolution layer (1, k, 3), ReLU activation, convolution layer (k, k, 3), ReLU activation, convolution layer (k, k, 3), three parameters of the conventional convolution respectively represents the input channel number; output channel number and convolution kernel size, here, k can be 16, 32, 48, 64 and so on. and in the output of the feature extraction module”;
Therefore, it is justified to interpreted different layers/components of the convolution network, especially if one layer/component output result to another one, as two networks with two pipeline process,
The combination of HE and WANG the limitations of “inputting the first reconstructed pixels into a correction network to obtain correction information of the target region” and “inputting the residual feature map and the correction information into a decoding network to obtain second reconstructed pixels of the enhancement layer of the target region”; for example, HE teaches that of “inputting the first reconstructed pixels into a correction network to obtain correction information of the target region” in fig. 1, the first reconstructed pixels is inputted into a convolutional neural network; page 16, “… the processing object of the convolutional neural network is the buffer video frame output by the basic layer decoding image buffer”; also in fig. 3 and fig. 4, the motion repair modules/compression damage repair module, is interpreted as a correction network and the output from these two modules are interpreted as the correction information of the target region; also in page 18-19, as: “include a motion repair module 310, a compression damage repair module 320 and a fusion module 330… The motion repair module 310 may generate an inner layer video frame having motion repair information… the compression damage repair module 320 can according to the compression damage feature between the previously reconstructed base layer video frame and the previously reconstructed enhancement layer video frame and the current enhancement layer video frame and the current base layer video frame between the compression damage feature similarity from the current base layer video frame generated with compression damage repair information…”.
The combination of HE and WANG teaches the limitations of “inputting the residual feature map and the correction information into a decoding network to obtain second reconstructed pixels of the enhancement layer of the target region” in fig. 1 the convolutional neural network is a decoding network and a second reconstructed pixels of the enhancement layer is the output of the convolutional neural network; from fig. 1, the two inputs to the convolutional neural works are the residual feature map and the correction information; page 16, “…from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame; at the same time from the enhancement layer decoding processing unit 122 out of the previously reconstructed high quality video, together into the trained convolutional neural network for inner quality lifting… to generate a high-quality video frame after quality improvement; in which, the high-quality video frame is interpreted as the second reconstructed pixels of the enhancement layer”; also in fig. 3 and fig. 4, in which, the residual feature map and the correction information from motion repair module and compression repair module are input to the fusion module (interpreted as a decoding network) to obtain second reconstructed pixels of the enhancement layer of the target region; as in page 20-21, “….in step S220, the fusion module 330 can perform high frequency feature repair to the inner layer video frame feature with motion repair information and the inner layer video frame feature with compressed damage repair information to generate the fused inner layer video frame feature, and then; The present base layer video frame is repaired by using the fused inner layer video frame feature to generate the inner layer video frame with enhanced quality. The fusion module 330 can repair the high frequency characteristics by the following operations to generate the fused inner layer video frame characteristics: cascading the inner layer video frame feature with motion repairing information and the inner layer video frame feature with the compressed damage repair information; performing multiple convolution operations on the cascade inner layer video frame feature with motion repair information and the inner layer video frame feature with the compressed damage repair information, wherein Each convolution operation in the plurality of convolution operations uses a different number of filters…” and page 27-28, “…the fusion module, as shown in FIG. 3, can be described as:
input respectively as the basic layer video frame of the current time i ; the reconstructed forward base layer video frame ; reconstructed forward enhancement layer video frame ; reconstructing the backward base layer video frame and reconstructed backward enhancement layer video frames”
Therefore, the combination of HE and WANG discloses the limitations of “inputting the first reconstructed pixels into a correction network to obtain correction information of the target region” and “inputting the residual feature map and the correction information into a decoding network to obtain second reconstructed pixels of the enhancement layer of the target region” in independent claims.
The applicant also argued that dependent claims should be allowed due to their dependency on independent claims.
Examiner’s Response:
As discussed above, the combination of HE and WANG discloses the limitations in independent claims.
Claim Rejections - 35 USC § 112
6. The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
7. Claim 1 and its dependent claims 2-17, 21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
For claim 1, it recites “a region enhancement layer”, in “A method for decoding a region enhancement layer”. However, it is not clear what is a region enhancement layer? Does it mean a region of enhancement layer or does it mean a special enhancement layer named “region enhancement layer”? or means something else?
Thus the scope of the claim and its dependent claims 2-17, 21 are unclear.
8. Claim 18 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention for the similar reason as for claim 1.
9. Claim 20 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention for the similar reason as for claim 1.
10. Claim 16 and its dependent claim 17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
For claim 16, it recites “the reconstructed image” in “parsing the base layer bitstream to obtain the reconstructed image of a second base layer of the image”. However, there is no antecedence basis for this limitation.
Thus the scope of the claim and its dependent claim 17 are unclear.
Examiner’s Note:
The “reconstructed image” in independent claim 1 is very general, it recites as:” obtaining first reconstructed pixels of a first base layer of only a target region of a plurality of regions in a reconstructed image”; however, the reconstructed image in claim 16 is very specific recited as: “parsing the base layer bitstream to obtain the reconstructed image of a second base layer of the image”; in which, the reconstructed image is an reconstructed image of a second base layer of the image; compared with these two limitations in claim 1 and 16, it can easily tell that the reconstructed image in claim 1 is for first base layer, and the second reconstructed image in claim 16 is of a reconstructed image of second base layer. Therefore, it is not justified to state that "a reconstructed image" from claim 1 provides antecedent basis for "the reconstructed image" in claim 16, since they refer to different things.
Claim Rejections - 35 USC § 103
. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
11. Claims 1-2, 18, 20 are rejected are rejected under 35 U.S.C. 103 as being unpatentable over HE et al. (CN 112702604) and in view of WANG et al. (CN 101702963).
Examiner’s Note:
. HE’s convolution network, as shown in fig. 4, as many components and different layers, also suggested in page 20-21, as: “that the structure and various parameters of the convolutional neural network shown in FIG. 4 and described in the following description and various parameters are used for implementing example of the operation of the convolutional neural network and each module described above with reference to FIG. 2 to FIG. 3… FIG. 4 (a) shows the specific structure of the motion repairing module according to the exemplary embodiment of the present disclosure. As shown in FIG. 4 (a), the motion repair module may include a feature extraction module, multi-scale feature extraction operator, space attention set mechanism and conventional convolution module… As shown in FIG. 4 (a), the input of the characteristic extraction module is sent to the convolution layer (1, k, 3), ReLU activation, convolution layer (k, k, 3), ReLU activation, convolution layer (k, k, 3), three parameters of the conventional convolution respectively represents the input channel number; output channel number and convolution kernel size, here, k can be 16, 32, 48, 64 and so on. and in the output of the feature extraction module”; Therefore, it is justified to interpreted different layers/components of the convolution network, especially if one layer/component output result to another one, as two networks.
Regarding claim 1, HE teaches a method for decoding a region enhancement layer (fig. 1, 120; page 16, … hierarchical video decoding device 120) and comprising:
obtain first reconstructed pixels of a base layer in a reconstructed image (fig. 1, the base layer decoder 121 obtained first reconstructed pixels of a base layer in a reconstructed image ; page 16, …obtain the reconstructed low quality video frame and reconstructed high quality video frame, then, from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame);
input the first reconstructed pixels into a correction network to obtain correction information of the target region (fig. 1, the first reconstructed pixels is inputted into a convolutional neural network; page 16, … the processing object of the convolutional neural network is the buffer video frame output by the basic layer decoding image buffer; also in fig. 3 and fig. 4, the motion repair modules/compression damage repair module, is interpreted as a correction network and the output from these two modules are interpreted as the correction information of the target region; also in page 18-19, as: “include a motion repair module 310, a compression damage repair module 320 and a fusion module 330… The motion repair module 310 may generate an inner layer video frame having motion repair information… the compression damage repair module 320 can according to the compression damage feature between the previously reconstructed base layer video frame and the previously reconstructed enhancement layer video frame and the current enhancement layer video frame and the current base layer video frame between the compression damage feature similarity from the current base layer video frame generated with compression damage repair information…”);
obtain an enhancement layer bitstream of the target region (fig. 1, the enhanced layer decoder 122 obtain enhancement layer bitstream of the target region);
decode the enhancement layer bitstream to obtain a residual feature map of an enhancement layer of the target region (fig. 1, the output from the inner layer video frame has residual feature map of an enhancement layer of the target region; page 16, … into the trained convolutional neural network for inner quality lifting, to obtain the inner layer video frame quality after lifting, and then sending it into the enhancement layer decoding image buffer… use the previous enhanced layer reconstruction video frame as the convolutional neural network reference information; after performing motion repair and compression repair by the convolutional neural network, obtaining the inner layer video frame; in which, motion repair and compression repair is interpreted as a residual feature map);
and input the residual feature map and the correction information into a decoding network to obtain second reconstructed pixels of the enhancement layer of the target region (fig. 1, the convolutional neural network is a decoding network and a second reconstructed pixels of the enhancement layer is the output of the convolutional neural network; from fig. 1, the two inputs to the convolutional neural works are the residual feature map and the correction information; page 16, …from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame; at the same time from the enhancement layer decoding processing unit 122 out of the previously reconstructed high quality video, together into the trained convolutional neural network for inner quality lifting… to generate a high-quality video frame after quality improvement; in which, the high-quality video frame is interpreted as the second reconstructed pixels of the enhancement layer; also in fig. 3 and fig. 4, in which, the residual feature map and the correction information from motion repair module and compression repair module are input to the fusion module (interpreted as a decoding network) to obtain second reconstructed pixels of the enhancement layer of the target region; as in page 20-21, “….in step S220, the fusion module 330 can perform high frequency feature repair to the inner layer video frame feature with motion repair information and the inner layer video frame feature with compressed damage repair information to generate the fused inner layer video frame feature, and then; The present base layer video frame is repaired by using the fused inner layer video frame feature to generate the inner layer video frame with enhanced quality. The fusion module 330 can repair the high frequency characteristics by the following operations to generate the fused inner layer video frame characteristics: cascading the inner layer video frame feature with motion repairing information and the inner layer video frame feature with the compressed damage repair information; performing multiple convolution operations on the cascade inner layer video frame feature with motion repair information and the inner layer video frame feature with the compressed damage repair information, wherein Each convolution operation in the plurality of convolution operations uses a different number of filters…” and page 27-28, “…the fusion module, as shown in FIG. 3, can be described as:
input respectively as the basic layer video frame of the current time i ; the reconstructed forward base layer video frame ; reconstructed forward enhancement layer video frame ; reconstructing the backward base layer video frame and reconstructed backward enhancement layer video frames” ).
It is noticed that HE does not disclose explicitly of the target region is only a target region of a plurality of regions in a reconstructed image.
WANG discloses of the target region is only a target region of a plurality of regions in a reconstructed image (fig. 3, AB, CD are only a target region of a plurality of regions in a reconstructed image).
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to incorporate the technology that the target region is only a target region of a plurality of regions in a reconstructed image as a modification to the method for the benefit of that focus on some specific regions to reduce the calculation of motion search (page 11).
Regarding claim 18, HE teaches a decoder (fig. 1, 120; page 16, … hierarchical video decoding device 120) comprising:
a memory configured to store instructions (page 13, … at least one memory storing computer-executable instructions);
and one or more processors coupled to the memory and configured to execute the instructions to cause the decoder to (page 13, …at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the hierarchical video coding method and/or hierarchical video decoding method):
obtain first reconstructed pixels of a base layer in a reconstructed image (fig. 1, the base layer decoder 121 obtained first reconstructed pixels of a base layer in a reconstructed image ; page 16, …obtain the reconstructed low quality video frame and reconstructed high quality video frame, then, from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame);
input the first reconstructed pixels into a correction network to obtain correction information of the target region (fig. 1, the first reconstructed pixels is inputted into a convolutional neural network; page 16, … the processing object of the convolutional neural network is the buffer video frame output by the basic layer decoding image buffer; also in fig. 3 and fig. 4, the motion repair modules/compression damage repair module, is interpreted as a correction network and the output from these two modules are interpreted as the correction information of the target region; also in page 18-19, as: “include a motion repair module 310, a compression damage repair module 320 and a fusion module 330… The motion repair module 310 may generate an inner layer video frame having motion repair information… the compression damage repair module 320 can according to the compression damage feature between the previously reconstructed base layer video frame and the previously reconstructed enhancement layer video frame and the current enhancement layer video frame and the current base layer video frame between the compression damage feature similarity from the current base layer video frame generated with compression damage repair information…”);
obtain an enhancement layer bitstream of the target region (fig. 1, the enhanced layer decoder 122 obtain enhancement layer bitstream of the target region);
decode the enhancement layer bitstream to obtain a residual feature map of an enhancement layer of the target region (fig. 1, the output from the inner layer video frame has residual feature map of an enhancement layer of the target region; page 16, … into the trained convolutional neural network for inner quality lifting, to obtain the inner layer video frame quality after lifting, and then sending it into the enhancement layer decoding image buffer… use the previous enhanced layer reconstruction video frame as the convolutional neural network reference information; after performing motion repair and compression repair by the convolutional neural network, obtaining the inner layer video frame; in which, motion repair and compression repair is interpreted as a residual feature map);
and input the residual feature map and the correction information into a decoding network to obtain second reconstructed pixels of the enhancement layer of the target region (fig. 1, the convolutional neural network is a decoding network and a second reconstructed pixels of the enhancement layer is the output of the convolutional neural network; from fig. 1, the two inputs to the convolutional neural works are the residual feature map and the correction information; page 16, …from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame; at the same time from the enhancement layer decoding processing unit 122 out of the previously reconstructed high quality video, together into the trained convolutional neural network for inner quality lifting… to generate a high-quality video frame after quality improvement; in which, the high-quality video frame is interpreted as the second reconstructed pixels of the enhancement layer; also in fig. 3 and fig. 4, in which, the residual feature map and the correction information from motion repair module and compression repair module are input to the fusion module (interpreted as a decoding network) to obtain second reconstructed pixels of the enhancement layer of the target region; as in page 20-21, “….in step S220, the fusion module 330 can perform high frequency feature repair to the inner layer video frame feature with motion repair information and the inner layer video frame feature with compressed damage repair information to generate the fused inner layer video frame feature, and then; The present base layer video frame is repaired by using the fused inner layer video frame feature to generate the inner layer video frame with enhanced quality. The fusion module 330 can repair the high frequency characteristics by the following operations to generate the fused inner layer video frame characteristics: cascading the inner layer video frame feature with motion repairing information and the inner layer video frame feature with the compressed damage repair information; performing multiple convolution operations on the cascade inner layer video frame feature with motion repair information and the inner layer video frame feature with the compressed damage repair information, wherein Each convolution operation in the plurality of convolution operations uses a different number of filters…” and page 27-28, “…the fusion module, as shown in FIG. 3, can be described as:
input respectively as the basic layer video frame of the current time i ; the reconstructed forward base layer video frame ; reconstructed forward enhancement layer video frame ; reconstructing the backward base layer video frame and reconstructed backward enhancement layer video frames” ).
It is noticed that HE does not disclose explicitly of the target region is only a target region of a plurality of regions in a reconstructed image.
WANG discloses of the target region is only a target region of a plurality of regions in a reconstructed image (fig. 3, AB, CD are only a target region of a plurality of regions in a reconstructed image).
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to incorporate the technology that the target region is only a target region of a plurality of regions in a reconstructed image as a modification to the decoder for the benefit of that focus on some specific regions to reduce the calculation of motion search (page 11).
Regarding claim 20, HE teaches a computer program product (fig. 9) comprising computer-executable instructions that are stored on a non-transitory computer-readable storage medium and that, when executed by one or more processors (page 13, …at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the hierarchical video coding method and/or hierarchical video decoding method), cause an apparatus (fig. 1, 120; page 16, … hierarchical video decoding device 120) to:
obtain first reconstructed pixels of a base layer in a reconstructed image (fig. 1, the base layer decoder 121 obtained first reconstructed pixels of a base layer in a reconstructed image ; page 16, …obtain the reconstructed low quality video frame and reconstructed high quality video frame, then, from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame);
input the first reconstructed pixels into a correction network to obtain correction information of the target region (fig. 1, the first reconstructed pixels is inputted into a convolutional neural network; page 16, … the processing object of the convolutional neural network is the buffer video frame output by the basic layer decoding image buffer; also in fig. 3 and fig. 4, the motion repair modules/compression damage repair module, is interpreted as a correction network and the output from these two modules are interpreted as the correction information of the target region; also in page 18-19, as: “include a motion repair module 310, a compression damage repair module 320 and a fusion module 330… The motion repair module 310 may generate an inner layer video frame having motion repair information… the compression damage repair module 320 can according to the compression damage feature between the previously reconstructed base layer video frame and the previously reconstructed enhancement layer video frame and the current enhancement layer video frame and the current base layer video frame between the compression damage feature similarity from the current base layer video frame generated with compression damage repair information…”);
obtain an enhancement layer bitstream of the target region (fig. 1, the enhanced layer decoder 122 obtain enhancement layer bitstream of the target region);
decode the enhancement layer bitstream to obtain a residual feature map of an enhancement layer of the target region (fig. 1, the output from the inner layer video frame has residual feature map of an enhancement layer of the target region; page 16, … into the trained convolutional neural network for inner quality lifting, to obtain the inner layer video frame quality after lifting, and then sending it into the enhancement layer decoding image buffer… use the previous enhanced layer reconstruction video frame as the convolutional neural network reference information; after performing motion repair and compression repair by the convolutional neural network, obtaining the inner layer video frame; in which, motion repair and compression repair is interpreted as a residual feature map);
and input the residual feature map and the correction information into a decoding network to obtain second reconstructed pixels of the enhancement layer of the target region (fig. 1, the convolutional neural network is a decoding network and a second reconstructed pixels of the enhancement layer is the output of the convolutional neural network; from fig. 1, the two inputs to the convolutional neural works are the residual feature map and the correction information; page 16, …from the basic layer decoding processing unit 121 of the basic layer decoding image buffer from the previous and the current reconstructed low quality video frame; at the same time from the enhancement layer decoding processing unit 122 out of the previously reconstructed high quality video, together into the trained convolutional neural network for inner quality lifting… to generate a high-quality video frame after quality improvement; in which, the high-quality video frame is interpreted as the second reconstructed pixels of the enhancement layer; also in fig. 3 and fig. 4, in which, the residual feature map and the correction information from motion repair module and compression repair module are input to the fusion module (interpreted as a decoding network) to obtain second reconstructed pixels of the enhancement layer of the target region; as in page 20-21, “….in step S220, the fusion module 330 can perform high frequency feature repair to the inner layer video frame feature with motion repair information and the inner layer video frame feature with compressed damage repair information to generate the fused inner layer video frame feature, and then; The present base layer video frame is repaired by using the fused inner layer video frame feature to generate the inner layer video frame with enhanced quality. The fusion module 330 can repair the high frequency characteristics by the following operations to generate the fused inner layer video frame characteristics: cascading the inner layer video frame feature with motion repairing information and the inner layer video frame feature with the compressed damage repair information; performing multiple convolution operations on the cascade inner layer video frame feature with motion repair information and the inner layer video frame feature with the compressed damage repair information, wherein Each convolution operation in the plurality of convolution operations uses a different number of filters…” and page 27-28, “…the fusion module, as shown in FIG. 3, can be described as:
input respectively as the basic layer video frame of the current time i ; the reconstructed forward base layer video frame ; reconstructed forward enhancement layer video frame ; reconstructing the backward base layer video frame and reconstructed backward enhancement layer video frames” ).
It is noticed that HE does not disclose explicitly of the target region is only a target region of a plurality of regions in a reconstructed image.
WANG discloses of the target region is only a target region of a plurality of regions in a reconstructed image (fig. 3, AB, CD are only a target region of a plurality of regions in a reconstructed image).
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to incorporate the technology that the target region is only a target region of a plurality of regions in a reconstructed image as a modification to the computer program product for the benefit of that focus on some specific regions to reduce the calculation of motion search (page 11).
Regarding claim 2, the combination of HE and WANG teaches the limitations recited in claim 1 as discussed above. In addition, HE further discloses that the correction information comprises pixel values of the target region or feature values of the target region (page 3, … generating an inner layer video frame feature having motion repair information… feature having compression damage repair information).
12. Claim 16 is rejected are rejected under 35 U.S.C. 103 as being unpatentable over Claims 1, 6-8 are rejected are rejected under 35 U.S.C. 103 as being unpatentable over HE et al. (CN 112702604) and in view of WANG et al. (CN 101702963). and further in view of CUI et al. (CN 114586364).
Regarding claim 16, the combination of HE and WANG teaches the limitations recited in claim 1 as discussed above. In addition, HE further discloses that obtaining a base layer bitstream of an image to which the target region belongs (fig. 1, the base layer decoder 121 does it). WANG further discloses that wherein the target region is one of the at least one region (fig. 3/fig. 4).
The motivation of combination is the same as in claim 1’s rejection.
It is noticed that HE does not disclose explicitly of parsing the base layer bitstream to obtain the reconstructed image of a second base layer of the image; and determining at least one region to be enhanced based on the reconstructed image.
CUI discloses of parsing the base layer bitstream to obtain the reconstructed image of a second base layer of the image (fig. 15/fig. 16; page 34, the input picture can be divided into four sub-regions. The right upper sub-region may be encoded as two layers, i.e., the layer 1 and the layer 4, and the lower right sub-region may be encoded as two layers, i.e., the layer 3 and the layer 5. In this case, layer 4 can perform motion compensation prediction reference layer 1, and layer 5 can perform motion compensation with reference to layer 3; in which, since layer 5 (second base layer) used layer 3 (first base layer) as reference, it is reconstructed by parsing the first base layer bitstream));
and determining at least one region to be enhanced based on the reconstructed image (fig. 11; page 31, The enhanced CSPS layer may refer to reconstructed pixels and motion vectors of the base layer corresponding to the same region).
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to incorporate the technology that parsing the base layer bitstream to obtain the reconstructed image of a second base layer of the image; and determining at least one region to be enhanced based on the reconstructed image as a modification to the system for the benefit of that enhance the quality of a region (page 31).
13. Claim 17 is rejected are rejected under 35 U.S.C. 103 as being unpatentable over Claims 1, 6-8 are rejected are rejected under 35 U.S.C. 103 as being unpatentable over HE et al. (CN 112702604) and in view of WANG et al. (CN 101702963) and further in view of CUI et al. (CN 114586364) and further in view of QI et al. (CN 110151133).
Regarding claim 16, the combination of HE, WANG and CUI teaches the limitations recited in claim 1 as discussed above. In addition, WANG further discloses that dividing the reconstructed image to obtain a plurality of regions (fig. 3/fig. 4).
The motivation of combination is the same as in claim 1’s rejection.
It is noticed that HE does not disclose explicitly of determining as the at least one region a first region that is of the regions and that has a variance greater than a first threshold; or determining as the at least one region a second region that is of the regions and that has a threshold proportion of pixels whose gradients are greater than a second threshold.
QI discloses of determining as the at least one region a first region that is of the regions and that has a variance greater than a first threshold (page 11, determining characteristic threshold value t, the image to be reconstructed is divided into target area Ω and the background area Ω b two parts; when the area of uai <t is Ω b, when the area of the uai ≥ t to Ω t); or determining as the at least one region a second region that is of the regions and that has a threshold proportion of pixels whose gradients are greater than a second threshold.
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to incorporate the technology that determining as the at least one region a first region that is of the regions and that has a variance greater than a first threshold; or determining as the at least one region a second region that is of the regions and that has a threshold proportion of pixels whose gradients are greater than a second threshold as a modification to the method for the benefit of that divide image according to different criteria (page 11).
14. Conclusion
. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
15. Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZAIHAN JIANG whose telephone number is (571)272-1399. The examiner can normally be reached on flexible.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath Perungavoor can be reached on (571)272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-270-0655.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZAIHAN JIANG/Primary Examiner, Art Unit 2488
Prosecution Insights