DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This communication is in response to the Preliminary Amendment filed on 10/25/2024. Claims 1-19 were originally filed. Claims 1-19 are amended. Claims 20 is added. Claims 1-20 are currently pending.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/16/2025 in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
The information disclosure statement (IDS) submitted on 06/19/2025 in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by HU ZHIHAO et al., “FVC: A New Framework towards Deep Video Compression in Feature Space”, 2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), IEEE, 20 June 2021 (2021-06-20), pages 1502-1511, XP034010475, DOI: 10.1109, CVPR46437.2021.00155 (Applicant Admitted Prior art with an IDS), hereinafter “Hu”
Regarding claims 1 and 8. (Currently Amended) Hu discloses a method (Figure 1) comprising:
Parsing a bitstream to obtaining a first feature map of a first image frame and a second feature map of a reference frame of the first image frame (Figure 1; feature extraction in figure 1 for input frame and a frame from the decoded frame buffer, i.e. reference frame);
obtaining, based on the first feature map and the second feature map, a first optical flow set, wherein the first optical flow set comprises one or more feature domain optical flows and corresponds to the (first) second feature map, and wherein each of the one or more feature domain optical flows indicates motion information between the first feature map and the (first) second feature map (page 1504 Section 3.1, page 1505 section 3.2, and page 1509 section 4.3; Figures 1, and 9c-9d; “section 3.1. overview. … Deformable Compensation. This procedure consists of three steps: motion estimation, motion compression and motion compensation. Specifically, based on Fₗ and Fro{, we perform motion estimation by using a lightweight net- work, and the output offset map Oₜ will be compressed by using the newly proposed motion compression network be- fore being transmitted to the decoder side. Finally, given the reconstructed offset map O₁ and the feature we can generate the predicted feature F by using deformable convolution in the motion compensation procedure. More details can be found in Section 3.2.”, “section 3.2. Deformable Compensation … Given the features
F
t
-
1
r
e
f
and Ft respectively from the previous reconstructed frame and the current frame, the de- formable compensation procedure aims at generating the predicted feature Fₜ at the current time step. The whole network architecture is shown in Fig. 3. Specifically, we first produce the deformable offset map Ot by feeding
F
t
-
1
r
e
f
and Ft into a 2-layer motion estimation network. Inspired by ... we propose an auto-encoder style network to com- press this offset map Ot, where both the encoder and the decoder consist of a set of Resblocks ... The offset map Ot is transformed to the latent space through the encoder and then quantized. After that, the decoder will convert the quantized latent representations back to the reconstructed offset map Oₜ.”, wherein the offset map corresponds to the feature domain optical flows indicating motion information between each element of the feature map of the reference frame and feature map of the current frame. There is one offset per each element of the feature map of the reference frame, see e.g. figure 9.c and d, i.e. there is a plurality of optical flows);
processing, based on the one or more feature domain optical flows, the (first) second feature map to obtain one or more first intermediate feature maps corresponding to the second first-feature map (page 1504 Section 3.1 and page 1505 section 3.2; figure 1; “section 3.1. overview. … Deformable Compensation. This procedure consists of three steps: motion estimation, motion compression and motion compensation. Specifically, based on Fₗ and Fro{, we perform motion estimation by using a lightweight net- work, and the output offset map Oₜ will be compressed by using the newly proposed motion compression network be- fore being transmitted to the decoder side. Finally, given the reconstructed offset map O₁ and the feature we can generate the predicted feature F by using deformable convolution in the motion compensation procedure. More details can be found in Section 3.2.”, “Multi-frame Feature Fusion. In this procedure, the extracted feature representations
F
t
-
1
r
e
f
,
F
t
-
2
r
e
f
and
F
t
-
3
r
e
f
from three previous reconstructed frames as well as the initial re- constructed feature representation Fₜ are fused to produce the final reconstructed feature Ft. More details will be introduced in Section 3.3.”, wherein a predicted feature map is obtained by compensating the feature map of the reference frame with the offset map with deformable convolution. Notably, each element feature map of the reference frame is compensated, lead to plurality of intermediate elements of the predicted feature map);
fusing the one or more first intermediate feature maps to obtain a first predicted feature map of the first image frame (page 1509 section 4.3; Figure 9g; wherein each intermediate element of the predicted feature map are put together to obtain the predicted feature map, see figure 9.g ); and
encoding, based on the first predicted feature map, the first image frame to obtain a bitstream (page 1504 Section 2.3; figure 1; please see the residual generation and compression in figure 1).
Regarding claims 2 and 9. (Currently Amended) Hu discloses the method wherein processing the first feature map comprises:
parsing the bitstream to obtain the first feature map (implicit); and
performing, based on a first feature domain optical flow of the one or more feature domain optical flows, warping on the first feature map to obtain the one or more intermediate feature maps, wherein the one or more first intermediate feature maps correspond to the first feature domain optical flow (wherein the feature map of the reference frame is implicitly transmitted/parsed from the bitstream).
Regarding claims 3 and 10. (Currently Amended) Hu discloses the method wherein the first optical flow set further corresponds to a third feature map of the reference frame, wherein before decoding/encoding the first image frame, the method further comprises:
processing, based on the one or more feature domain optical flows, the third feature map, to obtain one or more second intermediate feature maps corresponding to the third feature map (wherein reuse the optical flow obtained for one feature map for another feature map, e.g. corresponding to another channel); and
fusing the one or more second intermediate feature maps to obtain a second predicted feature map of the first image frame (wherein reuse the optical flow obtained for one feature map for another feature map, e.g. corresponding to another channel), and
wherein decoding/encoding the first image frame comprises decoding/encoding, based on the first predicted feature map and the second predicted feature map, the first image frame to obtain the first image/bitstream (wherein reuse the optical flow obtained for one feature map for another feature map, e.g. corresponding to another channel).
Regarding claims 4-5 and 11-12. (Currently Amended) Claims 4-5 and 11-12 have similar limitations as to those treated in the above rejections, and are met by the references as discussed above, and has been rejected for the same reasons of anticipations as used in the rejection to claims discussed above.
Regarding claims 6-7. (Currently Amended) The method of claim 1, further comprising:
obtaining the second feature map of the first image (Sections 3.1 and 3.3; wherein the enhancement of the features using previous features maps, see Multi-frame Feature Fusion module in figure 1, sub section "Multi-frame Feature Fusion);
obtaining an enhanced feature map based on the first feature map, the second feature map, and the first predicted feature map (Sections 3.1 and 3.3; wherein the enhancement of the features using previous features maps, see Multi-frame Feature Fusion module in figure 1, sub section "Multi-frame Feature Fusion); and
processing, based on the enhanced feature map, the first image, to obtain a second image, wherein a second definition of the second image is higher than a first definition of the first image (Sections 3.1 and 3.3; wherein the enhancement of the features using previous features maps, see Multi-frame Feature Fusion module in figure 1, sub section "Multi-frame Feature Fusion);
wherein processing the first image comprises: obtaining, based on the enhanced feature map, an enhancement layer image of the first image (Sections 3.1 and 3.3; wherein the enhancement of the features using previous features maps, see Multi-frame Feature Fusion module in figure 1, sub section "Multi-frame Feature Fusion); and
reconstructing, based on the enhancement layer image, the first image to obtain the second image (Sections 3.1 and 3.3; wherein the enhancement of the features using previous features maps, see Multi-frame Feature Fusion module in figure 1, sub section "Multi-frame Feature Fusion).
Regarding claims 13-19. Apparatus claims 13-19 are drawn to apparatus corresponding to the method of using same as claimed in claims 1-7 and/or 8-12. Therefore, apparatus claims 13-19 correspond to method claims 1-7 and/or 8 -12 and are rejected for the same reasons of anticipation as used above.
Allowable Subject Matter
Claim 18 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ASMAMAW G TARKO whose telephone number is (571)272-7493. The examiner can normally be reached M-F: 8am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chris Kelley can be reached at (571) 272-7331. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ASMAMAW G TARKO/ Primary Examiner, Art Unit 2482
Prosecution Insights