Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Summary
This office action for US Patent application 18/994221 is responsive to communications filed on May 26th, 2026. Currently, claims 1-16 are pending are presented for examination.
Response to Arguments
2. On pages 10-15, applicant’s arguments with respect to claims 1, 11 and 16 have been considered. The applicant is arguing that Park and Mar do not teach the newly added limitations “determining whether to perform retraining of the deep learning-based in-loop filter based on a parameter of the current block”. While applicant’s arguments have been considered and understood, the examiner respectfully disagrees. In response, the independent claims 1, 11 and 16 have been amended to add limitations disclosed in paragraphs 0149-0170 but the applicant did not explain what “a parameter of the current block” is. The examiner searched through the specification and couldn’t find any material which clearly describes “a parameter of the current block”. A parameter of the current block can be interpreted as a desired training error value of a reconstruction block of a current block and a target block to determine whether to perform retraining a NN based in-loop filter. This training error can be used as flag to determine whether to perform retraining of the NN based in-loop filter. This feature is well-known in the arts. In the newly amended claims, one limitation is about to determine whether to perform retraining of the deep-learning based in-loop filter based on a parameter of the current block and another limitation is about selecting a block for retraining of the deep learning-based in-loop filter from the reconstruction block, based on whether to perform retraining of the deep learning based in-loop filter which can be determined not based on a parameter of the current block. This is how the examiner interprets the claim language. The applicant is requested to clarify this subject matter which may require a consistent language to avoid confusion. Because of these reasons, the examiner does not think the new limitations add anything new into the previous claims and they are covered by Park and Ma.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-11, 13-16 is/are rejected under 35 U.S.C §103 unpatentable over PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1).
Regarding claim 1, PARK et al. (US 20200120340 A1) meets the claim limitations, as follows:
A method for reconstructing a current block, performed by a video decoding device, the method comprising:
generating a reconstruction block for the current block from a bitstream [i.e. reconstructed image (block); Fig. 2, 4, 12];
generating an output block by inputting the reconstruction block to a deep learning- based in-loop filter [i.e. input the reconstructed image to a DNN based in-loop filter to generated the filtered reconstructed image; paragraph. 0123-0124, Fig. 2, 4, 12], wherein the deep learning-based in-loop filter is pre-trained to generate an output that approximates original block from the reconstruction block [i.e. smallest error between original data and filtered reconstructed data; paragraph. 0123-0124, 0134; Fig. 4];
determining whether to perform retraining of the deep learning-based in-loop filter based on a parameter of the current block [i.e. the in-loop filtering is trained based on the reconstructed image and the original data until an error value is satisfied. This concept is well-known in the art; paragraph. 0134, 0145];
selecting a block for retraining of the in-loop filter from the reconstruction block [i.e. the reconstructed image is used as an input to train the DNN filter model; paragraph. 0145, Fig. 4, 12]; and
retraining the deep learning-based in-loop filter by using the selected reconstruction block and a target block [i.e. the in-loop filtering is trained based on the reconstructed image and the original data. The DNN based in-loop filter is always be trained with new data set to improve its accuracy. This concept is well-known in the art; paragraph. 0134, 0145].
Although retraining a DNN is a normal concept. However, PARK et al. (US 20200120340 A1) does not disclose it explicitly.
In the same field of endeavor, Ma et al. (US 20230319314 A1) discloses about a neural network based in-loop filter being trained and updated frequently [i.e. paragraph. 0187-0188]
It would have been obvious to one with ordinary skill in the art at the time of invention to modify the teachings of PARK et al. (US 20200120340 A1) with Ma et al. (US 20230319314 A1) in order to create a method as the claimed limitation.
Regarding claim 3, PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1) discloses the following claim limitations as set forth in claim 1.
Furthermore, PARK et al. (US 20200120340 A1) discloses the claim limitation as follows.
The method of claim 1, further comprising:
decoding a retraining flag from the bitstream; and
checking the retraining flag,
wherein the retraining of the deep learning-based in-loop filter is determined to be performed when the retraining flag is true[i.e. retraining a NN based in-loop filter is performed to fine tune its parameters iteratively until an error value is satisfied. This is well-known in the art; paragraph. 0134].
Regarding claim 4, PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1) discloses the following claim limitations as set forth in claim 3.
Furthermore, PARK et al. (US 20200120340 A1) discloses the claim limitation as follows.
The method of claim 1, further comprising:
The method of claim 3, when the retraining flag is true, further including:
generating an improved output block by inputting the selected reconstruction block to the retrained deep learning-based in-loop filter [i.e. improve output of a NN based in-loop filter based on smallest error between original data and filter reconstructed data; paragraph. 0134].
Regarding claim 5, PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1) discloses the following claim limitations as set forth in claim 1.
Furthermore, PARK et al. (US 20200120340 A1) discloses the claim limitation as follows.
The method of claim 1, further comprising:
The method of claim 1, wherein selecting the block for retraining comprises:
determining whether to select the reconstruction block as a block for retraining based on parameters used for compression of the reconstruction block [i.e. reconstructed data with different parameters is used to train a NN based in-loop filter to improve its compression; paragraph. 0133-0134, Fig. 7].
Regarding claim 6, PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1) discloses the following claim limitations as set forth in claim 1.
Furthermore, PARK et al. (US 20200120340 A1) discloses the claim limitation as follows.
The method of claim 1, further comprising: decoding section information to which the retraining is applied from the bitstream, wherein, selecting the block for retraining comprises: when the reconstruction block is included in the section information, selecting the reconstruction block as a block for retraining [i.e. the reconstructed data decoded from a bitstream is used to train a NN based in-loop filter; paragraph. 0094, Fig. 2, 4].
Regarding claim 7, PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1) discloses the following claim limitations as set forth in claim 1.
Furthermore, PARK et al. (US 20200120340 A1) discloses the claim limitation as follows.
The method of claim 1, wherein selecting the block for retraining comprises: generating an output by inputting the reconstruction block to a pre-trained discriminator and determining whether to select the reconstruction block as a block for retraining based on the output of the pre-trained discriminator [i.e. Fig. 4, paragraph. 0133-0134].
Regarding claim 8, PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1) discloses the following claim limitations as set forth in claim 1.
Furthermore, PARK et al. (US 20200120340 A1) discloses the claim limitation as follows.
The method of claim 1, wherein the target block is an output block corresponding to the selected reconstruction block, a block co-located with the current block in a reference frame of the current block, a reference block indicated by a motion vector of the current block, or a prediction block searched for in a current frame according to template matching based on a template of the current block [i.e. the original data is an output corresponding to the reconstructed data; paragraph. 0134, Fig. 4].
Regarding claim 9, PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1) discloses the following claim limitations as set forth in claim 1.
Furthermore, PARK et al. (US 20200120340 A1) discloses the claim limitation as follows.
The method of claim 1, wherein retraining the deep learning-based in-loop filter comprises:
defining a loss function based on a difference between the selected reconstruction block and the target block and updating parameters of the in-loop filter in a direction of reducing the loss function [i.e. training a neural network with updating parameters to reduce an error between filtered reconstructed data with original data; paragraph. 0130, 0133-0134].
Regarding claim 10, PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1) discloses the following claim limitations as set forth in claim 1.
Furthermore, PARK et al. (US 20200120340 A1) discloses the claim limitation as follows.
The method of claim 1, wherein the deep learning-based in-loop filter includes a first deep learning module and a second deep learning module that include same parameters in an initial state [i.e. a DNN based in-loop filter usually has different hidden layers (modules). Each layer is initialized with weight parameters which can be the same or random. This is well-known in the art; paragraph. 0125, 0128, Fig. 5];
wherein generating the output block comprises:
generates the output block from the reconstruction block by using the first deep learning module [i.e. generate the output from the output layer; paragraph. 0128, Fig. 4],
wherein retraining the deep learning-based in-loop filter comprises:
updating parameters of the second deep learning module by using the selected reconstruction block and the target block [i.e. it is well-known in the art to update or freeze parameters of a certain layer as an input layer or a hidden layer in training a DNN; paragraph. 0128, Fig. 4].
Regarding claim 11, all the claim limitations which are set forth and rejected as per discussion for claim 1.
Regarding claim 13, all the claim limitations which are set forth and rejected as per discussion for claim 3.
Regarding claim 14, all the claim limitations which are set forth and rejected as per discussion for claim 3.
Regarding claim 15, all the claim limitations which are set forth and rejected as per discussion for claim 8.
Regarding claim 16, all the claim limitations which are set forth and rejected as per discussion for claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2, 12 is/are rejected under 35 U.S.C §103 unpatentable over PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1) further in view of SALEHIFAR et al. (US 20210099710 A1).
Regarding claim 2, PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1) discloses the following claim limitations as set forth in claim 1.
However, PARK et al. (US 20200120340 A1) in view of Ma et al. (US 20230319314 A1) does not disclose about an in-loop filter flag.
In the same field of endeavor, SALEHIFAR et al. (US 20210099710 A1) discloses the deficient claim limitations, as follows:
The method of claim 1, further comprising:
decoding an in-loop filter flag from the bitstream [i.e. a flag representing whether a deblocking filter is available may be signaled; paragraph. 0122]; and
checking the in-loop filter flag,
wherein, when the in-loop filter flag is true, generating the output block is performed [i.e. if a value of the deblocking filter available flag is 1, the deblocking filter is available; paragraph. 0122].
It would have been obvious to one with ordinary skill in the art at the time of invention to modify the teachings of PARK et al. (US 20200120340 A1) and Ma et al. (US 20230319314 A1) and SALEHIFAR et al. (US 20210099710 A1) in order to create a method as the claimed limitation.
Regarding claim 12, all the claim limitations which are set forth and rejected as per discussion for claim 1.
Bitstream Rejections
The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claim 16 is rejected under 35 U.S.C 102(a)(1) as being anticipated by Lee et al. (20200244956).
Claim 16’s recitation of “computer-readable recording medium storing a bitstream…” is a product by process claim limitation where the product is the bitstream and the process is the method steps to generate the bitstream. MPEP §2113 recites “Product-by-Process claims are not limited to the manipulations of the recited steps, only the structure implied by the steps”. Thus, the scope of the claim is the storage medium storing the bitstream (with the structure implied by the method steps).
“To be given patentable weight, the printed matter and associated product must be in a functional relationship. A functional relationship can be found where the printed matter performs some function with respect to the product to which it is associated”. MPEP §2111.05(I)(A). When a claimed “computer-readable medium merely serves as a support for information or data, no functional relationship exists. MPEP §2111.05(III). The storage medium storing the claimed bitstream in claim 16 merely services as a support for the storage of the bitstream and provides no functional relationship between the stored bitstream and storage medium. Therefor the structure bitstream, which scope is implied by the method steps, is non-functional descriptive material and given no patentable weight. MPEP §2111.05(III). Thus, the claim scope is just a storage medium storing data and is anticipated by Lee recites a storage medium storing a bitstream (Fig. 1-2, the bitstream from Fig. 1 is stored in Fig. 2, 210; paragraph. 0002, a method and apparatus for encoding/decoding an image and a recording medium storing a bitstream).
New Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications form the examiner should be directed to Nam Pham, whose can be contacted by phone at (571)270-7352. The examiner can normally be reached on Mon—Thurs.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, CZEKAJ DAVID, can be reached on (571)272-7327.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) AT 866-217-9197 (too free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NAM D PHAM/ Primary Examiner, Art Unit 2487
Prosecution Insights