DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) was filed on 08/05/2026. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant's arguments filed August 11, 2026 have been fully considered but they are not persuasive.
In the non-final action, claims 1, 16 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Dai WO 2024/077574 A1 (hereinafter referred to as “Dai”) in view of Singh et al., US 20220272355 A1 (hereinafter referred to as “Singh”).
In Applicant’s response independent claims 1, 16 and 20 were amended. For example, independent claim 1, as amended, recites:
a plurality of convolution layers that are provided in parallel and have different convolution kernel sizes, input ends of the plurality of convolution layers being connected to each other and serving as an input end of the first residual block, each of the plurality of convolution layers receiving input data of the first residual block."
Support the for the amendment was found in the Specification. No new matter was added.
Dai discloses the plurality of convolution layers that are provided in parallel
In the Remarks, page 9, Applicant argues Dai does not disclose the plurality of convolution layers that are provided in parallel. More specifically, Applicant submits:
The Office Action concedes that Dai does not disclose the plurality of convolution layers that are provided in parallel and have different convolution kernel sizes…Applicant respectfully submits that any such reliance is unsupported because Dai's residual blocks are purely sequential: each comprises "a convolutional layer whose convolutional core size is 1x1, an ReLU layer, a convolutional layer whose convolutional core size is 1x1, and a convolutional layer whose convolutional core size is 3x3." Dai, paragraph [0051]. Dai's alternative attention residual block is likewise "sequentially connected." Dai, paragraph [0054].
The Examiner respectfully disagrees, Dai, in Fig. 3A, clearly illustrates convolution layers of filter size 3x3 provided in parallel.
As shown in the Fig. 3A each of the aforementioned convolution layers is followed by a convolutional layer whose convolutional core size is 1x1, an ReLU layer, a convolutional layer whose convolutional core size is 1x1, and a convolutional layer whose convolutional core size is 3x3. This does not imply that the convolution layers are sequential and not parallel, as recited in the claims.
Applicant further submits that Dai's alternative attention residual block is allegedly "sequentially connected.”
The Examiner respectfully disagrees. The Examiner has reviewed English interpretation of the reference and has not found evidence explicitly stating that the aforementioned convolutional layers (3x3 Convo) are sequentially connected. Term “sequentially connected” was not found in the reference.
For at least the aforementioned reasons, The Examiner submits that Dai discloses the plurality of convolution layers that are provided in parallel, as recited in the claims.
Singh discloses the plurality of convolution layers that are provided in parallel and have different convolution kernel sizes.
The Examiner agrees with the Applicant with respect to Dai failing to disclose both “[1] the plurality of convolution layers that are provided in parallel and [2] have different convolution kernel sizes.”
Assuming arguendo, in the case that Dai does not disclose parallel convolution layers and further does not disclose different convolution kernel sizes, Singh makes up for the deficiencies of Dai.
As stated in the non-final rejection, pages 3-4, the Examiner respectfully submits, Singh discloses both the “[1] the plurality of convolution layers that are provided in parallel and [2] have different convolution kernel sizes.”
Singh discloses a plurality of convolution layers that are provided in parallel and have different convolution kernel sizes. Singh., Figure 2DFigs. 8A and 10, discloses convolutional layer 810 applying a 5×5 convolutional filter and the parallel convolutional layer 811 processes the input chroma (U and V) applies a 3×3 convolutional filter. ¶ [0137].
Singh discloses input ends of the plurality of convolution layers being connected to each other and serving as an input end of the first residual block, each of the plurality of convolution layers receiving input data of the first residual block.
Applicant argues, Remarks page 10, Singh “[b]ecause layers 810 and 811 do not share a common input end and do not reside in a residual block, Singh provides no teaching or suggestion of the claimed parallel-branch residual block structure.”
The Examiner respectfully disagrees. Singh discloses “FIG. 8A is a diagram illustrating an example of a machine-learning based architecture with YUV (e.g., YUV 4:2:0) residuals.” Each of the plurality of convolution layers, 810 and 811, having different sizes (i.e., 5x5 and 3x3), receive input data of the first residual block (i.e., YUV residuals). The YUV residuals serve as a common input end for parallel convolution layers 810 and 810.
For at least the abovementioned reasons Dai in view of Singh disclose “a plurality of convolution layers that are provided in parallel and have different convolution kernel sizes, input ends of the plurality of convolution layers being connected to each other and serving as an input end of the first residual block, each of the plurality of convolution layers receiving input data of the first residual block."
Accordingly claims 1, 16 and 20 are unpatentable over the cited references. Claims 2-15, 17-19 depend from claims 1 and 16 respectively and are therefore unpatentable at least by virtue of their dependency.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
1. Claims 1-4 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Dai WO 2024/077574 A1 (hereinafter referred to as “Dai”). See English equivalent EP 4 604 524 A1 in view of Singh et al., US 20220272355 A1 (hereinafter referred to as “Singh”).
Regarding claim 1 (Currently Amended), Dai discloses a filtering method based on a neural network loop filter (NNLF) (Dai: Fig. 3A, Abstract NNLF), the method comprising:
acquiring input data of the NNLF, the input data including at least a target image (Dai: Fig. 3A);
inputting the input data into the NNLF (Dai: Fig. 3A illustrates a residual block being inputted into a NNLF), the NNLF including a first residual unit configured to extract image feature information (Dai: Dai Fig. 4 discloses that the luminance component and chrominance component of the reconstructed image are input separately), the first residual unit including a plurality of sequentially-connected residual blocks (Dai: multiple residual blocks (ResBlock) connected in sequence), and a first residual block among the plurality of residual blocks including a plurality of convolution layers that are provided in parallel (Dai: Fig. 3A); and
processing the target image using the NNLF to obtain a filtered image (Dai: Fig. 3A output YUV).
Dai does not disclose the plurality of convolution layers that are provided in parallel and have different convolution kernel sizes and input ends of the plurality of convolution layers being connected to each other and serving as an input end of the first residual block, each of the plurality of convolution layers receiving input data of the first residual block.
However, Singh discloses a plurality of convolution layers that are provided in parallel and have different convolution kernel sizes (Singh: convolutional layer 810 applying a 5×5 convolutional filter and the parallel convolutional layer 811 processes the input chroma (U and V) applies a 3×3 convolutional filter. Figs. 8A, and 10 ¶ [0137]); and input ends of the plurality of convolution layers being connected to each other and serving as an input end of the first residual block, each of the plurality of convolution layers receiving input data of the first residual block (Singh: “FIG. 8A is a diagram illustrating an example of a machine-learning based architecture with YUV (e.g., YUV 4:2:0) residuals.” Each of the plurality of convolution layers (i.e., 810 and 811) receive input data of the first residual block (i.e., YUV residuals).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dai such that the plurality of convolution layers that are provided in parallel and have different convolution kernel sizes and input ends of the plurality of convolution layers being connected to each other and serving as an input end of the first residual block, each of the plurality of convolution layers receiving input data of the first residual block, as taught by Singh, in order to improve feature representation and increase accuracy for complex objects.
Regarding claim 2 (Currently amended), Dai discloses the method according to claim 1, wherein the input data to the NNLF further includes at least one piece of the following information: boundary strength information, a slice-level quantization parameter (QP), a basic QP, a frame type of the target image, and a predicted image corresponding to the target image when the target image is a reconstructed image (Dai: As shown in Fig. 3A , the input of the NNLF1 filter includes QP information, and frame type information. The QP information includes the default baseline quantization parameter (BaseQP: Base Quantization Parameter) in the encoding profile and the slice quantization parameter (SliceQP: Slice Quantization Parameter) of the current slice, and the frame type information includes the slice type (SliceType), i.e., the type of frame to which the current slice belongs).
Regarding claim 3 (Currently amended), Dai discloses the method according to claim 1, wherein the plurality of convolution layers comprises a first convolution layer and a second convolution layer (Dai: As shown in Fig 3A); and
the first residual block further comprises: a first activation function layer sequentially connected to the first convolution layer (Dai: RelU connected to first convolution layer as shown in Fig 3A), a second activation function layer sequentially connected to the second convolution layer (Dai: RelU connected to second convolution layer as shown in Fig 3A), and a third convolution layer and a fourth convolution layer sequentially connected to the third convolution layer (Dai: a third and fourth convolution layer downstream from the first and second convolution layers the Fig. 3A), wherein the first convolution layer and the first activation function layer are connected to the second convolution layer and the second activation function layer in parallel (Dai: First and second layers are parallel as shown in Fig. 3A) and then connected to an input end of the third convolution layer (Dai: as shown in Fig. 3A); and the input data of the first residual block and output data of the fourth convolution layer are superimposed as output data of the first residual block (Dai: as shown in Fig. 3A and 5).
Regarding claim 4 (Original), Dai does not disclose the method according to claim 3, wherein a convolution kernel size of the first convolution layer is n×n; and a convolution kernel size of the second convolution layer is m×m, m and n are positive integers, and m≠n.
However, Singh discloses a convolution kernel size of the first convolution layer is n×n; and a convolution kernel size of the second convolution layer is m×m, m and n are positive integers, and m≠n (Singh: convolutional layer 810 applying a 5×5 convolutional filter and the parallel convolutional layer 811 processes the input chroma (U and V) applies a 3×3 convolutional filter. Figs. 8A, 10 ¶ [0137]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dai such that the plurality of convolution layers that are provided in parallel and have different convolution kernel sizes, as taught by Singh, in order to improve feature representation and increase accuracy for complex objects.
Regarding claim 11 (Currently amended), Dai discloses the method according to claim 3, wherein the fourth convolution layer comprises any one of the following: a convolution layer having a convolution kernel size of k×k (Dai: See Fig. 3A fourth convolution layer. The claim only requires one of the alternatives); two sub-convolution layers obtained by decomposing the convolution layer having the convolution kernel size of k×k through the tensor decomposition; two sub-convolution layers obtained by decomposing the convolution layer having the convolution kernel size of k×k through the DSC; and three sub-convolution layers obtained by decomposing the convolution layer having the convolution kernel size of k×k through the tensor decomposition and the depth-wise separable convolution DSC; wherein k is a positive integer.
Regarding claim 12 (Original), Dai discloses the method according to claim 1, wherein the first residual unit is configured to extract image feature information of one of a luminance component and a chrominance component of the target image (Dai: Fig. 4C: separation of Y and UV); and the NNLF further comprises a second residual unit configured to extract image feature information of the other of the luminance component and the chrominance component of the target image, wherein the second residual unit includes a plurality of sequentially-connected residual blocks (Dai: Fig. 4C: separation of Y and UV; Singh: also see Y and UV extraction Figs. 8-10).
Regarding claim 13 (Original), Dai discloses the method according to claim 12, wherein the first residual unit is configured to extract the image feature information of the luminance component of the target image, and the second residual unit is configured to extract the image feature information of the chrominance component of the target image; and the second residual unit includes at least one of the following: a second residual block having a structure the same as that of the first residual block, and other residual blocks except the second residual block (Dai: Fig. 4C: separation of Y and UV; Singh: also see Y and UV extraction Figs. 8-10).
Regarding claim 16 (Currently amended), claim 1 is sustainably similar to claim 16. Therefore, claim 16 is rejected for the same reasons as claim 1.
Regarding claim 17 (Currently amended), claim 2 is sustainably similar to claim 17. Therefore, claim 17 is rejected for the same reasons as claim 2.
Regarding claim 18 (Currently amended), claim 3 is sustainably similar to claim 18. Therefore, claim 18 is rejected for the same reasons as claim 3.
Regarding claim 19 (Original), claim 12 is sustainably similar to claim 19. Therefore, claim 19 is rejected for the same reasons as claim 11.
Regarding claim 20 (Currently amended), claim 1 is sustainably similar to claim 20. Therefore, claim 20 is rejected for the same reasons as claim 1.
2. Claims 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Dai in view of Singh and Boyd et al., US 20230008622 A1 (hereinafter referred to as “Boyd”).
Regarding claim 5 (Original), Dai discloses and a convolution kernel size of the second convolution layer is m×m, m and n are positive integers, and m≠n (Dai: Fig. 3A).
Dai and Singh do not disclose the method according to claim 3, wherein the first convolution layer comprises: two sub-convolution layers obtained by decomposing a convolution layer having a convolution kernel size of n×n through tensor decomposition;
However, Boyd discloses two sub-convolution layers obtained by decomposing a convolution layer having a convolution kernel size of n×n through tensor decomposition (Boyd: Abstract).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dai and Singh with two sub-convolution layers obtained by decomposing a convolution layer having a convolution kernel size of n×n through tensor decomposition, as taught by Boyd, in order to reduce the kernel size, thereby reducing computational cost.
Regarding claim 6 (Original), Dai and Singh do not disclose the method according to claim 5, wherein the two sub-convolution layers comprise: a first sub-convolution layer having a convolution kernel size of 1×n, and a second sub-convolution layer having a convolution kernel size of n×1.
However, Boyd discloses, the two sub-convolution layers comprise: a first sub-convolution layer having a convolution kernel size of 1×n, and a second sub-convolution layer having a convolution kernel size of n×1 (Boyd: an example convolution where an input tensor has a width of 18 activations and the kernel is a 3×3 kernel, the kernel may be decomposed into 1×3 kernel vectors ¶ [0023]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dai and Singh with the two sub-convolution layers comprise: a first sub-convolution layer having a convolution kernel size of 1×n, and a second sub-convolution layer having a convolution kernel size of n×1, as taught by Boyd, in order to reduce the kernel size, thereby reducing computational cost.
Regarding claim 9 (Original), Dai discloses a convolution kernel size of the second convolution layer is m×m, m and n are positive integers, and m≠n (Dai: Fig. 3A).
Dai and Singh do not disclose the method according to claim 3, wherein the first convolution layer comprises: three sub-convolution layers obtained by decomposing a convolution layer having a convolution kernel size of n×n through tensor decomposition and DSC.
Boyd discloses wherein the first convolution layer comprises: three sub-convolution layers obtained by decomposing a convolution layer having a convolution kernel size of n×n through tensor decomposition and DSC (Boyd: an example convolution where an input tensor has a width of 18 activations and the kernel is a 3×3 kernel, the kernel may be decomposed into 1×3 kernel vectors ¶ [0023]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dai and Singh with the first convolution layer comprises: three sub-convolution layers obtained by decomposing a convolution layer having a convolution kernel size of n×n through tensor decomposition and DSC, as taught by Boyd, in order to reduce the kernel size, thereby reducing computational cost.
Regarding claim 10 (Original), Dai and Singh do not disclose the method according to claim 9, wherein the three sub-convolution layers comprise: a fifth sub-convolution layer having a convolution kernel size of 1×n and performing group convolution, a sixth sub-convolution layer having a convolution kernel size of n×1 and performing group convolution, and a seventh sub-convolution layer having a convolution kernel size of 1×1.
However, Boyd discloses a fifth sub-convolution layer having a convolution kernel size of 1×n and performing group convolution, a sixth sub-convolution layer having a convolution kernel size of n×1 and performing group convolution, and a seventh sub-convolution layer having a convolution kernel size of 1×1 (Boyd: Abstract).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dai and Singh with a fifth sub-convolution layer having a convolution kernel size of 1×n and performing group convolution, a sixth sub-convolution layer having a convolution kernel size of n×1 and performing group convolution, and a seventh sub-convolution layer having a convolution kernel size of 1×1, as taught by Boyd, in order to reduce the kernel size, thereby reducing computational cost.
3. Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Dai in view of Singh and Zhang et al., US 20240193723 A1 (hereinafter referred to as “Zhang”).
Regarding claim 14 (Currently amended), Dai and Singh do not explicitly disclose the method according to claim 12, wherein the NNLF further comprises: a shallow feature extraction unit, wherein the shallow feature extraction unit comprises at least one convolution layer and is configured to extract shallow feature information of the input data the NNLF and input the shallow feature information into the first residual unit and the second residual unit.
However, Zhang discloses a shallow feature extraction unit, wherein the shallow feature extraction unit comprises at least one convolution layer and is configured to extract shallow feature information of the input data and input the shallow feature information into the first residual unit and the second residual unit (Zhang: in the shallow layer feature extraction network, an inputted noisy image first passes through a 3×3 deep convolutional layer in the shallow layer feature extraction network. ¶ [0041]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dai and Singh with wherein the shallow feature extraction unit comprises at least one convolution layer and is configured to extract shallow feature information of the input data and input the shallow feature information into the first residual unit and the second residual unit, as taught by Zhang. In this way, feature information of the inputted noisy image is initially extracted and the quality of the feature image is improved.
4. Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Dai in view of Singh and Wang et al., US 20220148131 A1 (hereinafter referred to as “Wang”).
Regarding claim 15 (Original), Dai and Singh do not explicitly disclose the method according to claim 12, wherein the NNLF further comprises: a first feature mapping unit configured to map the image feature information outputted by the first residual unit, and a second feature mapping unit configured to map the image feature information outputted by the second residual unit, wherein the first feature mapping unit is connected to the first residual unit, and the second feature mapping unit is connected to the second residual unit.
However, Wang discloses a first feature mapping unit configured to map the image feature information outputted by the first residual unit, and a second feature mapping unit configured to map the image feature information outputted by the second residual unit, wherein the first feature mapping unit is connected to the first residual unit, and the second feature mapping unit is connected to the second residual unit (Wang: yielding 64 feature maps of LRi, 64 feature maps of LRi-1 Figs. 6-8 ¶ [0169]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dai and Singh with a first feature mapping unit configured to map the image feature information outputted by the first residual unit, and a second feature mapping unit configured to map the image feature information outputted by the second residual unit, wherein the first feature mapping unit is connected to the first residual unit, and the second feature mapping unit is connected to the second residual unit, as taught by Wang. The modification allows neural networks to build a hierarchical representation of image data that improves computational efficiency, allowing the neural network to excel at feature recognition tasks.
5. Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Dai in view of Singh and Poudel Karmatha et al., US 20200134772 A1 (hereinafter referred to as “Poudel Karmatha”).
Regarding claim 7 (Original), Dai discloses the method according to claim 3, a convolution kernel size of the second convolution layer is m×m, m and n are positive integers, and m≠n (Dai: Fig. 3A).
Dai and Singh do not disclose the first convolution layer comprises: two sub-convolution layers obtained by decomposing a convolution layer having a convolution kernel size of n×n through depth-wise separable convolution (DSC)
However, Poudel Karmatha discloses wherein the first convolution layer comprises: two sub-convolution layers obtained by decomposing a convolution layer having a convolution kernel size of n×n through depth-wise separable convolution (DSC) (Poudel Karmatha: Fig 3).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dai and Singh with the first convolution layer comprises: two sub-convolution layers obtained by decomposing a convolution layer having a convolution kernel size of n×n through depth-wise separable convolution (DSC), as taught by Poudel Karmatha. The modification is known to be beneficial because depth-wise separable convolutions significantly reduce computational cost, block size, and energy consumption compared to standard convolutions.
Regarding claim 8 (Original), Dai and Singh do not disclose the method according to claim 7, wherein the two sub-convolution layers comprise: a third sub-convolution layer having a convolution kernel size of n×n and performing group convolution, and a fourth sub-convolution layer having a convolution kernel size of 1×1.
However, Poudel Karmatha discloses the two sub-convolution layers comprise: a third sub-convolution layer having a convolution kernel size of n×n and performing group convolution, and a fourth sub-convolution layer having a convolution kernel size of 1×1(Poudel Karmatha: Fig 3).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dai and Singh with wherein the two sub-convolution layers comprise: a third sub-convolution layer having a convolution kernel size of n×n and performing group convolution, and a fourth sub-convolution layer having a convolution kernel size of 1×1, as taught by Poudel Karmatha. The modification is known to be beneficial because depth-wise separable convolutions significantly reduce computational cost, block size, and energy consumption compared to standard convolutions.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DEIRDRE L BEASLEY whose telephone number is (571)270-0452. The examiner can normally be reached Monday-Friday 8 a.m. -5 p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chris Kelley can be reached at (571) 272-7331. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DLB/Patent Examiner, Art Unit 2482
/CHRISTOPHER S KELLEY/Supervisory Patent Examiner, Art Unit 2482