DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8, and 12-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over D1 (WO 2022/057837 A1) in view of D2 (U.S. PG-PUB NO. 2023/0419447) and further in view of D3 (U.S. PG-PUB NO. 2023/0325974).
-D1 discloses a method for video processing, comprising: receiving an input image (S101, acquiring an image to be processed, page 5); processing the input image by a first convolution layer (Input the image to be processed into the convolution layer for convolution processing to obtain the initial feature map, page 6); processing the input image by multiple Multi-mixed Scale and Depth Information with Attention Blocks (MMSDABs), concatenating outputs of the MMSDABs to form a concatenated image (Taking the initial feature map as the input of the first concatenated block and the output of the N-1th first convolutional layer as the input of the Nth concatenated block, the multi-scale feature extraction is performed by using the concatenated block, and the output intermediate feature map, page 6); processing the concatenated image (Take the output of the last first convolutional layer as the reconstructed feature map, page 6) and processing the intermediate image to generate an output image (S103, using the sub-pixel convolution layer of the image reconstruction model to amplify the reconstructed feature map to obtain a reconstructed image, page 5).
D1 is silent to teaching that applied to a decoding processor and wherein each of the MMSDABs includes more than two convolution branches sharing convolution parameters; processing by a second convolution layer to form an intermediate image, and by a third convolutional layer and a pixel shuffle layer. However, the claimed limitation is well known in the art as evidenced by D2.
In the same field of endeavor, D2 teaches applied to a decoding processor (decoded video, [0020]) and wherein each of the MMSDABs includes more than two convolution branches sharing convolution parameters (LiDeR takes N (i.e., a given number of) frames with resolutions W×H and produces N frames with resolutions U*W×U*H where U is the upscaling factor. Each frame may be processed with the same layers in parallel, [0021]); processing by a second convolution layer to form an intermediate image (the 1×1 convolution operation 208 may be applied at the end of block 200 to combine all the previous information within the block and extract a compressed feature map, [0023]), and by a third convolutional layer and a pixel shuffle layer (Network 100 consists of DenseRes blocks 104 and 106 (e.g., a combination Dense network and Residual network, such as DenseRes block 200 in FIG. 2) followed by a convolution operation 108 configured to prepare feature maps for a pixel shuffle layer 112, which may be applied at or towards the end to upscale the input to the target resolution, [0022]).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of D1 with the teaching of D2 in order to provide a lightweight dense residual network for video super-resolution on mobile devices.
The combination is silent to teaching that wherein a second convolution kernel size of the second convolution layer is smaller than a first convolution kernel size of the first convolution layer. However, the claimed limitation is well known in the art as evidenced by D3.
In the same field of endeavor, D3 teaches wherein a second convolution kernel size of the second convolution layer is smaller than a first convolution kernel size of the first convolution layer (F0 is preferably obtained by a 3×3 convolutional layer with LeakyReLU, [0052]).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of the combination with the teaching of D3 in order to reduce the enlarged channel count at minimal cost.
-Regarding claim 2, the combination further discloses the input image is received by a first part of a Multi-mixed Scale and Depth Information with Attention Neural Network (MMSDANet) (D1, the feature extraction network includes convolutional layers, multiple concatenated blocks and multiple first convolutional layers, multiple concatenated blocks, page 5).
-Regarding claim 3, the combination further discloses the first convolution layer is a "3x3" convolution layer (D1, F0 is preferably obtained by a 3×3 convolutional layer with LeakyReLU, [0052]), and wherein the first convolution layer is included in the first part of the MMSDANet (D1, Input the image to be processed into the convolution layer for convolution processing to obtain the initial feature map, page 6).
-Regarding claim 4, the combination further discloses the multiple MMSDABs are included in a second part of the MMSDANet, and wherein the second part of the MMSDANet includes a concatenation module (D1, Perform channel stacking on the initial feature map and the intermediate feature map output by each concatenated block before the Nth first convolutional layer, and input the Nth first convolutional layer for convolution processing after stacking, page 6).
-Regarding claim 5, the combination further discloses the multiple MMSDABs include 8 MMSDABs (D2, LiDeR, as described herein, can process up to ten (10) or more frames concurrently, [0024]).
-Regarding claim 6, the combination further discloses he second convolution layer is an "1x1" convolution layer, and therein the second convolution layer is included in the second part of the MMSDANet (D2, the 1×1 convolution operation 208 may be applied at the end of block 200 to combine all the previous information within the block and extract a compressed feature map, [0023]).
-Regarding claim 7, the combination further discloses the third convolution layer is a "3x3" convolution layer, and wherein the third convolution layer is included in a third part of the MMSDANet (D3, the deep features Fd is preferably reshaped using a 3×3 convolutional layer to obtain a residual image, [0054]).
-Regarding claim 8, the combination further discloses each of the MMSDABs includes a first layer, a second layer, and a third layer (D1, the concatenated block includes multiple residual blocks and multiple second convolution layers, and multiple residual blocks and multiple second convolution layers are alternately arranged, page 6).
-Regarding claim 12, D1 discloses a system for video processing, the system comprising: a processor (2120-processor, page 4); and a memory configured to store instructions (2110-storage medium, page 4), when executed by the processor, to: receive an input image (S101, acquiring an image to be processed, page 5); process the input image by a first convolution layer (Input the image to be processed into the convolution layer for convolution processing to obtain the initial feature map, page 6); process the input image by multiple Multi-mixed Scale and Depth Information with Attention Blocks (MMSDABs); concatenate outputs of the MMSDABs to form a concatenated image (Taking the initial feature map as the input of the first concatenated block and the output of the N-1th first convolutional layer as the input of the Nth concatenated block, the multi-scale feature extraction is performed by using the concatenated block, and the output intermediate feature map, page 6); process the concatenated image (Take the output of the last first convolutional layer as the reconstructed feature map, page 6); process the intermediate image and generate an output image (S103, using the sub-pixel convolution layer of the image reconstruction model to amplify the reconstructed feature map to obtain a reconstructed image, page 5).
D1 is silent to teaching that wherein each of the MMSDABs includes more than two convolution branches sharing convolution parameters; by a second convolution layer to form an intermediate image; by a third convolutional layer and a pixel shuffle layer. However, the claimed limitation is well known in the art as evidenced by D2.
In the same field of endeavor, D2 teaches wherein each of the MMSDABs includes more than two convolution branches sharing convolution parameters (LiDeR takes N (i.e., a given number of) frames with resolutions W×H and produces N frames with resolutions U*W×U*H where U is the upscaling factor. Each frame may be processed with the same layers in parallel, [0021]); by a second convolution layer to form an intermediate image (the 1×1 convolution operation 208 may be applied at the end of block 200 to combine all the previous information within the block and extract a compressed feature map, [0023]); by a third convolutional layer and a pixel shuffle layer (Network 100 consists of DenseRes blocks 104 and 106 (e.g., a combination Dense network and Residual network, such as DenseRes block 200 in FIG. 2) followed by a convolution operation 108 configured to prepare feature maps for a pixel shuffle layer 112, which may be applied at or towards the end to upscale the input to the target resolution, [0022]).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of D1 with the teaching of D2 in order to provide a lightweight dense residual network for video super-resolution on mobile devices.
The combination is silent to teaching that wherein a second convolution kernel size of the second convolution layer is smaller than a first convolution kernel size of the first convolution layer. However, the claimed limitation is well known in the art as evidenced by D3.
In the same field of endeavor, D3 teaches wherein a second convolution kernel size of the second convolution layer is smaller than a first convolution kernel size of the first convolution layer (F0 is preferably obtained by a 3×3 convolutional layer with LeakyReLU, [0052]).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of the combination with the teaching of D3 in order to reduce the enlarged channel count at minimal cost.
-Regarding claim 13, the combination further discloses the input image is received by a first part of a Multi-mixed Scale and Depth Information with Attention Neural Network (MMSDANet) (D1, the feature extraction network includes convolutional layers, multiple concatenated blocks and multiple first convolutional layers, multiple concatenated blocks, page 5).
-Regarding claim 14, the combination further discloses the first convolution layer is a "3x3" convolution layer (D1, F0 is preferably obtained by a 3×3 convolutional layer with LeakyReLU, [0052]), and wherein the first convolution layer is included in the first part of the MMSDANet (D1, Input the image to be processed into the convolution layer for convolution processing to obtain the initial feature map, page 6).
-Regarding claim 15, the combination further discloses the multiple MMSDABs is included in a second part of the MMSDANet, and wherein the second part of the MMSDANet includes a concatenation module (D1, Perform channel stacking on the initial feature map and the intermediate feature map output by each concatenated block before the Nth first convolutional layer, and input the Nth first convolutional layer for convolution processing after stacking, page 6).
-Regarding claim 16, the combination further discloses the multiple MMSDABs include 8 MMSDABs (D2, LiDeR, as described herein, can process up to ten (10) or more frames concurrently, [0024]).
-Regarding claim 17, the combination further discloses the second convolution layer is a "1x1" convolution layer, therein the second convolution layer is included in the second part of the MMSDANet (D2, the 1×1 convolution operation 208 may be applied at the end of block 200 to combine all the previous information within the block and extract a compressed feature map, [0023]), wherein the third convolution layer is a "3x3" convolution layer, and wherein the third convolution layer is included in a third part of the MMSDANet (D3, the deep features Fd is preferably reshaped using a 3×3 convolutional layer to obtain a residual image, [0054]).
-Regarding claim 18, the combination further discloses each of the MMSDABs includes a first layer, a second layer, and a third layer (D1, the concatenated block includes multiple residual blocks and multiple second convolution layers, and multiple residual blocks and multiple second convolution layers are alternately arranged, page 6).
Claim(s) 9 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over D1 (WO 2022/057837 A1) in view of D2 (U.S. PG-PUB NO. 2023/0419447), D3 (U.S. PG-PUB NO. 2023/0325974) and further in view of D4.
-Regarding claim 9, the combination is silent to teaching that the first layer includes three convolutional layers with different dimensions. However, the claimed limitation is well known in the art as evidenced by D4.
In the same field of endeavor, D4 teaches the first layer includes three convolutional layers with different dimensions (the weighted aggregation of N layers, the subsequent one or more convolutional layers (Conv 2, Conv 3, and the like) reduce the outputs to Nx h x w, wherein the choice of 3x3, [0034]).
Therefore, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to combine the teaching of the combination with the teaching of D4 in order to improve the detection and classification of objects across scale.
-Regarding claim 10, the combination further discloses the second layer includes one "1x1" convolutional layer (D4, he fusion network employs a 1 x 1 convolutional network, [0031]) and two "3x3" convolutional layers (D4, the weighted aggregation of N layers, the subsequent one or more convolutional layers (Conv 2, Conv 3, and the like) reduce the outputs to Nx h x w, wherein the choice of 3x3, [0034]).
Allowable Subject Matter
Claims 11 and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 20 is allowed.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PING Y HSIEH whose telephone number is (571)270-3011. The examiner can normally be reached Monday-Friday, 9am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PING Y HSIEH/ Primary Examiner, Art Unit 2664
Prosecution Insights