DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-8 are rejected under 35 U.S.C. 103 as being unpatentable over D11 and further in view of D2.2
With regard to claim 1, D1 teach method for implementing a lightweight dense residual network to achieve super- resolution performance, comprising (see abstract): receiving an input comprising a number of frames at a lower resolution (see fig. 2: input low resolution images); generating a first output, by a network comprising two or more DenseRes blocks and an additional convolution operation, each DenseRes block comprising two or more layers, each of the two or more layers comprising a convolution operation and a rectified linear activation function (ReLU) operation, and a 1x1 convolution operation, the first output comprising a plurality of feature maps (see fig. 1, 2); and generating a second output comprising the number of frames at a higher resolution, the higher resolution relative to the lower resolution by an upscaling factor (see fig. 2: outputting high resolution image).
D1 fails to explicitly teach up-sampling the first output by a pixel shuffle layer in the network, however D2 teaches the missing feature (see §3 last ¶). One skilled in the art before the effective filing date would have found it obvious to combine the teachings to arrive at the claimed invention. In particular, it would have been obvious to incorporate pixel shuffle layer as taught by D2 into the configuration of D1 yielding predictable and enhanced results. The motivation would have been to improve computation efficiency by reorganizing low resolution feature channels in a high-resolution image.
D1 and D2 fail to explicitly teach inputting a plurality of low-resolution frames and outputting plurality of high-resolution frames. However, one skilled in the art would have found it obvious to process plurality of low-resolution images using the network of D1 yielding predictable and enhanced results. The motivation would have been to implement the super resolution network on video images, for example, where a plurality of images needs to be up-sampled to higher resolution images.
With regard to claim 5, D1 teach method of claim 1, but fails to explicitly teach wherein the network is configured to process ten or more frames concurrently. However, one skilled in the art would have found it obvious to implement the network of D1 on video images and be further motivated to process plurality of frames concurrently by utilizing plurality of denseres blocks in order to process plurality of frames of video, providing seamless high resolutions images for video viewing.
With regard to claim 6, D1 fails to explicitly teach wherein the ten or more frames comprises every sixth frame in 60FPS video. However, it would have been obvious for one skilled in the art to process only every nth frame because this would result in reduced processing load and faster video effects.
With regard to claim 7, D1 teach method of claim 1, wherein a residual connection from a previous layer of the two or more layers in the network propagates a feature map from the previous layer to one or more upcoming layers (see fig. 1, 2).
With regard to claim 8, D1 teach method of claim 1, wherein the 1x1 convolution operation is configured to extract a compressed feature map from two or more feature maps within a DenseRes block (see fig. 1, 2).
Claims 2-4 are rejected under 35 U.S.C. 103 as being unpatentable over D1 in view of D2 and further in view of D3.3
With regard to claim 2, D1 and D2 teach method of claim 1, but fail to explicitly teach wherein the network comprises an additional ReLU operation clipped to have a maximum value of 1, the additional ReLU operation implemented before the pixel shuffle layer. However, D3 teach the missing feature (see abstract, fig. 3, § 3.2 last 3 paragraphs: ReLU clipped at a maximum value of 1).
One skilled in the art before the effective filing date would have found it obvious to combine the teachings to arrive at the claimed invention. In particular, it would have been obvious to incorporate known teachings of clipped ReLU as taught by D3 into the configuration of D1 yielding predictable and enhance results. The motivation would have been to improve reconstruction quality and runtime.
With regard to claim 3, D3 teach wherein the network is configured to run in real-time on a mobile device (see abstract, § 2 ¶ 1: mobile devices). The motivation for combining the references would have been the same as stated above. In addition, D1 teaches that the network improves performance, suggesting that it can be used on lower performance devices, such as mobile devices.
With regard to claim 4, D1 teach method of claim 1, wherein the second output provides for video super-resolution see fig. 1, 2: super resolution images). D3 teach specifically teaches mobile devices as noted above. In addition, D1 teaches that the network improves performance, suggesting that it can be used on lower performance devices, such as mobile devices.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AVINASH YENTRAPATI whose telephone number is (571)270-7982. The examiner can normally be reached on 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached on (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AVINASH YENTRAPATI/Primary Examiner, Art Unit 2672
1 Zhang, Yulun, et al. "Residual dense network for image super-resolution." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
2 Purohit, Kuldeep, Srimanta Mandal, and A. N. Rajagopalan. "Scale-recurrent multi-residual dense network for image super-resolution." Proceedings of the European Conference on Computer Vision (ECCV) Workshops. 2018.
3 Ayazoglu, Mustafa. "Extremely lightweight quantization robust real-time single-image super resolution for mobile devices." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.