Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
A. Claim 1-3, 5, 9-13 and 15 are rejected under 35 USC 103 as being unpatentable over Lv et al. (“MBLLEN: Low-light Image/Video Enhancement Using CNNs”) in view of Delbracio et al. (Projected Distribution Loss for Image Enhancement, arXiv:2012.09289v2 [cs.CV] 17 May 2021) and Yao et al. (US 2024/0005649).
With respect to claim 1, Lv. Et al. teach a method for training an image enhancement model (Fig. 3, Data flow for training)
wherein the image enhancement model comprises an enhancement module configured to enhance brightness and contrast (Abstract, To address difficulty in handling various factors simultaneously including brightness, contrast, artifacts and noise for low-light image enhancement, we propose the multi-branch low-light enhancement network (MBLLEN)), and
the enhancement module comprises convolution branches in one-to-one correspondence with a plurality of preset brightness intervals, the enhancement module is configured to input pixels of an image input to the enhancement module to corresponding convolution branches according to brightness intervals to which the pixels belong (Fig 2 pluralities of EM(enhancement module) with different brightness);subject the pixels to convolution processing by a first convolution unit in each of the convolution branches (Fig 2; right side (CONV in EM)),
merge images output from the respective convolution branches, and subject to convolution processing by a second convolution unit ((Fig 2 (FM (fusion module), page 4, 3.1 Network Architecture, FM, It accepts the outputs of all EM sub-nets to produce the finally enhanced image. We concatenate all the outputs from EM in the color channel dimension and use a convolution kernel to merge them.) ; and the method comprises:
inputting a sample image to the image enhancement model, and acquiring a result image output by the image enhancement model (Fig. 3, Low-Light, Enhanced result);
calculating losses comprising an image loss of the result image relative to a Ground Truth image (Fig. 3, Loss based on Enhanced result and Ground Truth)
adjusting the enhancement module according to the losses (Fig. 3, Loss goes back to EM (enhancement module)).
Lv. Et al. teach do not teach a first constraint loss of brightness histogram constraint of each of the convolution branches of an image output from each of the convolution branches relative to the Ground Truth image;
in a case where a training end condition is not met, returning to the operation of inputting the sample image to the image enhancement model.
Delbracio et al. teach a loss of brightness histogram constraint (distribution of pixel value) of the convolution of an image output from the convolution relative to the Ground Truth image (Fig. 1);
At the time of effective filing, it would have been obvious to a person of ordinary skill in the art to determine loss based on histogram of enhanced image and ground truth image in the method of Lv. Et al.
The suggestion/motivation for doing so would have been that to allow recovering detain (texture/grain) without forcing them to be located in the exact same spatial position as in the reference image (Delbracio et al., page 1 right col. last paragraph).
Yao et al. teach in a case where a training end condition is not met, returning to the operation of inputting the sample image to the image enhancement model. (Fig. 7, para [0071], decision operation 706, where a determination is made as to whether training is complete. Such a determination may be made using any suitable technique or techniques. In some embodiments, training is complete when the loss function or optimization function defined at operation 704 has a loss or error less than a threshold, when loss does not met the condition, it goes back to applying current CNN to subset of training images (sample images)).
At the time of effective filing, it would have been obvious to a person of ordinary skill in the art to repeat training process when loss does meet required condition in the method of Lv. Et al.
The suggestion/motivation for doing so would have been that to get better enhanced image output.
Therefore, it would have been obvious to combine Delbracio et al. and Yao et al. with Lv. Et al. to obtain the invention as specified in claim 1.
With respect to claim 2, Lv. Et al. teach a sampling section provided after the first convolution unit and comprising a plurality of sampling units configured to perform sampling; and inputs to each sampling unit are from a convolution branch where the sampling unit is located and at least one of other convolution branches (Fig 2. Each WxHx32 is interpreted as sample that in inputted to EM (enhancement module that perform convolution).
With respect to claim 3, Delbracio et al. teach the losses further comprise: a second constraint loss of brightness histogram constraint of each of the convolution branches of an image input to the sampling section relative to the image input to the enhancement module (Fig. 1 each layer (branch ) has leaning losses (1..m)).
With respect to claim 5, Lv. Et al. teach that the sampling section comprises a down-sampling unit configured to perform down-sampling, and an up-sampling unit provided after the down-sampling unit and configured to perform up-sampling (Fig. 2 right side EM#n CONV, DECONV).
With respect to claim 9, claim 9 is rejected same reason as claim 1 above.
With respect to claim 10, claim 9 is rejected same reason as claim 1 above.
With respect to claim 11, claim 9 is rejected same reason as claim 1 above.
With respect to claim 12, claim 12 is rejected same reason as claim 2 above.
With respect to claim 13, claim 13 is rejected same reason as claim 3 above.
With respect to claim 15, claim 15 is rejected same reason as claim 5 above.
B. Claim 6, 7, 16 and 17 are rejected under 35 USC 103 as being unpatentable over Lv et al. (“MBLLEN: Low-light Image/Video Enhancement Using CNNs”) in view of Delbracio et al. (Projected Distribution Loss for Image Enhancement, arXiv:2012.09289v2 [cs.CV] 17 May 2021) and Yao et al. (US 2024/0005649) and in further view of Xiang et al. (US 2024/0307018).
With respect to claim 6, Lv et al., Delbracio et al. and Yao et al. teach all the limitations of claim 5 as applied above from which claim 6 respectively depend.
Lv et al., Delbracio et al. and Yao et al. do not teach that the down-sampling unit is configured to perform residual down-sampling; and the up-sampling unit is configured to perform residual up-sampling.
Xiang et al. teach the down-sampling unit is configured to perform residual down-sampling; and the up-sampling unit is configured to perform residual up-sampling (Fig. 2 Series of RSUs, para [0045] residual U-block and [0048], down-sampling and upsampling).
At the time of effective filing, it would have been obvious to a person of ordinary skill in the art to use residual down-sampling and up-sampling in the method of Lv et al., Delbracio et al. and Yao et al.
The suggestion/motivation for doing so would have been that to increase networks’ s depth without a massive increase in computational cost.
Therefore, it would have been obvious to combine Xiang et al. with Lv et al., Delbracio et al. and Yao et al.to obtain the invention as specified in claim 6.
With respect to claim 7, Xiang et al. teach each of the convolution branches further comprises: a short-cut connection between an input terminal of the sampling section and an output terminal of the sampling section, and the short-cut connection is configured to input an image input to the sampling section to the output terminal of the sampling section (para [0044], skip connection).
With respect to claim 16, claim 16 is rejected same reason as claim 6 above.
With respect to claim 17, claim 17 is rejected same reason as claim 7 above.
Allowable Subject Matter
1. Claims 4, 8, 14 and 18 are objected to as being dependent upon a rejected base claim, but would be allowable of rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Randolph Chu whose telephone number is 571-270-1145. The examiner can normally be reached on Monday to Thursday from 7:30 am - 5 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached on (571) 272-7778.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/RANDOLPH I CHU/
Primary Examiner, Art Unit 2667
Prosecution Insights