DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/11/2025 has been made record of and considered by the examiner.
Claim Objections
Claims 2 and 13 are objected to because of the following informalities: “the consistency loss is calculated in a first convolutional layer in the first neural network model and a second convolutional layer in the second neural network model”, particularly “is calculated in,” may lead to ambiguity. The examiner respectfully suggests amending to something along the lines of: “is calculated based on VGG features at a first convolutional layer” Appropriate correction is required.
Claims 12 is objected to because of the following informalities: “one or more processors; and one or more computer storage media for storing one or more computer-readable instructions, wherein the processor is configured to drive the computer storage media to execute the following tasks:”, should be corrected to “one or more processors; and one or more computer storage media for storing one or more computer-readable instructions, wherein the one or more processors are configured to drive the computer storage media to execute the following tasks:”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-22 are rejected under 35 U.S.C. 103 as being unpatentable over Chen (‘Seeing Motion in the Dark’), and in further view of Zhang (CN115619674A).
Consider claims 1and 12, Chen discloses a device for denoising dynamic video (Abstract; “we train a Siamese network on static raw videos, for which ground truth is available, such that the network generalizes to videos of dynamic scenes at test time”), comprising:
one or more processors (4.1.Implementation details; TensorFlow, Nvidia Tesla V100 GPU with 32 GB of memory); and
one or more computer storage media for storing one or more computer-readable instructions, wherein the processor is configured to drive the computer storage media to execute the following tasks (4.1.Implementation details; TensorFlow, Nvidia Tesla V100 GPU with 32 GB of memory):
obtaining a first image frame and a second image frame (FIG. 3, 3. Dark Raw Video Dataset; 4. Method; “First, the raw Bayer video frames are preprocessed. The preprocessing includes Bayer to raw RGB conversion, … The result after these preprocessing stages is shown in Fig.3 as “Raw RGB video”… For training, two frames from a static sequence in DRV are sampled at random and are fed to the network in Siamese mode.”); and
inputting the first image frame and the second image frame to a first neural network model and a second neural network model, respectively (4. Method; “For training, two frames from a static sequence in DRV are sampled at random and are fed to the network in Siamese mode. Let
Y
1
^
and
Y
2
^
denote these two frames”), to generate a first optimized image frame and a second optimized image frame, wherein the first neural network model and the second neural network model are trained using a consistency loss (4.2 Method, Equations 2-4; “The loss for this training pair is defined as … where Lr is referred to as the the recovery loss and Lc is called the self-consistency loss… To alleviate temporal instability, we use the self-consistency loss, which encourages the two outputs to be close to each other. The network produces output in sRGB space.”).
Chen’s (‘Seeing Motion in the Dark’) (2019)
Figure 3:
PNG
media_image1.png
185
685
media_image1.png
Greyscale
Current Application 18/969,613
Figure 2:
[0039] “use the consistency loss LC during the training process to compare the continuous image frames 222 and 224 and reduce the difference between the feature maps corresponding to different image frames”
PNG
media_image2.png
337
988
media_image2.png
Greyscale
In related art, Zhang clearly supports: inputting the first image frame and the second image frame to a first neural network model (Zhang ¶49; Backbone 1) and a second neural network model (Zhang ¶52; Backbone 2), respectively.
Zhang additionally discloses, one or more computer storage media for storing one or more computer-readable instructions, wherein the processor is configured to drive the computer storage media to execute the following tasks (Zhang ¶19, 101-102):
inputting the first image frame and the second image frame to a first neural network model (Zhang ¶49; Backbone 1) and a second neural network model (Zhang ¶52; Backbone 2), respectively, to generate a first optimized image frame and a second optimized image frame (Zhang ¶40-41; since the training set video is split into frames, the unsupervised denoising network model is trained to denoise each frame of video image separately… dual-input twin curve estimation network includes a first network model and a second network model with shared weights” ¶47-57; ‘After the training frame image pair (vi, vi+1) is obtained through the denoising network model (Vi, Vi+1) as the input of the twin-input twin curve estimation network model, the output enhancement parameter map is paired… enhanced video frame image pair (Yi, Yi+1) is finally output”), wherein the first neural network model and the second neural network model are trained using a consistency loss (Zhang ¶30-32; “The image pair is iteratively enhanced, … and inter-frame consistency regularization loss function are calculated, and the low-illumination enhancement network model is trained and saved”; ¶89; “the double-input twin curve estimation network calculates the inter-frame consistency regularization loss function for the adjacent frame enhancement results, and encourages the adjacent frame enhanced image features to be close to each other, which can alleviate temporal instability and prevent frame loss when enhancing video. Inconsistency problems between frames, such as flickering and artifacts, so using twin-input twin curve estimation network and frame-to-frame consistency regularized loss function during training can effectively encourage the network to produce temporally stable outputs”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to incorporate the clear network support of Zhang into the denoising method of Chen to further encourage the two outputs to be similar to each other. As stated by Chen, “To alleviate temporal instability, we use the self-consistency loss, which encourages the two outputs to be close to each other (Chen 4. Method).” Zhang further states, “by encouraging adjacent frames to enhance image features close to each other, temporal instability can be alleviated and prevent inter-frame inconsistencies when enhancing video, such as flickering and artifacts. Therefore, dual-input twin curve estimation is used during training (Zhang ¶162).”
Consider claims 2 and 13, Chen, as modified by Zhang, discloses the claimed invention wherein the consistency loss is calculated in a first convolutional layer in the first neural network model and a second convolutional layer in the second neural network model (Chen 4. Method, 4.1 Implementation Details; Zhang ¶81, FIG. 3).
Consider claims 3 and 14, Chen, as modified by Zhang, discloses the claimed invention:
PNG
media_image3.png
318
595
media_image3.png
Greyscale
(Chen Eq. 4; Zhang ¶82-83, Eq. 9)
Consider claims 4 and 15, Chen, as modified by Zhang, discloses the claimed invention further comprising: using a recovery loss to promote the first optimized image frame and the second optimized image frame to be close to a real image frame (Chen FIG. 3, 4. Method, Eq. 3).
Consider claims 5 and 16, Chen, as modified by Zhang, discloses the claimed invention:
PNG
media_image4.png
331
613
media_image4.png
Greyscale
(Chen FIG. 3, 4. Method, Eq. 3)
Consider claims 6 and 17, Chen, as modified by Zhang, discloses the claimed invention wherein the first image frame and the second image frame are two consecutive image frames randomly sampled from a pre-processed dynamic video (Chen FIG. 3, 4. Method, Eq. 3; Zhang ¶94).
Consider claims 7 and 18, Chen, as modified by Zhang, discloses the claimed invention wherein the pre-processed dynamic video is a dynamic video obtained through pre-processing (Chen FIG. 3, 4. Method, Eq. 3).
Consider claims 8 and 19, Chen, as modified by Zhang, discloses the claimed invention wherein the pre-processing comprises Bayer to raw RGB conversion, black level subtraction, binning and global digital gain (Chen FIG. 3, 4. Method, Eq. 3).
Consider claims 9 and 20, Chen, as modified by Zhang, discloses the claimed invention wherein the first image frame and the second image frame are input to the first neural network model and the second neural network model in a Siamese mode (Chen FIG. 3, 4. Method, Eq. 3).
Consider claims 10 and 21, Chen, as modified by Zhang, discloses the claimed invention wherein the first neural network model and the second neural network model are deep Siamese network models (Chen 1. Introduction; 4. Method; Zhang ¶41, 97).
Consider claims 11 and 22, Chen, as modified by Zhang, discloses the claimed invention wherein the first neural network model and the second neural network model are based on a convolutional neural network (CNN) model (Chen FIG. 3, 4. Method, 4.1 Implementation details; Zhang ¶43-45, 81).
Relevant Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. US 2020/0357099 A1 discloses techniques of inpainting video content.
US 2023/0063209 A1 discloses neural network training based on consistency loss.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ASHLEY HYTREK whose telephone number is (703)756-4562. The examiner can normally be reached M-F 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Steve Koziol can be reached at (408)918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ASHLEY HYTREK/ Examiner, Art Unit 2665
/Stephen R Koziol/ Supervisory Patent Examiner, Art Unit 2665
Prosecution Insights