DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 17 November 2025 have been fully considered but they are not persuasive.
Applicant argues that Oh does not appear to disclose "use ... one or more neural networks to generate ... blending weights," nor a neural network that uses masks. Examiner notes that Kalantari is cited for this claim limitation, not Oh. Applicant argues that Kalantari’s neural network appears to take aligned image stacks as inputs rather than computed "pixel value masks" from comparisons. Examiner notes that the aligned image stacks are inputs to the network in Kalantari, but also notes that claim 1 requires the blending weights to be based, at least in part, on the one or more pixel value masks, not direct inputs to the network. In Kalantari, the initial blending parameters mask the values of input images in order to combine them, and those combined images are used to refine the blending weights in the network. Thus the weights are generated by the network based, at least in part, on the triangle merge masks that are used to initially combine images.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 1, 4, 7, 10, 13, 16, 19, 22, 25, 28, 32, 34, and 36 are rejected under 35 U.S.C. 103 as being unpatentable over Kalantari in view of Oh (U.S. Publication 2020/0027202) and further in view of Hu (U.S. Publication 2020/0265567).
As to claim 1, Kalantari discloses one or more processors (fig. 12, element 1202), comprising circuitry to use one or more neural networks to generate one or more blending weights to images based, at least in part, on one or more pixel value masks (p. 3, section 0048-p. 4, section 0063; p. 5, sections 0078-0079; various methods that output, using a convolutional neural network, blending weights that are used to blend images are described; the training set includes initial blending parameters that implement a triangle merge that can read on broadly recited “pixel value masks” for the images).
Kalantari does not disclose, but Oh discloses comparing one or more pixel values corresponding to one or more locations of a prior frame of a video to one or more pixel values of one or more groups of pixels of a window around a corresponding location of a current frame of the video, to generate one or more pixel value masks based on the corresponding pixel values (p. 1, section 0009-p. 2, section 0011; p. 2, section 0031; p. 5, sections 0053-0055; p. 6, section 0065; difference metrics between frames and variance for a previous frame pixel are calculated and used to control blending of pixels, acting as a mask; the SAD difference metric is determined by comparing a 3x3 window around a pixel of a prior frame to a 3x3 window of that same pixel location in a current frame), wherein the one or more blending weights are to be used to blend pixels of at least two frames of video (p. 6, section 0067-p. 7, section 0074; the value alpha, which is based on the difference metric that acts as a mask, acts as a blending weight to blend a current frame with a previous filtered frame). The motivation for this is to reduce the potential of replacing a current frame with a noisy previous output. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Kalantari to compare one or more pixel values corresponding to one or more locations of a prior frame of a video to one or more pixel values of one or more groups of pixels of a window around a corresponding location of a current frame of the video, to generate one or more pixel value masks based on the corresponding pixel values, wherein the one or more blending weights are to be used to blend pixels of at least two frames of video in order to reduce the potential of replacing a current frame with a noisy previous output as taught by Oh.
Kalantari does not disclose, but Hu discloses the one or more blending weights generated by the neural network are to be used to blend pixels of at least two frames of the video (p. 1, section 0002; p. 12, section 0113-p. 13, section 0115; p. 16, section 0138; p. 18, section 0156; blending weight maps are created using a neural network and modified using luminance/luma values and motion values to not blur with specific images; the maps are used to blend pixels of multiple frame images captured in sequence, reading on frames of a video). The motivation for this is to control blending when blending would result in too high of a blur. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Kalantari and Oh to use blending weights generated by the neural network to blend pixels of at least two frames of the video in order to control blending when blending would result in too high of a blur as taught by Hu.
As to claim 4, Kalantari does not disclose, but Oh does disclose wherein the one or more blending weights are further determined based, at least in part, upon luma values for one or more of the current frame and the prior frame (p. 4, sections 0050-0051; the SAD masks and alpha blending weights are determined based on each channel in an image, including luma and chroma color channels in a YUV format). Motivation for the combination is given in the rejection to claim 1.
As to claim 7, see the rejection to claim 1.
As to claim 10, see the rejection to claim 4.
As to claim 13, see the rejection to claim 1.
As to claim 16, see the rejection to claim 4.
As to claim 19, see the rejection to claim 1. Further, Kalantari discloses a non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to execute the method claimed (p. 9-10, section 0146).
As to claim 22, see the rejection to claim 4.
As to claim 25, see the rejection to claim 1.
As to claim 28, see the rejection to claim 4.
As to claim 32, Kalantari does not disclose, but Oh does disclose wherein the one or more blending weights for at least two frames of the video are further generated based, at least in part, on information that indicates one or more changes of one or more pixels of a current frame within a window around a corresponding location relative to one or more pixel values at the corresponding location of the prior frame (p. 1, section 0009-p. 2, section 0011; p. 2, section 0031; p. 5, sections 0053-0055; p. 6, section 0065-p. 7, section 0074; difference metrics between frames and variance for a previous frame pixel are calculated and used to control blending of pixels, acting as a mask; the SAD difference metric is determined by comparing a 3x3 window around a pixel of a prior frame to a 3x3 window of that same pixel location in a current frame; the value alpha, which is based on the different metric that acts as a mask, acts as a blending weight to blend a current frame with a previous filtered frame). Motivation for the combination is given in the rejection to claim 1.
As to claim 34, Kalantari does not disclose, but Oh does disclose wherein an individual pixel value mask of the one or more pixel value masks includes data that specifies an amount that each pixel of an individual image of the two or more frames is different from a corresponding pixel of another image of the two or more frames (p. 1, section 0009-p. 2, section 0011; p. 2, section 0031; p. 5, sections 0053-0055; p. 6, section 0065-p. 7, section 0074; difference metrics between frames and variance for a previous frame pixel are calculated and used to control blending of pixels, acting as a mask; the SAD difference metric is a mask that is determined by comparing a 3x3 window around a pixel of a prior frame to a 3x3 window of that same pixel location in a current frame, and adding up the sum of differences to represent how different a pixel is to its corresponding pixel from the other frame). Motivation for the combination is given in the rejection to claim 1.
As to claim 36, Kalantari does not disclose, but Oh does disclose wherein the one or more pixel value masks are determined at least in part by determining differences between corresponding pixel values at one or more locations of the prior frame and pixel values of the one or more groups of pixels within the window around the corresponding location of the current frame (p. 1, section 0009-p. 2, section 0011; p. 2, section 0031; p. 5, sections 0053-0055; p. 6, section 0065-p. 7, section 0074; difference metrics between frames and variance for a previous frame pixel are calculated and used to control blending of pixels, acting as a mask; the SAD difference metric is a mask that is determined by comparing a 3x3 window around a pixel of a prior frame to a 3x3 window of that same pixel location in a current frame, and adding up the sum of differences to represent how different a pixel is to its corresponding pixel from the other frame). Motivation for the combination is given in the rejection to claim 1.
Claims 2, 8, 14, 20, and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Kalantari in view of Oh and Hu and further in view of Wang (U.S. Publication 2017/0310972).
As to claim 2, Kalantari does not disclose, but Wang does disclose wherein the circuitry is further to determine a mean and a standard deviation for one or more groups of pixels within a window around a corresponding location of a current frame, and to calculate, relative to the standard deviation, one or more variances between one or more pixel values at one or more locations of the prior frame of the video and the mean of the windowed group of pixels of the current frame (fig. 6; p. 11, section 0091-p. 12, section 0093; p. 15, section 0112; mean and standard deviation are determined for pixels in a 4x4 window around a frame location; variance is calculated with respect to prior window means to evaluate pixel classification), wherein the one or more pixel value masks comprise the calculated one or more variances (p. 11, section 0091-p. 12, section 0093; p. 14, sections 0106-0107; based on the variances, it is determined whether pixels are background; a binary mask is set for each location that indicates whether the pixels are background, which is based on the determination that is based on variance). The motivation for this is to improve coding efficiency (p. 15, section 0115). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Kalantari, Oh, and Hu to determine a mean and a standard deviation for one or more groups of pixels within a window around a corresponding location of a current frame, and to calculate, relative to the standard deviation, one or more variances between one or more pixel values at one or more locations of the prior frame of the video and the mean of the windowed group of pixels of the current frame wherein the one or more pixel value masks comprise the calculated one or more variances in order to improve coding efficiency as taught by Wang.
As to claim 8, see the rejection to claim 2.
As to claim 14, see the rejection to claim 2.
As to claim 20, see the rejection to claim 2.
As to claim 26, see the rejection to claim 2.
Claims 3, 9, 15, 21, and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Kalantari in view of Oh and Hu and further in view of Smolic (U.S. Publication 2016/0353164).
As to claim 3, Kalantari does not disclose, but Oh discloses that the masks are generated from comparing one or more pixel values at one or more locations of the prior frame to pixel values of one or more groups of pixels within a window around a corresponding location of the current frame, as discussed above in the rejection to claim 1. Kalantari further does not disclose, but Smolic does disclose wherein the one or more circuits are further to apply one or more scale factors to the one or more pixel value masks, the one or more scale factors being user configurable (p. 5, section 0058; a user can choose a factor to scale a pixel by in a mask used to blend between pixel values). The motivation for this is to allow for more artistic freedom (p. 2, section 0027; p. 5, section 0059). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Kalantari, Oh, and Hu to apply user configurable scale factors to pixel value masks in order to allow for more artistic freedom as taught by Smolic.
As to claim 9, see the rejection to claim 3.
As to claim 15, see the rejection to claim 3.
As to claim 21, see the rejection to claim 3.
As to claim 27, see the rejection to claim 3.
Claims 5, 11, 17, 23, and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Kalantari in view of Oh and Hu and further in view of Sullivan (U.S. Publication 2008/0232452).
As to claim 5, Kalantari does not disclose, but Sullivan does disclose wherein the one or more circuits are further to utilize a parameterized kernel filter to determine one or more pixel values for one or more upsampled images of the video (p. 9, section 0113-p. 10, section 0116; p. 17, section 0198; a filter with a parameterized kernel is selected to upsample an image of the video). The motivation for this is to allow a high degree of freedom while keeping syntax overhead very low. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Kalantari, Oh, and Hu to utilize a parameterized kernel filter to determine one or more pixel values for one or more upsampled images of the video in order to allow a high degree of freedom while keeping syntax overhead very low as taught by Sullivan.
As to claim 11, see the rejection to claim 5.
As to claim 17, see the rejection to claim 5.
As to claim 23, see the rejection to claim 5.
As to claim 29, see the rejection to claim 5.
Claims 6, 12, 18, 24, and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Kalantari in view of Oh and Hu and further in view of Cai (U.S. Publication 2021/0150674).
As to claim 6, Oh discloses processing including current and prior frames as noted in the rejection to claim 1. The combination of Kalantari, Oh, and Hu does not disclose, but Cai does disclose wherein the one or more circuits are further to reduce a resolution of the frame before generating the one or more blending weights (fig. 4; fig. 9; p. 3, section 0038; p. 5, sections 0053-0055; the images are downsampled in resolution before weights are determined for blending). The motivation for this is to increase the perceptive field and remove low frequency components of noise. It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Kalantari, Oh, and Hu to reduce a resolution of a frame before determining the one or more blending weights in order to increase the perceptive field and remove low frequency components of noise as taught by Cai.
As to claim 12, see the rejection to claim 6.
As to claim 18, see the rejection to claim 6.
As to claim 24, see the rejection to claim 6.
As to claim 30, see the rejection to claim 6.
Claim 33 is rejected under 35 U.S.C. 103 as being unpatentable over Kalantari in view of Oh and Hu and further in view of Weinzaepfel (U.S. Publication 2020/0364509).
As to claim 33, Kalantari does not disclose, but Weinzaepfel discloses wherein the one or more pixel masks include a binary mask (p. 2-3, section 0044; p. 3, section 0057; p. 9-10, section 0177; a neural network has training input including a segmentation mask, which indicates a binary segmentation) that indicates whether history data corresponding to the prior frame is within the window of a corresponding location of an image of the current frame (p. 9-10, section 0177; the mask is used to indicate whether a bounding box contains data for an object of interest in a previous/historical frame and then in subsequent frames; a bounding box can be visualized as a window surrounding location of a pixel or pixels). The motivation for this is to provide a robust, stable, and precise visual localization method (p. 2, sections 0019-0021). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Kalantari, Oh, and Hu to use a binary mask that indicates whether history data corresponding to the prior frame is within the window of a corresponding location of an image of the current frame in order to provide a robust, stable, and precise visual localization method as taught by Weinzaepfel.
Claim 35 is rejected under 35 U.S.C. 103 as being unpatentable over Kalantari in view of Oh and Hu and further in view of Chitiboi (U.S. Publication 2021/0383537).
As to claim 35, Kalantari does not disclose, but Oh discloses the circuitry further to generate an image based, at least in part, on the one or more blending weights, a current image of the current frame and a prior image of the prior frame and add the generated image to the video (p. 1, section 0009-p. 2, section 0011; p. 5, sections 0053-0055; the current and previous frames are blended and added to the collection of frames using the weights) and the one or more pixel value masks are generated based at least in part on comparing one or more pixel values at one or more locations of the prior frame to pixel values of one or more groups of pixels within the window around the corresponding location of the current frame of the video (p. 1, section 0009-p. 2, section 0011; p. 2, section 0031; p. 5, sections 0053-0055; p. 6, section 0065; difference metrics between frames and variance for a previous frame pixel are calculated and used to control blending of pixels, acting as a mask; the SAD difference metric is determined by comparing a 3x3 window around a pixel of a prior frame to a 3x3 window of that same pixel location in a current frame). Motivation for the combination is given in the rejection to claim 1.
Kalantari does not disclose, but Chitiboi does disclose wherein:
the one or more pixel value masks are input to the one or more neural networks (p. 1, section 0007; p. 2, section 0024; p. 2-3, section 0029; masks with values for each pixel are input to the generator neural network);
and the one or more blending weights are further generated, based at least in part on inputting at least one frame of the video to one or more neural networks (p. 3, section 0033; p. 4, section 0047-p. 5, section 0048; weights for blending the network outputs are generated based on training with different images input to the network). The motivation for this is to train machine learning systems for automatic segmentation, reducing time spent by humans in reading images (p. 1, section 0002). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Kalantari, Oh, and Hu to have the one or more pixel value masks input to the one or more neural networks and have the one or more blending weights be further generated, based at least in part on inputting at least one frame of the video to one or more neural networks in order to train machine learning systems for automatic segmentation, reducing time spent by humans in reading images as taught by Chitiboi.
Claim 37 is rejected under 35 U.S.C. 103 as being unpatentable over Kalantari in view of Oh and Hu and further in view of Cai and Park (U.S. Publication 2021/0374461).
As to claim 37, Kalantari does not disclose, but Oh discloses wherein the generated one or more blending weights are per-pixel blending weights and the circuitry is further to blend corresponding pixels of the current frame and the prior frame using the one or more blending weights (p. 1, section 0009-p. 2, section 0011; p. 2, section 0031; p. 5, sections 0053-0055; p. 6, section 0065-0068; each pixel is blended with previous frame pixels based on alpha weights for the pixel). Motivation for the combination is given in the rejection to claim 1.
Further, Kalantari does not disclose, but Cai discloses that the blending weights are generated at an output resolution that is higher than a resolution of the current frame (fig. 4; fig. 9; p. 3, section 0038; p. 5, sections 0053-0055; the image frames are downsampled in resolution before weights are determined for blending; the blending weights corresponding to the fine scale images would be a higher resolution than the current downsampled image frame). Motivation for the combination is given in the rejection to claim 6.
Further, Kalantari does not disclose, but Park discloses that the one or more pixel value masks, the current frame, and the prior frame are provided as input to the one or more neural networks (fig. 17; p. 3, section 0056; p. 9, section 0183-0184; a current frame, prior frame and generated first mask are inputs to the neural networks; a second mask is generated from them to weigh image pixels in a filter). The motivation for this is to filter to reduce noise in an image (p. 7, section 0139). It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to modify Kalantari, Oh, Hu, and Cai to have the one or more pixel value masks, the current frame, and the prior frame provided as input to the one or more neural networks in order to filter to reduce noise in an image as taught by Park.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON M RICHER whose telephone number is (571)272-7790. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON M RICHER/Primary Examiner, Art Unit 2617