DETAILED ACTION
This Office Action is in response to the Amendment filed on 06/24/2026.
In the filed response, Claims 1, 2, 5-7, 9, 11, 14-16, and 20 have been amended, where Claims 1, 11, and 16 are independent claims.
Accordingly, Claims 1-20 have been examined and are pending. This Action is made FINAL.
Response to Arguments
1. Applicant’s arguments with respect to the instant claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
2. After carefully considering Applicant’s response, the work of Staranowicz et al. US 2018/0315174 A1, hereinafter referred to as Staranowicz, is brought in to address the newly amended limitation “masking the synthetic frame by multiplying the synthetic frame by a mask generating a masked synthetic frame” given its broadest reasonable interpretation (BRI). Staranowicz, in particular, teaches multiplying an interpolated image by an occlusion mask (i.e. binary mask), where the interpolated image can be considered synthetic as noted in ¶0033 of the filed specification. See for e.g. ¶0066. Further, the interpolation algorithms are based on optical flow (¶0038) which calculate both forward and backward optical flows for frames in a video sequence (¶0044). Also relied on is the work of Sekkappan et al. US 2024/0127572 A1, hereinafter referred to as Sekkappan, which more clearly describes using the calculated forward and backward optical flows in a consistency check to determine whether the flows disagree by more than a threshold (e.g. ¶0025).
3. Other prior art found that is deemed relevant include the work of Chi et al. US 10,958,869 B1, hereinafter referred to as Chi. For e.g., Chi describes video frame interpolation methods via a vector-based approach that rely on estimating optical flow (e.g. col. 1 lines 39-67, col. 2 lines 1-10 and col. 6 lines 50-67). By Chi’s approach, an interpolated intermediate frame may be generated which can be construed as being synthesized (col. 6 lines 24-27). Chi further describes applying high and low motion masks (i.e. binary masks) to the candidate interpolated frame in order to help refine errors (e.g. col. 9 lines 13-25). Also note Staranowicz et al. US 2018/0061012 A1 which pertain to methods for video image post processing for correcting artifacts. Please refer to PTO 892.
4. Applicant’s response/amendments related to the objection to the specification are acknowledged. As such, the objection is withdrawn.
5. Applicant’s amendments to the drawings are acknowledged and appreciated.
6. Applicant’s response/amendments related to the claim rejections under 35 U.S.C. 112(a) are acknowledged. As such, the rejections are withdrawn.
7. The Examiner is available to discuss the matters of this office action to help move the Instant Application forward. Please refer to the conclusion to this office action regarding scheduling interviews.
8. In light of the foregoing, Claims 1-20 have been examined and are pending.
Claim Objections
9. Claim 9 is objected to because of the following informalities: the claim recites “adding the masked synthetic frame to an accumulator; and adding the mask used in masking the synthetic frame to an inclusion counter.” As claimed, the terms “accumulator” and “inclusion counter” are vague since their functions are not clearly specified in the context of the masked synthetic frame. According to for e.g. ¶0038 of the filed specification, said accumulator appears to be used for recording the sum of pixels when generating composite frames. Regarding said inclusion counter, ¶0184 of the filed specification shows said inclusion counter may be incremented for pixels of the synthetic frame included in the accumulator. Recommend clarifying the foregoing limitations to help strengthen the claim. Appropriate correction is required.
Claim Rejections - 35 USC § 102
10. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim16 is rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Staranowicz et al. US 2018/0315174 A1, hereinafter referred to as Staranowicz, where Staranowicz discloses methods for detecting and removing artifacts using frame interpolation techniques (e.g. ¶0003)
Regarding claim 16, Given the broadest reasonable interpretation (BRI) of the following limitations, Staranowicz’s teaches and/or suggests “comprising: extracting a plurality of frames from the video; generating a plurality of optical flow files based on the video [See for e.g. ¶0044 with respect to calculating forward and backward optical flows for selected frames of data from a video sequence. Calculated optical flows of higher order are construed to generate data files that can be used for performing more accurate motion estimation]; generating a plurality of synthetic frames corresponding to that mimic a frame of the video based on an optical flow estimation of the video in the plurality of optical flow files Although a ‘synthetic frame’ is not ‘explicitly’ disclosed, the filed specification (e.g. ¶0033) shows that linear interpolation can be construed as a means for generating a synthetic frame. As such, please see ¶0038 regarding interpolation algorithms based on optical flow. Also note ¶0044, where generated interpolated frame(s) may be realized based on calculated optical flows of higher order]; performing a selective averaging of portions of the plurality of synthetic frames and the plurality of frames generating a plurality of composite frames [See for e.g. ¶0038 and ¶0050 regarding linear blending/weighted averaging of frames]; and compiling a denoised video based on the plurality of composite frames.” [Based on Staranowicz’s frame interpolation methods, artifacts can be detected and removed (e.g. ¶0003)]
Claim Rejections - 35 USC § 103
11. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Staranowicz, in further view of Izadi et al. US 2023/0119747 A1, hereinafter referred to as Izadi.
Regarding claim 1, Given the broadest reasonable interpretation (BRI) of the following limitations, Staranowicz’s teaches and/or suggests “A method of denoising a video, comprising: estimating optical flow in the video [See fig. 2A regarding calculated optical flow from a video sequence, where forward and backward optical flows may be determined (e.g. ¶0044)]; generating synthetic frame corresponding to a frame of the video based on a neighboring frame of the frame and the optical flow [Although a ‘synthetic frame’ is not ‘explicitly’ disclosed, the filed specification (e.g. ¶0033) shows that linear interpolation can be construed as a means for generating a synthetic frame. As such, please see ¶0038 regarding interpolation algorithms based on optical flow. Also note ¶0044, where based on higher order optical flow calculations involving three or more sets of frames, more accurate motion estimation of pixels in a generated interpolated frame(s) may be realized]; masking the synthetic frame by multiplying the synthetic frame by a mask generating a masked synthetic frame [See for e.g. ¶0066 where an occlusion mask (i.e. a binary mask) and the interpolated image are multiplied together to yield a temporary image, which can be construed as a “masked synthetic frame” given its BRI]; generating a composite frame based on the masked synthetic frame and the frame [In ¶0066, the final step is to combine the interpolated frame and a smoothed version of the temporary image (above) to create an interpolated frame that does not have any holes or missing image data along for e.g., the edges, i.e. the combined/composite image can be construed as a denoised image.]; and encoding the composite frame into a denoised video.” [Although Staranowicz does not explicitly refer to encoding the combined image above, it is believed this would be within the level of skill in the art. Please see for e.g. the work of Izadi below for corresponding support] Although Staranowicz’s method for generating a noise removed interpolated frame of data is deemed relevant given the BRI of the limitation, it does not explicitly address encoding this frame. Even though this would be considered within the level of skill in the art in order to facilitate storing and/or transmitting said frames, the work of Izadi from the same or similar field of endeavor is brought in to address the foregoing feature. [See for e.g. ¶0089 regarding an encoded denoised image either as a single image or as part of a sequence of images] Recognizing Izadi’s teachings for encoding a denoised image(s), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the work of Staranowicz for detecting and removing artifacts from generated interpolated frames of data (e.g. ¶0003), to add the teachings of Izadi as above for denoising image data before coding so that not only the efficiency and accuracy of coding can be improved but also the visual quality of the image data itself (e.g. ¶0022).
Regarding claim 6, Staranowicz and Izadi teach and/or suggest all the limitations of claim 1 and are analyzed as previously discussed with respect to that claim. Staranowicz further teaches and/or suggests “where masking the synthetic frame comprises: determining edges of the frame; and generating the mask based on the edges.” [See for e.g. ¶0075 with respect to Staranowicz’s edge detection module which applies corrections to the interpolated frame]
Claims 2, 3, 5, and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Staranowicz et al. US 2018/0315174 A1, in view of Izadi, and in further view of Sekkappan et al. US 2024/0127572 A1, hereinafter referred to as Sekkappan.
Regarding claim 2, Staranowicz and Izadi teach and/or suggest all the limitations of claim 1 and are analyzed as previously discussed with respect to that claim. Given the BRI of the following limitations, Staranowicz further teaches and/or suggests “where masking the synthetic frame [Please see claim 1 above] comprises: calculating differences between pixel values of the synthetic frame and the frame [¶0027 (also note ¶0064-¶0065) of Staranowicz describes using calculated forward and backward optical flows between frames for generating occlusion masks. Although not explicit, it is believed differences between the forward and backward mappings can indicate the presence of occlusions. Please see Sekkappan below for further support regarding a forward-backward consistency check for detecting occlusions]; and generating a mask based on the differences” [Same as above] Although Staranowicz appears to suggest the aforementioned features based on the calculated optical flows (where Izadi does not), the work of Sekkappan from the same or similar field of endeavor is relied on to more clearly address “calculating differences between pixel values of the synthetic frame and the frame [See for e.g. ¶0025 regarding a comparison unit for comparing forward and backward flow vectors between image pixels in a forward-backward consistency check to see whether they disagree by more than a threshold]; and generating the mask based on the differences.” [As a result of the consistency check performed, inaccurate flow vectors may be identified which indicate the presence of occlusions. Generating an occlusion mask/map would be within the level of skill in the art since Sekkappan’s approach is performed for each pixel (e.g. ¶0051)] Recognizing Sekkappan’s teachings for performing real-time occlusion detection between frames for a video streaming system (abstract), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Staranowicz’s frame interpolation techniques based on calculated optical flows (¶0038) along with Izadi’s noise reduction techniques (abstract), to add the teachings of Sekkappan as above for employing a lightweight, fast, real or near-real time method without delay to detect occluded pixels between frames of a video sequence (e.g. ¶0016).
Regarding claim 3, Staranowicz, Izadi, and Sekkappan teach and/or suggest all the limitations of claim 2 and are analyzed as previously discussed with respect to that claim. Since Staranowicz and Izadi do not appear to address the features of claim 3, the work of Sekkappan from the same or similar field of endeavor is brought in to further teach and/or suggest “where generating the mask comprises comparing the differences to a difference threshold.” [See ¶0025 regarding a comparison unit for comparing forward and backward flow vectors between image pixels in a forward-backward consistency check to see whether they disagree by more than a threshold] The motivation for combining Staranowicz, Izadi, and Sekkappan has been discussed in connection with claim 2, above.
Regarding claim 5, Staranowicz and Izadi teach and/or suggest all the limitations of claim 1, and are analyzed as previously discussed with respect to that claim. Since Staranowicz and Izadi do not appear to address the features of claim 5, the work of Sekkappan from the same or similar field of endeavor is brought in to further teach and/or suggest “where masking the synthetic frame comprises: calculating differences between a luminance component of pixels of the synthetic frame and the frame [Noting Sekkappan’s teachings may be applied to frame interpolation (e.g. ¶0018), ¶0033 for example, shows that a difference can be determined in luma values between source and destination pixels by more than a threshold]; and generating the mask based on the differences.” [By evaluating luma differences for each pixel, one can determine which pixels are valid/not valid. Although a mask is not ‘explicitly’ mentioned, generating a mask/map based on the pixel-wise differences at each pixel (x,y) due to the absence/presence of an occlusion, would be considered within the level of skill in the art] Although Sekkappan’s approach checks both luma and chroma values, it shows that by evaluating the differences in these values, the validity of a flow vector can be determined which, in turn, relates to the presence or absence of occlusions. Thus, Sekkappan’s teachings are deemed relevant given the BRI of the aforementioned features. As such, the motivation for combining Staranowicz, Izadi, and Sekkappan has been discussed in connection with claim 2, above.
Regarding claim 7, Staranowicz and Izadi teach and/or suggest all the limitations of claim 1 and are analyzed as previously discussed with respect to that claim. Staranowicz and Izadi, however, do not appear to address the features of claim 7. For this reason, the work of Sekkappan from the same or similar field of endeavor is brought in to further teach and/or suggest “where masking the synthetic frame comprises: estimating areas of occluded motion based on the optical flow [See Sekkappan’s occlusion detection pipeline 104 which includes an optical flow engine for determining optical flow between image pairs (e.g. ¶0021-¶0025 and ¶0064-¶0065)]; and generating a mask based on the areas of occluded motion.” [A comparison unit of the occlusion detection pipeline 104 compares pixel flow vectors between image frames from which valid/not valid flow vectors can be determined, where the latter can correspond to occluded pixels. Although a mask is not ‘explicitly’ mentioned, generating a mask/map based on determined valid/not valid flow vectors at each pixel (x,y) due to the absence/presence of occlusions, would be considered within the level of skill in the art] The motivation for combining Staranowicz, Izadi, and Sekkappan has been discussed in connection with claim 2, above.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Staranowicz, in view of Izadi, and in further view of Jiang et al. US 10,776,688 B2, hereinafter referred to as Jiang.
Regarding claim 8, Staranowicz and Izadi teach and/or suggest all the limitations of claim 1 and are analyzed as previously discussed with respect to that claim. Staranowicz and Izadi, however, do not appear to address the features of claim 8. As such the work of Jiang from the same or similar field of endeavor is relied on to teach and/or suggest “where generating the synthetic frame comprises warping the neighboring frame based on the optical flow.” [See col. 2 lines 9-16 with respect to a flow interpolation neural network, where first and second neighboring frames are warped according to the intermediate forward and backward optical flow data, respectively] Recognizing Jiang’s teachings for performing video interpolation using optical flow (col. 1 lines 14-16), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Staranowicz’s frame interpolation techniques based on calculated optical flows (¶0038) along with Izadi’s noise reduction techniques (abstract), to add Jiang’s frame interpolation techniques as above to help address artifacts around boundaries of moving objects associated with prior works (e.g. col. 1 lines 46-51).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Staranowicz, in view of Izadi, and in further view of Qiu WO 2024/131035 A1, hereinafter referred to as Qiu.
Regarding claim 17, Staranowicz and Izadi teach and/or suggest all the limitations of claim 16 and are analyzed as previously discussed with respect to that claim. However, both Staranowicz and Izadi do not appear to address ‘scaled frames of the video’ as claimed. Qiu on the other hand from the same or similar field of endeavor is brought in to teach and/or suggest “further comprising generating scaled frames of the video, where generating the plurality of optical flow files is based on the scaled frames of the video.” [First and second video frames can be scaled to reduce the size of the frames which improves the analysis speed during subsequent optical flow analysis. See 8th paragraph on pg. 7] Recognizing Qiu’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Staranowicz’s frame interpolation techniques based on calculated optical flows (¶0038) along with Izadi’s noise reduction techniques (abstract), to add the video frame interpolation method of Qiu as above which enables the accuracy of generating a composite frame on the basis of two consecutive video frames to be improved, thereby improving frame interpolation quality and the frame interpolation effect (e.g. abstract).
Allowable Subject Matter
12. Claims 4, 9-10, and 18-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. In light of the specification, the Examiner finds the claimed invention to be patentably distinct from the prior art of records. The prior art of record, taken individually or in combination fail to explicitly teach or render obvious within the context of the respective independent claims the limitations:
4. (Original) The method of claim 3, further comprising selecting the difference threshold based on a temporal distance between a first neighboring frame and the frame.
9. (Currently Amended) The method of claim 1, further comprising: adding the masked synthetic frame to an accumulator; and adding the mask used in masking the synthetic frame to an inclusion counter.
10. (Original) The method of claim 9, where generating the composite frame is based on dividing the accumulator by the inclusion counter.
18. The method of clam 16, where generating the plurality of synthetic frames comprises: warping a first frame of the plurality of frames generating a first synthetic frame based on a first optical flow file of the plurality of optical flow files; and warping the first synthetic frame generating a second synthetic frame based on a second optical flow file of the plurality of optical flow files.
19. The method of clam 18, where generating the plurality of composite frames comprises generating a first composite frame based on the first synthetic frame and a second frame of the plurality of frames, the second frame temporally adjacent to the first frame.
20. (Currently Amended) The method of claim 16, where performing the selective averaging of portions of the plurality of synthetic frames and the plurality of frames comprises: selecting a first synthetic frame of the plurality of synthetic frames that mimic mimics a first frame of the plurality of frames; calculating a difference between a first luminance component of the first synthetic frame and a second luminance component of the first frame; generating a mask by comparing the difference with a threshold; applying the mask to the first synthetic frame generating a masked synthetic frame; adding the first synthetic frame and the first frame to an accumulator; adding the mask to an inclusion counter; and generating a composite frame based on the accumulator and the inclusion counter.
13. Claim 11-15 are allowed.
The following is a statement of reasons for the indication of allowable subject matter: given the broadest reasonable interpretation, the art of record does not appear to reasonably teach and/or suggest, either alone or in combination, the first and second iterations for generating corresponding synthetic frames as claimed, when considering the claim as a whole, i.e. “generating a first-iteration synthetic frame of the synthetic frames corresponding to a first frame of the frames of the video based on warping a previous temporal frame of the first frame using first object motion of the object motion between the previous temporal frame and the first frame; and generating a second-iteration synthetic frame of the synthetic frames corresponding to a next temporal frame following the first frame based on warping the first-iteration synthetic frame using second object motion of the object motion between the first frame and the next temporal frame”. Although Staranowicz’s multi-pass operations can be construed as a plurality of iterations, the disclosed frame interpolation techniques do not appear to generate the first and second iterations corresponding first-iteration and second-iteration synthetic frames as claimed. The remaining art of record, notably Izadi, Sekkappan, and Jiang, along with prior art Chi, are also silent regarding the aforementioned features.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RICHARD A HANSELL JR. whose telephone number is (571)270-0615. The examiner can normally be reached Mon - Fri 10 am- 7 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jamie Atala can be reached at 571-272-7384. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RICHARD A HANSELL JR./Primary Examiner, Art Unit 2486
Prosecution Insights