CTNF 19/094,504 CTNF 82198 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al (US 2020/0012940) and further in view of Winder (US 2004/025230) . For claim 1, Liu et al teach a head-mounted display device comprising: at least one display (e.g. paragraph 173: display device); instructions (e.g. paragraph 153: processor circuitry); and at least one programmable circuit (e.g. paragraph 153: processor circuitry ) to be programmed based on the instructions to: warp a first frame of a first video (e.g. paragraph 4: Video frame interpolation is a basic video processing technique that is used to generate intermediate frames between any two consecutive original frames. Video frame interpolation algorithms typically estimate optical flow or its variations and use them to warp and blend original frames to produce interpolation results); warp a second frame of the first video (e.g. paragraph 4: Video frame interpolation is a basic video processing technique that is used to generate intermediate frames between any two consecutive original frames. Video frame interpolation algorithms typically estimate optical flow or its variations and use them to warp and blend original frames to produce interpolation results); synthesize, with a neural network, a third frame based on the first warped frame and the second warped frame, the third frame corresponding to an intermediate frame between the first frame and the second frame (e.g. paragraph 93: An appearance flow operation may employ a deep convolutional neural network to estimate flows and then may use them to warp input pixels to create a novel view. This operation can warp individual input frames and blend them together to produce a frame between the input ones ); and output a second video including the first frame, the third frame and the second frame to the at least one display (e.g. paragraph 64: videos that are widely available online may be used to train the neural network. To make it easy to reproduce results). Liu et al do not further disclose: warp a first frame of a first video based on first motion data associated with the first frame to determine a first warped frame; warp a second frame of the first video based on second motion data associated with the second frame to determine a second warped frame. Winder teaches warp a first frame of a first video based on first motion data associated with the first frame to determine a first warped frame; warp a second frame of the first video based on second motion data associated with the second frame to determine a second warped frame (e.g. paragraph 42: At a given pyramid level, warping consists of warping the past reference frame forward in time, and warping the future reference frame backward in time, using the global motion information. Local motion is then estimated at that level as a correction to the global motion information. Also paragraph 95: For example, the local motion estimator (472) creates a vector for each pixel or block of pixels (e.g., a 2.times.2 or 4.times.4 block depending on frame size) within the frames. The global estimation parameters for a source frame interval are the starting point for the local motion estimation. ). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Winder into the teaching of Liu et al for a media playback device to use frame interpolation with motion analysis in real time to increase the frame rate of streamed video for playback (e.g. paragraph 1, Winder) and to improve the quality of frame interpolation (e.g. paragraph 6, Winder). Claim 8 is rejected for the same reasons as discussed in claim 1 above, wherein figure 27 shows interface circuitry 2718. Claim 15 is rejected for the same reasons as discussed in claim 1 above. For claims 2, 9 and 16, Liu et al do not further disclose the first frame is associated with a first time, the second frame is associated with a second time after the first time, the first warped frame is associated with a third time between the first time and the second time, the second warped frame is associated with the third time, and the third frame is associated with the third time. Winder teaches the first frame is associated with a first time, the second frame is associated with a second time after the first time, the first warped frame is associated with a third time between the first time and the second time, the second warped frame is associated with the third time, and the third frame is associated with the third time (e.g. figure 4a: step 410 identify source frames at T1 and T2; identify time T1 +t for new output frame). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Winder into the teaching of Liu et al for a media playback device to use frame interpolation with motion analysis in real time to increase the frame rate of streamed video for playback (e.g. paragraph 1, Winder) and to improve the quality of frame interpolation (e.g. paragraph 6, Winder). For claims 4, 11 and 18, Liu et al do not further disclose the first motion data includes at least one of first optical flow data or a first displacement map, and the second motion data includes at least one of second optical flow data or a second displacement map. Winder teaches the first motion data includes at least one of first optical flow data or a first displacement map, and the second motion data includes at least one of second optical flow data or a second displacement map (e.g. paragraph 38: “ local optical flow based motion estimation”). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Winder into the teaching of Liu et al for a media playback device to use frame interpolation with motion analysis in real time to increase the frame rate of streamed video for playback (e.g. paragraph 1, Winder) and to improve the quality of frame interpolation (e.g. paragraph 6, Winder). For claims 5, 12 and 19, Liu et al teach the neural network is a first neural network, and one or more of the at least one programmable circuit is to determine, with a second neural network (e.g. paragraph 93: An appearance flow operation may employ a deep convolutional neural network to estimate flows and then may use them to warp input pixels to create a novel view. This operation can warp individual input frames and blend them together to produce a frame between the input ones ). Liu et al do not further disclose the first motion data and the second motion data based on the first image and the second image. Winder teaches the first motion data and the second motion data based on the first image and the second image (e.g. paragraph 42: At a given pyramid level, warping consists of warping the past reference frame forward in time, and warping the future reference frame backward in time, using the global motion information. Local motion is then estimated at that level as a correction to the global motion information. Also paragraph 95: For example, the local motion estimator (472) creates a vector for each pixel or block of pixels (e.g., a 2.times.2 or 4.times.4 block depending on frame size) within the frames. The global estimation parameters for a source frame interval are the starting point for the local motion estimation. ). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Winder into the teaching of Liu et al for a media playback device to use frame interpolation with motion analysis in real time to increase the frame rate of streamed video for playback (e.g. paragraph 1, Winder) and to improve the quality of frame interpolation (e.g. paragraph 6, Winder). For claims 6 and 13, Liu et al teach one or more of the at least one programmable circuit is to warp the first frame and the second frame with a third neural network(e.g. paragraph 93: An appearance flow operation may employ a deep convolutional neural network to estimate flows and then may use them to warp input pixels to create a novel view. This operation can warp individual input frames and blend them together to produce a frame between the input ones ). For claims 7, 14 and 20, Liu et al teach the neural network includes: at least a first neural network layer to determine the first motion data and the second motion data based on the first image and the second image; at least a second neural network layer to warp the first frame based on the first motion data and warp the second frame based on the second motion data; and at least a third neural network layer to synthesize the third frame based on the first warped frame and the second warped frame. (e.g. paragraph 93: An appearance flow operation may employ a deep convolutional neural network to estimate flows and then may use them to warp input pixels to create a novel view. This operation can warp individual input frames and blend them together to produce a frame between the input ones ). For claims 3, 10 and 17, Liu et al do not further disclose the first frame is associated with a first perspective, the second frame is associated with a second perspective different from the first perspective, the first warped frame is associated with a third perspective between the first perspective and the second perspective, the second warped frame is associated with and the third perspective, and the third frame is associated with the third perspective. Winder et al teach the first frame is associated with a first perspective, the second frame is associated with a second perspective different from the first perspective, the first warped frame is associated with a third perspective between the first perspective and the second perspective, the second warped frame is associated with and the third perspective, and the third frame is associated with the third perspective (e.g. figure 4a: step 410 identify source frames at T1 and T2; identify time T1 +t for new output frame). It would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Winder into the teaching of Liu et al for a media playback device to use frame interpolation with motion analysis in real time to increase the frame rate of streamed video for playback (e.g. paragraph 1, Winder) and to improve the quality of frame interpolation (e.g. paragraph 6, Winder). Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAQUAN ZHAO whose telephone number is (571)270-1119. The examiner can normally be reached M-Thur: 7:00 am-5:00 pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thai Tran can be reached on 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. Email: daquan.zhao1@uspto.gov. Phone: (571)270-1119 /DAQUAN ZHAO/Primary Examiner, Art Unit 2484 Application/Control Number: 19/094,504 Page 2 Art Unit: 2484 Application/Control Number: 19/094,504 Page 3 Art Unit: 2484 Application/Control Number: 19/094,504 Page 4 Art Unit: 2484 Application/Control Number: 19/094,504 Page 5 Art Unit: 2484 Application/Control Number: 19/094,504 Page 6 Art Unit: 2484 Application/Control Number: 19/094,504 Page 7 Art Unit: 2484 Application/Control Number: 19/094,504 Page 8 Art Unit: 2484 Application/Control Number: 19/094,504 Page 9 Art Unit: 2484
Prosecution Insights