Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 31, 33-38, 40-45 and 48-50 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (Chinese machine translated Patent publication: CN 109118431. “Wang”) as modified by Wang et al. US Patent Publication: 2020211206, “Wang2”)
Regarding claim 31, Wang teaches, a method to cause one or more neural networks to generate one or more images based, at least in part, on:
one or more optical flow terms generated at least partially dependently on one or more image reconstruction terms and at least one of: one or more optical flow terms or one or more image reconstruction terms. ((Abstract The invention claims a multi-memory-based mixing loss of video super-resolution reconstruction method, comprising optical network and image reconstructing network two parts. in the optical network, the multi-frame input, calculating the optical flow between the present frame and the reference frame, and uses the light stream as movement compensation possible compensation to the current frame and the reference frame are similar. In the image reconstruction network orderly inputting the compensated multiple frame network, network by multi-memory residual extracting image characteristic, which makes the following input frame can receive the characteristic pattern information of the preceding frame. Finally, the output of the low resolution character image after carrying out sub-pixel amplification, and enlarged with bicubic interpolation of image to obtain the final high resolution video frame. The training process uses a mixing loss function, simultaneously training the optical flow network and image reconstructing network.” Wang inputs multiple images through optical flow network and reconstruction network to generate final high resolution image using shared loss function or mixed loss function. The training is at least partially dependently because both network are training using shared loss function. )
But Wang is silent that the method is performed by a circuit or processor and , Wang doesn’t expressly teach, at least one of: one or more independently generated optical flow terms or one or more independently generated image reconstruction terms.
However Wang2 teaches, similar methods of optical flow network and construction of image is performed one or more circuits controlled by a processor ( 16. A system for image processing, the system comprising: one or more processors; and a non-transitory computer-readable medium or media, communicatively coupled to at least one of the one or more process, comprising: an optical flow network to predict a forward optical flow from a first image to a second image of a scene; “)
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wang to have included a processor comprising circuitry to cause one or more network to generate images as taught by Wang2 for purpose of using Wang’s invention to perform Wang in alternative environment.
Wang2 additionally teaches, image generation based on at least one of: one or more independently generated optical flow terms or one or more independently generated image reconstruction terms. (optical flow term is independently generated. . [0026] In one or more embodiments, the motion network 120, the optical flow network 130, the depth network 140, and the HMP 150 may be trained individually or jointly using one or more losses associated with at least one of motion consistency, synthesis, and smoothness.) and
it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wang as modified by Wang2 to additionally include image generation based on at least one of: one or more independently generated optical flow terms or one or more independently generated image reconstruction terms as additionally taught by Wang2.
The motivation to include the independent training to have next training (which is based shared loss) in training sequence faster and accurate.
Regarding claim 38, Wang teaches, a method to cause one or more neural networks to generate one or more images based, at least in part, on:
one or more image reconstruction terms generated at least partially dependently on one or more one or more optical flow terms and at least one of: one or more optical flow terms or one or more image reconstruction terms. ((Abstract The invention claims a multi-memory-based mixing loss of video super-resolution reconstruction method, comprising optical network and image reconstructing network two parts. in the optical network, the multi-frame input, calculating the optical flow between the present frame and the reference frame, and uses the light stream as movement compensation possible compensation to the current frame and the reference frame are similar. In the image reconstruction network orderly inputting the compensated multiple frame network, network by multi-memory residual extracting image characteristic, which makes the following input frame can receive the characteristic pattern information of the preceding frame. Finally, the output of the low resolution character image after carrying out sub-pixel amplification, and enlarged with bicubic interpolation of image to obtain the final high resolution video frame. The training process uses a mixing loss function, simultaneously training the optical flow network and image reconstructing network.” Wang inputs multiple images through optical flow network and reconstruction network to generate final high resolution image using shared loss function or mixed loss function. The training is at least partially dependently because both network are training using shared loss function. )
But Wang is silent that the method is performed by a circuit or processor and , Wang doesn’t expressly teach, at least one of: one or more independently generated optical flow terms or one or more independently generated image reconstruction terms.
However Wang2 teaches, similar methods of optical flow network and construction of image is performed one or more circuits controlled by a processor ( 16. A system for image processing, the system comprising: one or more processors; and a non-transitory computer-readable medium or media, communicatively coupled to at least one of the one or more process, comprising: an optical flow network to predict a forward optical flow from a first image to a second image of a scene; “)
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wang to have included a processor comprising circuitry to cause one or more network to generate images as taught by Wang2 for purpose of using Wang’s invention to perform Wang in alternative environment.
Wang2 additionally teaches, image generation based on at least one of: one or more independently generated optical flow terms or one or more independently generated image reconstruction terms. (optical flow term is independently generated. . [0026] In one or more embodiments, the motion network 120, the optical flow network 130, the depth network 140, and the HMP 150 may be trained individually or jointly using one or more losses associated with at least one of motion consistency, synthesis, and smoothness.) and
it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wang as modified by Wang2 to additionally include image generation based on at least one of: one or more independently generated optical flow terms or one or more independently generated image reconstruction terms as additionally taught by Wang2.
The motivation to include the independent training to have next training (which is based shared loss) in training sequence faster and accurate.
Wang as modified by Wang2 teaches, one or more memory devices to store the one or more generated images.( See Wang claim 3, “ The method according to claim 1 the multi-memory loss based video super-resolution reconstruction method, wherein step 3, using multi-memory block, storing the characteristic information of the current frame to the next frame to the feature information fusion;”)
Regarding claim 45, Wang teaches, a method comprising using one or more neural networks to generate one or more images based, at least in part, on:
at least one of:
using the one or more neural networks to generate one or more optical flow terms at least partially dependently on one or more image reconstruction terms; or
using the one or more neural networks to generate the one or more image reconstruction terms at least partially dependently on the one or more optical flow terms..((Abstract The invention claims a multi-memory-based mixing loss of video super-resolution reconstruction method, comprising optical network and image reconstructing network two parts. in the optical network, the multi-frame input, calculating the optical flow between the present frame and the reference frame, and uses the light stream as movement compensation possible compensation to the current frame and the reference frame are similar. In the image reconstruction network orderly inputting the compensated multiple frame network, network by multi-memory residual extracting image characteristic, which makes the following input frame can receive the characteristic pattern information of the preceding frame. Finally, the output of the low resolution character image after carrying out sub-pixel amplification, and enlarged with bicubic interpolation of image to obtain the final high resolution video frame. The training process uses a mixing loss function, simultaneously training the optical flow network and image reconstructing network.” Wang inputs multiple images through optical flow network and reconstruction network to generate final high resolution image using shared loss function or mixed loss function. The training is at least partially dependently because both network are training using shared loss function. )
But Wang is silent that generating image based , at least in part, on generating at least one of: one or more independently generated image reconstruction terms or one or more independently generated optical flow terms;.
Wang2 additionally teaches, generating image based , at least in part, on generating at least one of: one or more independently generated image reconstruction terms or one or more independently generated optical flow terms;.. . [0026] In one or more embodiments, the motion network 120, the optical flow network 130, the depth network 140, and the HMP 150 may be trained individually or jointly using one or more losses associated with at least one of motion consistency, synthesis, and smoothness.) and
it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Wang to include generating image based , at least in part, on generating at least one of: one or more independently generated image reconstruction terms or one or more independently generated optical flow terms as taught by Wang2.
The motivation to include the independent training to have next training (which is based shared loss) in training sequence faster and accurate.
Regarding claim 33 and 40, Wang as modified by Wang2 teaches, wherein the one or more neural networks comprise one or more of an optical flow network or a reconstruction network. (Wang, Abstract The invention claims a multi-memory-based mixing loss of video super-resolution reconstruction method, comprising optical network and image reconstructing network two parts. in the optical network, the multi-frame input, calculating the optical flow between the present frame and the reference frame, and uses the light stream as movement compensation possible compensation to the current frame and the reference frame are similar. In the image reconstruction network orderly inputting the compensated multiple frame network, network by multi-memory residual extracting image characteristic, which makes the following input frame can receive the characteristic pattern information of the preceding frame. Finally, the output of the low resolution character image after carrying out sub-pixel amplification, and enlarged with bicubic interpolation of image to obtain the final high resolution video frame. The training process uses a mixing loss function, simultaneously training the optical flow network and image reconstructing network.” Wang inputs multiple images through optical flow network and reconstruction network to generate final high resolution image using shared loss function or mixed loss function.)
Regarding claims 34, 41 and 48 Wang as modified by Wang2 teaches, wherein the one or more neural networks comprise a fused network comprising both optical flow and reconstruction portions. ( Abstract The invention claims a multi-memory-based mixing loss of video super-resolution reconstruction method, comprising optical network and image reconstructing network two parts. in the optical network, the multi-frame input, calculating the optical flow between the present frame and the reference frame, and uses the light stream as movement compensation possible compensation to the current frame and the reference frame are similar. In the image reconstruction network orderly inputting the compensated multiple frame network, network by multi-memory residual extracting image characteristic, which makes the following input frame can receive the characteristic pattern information of the preceding frame. Finally, the output of the low resolution character image after carrying out sub-pixel amplification, and enlarged with bicubic interpolation of image to obtain the final high resolution video frame. The training process uses a mixing loss function, simultaneously training the optical flow network and image reconstructing network.” Wang inputs multiple images through optical flow network and reconstruction network to generate final high resolution image using shared loss function or mixed loss function.)
Regarding claims 35 and 42, Wang as modified by Wang2 teaches, wherein the one or more images comprise an upscaled image generated based, at least in part, on an input low resolution image. (Wang , “Abstract …..in the optical network, the multi-frame input, calculating the optical flow between the present frame and the reference frame, and uses the light stream as movement compensation possible compensation to the current frame and the reference frame are similar. In the image reconstruction network orderly inputting the compensated multiple frame network, network by multi-memory residual extracting image characteristic, which makes the following input frame can receive the characteristic pattern information of the preceding frame. Finally, the output of the low resolution character image after carrying out sub-pixel amplification, and enlarged with bicubic interpolation of image to obtain the final high resolution video frame.:)
Regarding claims 36, 43, 49, Wang as modified by Wang2 teaches, wherein the one or more images comprise one or more frames of a video (Wang Abstract ) but doesn’t expressly teach . one or more frames of a video game.
As frames of a video and frames of video game are analogous, an ordinary skilled person in the art would have been motivated to use Wang’s method to generate frames of a video game.
The motivation for the above is to enhance the applicability Wang’s image generation.
Regarding claims 37, 44 and 50, Wang as modified by Wang2 teaches, wherein the one or more neural networks are further to generate the one or more images based, at least in part, on one or more previously generated images. (See Wang claim 3, “ The method according to claim 1 the multi-memory loss based video super-resolution reconstruction method, wherein step 3, using multi-memory block, storing the characteristic information of the current frame to the next frame to the feature information fusion;”)
Allowable Subject Matter
Claim 32, 39, 46-47 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 32 and 47 are objected because the combination of prior art fails to expressly teach, wherein generating the one or more optical flow terms at least partially dependently comprises using an initial phase where reconstruction terms are ignored, and a subsequent phase where a contribution of the reconstruction terms is gradually increased.
Claim 39 and 46 are objected because the combination of prior art fails to expressly teach,, wherein generating the one or more image reconstruction terms at least partially dependently comprises using an initial phase where the optical flow terms are ignored, and a subsequent phase where a contribution of the optical flow terms is gradually increased.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Tapas Mazumder whose telephone number is (571)270-7466. The examiner can normally be reached M-F 8:00 AM-5:00 PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TAPAS MAZUMDER/Primary Examiner, Art Unit 2615
Prosecution Insights