DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1, 3, 8 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Mendez U.S. Patent Application 20160253819 in view of Varekamp U.S. Patent Application 20240406363.
Regarding claim 1, Mendez discloses a method comprising:
reprojecting a first frame comprising a first blurred texture (paragraph [0030]: FIG. 4, a flowchart illustrating an exemplary first motion blur estimation method 400 for estimating a motion blur of an image target is shown… At block 410, a first projected pose may be determined by measuring a pose of the image target frame in question and projecting the pose of the image target frame in question to screen space);
reprojecting a second frame comprising a second blurred texture (paragraph [0030]: At block 420, a second projected pose may be determined by measuring a pose of the image target frame directly before the image target frame in question and projecting the pose of the image target frame directly before the image target frame in question to screen space);
determining a motion approximation between the first reprojected frame and the second reprojected frame (paragraph [0030]: at block 430, a motion blur vector of the image target frame in question may be determined based at least in part on the difference between the second projected pose and the first projected pose… the result of the motion blur estimation for the current image target frame may be based on a plurality of previous pairs of successive image target frames); and
rendering an intermediate frame based at least in part on the motion approximation, the intermediate frame being rendered between the first reprojected frame and the second reprojected frame (paragraph [0034]: at block 730, one or more intermediate poses between the pose of the image target frame in question and the pose of the image target frame directly before it may be interpolated... At block 740, a plurality of unblurred hypothetical image target frames may be constructed, wherein one hypothetical image target frame may be constructed for each intermediate pose as well as the pose of the image target frame in question and the pose of the image target frame directly before it; paragraph [0023]: The image targets, on which the one or more virtual objects are superimposed, are then rendered frame-by-frame on the display device 121).
Mendez discloses all the features with respect to claim 1 as outlined above. However, Mendez fails to disclose reprojecting a first frame comprising a first blurred texture into a first reprojected frame using a first transformation associated with a viewing perspective of a user, reprojecting a second frame comprising a second blurred texture into a second reprojected frame using a second transformation associated with a viewing perspective of a user.
Varekamp discloses reprojecting a first frame comprising a first blurred texture into a first reprojected frame using a first transformation associated with a viewing perspective of a user, reprojecting a second frame comprising a second blurred texture into a second reprojected frame using a second transformation associated with a viewing perspective of a user (paragraph [0070]: When an image is synthesized at the target viewpoint of the virtual camera 108 by using images and depth maps corresponding to the projection planes 106d and 106c (first and second reprojected frame) (i.e. the projection planes with the most motion blur), the new virtual image may show a blurry texture of the object 102 which appears to be in the background).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Mendez’s to transform images associated with user viewpoint as taught by Varekamp, to handle motion blur when processing multi-view data.
Regarding claim 3, Mendez as modified by Varekamp discloses the method of claim 1, wherein the determining the motion approximation comprises applying linear interpolation between the first reprojected frame and the second reprojected frame (Mendez’s paragraph [0034]: at block 730, one or more intermediate poses between the pose of the image target frame in question and the pose of the image target frame directly before it may be interpolated).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Mendez’s to transform images associated with user viewpoint as taught by Varekamp, to handle motion blur when processing multi-view data.
Regarding claim 8, Mendez as modified by Varekamp discloses the method of claim 1, wherein the determining the motion approximation comprises applying extrapolation between the first reprojected frame and the second reprojected frame (Mendez’s paragraph [0030]: differences between previous pairs of successive image target frames (as projected to screen space) may be used to determine the motion blur vector of the current image target frame on an extrapolated curve, so that the result of the motion blur estimation for the current image target frame may be based on a plurality of previous pairs of successive image target frames).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Mendez’s to transform images associated with user viewpoint as taught by Varekamp, to handle motion blur when processing multi-view data.
Claim 20 recites the functions of the apparatus recited in claim 1 as medium steps. Accordingly, the mapping of the prior art to the corresponding functions of the apparatus in claim 1 applies to the medium steps of claim 20.
Claim 2, 4 and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Mendez U.S. Patent Application 20160253819 in view of Varekamp U.S. Patent Application 20240406363, and further in view of Sloan U.S. Patent Application 20070014470.
Regarding claim 2, Mendez as modified by Varekamp discloses generating the first and second blurred texture (Mendez's paragraph [0030]: FIG. 4, a flowchart illustrating an exemplary first motion blur estimation method 400 for estimating a motion blur of an image target is shown… At block 410, a first projected pose may be determined by measuring a pose of the image target frame in question and projecting the pose of the image target frame in question to screen space. At block 420, a second projected pose may be determined by measuring a pose of the image target frame directly before the image target frame in question and projecting the pose of the image target frame directly before the image target frame in question to screen space). However, Mendez as modified by Varekamp fails to disclose applying a convolution filter to the frame based on a kernel.
Sloan discloses applying a convolution filter to the frame based on a kernel (paragraph [0045]: use of a convolution filter kernel (e.g. Gaussian Blur). The larger the kernel, the more coverage of the filter, which makes it more blurred).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Mendez and Varekamp’s to apply a convolution filter as taught by Sloan, to allows blur process to be performed quickly and efficiently.
Regarding claim 4, Mendez as modified by Varekamp and Sloan discloses the method of claim 3, wherein the applying the linear interpolation comprises interpolating between a first Gaussian distribution corresponding to the first reprojected frame and a second Gaussian distribution corresponding to the second reprojected frame (Sloan’s paragraph [0045]: use of a convolution filter kernel (e.g. Gaussian Blur). The larger the kernel, the more coverage of the filter, which makes it more blurred; Mendez’s paragraph [0034]: at block 730, one or more intermediate poses between the pose of the image target frame in question and the pose of the image target frame directly before it may be interpolated).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Mendez and Varekamp’s to apply a convolution filter as taught by Sloan, to allows blur process to be performed quickly and efficiently.
Regarding claim 12, Mendez discloses a device comprising:
a memory (memory 135); and
one or more processors (processors 110) configured to:
reproject a plurality of previous frames having blurred textures (paragraph [0030]: FIG. 4, a flowchart illustrating an exemplary first motion blur estimation method 400 for estimating a motion blur of an image target is shown… At block 410, a first projected pose may be determined by measuring a pose of the image target frame in question and projecting the pose of the image target frame in question to screen space. At block 420, a second projected pose may be determined by measuring a pose of the image target frame directly before the image target frame in question and projecting the pose of the image target frame directly before the image target frame in question to screen space);
determine a motion approximation between the plurality of reprojected blurred frames by applying interpolation between a plurality of reprojected blurred frames (paragraph [0030]: at block 430, a motion blur vector of the image target frame in question may be determined based at least in part on the difference between the second projected pose and the first projected pose… the result of the motion blur estimation for the current image target frame may be based on a plurality of previous pairs of successive image target frames); and
render an intermediate frame based at least in part on the motion approximation, the intermediate frame being rendered between two reprojected blurred frames of the plurality of reprojected blurred frames (paragraph [0034]: at block 730, one or more intermediate poses between the pose of the image target frame in question and the pose of the image target frame directly before it may be interpolated... At block 740, a plurality of unblurred hypothetical image target frames may be constructed, wherein one hypothetical image target frame may be constructed for each intermediate pose as well as the pose of the image target frame in question and the pose of the image target frame directly before it; paragraph [0023]: The image targets, on which the one or more virtual objects are superimposed, are then rendered frame-by-frame on the display device 121).
Mendez discloses all the features with respect to claim 12 as outlined above. However, Mendez fails to disclose reprojecting a plurality of previous frames having blurred textures into respective ones of a plurality of reprojected blurred frames using one or more transformations associated with a viewing perspective of a user, and kernel distributions associated with respective ones of the blurred frames.
Varekamp discloses reprojecting a plurality of previous frames having blurred textures into respective ones of a plurality of reprojected blurred frames using one or more transformations associated with a viewing perspective of a user (paragraph [0070]: When an image is synthesized at the target viewpoint of the virtual camera 108 by using images and depth maps corresponding to the projection planes 106d and 106c (first and second reprojected frame) (i.e. the projection planes with the most motion blur), the new virtual image may show a blurry texture of the object 102 which appears to be in the background).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Mendez’s to transform images associated with user viewpoint as taught by Varekamp, to handle motion blur when processing multi-view data.
Mendez as modified by Varekamp discloses all the features with respect to claim 12 as outlined above. However, Mendez as modified by Varekamp fails to disclose kernel distributions associated with respective ones of the blurred frames.
Sloan discloses kernel distributions associated with respective ones of the blurred frames (paragraph [0045]: use of a convolution filter kernel (e.g. Gaussian Blur). The larger the kernel, the more coverage of the filter, which makes it more blurred).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Mendez and Varekamp’s to apply a convolution filter as taught by Sloan, to allows blur process to be performed quickly and efficiently.
Regarding claim 13, Mendez as modified by Varekamp and Sloan discloses the device of claim 12, further comprising:
generating a first blurred texture in a first frame of the plurality of previous frames by applying a first convolution filter to the first frame based on a first kernel; and generating a second blurred texture in a second frame of the plurality of previous frames by applying a second convolution filter to the second frame based on a second kernel (Sloan’s paragraph [0045]: use of a convolution filter kernel (e.g. Gaussian Blur). The larger the kernel, the more coverage of the filter, which makes it more blurred; Mendez’s paragraph [0034]: at block 730, one or more intermediate poses between the pose of the image target frame in question and the pose of the image target frame directly before it may be interpolated).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Mendez and Varekamp’s to apply a convolution filter as taught by Sloan, to allows blur process to be performed quickly and efficiently.
Regarding claim 14, Mendez as modified by Varekamp and Sloan discloses the device of claim 12, wherein the determining the motion approximation comprises applying linear interpolation between a first reprojected blurred frame of the plurality of reprojected blurred frames and a second reprojected blurred frame of the plurality of reprojected blurred frames, wherein the applying the linear interpolation comprises interpolating between a first Gaussian distribution corresponding to the first reprojected blurred frame and a second Gaussian distribution corresponding to the second reprojected blurred frame (Sloan’s paragraph [0045]: use of a convolution filter kernel (e.g. Gaussian Blur). The larger the kernel, the more coverage of the filter, which makes it more blurred; Mendez’s paragraph [0034]: at block 730, one or more intermediate poses between the pose of the image target frame in question and the pose of the image target frame directly before it may be interpolated).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Mendez and Varekamp’s to apply a convolution filter as taught by Sloan, to allows blur process to be performed quickly and efficiently.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Mendez U.S. Patent Application 20160253819 in view of Varekamp U.S. Patent Application 20240406363, and further in view of Lobete U.S. Patent Application 20210400170.
Regarding claim 11, Mendez as modified by Varekamp discloses all the features with respect to claim 1 as outlined above. However, Mendez as modified by Varekamp fails to disclose one or more of the first frame or the second frame is rendered at a first frame rate and the intermediate frame is rendered at a second frame rate corresponding to the first frame rate.
Lobete discloses one or more of the first frame or the second frame is rendered at a first frame rate and the intermediate frame is rendered at a second frame rate corresponding to the first frame rate (paragraph [0062]: if the display refresh rate is set to 60 FPS, and the frame rendering rate is reduced to 30 FPS, the interpolation operation may generate interpolated rendered frames (intermediate frame) for display at the display refresh rate of 60 FPS).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Mendez and Varekamp’s to render intermediate frame as taught by Lobete, to process rendered frames such that they may be displayed at a display refresh rate.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Mendez U.S. Patent Application 20160253819 in view of Varekamp U.S. Patent Application 20240406363, in view of Sloan U.S. Patent Application 20070014470, and further in view of Liu U.S. Patent Application 20230094297.
Regarding claim 19, Mendez as modified by Varekamp and Sloan discloses the motion approximation is determined based on a difference between at least two of the plurality of kernel distributions (Mendez’s paragraph [0030]: at block 430, a motion blur vector of the image target frame in question may be determined based at least in part on the difference between the second projected pose and the first projected pose… the result of the motion blur estimation for the current image target frame may be based on a plurality of previous pairs of successive image target frames; Sloan’s paragraph [0045]: use of a convolution filter kernel (e.g. Gaussian Blur). The larger the kernel, the more coverage of the filter, which makes it more blurred). However, Mendez as modified by Varekamp and Sloan fails to disclose kernel distributions being within two standard deviations.
Liu discloses kernel distributions being within two standard deviations (paragraph [0033]: where r is a radius of a blur kernel and σ is standard deviation of a normal distribution. Through the adjustment of these two parameters, different blur kernels are generated, which are subject to operation with clear image to obtain blurred images of different blurring levels. As shown in FIG. 4, the larger r and σ, the blurrier the generated image is).
Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Mendez, Varekamp and Sloan’s to apply kernel distributions as taught by Liu, to improve the detection of the focus performance of the image acquisition device.
Allowable Subject Matter
Claim 5-7, 9-10 and 15 -18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Claim 5 and 15 are about the first Gaussian distribution and the second Gaussian distribution are each located at a mean that corresponds to an offset value. Claim 9 and 17 are similar to claim 5, about the applying the extrapolation comprises extrapolating between a first Gaussian distribution corresponding to the first reprojected frame and a second Gaussian distribution corresponding to the second reprojected frame, wherein the first Gaussian distribution is offset by a first offset value and the second Gaussian distribution is offset by a second offset value different from the first offset value.
Mendez 20160253819, Varekamp 20240406363, Sloan 20070014470 and Kalkbrenner 20200300765 combined cannot teach these features perfectly. These limitations when read in light of the rest of the limitations in the claim and the claims to which it depends make the claim allowable subject matter.
Claim 6 depends on claim 5, are allowed base on same reason as claim 5.
Claim 10 depends on claim 9, are allowed base on same reason as claim 9.
Claim 18 depends on claim 17, are allowed base on same reason as claim 17.
Claim 7 and 16 are about the determining the motion approximation comprises determining a translated Gaussian distribution representing a result of the applied linear interpolation between the first Gaussian distribution and the second Gaussian distribution, the translated Gaussian distribution indicating the motion approximation.
Mendez 20160253819, Varekamp 20240406363, Sloan 20070014470 and Roding 20130116935 combined cannot teach these features perfectly. These limitations when read in light of the rest of the limitations in the claim and the claims to which it depends make the claim allowable subject matter.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Yi Yang whose telephone number is (571)272-9589. The examiner can normally be reached on Monday-Friday 9:00 AM-6:00 PM EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached on 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/YI YANG/
Primary Examiner, Art Unit 2616
Prosecution Insights