DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 5, 9, 11, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (Pub. No. US 2016/0148416) in view of Maciocci (Pub. No. US 2014/0146148).
Regarding claim 1, Wu teaches a method of displaying a video having corresponding depth information (3D coordinates), the method comprising: obtaining information indicating a time-varying (adjusted for each frame of video) number of focal planes (number M of focal planes) [Para. 23 “This speed is sufficient to render video in real-time.” And “The focal plane configuration may be adjusted for each frame of video or less frequently, for example every certain number of frames or for each scene.”; Para. 30 “These points can be obtained by a 3D camera or generated by a computer graphics engine, for example.”; and Para. 64 “In the original example with six focal planes, the multi-focal display might determine the number M of focal planes where M can be up to six.”].
Wu teaches the number of focal planes (number of focal planes) for images frames (frame of video) in the video [Para. 6 “Therefore, an important consideration for MFDs is the focal plane configuration, including the number of focal planes and the location of the focal planes (that is, distances from the viewer)”; and Para. 23 “The focal plane configuration may be adjusted for each frame of video or less frequently, for example every certain number of frames or for each scene”].
However, Wu doesn’t explicitly teach the number of focal planes being different for at least two different image frames in the video.
Maciocci teaches the number of focal planes (depth planes) being different for at least two different image frames in the video (video stream) [Para. 17 “in some embodiments a stream of individual plenoptic images are combined into a sequence of frames in order to create a 2-D or 3-D movie sequence containing multiple depth plane information”; “Different numbers of depth planes may be used for each scene of the movie.”; Para. 34 “The process 400 then repeats for each plenoptic frame in a sequence of plenoptic frames in a video stream. Therefore, each plenoptic frame in the video sequence re-defines the viewer's 2-D gaze coordinates, depth planes and focus area”].
It would have been obvious to one of ordinary skill in the art before the effective filling date to modify Wu’s focal plane placement module 140 and scene separation module 150 by using Maciocci’s teaching that different numbers of focal planes (depth planes) may be used for different scenes in a video stream, and that each plenoptic frame in a video sequence re-defines focal planes (depth planes), so that Wu’s per-frame focal plane configuration selects a different number M of focal planes or different image frames when video depth complexity changes. This modification improves Wu by using fewer focal planes for simple frames and more focal planes for complex frames, thereby reducing unnecessary rendering and power consumption while preserving depth quality when additional focal planes are needed.
Wu teaches for each image frame in the video (frame of video), mapping (separates) the image frame to a set (series) of image planes (2D images) having different associated depths (different focal planes) [Para. 23 “The focal plane configuration may be adjusted for each frame of video or less frequently, for example every certain number of frames or for each scene.”; Para. 22 “Scene separation module 150 separates the 3D scene into the constituent 2D images to be rendered.” Para 25 “MFD technology can represent a 3D scene by a series of 2D images at different focal planes due to a concept known as depth blending”].
Wu teaches wherein the number of image planes (2D images) in the set (series) is based on the number of focal planes (number M of focal planes) [Para. 21 “In this way, a 3D scene can be approximated by a series of 2D images rendered at the different focal planes.”; Para. 64 “In the original example with six focal planes, the multi-focal display might determine the number M of focal planes where M can be up to six. Less than the maximum number may be selected for various reasons, for example to reduce power consumption.”].
However, Wu doesn’t explicitly teach wherein the number of image planes in the set is based on the time-varying number of focal planes for the respective frame.
Maciocci teaches wherein the number of image planes in the set is based on the time-varying number (number of depth planes) of focal planes for the respective frame (currently displayed plenoptic frame) [Para. 38 “Therefore, each plenoptic frame in the video sequence re-defines the viewer's 2-D gaze coordinates, depth planes and focus area”; Para. 31 “For example, in a simple plenoptic frame with a person in the foreground and a tree in the background, the process block 420 may define only two depth planes. Conversely, in a more complex plenoptic frame comprising multiple depth areas, the process 400 at block 420 may define many more depth planes in an image, wherein each depth plane represents a different area of depth within the plenoptic image”; and Para. 34 “Therefore, each plenoptic frame in the video sequence re-defines the viewer's 2-D gaze coordinates, depth planes and focus area”. It clear that Maciocci teaches time-varying number of focal planes because Maciocci determines the number of depth planes for the currently displayed frame, allows simple frames to define fewer depth planes and complex frames to define many more, and repeats/re-defines those depth planes for each frame in the video sequence, so the focal-plane count changes from frame to frames over time. para. 31, 34 and 38.].
It would have been obvious to one of ordinary skill in the art before the effective filling date to modify Wu’s focal plane placement module 140 and scene separation module 150 by using Maciocci’s teaching that different numbers of focal planes (depth planes) may be used for different scenes in a video stream, and that each plenoptic frame in a video sequence re-defines focal planes (depth planes), so that Wu’s per-frame focal plane configuration selects a different number M of focal planes or different image frames when video depth complexity changes. This modification improves Wu by using fewer focal planes for simple frames and more focal planes for complex frames, thereby reducing unnecessary rendering and power consumption while preserving depth quality when additional focal planes are needed. and
Wu teaches rendering (renders) each of the image planes (2D images) in the set (series)at the corresponding focal plane (different focal planes) in a multi-focal plane display device (multi-focal display) [Para. 21 “FIG. 1 is a diagram of a multi-focal display 100 according to the present invention.”; Para. 25 “MFD technology can represent a 3D scene by a series of 2D images at different focal planes due to a concept known as depth blending.” And Para. 22 “Rendering engine 160 then renders the 2D images onto the display, in coordination with adjustment of the optical element 120 to effect the different focal planes.”].
Regarding claims 3 and 11, Wu teaches number of focal planes (number M of focal planes) [Para. 64].
However, Wu doesn’t explicitly teach wherein the information indicating the time-varying number of focal planes is determined based on content of the video.
Maciocci teaches wherein the information indicating the time-varying number of focal planes (depth planes) is determined based on content of the video [Para. 17 and 31].
It would have been obvious to one of ordinary skill in the art before the effective filling date to modify Wu’s focal plane placement module 140 and scene separation module 150 by using Maciocci’s teaching that different numbers of focal planes (depth planes) may be used for different scenes in a video stream, and that each plenoptic frame in a video sequence re-defines focal planes (depth planes), so that Wu’s per-frame focal plane configuration selects a different number M of focal planes or different image frames when video depth complexity changes. This modification improves Wu by using fewer focal planes for simple frames and more focal planes for complex frames, thereby reducing unnecessary rendering and power consumption while preserving depth quality when additional focal planes are needed.
Regarding claims 5 and 13, Wu teaches determining depths of the focal planes by selecting locations of focal planes [Para. 6].
However, Wu doesn’t explicitly teach the rest of claim limitations.
Maciocci teaches determining the depths of the focal planes (depth planes) based on content of the video [Para. 17, 31, and 34].
It would have been obvious to one of ordinary skill in the art before the effective filling date to modify Wu’s focal plane placement module 140 and scene separation module 150 by using Maciocci’s teaching that different numbers of focal planes (depth planes) may be used for different scenes in a video stream, and that each plenoptic frame in a video sequence re-defines focal planes (depth planes), so that Wu’s per-frame focal plane configuration selects a different number M of focal planes or different image frames when video depth complexity changes. This modification improves Wu by using fewer focal planes for simple frames and more focal planes for complex frames, thereby reducing unnecessary rendering and power consumption while preserving depth quality when additional focal planes are needed.
Claim 9 is rejected for the same reasons as claim 1 above. Furthermore, Wu teaches a display device configured to display a video having depth information [fig. 1 and corresponding description].
Claims 2 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (Pub. No. US 2016/0148416) in view of Maciocci (Pub. No. US 2014/0146148) further in view of Rodriguez et al. (Pub. No. US 2018/0053284).
Regarding claims 2 and 10, Wu teaches number of focal planes (number M of focal planes) [Para. 64] while Maciocci teaches determining the frame dependent number of focal planes (number of depth planes) [para. 38].
However, Wu in view of Maciocci doesn’t explicitly teach the wherein the information indicating the time-varying number of focal planes is obtained from a server.
Rodriguez teaches wherein the information (depth plane indicator data) indicating the time-varying number of focal planes (depth planes) is obtained from a server (server or base station) [Para. 194, 195, 257-260].
It would have been obvious to one of ordinary skill in the art before the effective filling date to modify Wu in view of Maciocci’s per frame depth plane count determination by using Rodriguez’s server transmitted light field video stream carrying information so that the local display controller obtains, from the server frame level data indicating which focal planes are active/inactive and thus the time varying number of focal planes of focal planes for the frame. This medication improves Wu by allowing focal plane count metadata to be generated and transmitted with the video stream.
Claims 4, 8, 12 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (Pub. No. US 2016/0148416) in view of Maciocci (Pub. No. US 2014/0146148) further in view of Bradski et al. (Pub. No. US 2015/0178939).
Regarding claims 4 and 12, Wu in view of Maciocci doesn’t explicitly teach the claim limitations.
However, Bradski teaches obtaining from a server (remote processing module) information indicating the depths of the focal planes [Para. 46, fig. 2, 3 and related description].
It would have been obvious to one of ordinary skill in the art before the effective filling date to modify Wu in view of Maciocci’s focal plane placement module, by obtaining the focal plane depth information from Bradski’s server after remote image/depth processing. This medication improves Wu by allowing focal plane depth metadata to be computed or retrieved remotely, thereby reducing local compute/storage requirements and supporting updated streamed depth plane configurations.
Regarding claims 8 and 16, Wu teaches rendering image planes (2D images) at corresponding focal planes (different focal planes) in multi-focal display [Para. 25].
However, Wu in view of Maciocci doesn’t explicitly teach the rest of claim limitations.
Bradski teaches wherein the image planes are rendered at the corresponding focal planes in an interleaved order (interleaved) such that, after at least two image planes are rendered at a corresponding two focal planes, at least a third image plane is rendered at a depth between the two focal planes [Para. 79, and 81].
It would have been obvious to one of ordinary skill in the art before the effective filling date to modify Wu in view of Maciocci’s focal plane placement module, by obtaining the focal plane depth information from Bradski’s server after remote image/depth processing. This medication improves Wu by allowing focal plane depth metadata to be computed or retrieved remotely, thereby reducing local compute/storage requirements and supporting updated streamed depth plane configurations.
Claims 6, 7, 14, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (Pub. No. US 2016/0148416) in view of Maciocci (Pub. No. US 2014/0146148) further in view of Jiang et al. (Pub. No. US 2002/0027610).
Regarding claims 6 and 14, Wu in view of Maciocci doesn’t explicitly teach the claim limitations.
However, Jiang teaches wherein mapping the image frame to the image planes is performed using a set of blending functions [Para. 7, 40 and 41].
It would have been obvious to one of ordinary skill in the art before the effective filling date to modify Wu in view of Maciocci’s scene separation module and depth bending process by using Jiang’s bending functions as plane weighting functions to assign pixel among focal plane image planes generated for a frame. This modification improves Wu producing smoother transitions between adjacent focal plane image planes.
Regarding claims 7 and 15, Wu in view of Maciocci doesn’t explicitly teach the claim limitations.
However, Jiang teaches wherein at least one of the blending functions (blending factors) is a sinusoidal blending function (stretched sinusoidal curve) [Para. 40 and 41].
It would have been obvious to one of ordinary skill in the art before the effective filling date to modify Wu in view of Maciocci’s scene separation module and depth bending process by using Jiang’s bending functions as plane weighting functions to assign pixel among focal plane image planes generated for a frame. This modification improves Wu producing smoother transitions between adjacent focal plane image planes.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-7 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-0101 (IN USA OR CANADA) or 571-272-1000.
/SOLOMON G BEZUAYEHU/ Primary Examiner, Art Unit 2666
Prosecution Insights