DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Currently, claims 1, 3-12, 14-20 & 93-94 are pending in the application. Claims 4-6, and 15-17 are amended. Claims 2, 13 21-92 are cancelled.
Information Disclosure Statement
The information disclosure statements (IDS) were submitted on 10/15/2025. The submission are in compliance with the provisions of 37 CFR § 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments / Amendments
Applicant’s arguments have been fully considered, but they are not persuasive, see discussion below.
Rejections under 35 U.S.C. § 103:
The applicant argued that Valli, Chupeau and Bai, fail to teach or render obvious: "transmitting over a network, in parallel, the plurality of encoded CDM data streams for the plurality of depth ranges, from a server to a client device to cause the client device to: reconstruct the depth map based on a plurality of decoded CDMs; and generate for display an image based on the reconstructed depth map," as recited by claim 1 (and similarly claim 12).
As to the above argument, Valli discloses generating a plurality of encoded CDM data streams for the plurality of depth ranges, wherein each respective CDM data stream is based at least in part on a respective CDM by performed in the time-multiplexing of MPFs in shifted positions. In step 502, an image of a scene is captured, and in step 504, a depth map of the scene is generated. In step 506, shifted stacks of focal planes are formed, and in step 508, the shifted stacks of focal planes are rendered for display; [0094]-[0096], the smallest and furthest depths (distances from the eye) are determined) ([0077] FIG. 5).
Furthermore, BAI teaches transmitting the plurality of depth ranges in parallel in transmitting plurality of depth maps and the plurality of 3D images during the 3D video call simultaneously [parallelly] in the uplink channel and the downlink and simultaneously upload and download the depth maps and the 3D images with large data size in 3D video call, and satisfy the efficient transmission of the plurality of 2D images, the plurality of depth maps, and the plurality of 3D images and the plurality of depth maps and the plurality of 3D images during the 3D video call, the uplink data and the downlink data are simultaneously transmitted in the uplink channel ([0080]; [0121])
Accordingly, Examiner maintains the rejection with regards to above arguments.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 7-12, 14, 17-20 & 93-94 are rejected under 35 U.S.C. 103 as being unpatentable over Valli et al. (US 20210133994, hereinafter Valli) in view of Chupeau et al. (US 20230362409, hereinafter Chupeau) and BAI et al ( US 20190342541, hereinafter BAI)
Regarding Claim 1, Valli discloses a method comprising:
accessing image data that comprises a texture data and a depth map (([0126] FIG. 15A, image content is captured with a camera (e.g. a depth camera);[0141], FIG. 17, selects a location (depth) for each of the focal planes based on the content of a depth map for the image to be rendered, and receives the depth map and the image to be rendered &capture texture and depth);
decomposing the depth map into a plurality of component depth maps (CDMs) for a plurality of depth ranges ([0127], FIG. 15B, 3D content is captured (step 1512), reconstructed (step 1514) and the image content is mapped to different stacks of image planes based on the depth map (step 1516); [0195], coordinates and scale of a tracked marker or other set of features are used for positioning and scaling of virtual objects, decomposed into focal planes; [0060] Parameters that characterize multi focal-plane (MFP) displays generally include the number of focal planes and the properties of depth blending function ), wherein each component depth map corresponds to a focal plane of multiple focal plane (MFP) decomposition of the image data ([0141] FIG. 17, forms (step 1712) and renders (step 1714) individual image planes and provides them to an multiple focal plane (MFP) display that loops over the display planes, adjusting the lens (or other adjustable display optics) for each display plane (step 1716) and displaying the image plane at the corresponding respective depth (step 1718));
generating a plurality of encoded CDM data streams for the plurality of depth ranges, wherein each respective CDM data stream is based at least in part on a respective CDM ([0077] FIG. 5, performed in the time-multiplexing of MPFs in shifted positions. In step 502, an image of a scene is captured, and in step 504, a depth map of the scene is generated. In step 506, shifted stacks of focal planes are formed, and in step 508, the shifted stacks of focal planes are rendered for display; [0094]-[0096], the smallest and furthest depths (distances from the eye) are determined); and
transmitting over a network, ([0126] FIG. 15A, image content is mapped to different stacks of image planes based on the depth map and different stacks of image planes are time-multiplexed (step 1508) and rendered (step 1510) for display to a user; [0141], FIG. 17, selects a location (depth) for each of the focal planes (step 1706)based on the content of a depth map for the image to be rendered and number and location of the focal planes are provided to a renderer 1708; [0206], rendering of multiple focal planes and multiple focal plane renderings), from a server ([0141], FIG. 17) to cause the client device to:
generate for display an image based on the reconstructed depth map ([0126] FIG. 15A, image content is mapped to different stacks of image planes are time-multiplexed and rendered (step 1510) for display to a user).
Valli does not explicitly disclose reconstruct the depth map based on a plurality of decoded CDMs.
Chupeau teaches reconstruct the depth map based on a plurality of decoded CDMs ([0048] FIG. 2, encoding, transmission and decoding of data representative of a sequence of 3D scenes are provided to a volumetric video rendering device for a 3DoF, 3Dof+ or 6DoF rendering and displaying; [0049], bit stream representative of a sequence of 3D scenes are read from a memory 22 and/or received from a network 22 by a decoder 23; [0118] FIG. 11, decoded to retrieve at least one atlas image and associated metadata to retrieve a number of depth layers and parameters representative of a depth quantization law at a view level in the metadata).
Therefore, it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of reconstruct the depth map based on a plurality of decoded CDMs as taught by Chupeau ([0048]) into the encoding & decoding system of Valli in order to effectively encode the data representative of the texture and the geometry of the three-dimensional scene for rendering the volumetric content on the end-user devices (Chupeau, [0001]).
Valli & Chupeau do not explicitly disclose transmitting the plurality of depth ranges in parallel.
BAI teaches transmitting the plurality of depth ranges in parallel ([0080] transmit plurality of depth maps and the plurality of 3D images during the 3D video call simultaneously [parallelly] in the uplink channel and the downlink; [0121], simultaneously upload and download the depth maps and the 3D images with large data size in 3D video call, and satisfy the efficient transmission of the plurality of 2D images, the plurality of depth maps, and the plurality of 3D images.)
Therefore, it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of transmitting the plurality of depth ranges in parallel as taught by BAI ([0080]) into the encoding & decoding system of Valli & Chupeau in order to enable increasing stereoscopic three-dimensional video effect and user experience, reducing data processing amount and power consumption of electronic device and shortening time delay of video communication process (BAI, [0043]).
Regarding Claim 3, Valli in view of Chupeau and BAI discloses the method of claim 1, Chupeau discloses wherein: the plurality of encoded CDM data streams are synchronized with the texture data; and wherein the transmitting over the network, ([0048] FIG. 2, encoding, transmission and decoding of data representative of a sequence of 3D scenes are provided to a volumetric video rendering device for a 3DoF, 3Dof+ or 6DoF rendering and displaying; [0049], bit stream representative of a sequence of 3D scenes are read from a memory 22 and/or received from a network 22 by a decoder 23; [0118] FIG. 11, decoded to retrieve at least one atlas image and associated metadata to retrieve a number of depth layers and parameters representative of a depth quantization law at a view level in the metadata).
BAI teaches transmitting the plurality of depth ranges in parallel ([0154] FIG. 16, server 1620 generate & encode immersive video content a for transmission to one or more client 1630 devices inputs from one or more optical cameras 1601 having depth sensors 1602 and parallel compute 1604 resources to decompose the video and depth data into point clouds 1605 and/or texture triangles 1606).
The same reason or rational of obviousness motivation applied as used above in claim 1.
Regarding Claim 7, Valli in view of Chupeau and BAI discloses the method of claim 1, Chupeau discloses further comprising: identifying a key depth range by performing object detection on the image data; selecting a key CDM of the plurality of CDMs that corresponds to the key depth range; wherein the generating the plurality of encoded CDM data streams comprises: encoding the key CDM at higher bit rate than at least one other CDM of the plurality of CDMs ([0074] area of small capacity (some bits) or to very large area (e.g. a whole program or large amount of received or decoded data); [0107], more demanding in terms of metadata bitrate, the number of patches in the atlas being of a larger order of magnitude than the number of layers in a MPI view, and the view parameters being infrequently updated while the patch list is regularly refreshed, typically per intra-period (e.g., every 32 frames)). The same reason or rational of obviousness motivation applied as used above in claim 1.
Regarding Claim 8, Valli in view of Chupeau and BAI discloses the method of claim 7, Valli discloses wherein the selection of the key CDM is performed during live streaming of the image data ([0128] image information may be captured and transmitted as real-time 3D data. This may affect to the formation of MFPs at the receiving site; [0202])
Regarding Claim 9, Valli in view of Chupeau and BAI discloses the method of claim 1, Valli discloses wherein the generating the plurality of encoded CDM data streams comprises: separately pre-encoding each CDM at a plurality of bit rates ([0074] area of small capacity (some bits) or to very large area (e.g. a whole program or large amount of received or decoded data); [0107], more demanding in terms of metadata bitrate, the number of patches in the atlas being of a larger order of magnitude than the number of layers in a MPI view, and the view parameters being infrequently updated while the patch list is regularly refreshed, typically per intra-period (e.g., every 32 frames)). The same reason or rational of obviousness motivation applied as used above in claim 1.
Regarding Claim 10, Valli in view of Chupeau and BAI discloses the method of claim 9, Valli discloses wherein the transmitting the plurality of encoded CDM data streams comprises: selecting a first bit rate of the plurality of bit rates for a first CDM of the plurality of CDMs; and selecting a second bit rate of the plurality of bit rates for a second CDM of the plurality of CDMs; transmitting data pre-encoded at the first bit rate for the first CDM; and transmitting data pre-encoded at the second bit rate for the second CDM ([0074] area of small capacity (some bits) or to very large area (e.g. a whole program or large amount of received or decoded data); [0107], more demanding in terms of metadata bitrate, the number of patches in the atlas being of a larger order of magnitude than the number of layers in a MPI view, and the view parameters being infrequently updated while the patch list is regularly refreshed, typically per intra-period (e.g., every 32 frames). The same reason or rational of obviousness motivation applied as used above in claim 1.
Regarding Claim 11, A method claim 11 of using the corresponding method claimed in claims 1, and the rejections of which are incorporated herein for the same reasons of as used above.
Regarding Claims 12, 14, 17-20, System claims 12-20 of using the corresponding method claimed in claims 1, 3 & 7-10, and the rejections of which are incorporated herein for the same reasons of as used above.
Regarding Claim 93, Valli in view of Chupeau and BAI discloses the method of claim 1, Valli discloses wherein the displayed image based on the plurality of MFPs is adjustable by a spatial light modulator (SLM) ([0166] using a blocker element, such as a spatial light modulator (SLM) to block the real-world view, background occlusion)
Regarding Claim 94, Valli in view of Chupeau and BAI discloses the method of claim 1, Valli discloses wherein each component depth map CDM corresponds to a phase function ([0089] FIG. 6, two sinusoidal functions, in opposite phases to produce blending functions for five focal planes; [0131], continuous sinusoidal functions in opposite phases as basic functions)
Claims 4-6 & 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Valli et al. (US 20210133994, hereinafter Valli) in view of Chupeau et al. (US 20230362409, hereinafter Chupeau) and BAI et al ( US 20190342541, hereinafter BAI) and Valli et al. (US 20210136354, hereinafter Valli_354)
Regarding Claim 4, Valli in view of Chupeau and BAI discloses the method of claim 1, but does not explicitly disclose wherein decomposing the depth map into the plurality of CDMs comprises- applying a reversible depth blending function to decompose the depth map into a plurality of overlapping component planes.
Valli_354 teaches wherein decomposing the depth map into the plurality of CDMs comprises- applying a reversible depth blending function to decompose the depth map into a plurality of overlapping component planes ([0192] FIG. 14 is a schematic diagram illustrating example focal-image planes viewed by an observer. FIG. 14, a version of which is included in Suyama, shows a top view 1400 and front views 1402, 1404 of two image planes 1406, 1408 viewed by an observer 1410 and a side view 1414 of how the planes are perceived as a 3D image by an observer 1412. In Suyama, two focal planes 1406, 1408 are formed, placed at different distances, and viewed from two eye points. This setup creates disparity and 3D perception produced by the sideways displacement of the two focal planes seen from the two eye-points. The displacement produces particular pixel value distributions at object edges, which the brain interprets as disparity and as depth variations, respectively. FIG. 14 shows perception of 3D images by viewing two overlapping focal planes formed from a scene. The two eyes see the focal planes from slightly different angles, which creates a synthetic stereo disparity between focal planes; [0197] FIG. 17 is a schematic plan view illustrating an example viewing of two focal planes with depth-based luminosity weighting functions. FIG. 17 is a plan view schematic 1700 that shows the principle of depth fusing of two focal planes 1704, 1706 seen from one eye-point 1702. Kurt Akeley, et al., A Stereo Display Prototype with Multiple Focal Distances, 23(3) ACM TRANSACTIONS ON GRAPHICS (TOG) 804-813 (2004) (“Akeley”) and X. Hu & H. Hua, Design and Assessment of a Depth-Fused Multi-Focal-Plane Display Prototype, 10(4) IEEE/OSA Journal of Display Technology 308-316 (2014) (“Hu & Hua”) describe how a depth blending effect has been noticed to work with two or more focal planes viewed monocularly by only one eye. FIG. 17)
PNG
media_image1.png
762
526
media_image1.png
Greyscale
Therefore, it would have been obvious to one ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of reversible depth blending function to decompose the depth map as taught by Valli_354 ([0192]) into the encoding & decoding system of Valli & Chupeau in order to achieve desired accuracy by using high number of multi-focal planes (MFPs), and by selecting and optimizing depth blending functions when forming the MFP (Valli_354 , [0158]).
Regarding Claim 5, Analogous rejection as the rejection of Claim 4 applies.
Regarding Claim 6, Valli in view of Chupeau and BAI, Valli_354 discloses the method of claim 4, Valli discloses wherein the reversible depth blending function is a set of sinusoid functions ([0074] , sinusoidal blending functions are employed. Such functions are easy to form and their position is easy to vary by changing their phase by a control variable).
Regarding Claims 15-16, System claims 15-16 of using the corresponding method claimed in claims 4-5, and the rejections of which are incorporated herein for the same reasons of as used above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Samuel D Fereja whose telephone number is (469)295-9243. The examiner can normally be reached 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DAVID CZEKAJ can be reached at (571) 272-7327. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAMUEL D FEREJA/Primary Examiner, Art Unit 2487