DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Preliminary Amendment
The Preliminary Amendment submitted on 01/23/2025 has been entered and made of record.
Status of Claims
This communication is in response to the Application Filed on 01/23/2025.
Claims 1–11 are pending in this application.
Drawings
The drawing(s) filed on 01/23/2025 are accepted by the Examiner.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/23/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claims 1, 2, 8 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Gausebeck et al. (US 2019/0026958 A1, hereafter, "Gausebeck") in view of Asija (See NPL attached, "3D PANO INPAINTING: SCENE CONSTRUCTION USING A SINGLE INPUT", hereafter, "Asija"), and further in view of Sharma et al. (See NPL attached, "Unsupervised Learning of Depth and Ego-Motion", hereafter, "Sharma").
Regarding claim 1, Gausebeck discloses a stereoscopic image generation method for panorama image (See Gausebeck, ¶ [0176], In some embodiments, these two or more panoramic images can be combined (e.g., by the stitching component 508 or at the capture device) to generate a single panoramic image with a wider vertical field-of-view than either image alone), comprising:
cropping a partial view frame from a panorama image (See Gausebeck, ¶ [0114], For example, the cropping component 510 can select a portion of the panoramic image corresponding to a narrower field-of-view. Note: Examiner is interpreting the cropped portion as the partial view frame);
[establishing an initial three-dimensional mesh in a spherical coordinate system for the partial view frame;
performing depth estimation on the partial view frame to obtain a target depth map;
updating the initial three-dimensional mesh of the partial view frame according to the target depth map to obtain a three-dimensional scene mesh; and
generating a side-by-side image comprising a left eye image and a right eye image by performing camera projection processing according to the three-dimensional scene mesh].
However, Gausebeck fail to teach establishing an initial three-dimensional mesh in a spherical coordinate system for the partial view frame; performing depth estimation on the partial view frame to obtain a target depth map; updating the initial three-dimensional mesh of the partial view frame according to the target depth map to obtain a three-dimensional scene mesh; and generating a side-by-side image comprising a left eye image and a right eye image by performing camera projection processing according to the three-dimensional scene mesh.
Asija, working in the same field of endeavor, teaches: establishing an initial three-dimensional mesh in a spherical coordinate system for the partial view frame (See Asija, [Pg. 28, ln. 1–5, 4.4 Coordinate Projection, To create a fully immersive virtual reality environment, it is important to ensure that the mesh conforms to the perspective of the camera, wrapping around its view. Our initial step involves scaling the width and height coordinates of the LDI to fit within the ranges of-π to π and
-
π
2
to
π
2
, respectively. These scaled values correspond to our horizontal angle (θ) and vertical angle (ϕ). [Pg. 25, Figure 4.1], our system uses a combination of inpainting and spherical projection to generate (1) a 3D mesh);
performing depth estimation on the partial view frame to obtain a target depth map (See Asija, [Pg. 26, ln. 17–20, 4.2.2 360 Monodepth], We adopt 360MonoDepth in our pipeline to address the challenge of accurately predicting spherical depth maps with aligned left and right edges. 360MonoDepth is specifically designed to estimate high-resolution depth maps from a single 360° monocular equirectangular input image).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s reference to establishing an initial three-dimensional mesh in a spherical coordinate system for the partial view frame; performing depth estimation on the partial view frame to obtain a target depth map based on the method of Asija’s reference. The suggestion/motivation would have been to improve the perception quality of the scene reconstruction (See Asija, [Pg. 37, 6.2 Qualitative Evaluation]).
However, Gausebeck and Asija fail to teach updating the initial three-dimensional mesh of the partial view frame according to the target depth map to obtain a three-dimensional scene mesh; and generating a side-by-side image comprising a left eye image and a right eye image by performing camera projection processing according to the three-dimensional scene mesh.
Sharma, working in the same field of endeavor, teaches: updating the initial three-dimensional mesh of the partial view frame according to the target depth map to obtain a three-dimensional scene mesh (See Sharma, [Pg. 12, ln. 17 and 18, A. Mesh Creation], The depth of each vertex is scaled by the corresponding depth in the predicted depth map. Note. That the vertex is scaled by the depth map which the examiner is interpreting at updating the mesh); and
generating a side-by-side image comprising a left eye image and a right eye image by performing camera projection processing according to the three-dimensional scene mesh (See Sharma, [Pg. 12, ln. 14 and 15, A. Mesh Creation], To create the stereo panorama from a single input panorama, we use the predicted depth map output by our trained network to create a dense mesh and then synthetically render the left- and right-eye panoramas. See also Fig. 13, Textured Mesh. See also Fig. 14, Generated Stereo panoramas).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s and Asija’s reference updating the initial three-dimensional mesh of the partial view frame according to the target depth map to obtain a three-dimensional scene mesh; and generating a side-by-side image comprising a left eye image and a right eye image by performing camera projection processing according to the three-dimensional scene mesh based on the method of Sharma’s reference. The suggestion/motivation would have been to accurately reconstruct stereo panorama images from single panorama image (See Sharma, [Pg. 12, V. APPLICATIONS: NEW VIEW SYNTHESIS AND STEREO PANORAMA CONVERSION]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Sharma with Gausebeck and Asija to obtain the invention as specified in claim 1.
Regarding claim 2, Gausebeck in view of Asija further in view of Sharma teaches the stereoscopic image generation method for panorama image as claimed in claim 1, wherein the step of cropping the partial view frame from the panorama image (See Gausebeck, ¶ [0114], For example, the cropping component 510 can select a portion of the panoramic image corresponding to a narrower field-of-view) comprises:
determining a field of view (FOV) (See Gausebeck, ¶ [0114], For example, the cropping component 510 can select a portion of the panoramic image corresponding to a narrower field-of-view); and
cropping the partial view frame from the panorama image according to the field of view (See Gausebeck, ¶ [0114], For example, the cropping component 510 can select a portion of the panoramic image corresponding to a narrower field-of-view).
Regarding claim 8, Gausebeck in view of Asija further in view of Sharma teaches the stereoscopic image generation method for panorama image as claimed in claim 1, [wherein the step of generating the side-by-side image comprising the left eye image and the right eye image by performing the camera projection processing on the three-dimensional scene mesh comprises:
projecting the spherical coordinate of each of the plurality of mesh vertices of the three- dimensional scene mesh onto a left eye pixel plane and a right eye pixel plane based on camera projection parameters to generate the left eye image and the right eye image; and
combining the left eye image and the right eye image in a side-by-side format to obtain the side-by-side image].
However, Gausebeck and Asija fail to teach wherein the step of generating the side-by-side image comprising the left eye image and the right eye image by performing the camera projection processing on the three-dimensional scene mesh comprises: projecting the spherical coordinate of each of the plurality of mesh vertices of the three- dimensional scene mesh onto a left eye pixel plane and a right eye pixel plane based on camera projection parameters to generate the left eye image and the right eye image; and combining the left eye image and the right eye image in a side-by-side format to obtain the side-by-side image.
Sharma, working in the same field of endeavor, teaches: wherein the step of generating the side-by-side image comprising the left eye image and the right eye image by performing the camera projection processing on the three-dimensional scene mesh (See Sharma, [Pg. 12, ln. 14 and 15, A. Mesh Creation], To create the stereo panorama from a single input panorama, we use the predicted depth map output by our trained network to create a dense mesh and then synthetically render the left- and right-eye panoramas. [Pg. 6, ln. 24], The inverse projection from the unit cylinder to a 3D point in the sensor coordinate system) comprises:
projecting the spherical coordinate of each of the plurality of mesh vertices of the three- dimensional scene mesh onto a left eye pixel plane and a right eye pixel plane based on camera projection parameters to generate the left eye image and the right eye image (See Sharma, [Pg. 12, ln. 14 and 15, A. Mesh Creation], To create the stereo panorama from a single input panorama, we use the predicted depth map output by our trained network to create a dense mesh and then synthetically render the left- and right-eye panoramas. [Pg. 6, ln. 24], The inverse projection from the unit cylinder to a 3D point in the sensor coordinate system); and
combining the left eye image and the right eye image in a side-by-side format to obtain the side-by-side image (See Sharma, [Pg. 12, ln. 14 and 15, A. Mesh Creation], To create the stereo panorama from a single input panorama, we use the predicted depth map output by our trained network to create a dense mesh and then synthetically render the left- and right-eye panoramas. [Pg. 6, ln. 24], The inverse projection from the unit cylinder to a 3D point in the sensor coordinate system).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s and Asija’s reference wherein the step of generating the side-by-side image comprising the left eye image and the right eye image by performing the camera projection processing on the three-dimensional scene mesh comprises: projecting the spherical coordinate of each of the plurality of mesh vertices of the three- dimensional scene mesh onto a left eye pixel plane and a right eye pixel plane based on camera projection parameters to generate the left eye image and the right eye image; and combining the left eye image and the right eye image in a side-by-side format to obtain the side-by-side image based on the method of Sharma’s reference. The suggestion/motivation would have been to accurately reconstruct stereo panorama images from single panorama image (See Sharma, [Pg. 12, V. APPLICATIONS: NEW VIEW SYNTHESIS AND STEREO PANORAMA CONVERSION]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Sharma with Gausebeck and Asija to obtain the invention as specified in claim 8.
Regarding claim 11, claim 11 is rejected the same as claim 1 and the arguments similar to that presented above for claim 1 are equally applicable to claim 11, and all of the other limitations similar to claim 1 are not repeated herein, but incorporated by reference. Furthermore, Gausebeck teaches a stereoscopic image display system, comprising: a stereoscopic display device; and at least one processor, coupled to the stereoscopic display device, and configured to (See Gausebeck, ¶ [0062], For example, the user device 130 can include but is not limited to: a desktop computer, a laptop computer, a mobile phone, a smartphone, a tablet personal computer (PC), a personal digital assistant (PDA), a heads-up display (HUD), a virtual reality (VR) headset, augmented reality (AR) headset or device, a standalone digital camera, or another type of wearable computing device. ¶ [0063], The computing device 104 can include or be operatively coupled to at least one memory 104 and at least one processor 124).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Gausebeck et al. (US 2019/0026958 A1, hereafter, "Gausebeck") in view of Asija (See NPL attached, "3D PANO INPAINTING: SCENE CONSTRUCTION USING A SINGLE INPUT", hereafter, "Asija") and Sharma et al. (See NPL attached, "Unsupervised Learning of Depth and Ego-Motion", hereafter, "Sharma"), and further in view of Wong et al. (US 11,830,163 B2, hereafter, "Wong").
Regarding claim 3, Gausebeck in view of Asija further in view of Sharma teaches the stereoscopic image generation method for panorama image as claimed in claim 2, [wherein the field of view is determined based on user input, an application setting, or metadata of the panorama image].
However, Gausebeck, Asija and Sharma fail to teach wherein the field of view is determined based on user input, an application setting, or metadata of the panorama image.
Wong, working in the same field of endeavor, teaches: wherein the field of view is determined based on user input, an application setting, or metadata of the panorama image (See Wong, [Col. 3, ln. 34–36], cropping the set of images into an appropriate horizontal and vertical field of view (e.g., a predetermined FOY, a calculated FOY, etc.)).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s, Asija’s and Sharma’s reference to wherein the field of view is determined based on user input, an application setting, or metadata of the panorama image based on the method of Wong’s reference. The suggestion/motivation would have been to enable everyday consumers to easily generate parallax-tolerant indoor panoramas (See Wong, [Col. 1, ln. 48–52]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Wong with Gausebeck, Asija and Sharma to obtain the invention as specified in claim 3.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Gausebeck et al. (US 20190026958 A1, hereafter, "Gausebeck") in view of Asija (See NPL attached, "3D PANO INPAINTING: SCENE CONSTRUCTION USING A SINGLE INPUT", hereafter, "Asija") and Sharma et al. (See NPL attached, "Unsupervised Learning of Depth and Ego-Motion", hereafter, "Sharma"), and further in view of Zhuang et al. (See NPL attached, "SPDET: Edge-Aware Self-Supervised", hereafter, "Zhuang").
Regarding claim 4, Gausebeck in view of Asija further in view of Sharma teaches the stereoscopic image generation method for panorama image as claimed in claim 1, [wherein the step of establishing the initial three-dimensional mesh in the spherical coordinate system for the partial view frame comprises:
mapping a plurality of pixel coordinates of the partial view frame to a plurality of cartesian coordinates in a cartesian coordinate system;
converting the plurality of cartesian coordinates of the partial view frame to a plurality of spherical coordinates in the spherical coordinate system; and
generating the initial three-dimensional mesh comprising a plurality of mesh vertices based on the plurality of spherical coordinates corresponding to the plurality of pixels of the partial view frame].
However, Gausebeck fail to teach wherein the step of establishing the initial three-dimensional mesh in the spherical coordinate system for the partial view frame comprises: mapping a plurality of pixel coordinates of the partial view frame to a plurality of cartesian coordinates in a cartesian coordinate system; converting the plurality of cartesian coordinates of the partial view frame to a plurality of spherical coordinates in the spherical coordinate system; and generating the initial three-dimensional mesh comprising a plurality of mesh vertices based on the plurality of spherical coordinates corresponding to the plurality of pixels of the partial view frame.
Asija, working in the same field of endeavor, teaches: wherein the step of establishing the initial three-dimensional mesh in the spherical coordinate system for the partial view frame (See Asija, [Pg. 28, ln. 1–5, 4.4 Coordinate Projection], To create a fully immersive virtual reality environment, it is important to ensure that the mesh conforms to the perspective of the camera, wrapping around its view. Our initial step involves scaling the width and height coordinates of the LDI to fit within the ranges of-π to π and
-
π
2
to
π
2
, respectively. These scaled values correspond to our horizontal angle (θ) and vertical angle (ϕ). [Pg. 25, Figure 4.1], our system uses a combination of inpainting and spherical projection to generate (1) a 3D mesh) comprises:
generating the initial three-dimensional mesh comprising a plurality of mesh vertices based on the plurality of spherical coordinates corresponding to the plurality of pixels of the partial view frame (See Asija, [Pg. 28, ln. 1–5, 4.4 Coordinate Projection], To create a fully immersive virtual reality environment, it is important to ensure that the mesh conforms to the perspective of the camera, wrapping around its view. Our initial step involves scaling the width and height coordinates of the LDI to fit within the ranges of-π to π and
-
π
2
to
π
2
, respectively. These scaled values correspond to our horizontal angle (θ) and vertical angle (ϕ). [Pg. 25, Figure 4.1], our system uses a combination of inpainting and spherical projection to generate (1) a 3D mesh).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s reference wherein the step of establishing the initial three-dimensional mesh in the spherical coordinate system for the partial view frame comprises: generating the initial three-dimensional mesh comprising a plurality of mesh vertices based on the plurality of spherical coordinates corresponding to the plurality of pixels of the partial view frame based on the method of Asija’s reference. The suggestion/motivation would have been to improve the perception quality of the scene reconstruction (See Asija, [Pg. 37, 6.2 Qualitative Evaluation]).
However, Gausebeck, Asija and Sharma fail to teach mapping a plurality of pixel coordinates of the partial view frame to a plurality of cartesian coordinates in a cartesian coordinate system; converting the plurality of cartesian coordinates of the partial view frame to a plurality of spherical coordinates in the spherical coordinate system.
Zhuang, working in the same field of endeavor, teaches: mapping a plurality of pixel coordinates of the partial view frame to a plurality of cartesian coordinates in a cartesian coordinate system (See Zhuang, [Pg. 12477, Col. 1, A. Panoramic Imaging Model], In the panoramic imaging model, objects in 3D space are projected on a unit sphere located at the camera center. As illustrated in Fig. 2(a), let the Cartesian coordinates of a point P be P =(x,y,z) ∈ R3, then its azimuth angle u ∈ [−π,π] and polar angle v ∈ [
-
π
2
to
π
2
] projected on the spherical surface);
converting the plurality of cartesian coordinates of the partial view frame to a plurality of spherical coordinates in the spherical coordinate system (See Zhuang, [Pg. 12477, Col. 1, A. Panoramic Imaging Model], In the panoramic imaging model, objects in 3D space are projected on a unit sphere located at the camera center. As illustrated in Fig. 2(a), let the Cartesian coordinates of a point P be P =(x,y,z) ∈ R3, then its azimuth angle u ∈ [−π,π] and polar angle v ∈ [
-
π
2
to
π
2
] projected on the spherical surface).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s, Asija’s and Sharma’s reference to mapping a plurality of pixel coordinates of the partial view frame to a plurality of cartesian coordinates in a cartesian coordinate system; converting the plurality of cartesian coordinates of the partial view frame to a plurality of spherical coordinates in the spherical coordinate system based on the method of Zhuang’s reference. The suggestion/motivation would have been to accurately reconstruct the scene and determine the depth (See Zhuang, [Table 1]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Zhuang with Gausebeck, Asija and Sharma to obtain the invention as specified in claim 4.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Gausebeck et al. (US 20190026958 A1, hereafter, "Gausebeck") in view of Asija (See NPL attached, "3D PANO INPAINTING: SCENE CONSTRUCTION USING A SINGLE INPUT", hereafter, "Asija"), Sharma et al. (See NPL attached, "Unsupervised Learning of Depth and Ego-Motion", hereafter, "Sharma") and Zhuang et al. (See NPL attached, "SPDET: Edge-Aware Self-Supervised", hereafter, "Zhuang"), and further in view of Marrinan et al. (See NPL attached, "Image Synthesis from a Collection of Depth Enhanced Panoramas", hereafter, "Marrinan").
Regarding claim 5, Gausebeck in view of Asija, Sharma further in view of Zhuang teaches the stereoscopic image generation method for panorama image as claimed in claim 4, [wherein the plurality of pixel coordinates of the partial view frame are mapped to the plurality of cartesian coordinates based on a preset reference depth].
However, Gausebeck, Asija, Sharma and Zhuang fail to teach wherein the plurality of pixel coordinates of the partial view frame are mapped to the plurality of cartesian coordinates based on a preset reference depth.
Marrinan, working in the same field of endeavor, teaches: wherein the plurality of pixel coordinates of the partial view frame are mapped to the plurality of cartesian coordinates based on a preset reference depth (See Marrinan, [Pg. 67, 4.1 2D to 3D Transformation], Each pixel’s azimuth, inclination, and depth defne its relative position to the capturing camera in spherical coordinates. Therefore, the 3D position of each point relative to its corresponding pixel’s capturing camera can be calculated by converting from spherical to Cartesian coordinates. Note: the system is based on reference depth as the depth is calculated later);
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s, Asija’s, Sharma’s and Zhuang’s reference to wherein the plurality of pixel coordinates of the partial view frame are mapped to the plurality of cartesian coordinates based on a preset reference depth based on the method of Marrinan’s reference. The suggestion/motivation would have been to accurately reconstruct the left and right image (See Marrinan, [Table 1]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Marrinan with Gausebeck, Asija, Sharma and Zhuang to obtain the invention as specified in claim 5.
Claims 6 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Gausebeck et al. (US 2019/0026958 A1, hereafter, "Gausebeck") in view of Asija (See NPL attached, "3D PANO INPAINTING: SCENE CONSTRUCTION USING A SINGLE INPUT", hereafter, "Asija") and Sharma et al. (See NPL attached, "Unsupervised Learning of Depth and Ego-Motion", hereafter, "Sharma"), and further in view of Du et al. (US 2024/0212184 A1, hereafter, "Du).
Regarding claim 6, Gausebeck in view of Asija further in view of Sharma teaches the stereoscopic image generation method for panorama image as claimed in claim 1, [wherein the step of updating the initial three-dimensional mesh of the partial view frame according to the target depth map to obtain the three-dimensional scene mesh comprises:
adjusting radial component of a first spherical coordinate of each of the mesh vertices in the initial three-dimensional mesh using the target depth map to obtain a second spherical coordinate of each of the plurality of mesh vertices in the three-dimensional scene mesh].
However, Gausebeck, Asija and Sharma fail to teach wherein the step of updating the initial three-dimensional mesh of the partial view frame according to the target depth map to obtain the three-dimensional scene mesh comprises: adjusting radial component of a first spherical coordinate of each of the mesh vertices in the initial three-dimensional mesh using the target depth map to obtain a second spherical coordinate of each of the plurality of mesh vertices in the three-dimensional scene mesh.
Du, working in the same field of endeavor, teaches: wherein the step of updating the initial three-dimensional mesh of the partial view frame according to the target depth map to obtain the three-dimensional scene mesh (See Sharma, [Pg. 12, ln. 17 and 18, A. Mesh Creation], The depth of each vertex is scaled by the corresponding depth in the predicted depth map. Note. That the vertex is scaled by the depth map which the examiner is interpreting at updating the mesh) comprises:
adjusting radial component of a first spherical coordinate of each of the mesh vertices in the initial three-dimensional mesh using the target depth map to obtain a second spherical coordinate of each of the plurality of mesh vertices in the three-dimensional scene mesh (See Du, ¶ [0037], For a W×H resolution output image, the differential mesh renderer 235, 240 can be configured to generate a spherical mesh following a UV pattern with 2H height segments and 2 W width segments. Next, vertices can be offset to the correct radius based on a Euclidean depth d from the depth prediction 225, 230).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s, Asija’s and Sharma’s reference to wherein the step of updating the initial three-dimensional mesh of the partial view frame according to the target depth map to obtain the three-dimensional scene mesh comprises: adjusting radial component of a first spherical coordinate of each of the mesh vertices in the initial three-dimensional mesh using the target depth map to obtain a second spherical coordinate of each of the plurality of mesh vertices in the three-dimensional scene mesh based on the method of Du’s reference. The suggestion/motivation would have been to accurately reconstruct 360 images (See Du, ¶ [0003–0005]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Du with Gausebeck, Asija and Sharma to obtain the invention as specified in claim 6.
Regarding claim 7, Gausebeck in view of Asija further in view of Sharma teaches the stereoscopic image generation method for panorama image as claimed in claim 6, [wherein the radial component of the second spherical coordinate of the first mesh vertex in the three-dimensional scene mesh is determined by adding the radial component of the first spherical coordinate of the first mesh vertex in the initial 3D mesh to a corresponding depth value from the target depth map].
However, Gausebeck, Asija and Sharma fail to teach wherein the radial component of the second spherical coordinate of the first mesh vertex in the three-dimensional scene mesh is determined by adding the radial component of the first spherical coordinate of the first mesh vertex in the initial 3D mesh to a corresponding depth value from the target depth map.
Du, working in the same field of endeavor, teaches: wherein the radial component of the second spherical coordinate of the first mesh vertex in the three-dimensional scene mesh is determined by adding the radial component of the first spherical coordinate of the first mesh vertex in the initial 3D mesh to a corresponding depth value from the target depth map (See Du, ¶ [0037], For a W×H resolution output image, the differential mesh renderer 235, 240 can be configured to generate a spherical mesh following a UV pattern with 2H height segments and 2 W width segments. Next, vertices can be offset to the correct radius based on a Euclidean depth d from the depth prediction 225, 230).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s, Asija’s and Sharma’s reference to wherein the radial component of the second spherical coordinate of the first mesh vertex in the three-dimensional scene mesh is determined by adding the radial component of the first spherical coordinate of the first mesh vertex in the initial 3D mesh to a corresponding depth value from the target depth map based on the method of Du’s reference. The suggestion/motivation would have been to accurately reconstruct 360 image (See Du, ¶ [0003–0005]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Du with Gausebeck, Asija and Sharma to obtain the invention as specified in claim 7.
Claims 9 is rejected under 35 U.S.C. 103 as being unpatentable over Gausebeck et al. (US 2019/0026958 A1, hereafter, "Gausebeck") in view of Asija (See NPL attached, "3D PANO INPAINTING: SCENE CONSTRUCTION USING A SINGLE INPUT", hereafter, "Asija") and Sharma et al. (See NPL attached, "Unsupervised Learning of Depth and Ego-Motion", hereafter, "Sharma"), and further in view of Yerushalmy et al. (US 2020/0221064 A1, hereafter, "Yeru").
Regarding claim 9, Gausebeck in view of Asija further in view of Sharma teaches the stereoscopic image generation method for panorama image as claimed in claim 1, [wherein the step of performing the depth estimation on the partial view frame to obtain the target depth map comprises:
performing the depth estimation on the partial view frame to obtain an initial depth map; and
generating the target depth map according to a previous depth map of a previous partial view frame and the initial depth map of the partial view frame when the field of view of the partial view frame overlaps with the field of view of the previous partial view frame of the panorama image].
However, Gausebeck, Asija and Sharma fail to teach wherein the step of performing the depth estimation on the partial view frame to obtain the target depth map comprises: performing the depth estimation on the partial view frame to obtain an initial depth map; and generating the target depth map according to a previous depth map of a previous partial view frame and the initial depth map of the partial view frame when the field of view of the partial view frame overlaps with the field of view of the previous partial view frame of the panorama image.
Asija, working in the same field of endeavor, teaches: wherein the step of performing the depth estimation on the partial view frame to obtain the target depth map comprises (See Asija, [Pg. 26, ln. 17–20, 4.2.2 360 Monodepth], We adopt 360MonoDepth in our pipeline to address the challenge of accurately predicting spherical depth maps with aligned left and right edges. 360MonoDepth is specifically designed to estimate high-resolution depth maps from a single 360° monocular equirectangular input image):
performing the depth estimation on the partial view frame to obtain an initial depth map (See Asija, [Pg. 26, ln. 17–20, 4.2.2 360 Monodepth], We adopt 360MonoDepth in our pipeline to address the challenge of accurately predicting spherical depth maps with aligned left and right edges. 360MonoDepth is specifically designed to estimate high-resolution depth maps from a single 360° monocular equirectangular input image).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s reference to wherein the step of performing the depth estimation on the partial view frame to obtain the target depth map comprises: performing the depth estimation on the partial view frame to obtain an initial depth map based on the method of Asija’s reference. The suggestion/motivation would have been to improve the perception quality of the scene reconstruction (See Asija, [Pg. 37, 6.2 Qualitative Evaluation]).
However, Gausebeck, Asija and Sharma fail to teach generating the target depth map according to a previous depth map of a previous partial view frame and the initial depth map of the partial view frame when the field of view of the partial view frame overlaps with the field of view of the previous partial view frame of the panorama image.
Yeru, working in the same field of endeavor, teaches: generating the target depth map according to a previous depth map of a previous partial view frame and the initial depth map of the partial view frame when the field of view of the partial view frame overlaps with the field of view of the previous partial view frame of the panorama image (See Yeru, ¶ [0010], wherein the first FOV and the second FOV overlap over an overlap region, calculating a stereoscopic depth map in the overlap region using respective image information provided by the first and second cameras, obtaining a first camera 2 sub-pixel phase detection (2PD) disparity map in the entire first FOV, and improving the stereoscopic depth map or the 2PD depth map in at least the overlap region using the stereoscopic depth map in the overlap region and/or the first camera 2PD disparity map in the entire first FOV).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s, Asija’s and Sharma’s reference to generating the target depth map according to a previous depth map of a previous partial view frame and the initial depth map of the partial view frame when the field of view of the partial view frame overlaps with the field of view of the previous partial view frame of the panorama image based on the method of Yeru’s reference. The suggestion/motivation would have been to accurately generate a depth map (See Yeru, ¶ [0009]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Yeru with Gausebeck, Asija and Sharma to obtain the invention as specified in claim 9.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Gausebeck et al. (US 20190026958 A1, hereafter, "Gausebeck") in view of Asija (See NPL attached, "3D PANO INPAINTING: SCENE CONSTRUCTION USING A SINGLE INPUT", hereafter, "Asija") and Sharma et al. (See NPL attached, "Unsupervised Learning of Depth and Ego-Motion", hereafter, "Sharma"), and further in view of Aflaki (US 20180176545 A1, hereafter, "Aflaki").
Regarding claim 10, Gausebeck in view of Asija further in view of Sharma teaches the stereoscopic image generation method for panorama image (See Gausebeck, ¶ [0176], In some embodiments, these two or more panoramic images can be combined (e.g., by the stitching component 508 or at the capture device) to generate a single panoramic image with a wider vertical field-of-view than either image alone) as claimed in claim 1, further comprising:
[performing a stereoscopic display operation using a stereoscopic display device according to the side-by-side image].
However, Gausebeck, Asija and Sharma fail to teach performing a stereoscopic display operation using a stereoscopic display device according to the side-by-side image.
Aflaki, working in the same field of endeavor, teaches: performing a stereoscopic display operation using a stereoscopic display device according to the side-by-side image (See Aflaki, ¶ [0070], The HMD may comprise two digital screens for displaying stereoscopic video images of the virtual world in front of respective eyes of the user).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Gausebeck’s, Asija’s and Sharma’s reference to performing a stereoscopic display operation using a stereoscopic display device according to the side-by-side image based on the method of Aflaki’s reference. The suggestion/motivation would have been to accurately display the panorama images (See Aflaki, ¶ [0002 and 0003]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Aflaki with Gausebeck, Asija and Sharma to obtain the invention as specified in claim 10.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ravasz et al. (US 20240320930 A1) teaches the present disclosure generally relates to methods and interfaces for capturing media with a camera application. In some examples, a computer system captures media based on gaze, modifies video playback to improve viewing comfort, and/or surfaces a view setting for media playback based on media stability characteristics. In some examples, a computer system displays one or more of: a camera preview with a level indicator, a camera preview for spatial media capture with prompts to improve capture quality, a camera preview for spatial media capture with a camera movement indicator, and/or a camera preview for media capture with viewpoint stability guidance.
Tan et al. (US 20190238825 A1) teaches in the case of virtual reality content processing, the captured images and video frames are analyzed and stitched together to form a sphere vision. The resulted sphere image can be stored in a format called equirectangular panorama image. The format is widely used for most virtual reality data processing. If the virtual reality content is in stereoscopic 3D pairs, the two equirectangular panorama images are generated. For transmission, the two equirectangular images can be transmitted separately in two channels or arranged in side-by-side fashion or over-and-under fashion and sent in one channel.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DION J SATCHER whose telephone number is (703)756-5849. The examiner can normally be reached Monday - Thursday 5:30 am - 2:30 pm, Friday 5:30 am - 9:30 am PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DION J SATCHER/ Patent Examiner, Art Unit 2676
/SHEFALI D GORADIA/ Primary Patent Examiner, Art Unit 2676
Prosecution Insights