DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
2. Acknowledgement is made of amendment filed on March 17, 2026, in which claims 1, 4, 16 and 19 are amended, claims 3 and 18 are canceled, and claims 1, 2, 4-17, 19 and 20 are still pending.
Response to Arguments
3. Applicant's arguments, filed on March 17, 2026, with respect to Claims 1, 2, 4-17, 19 and 20 have been fully considered but they are not persuasive.
4. With regards to arguments for independent claims 1 and 16, applicants argue that Ziegler et al. (US 2021/0082185 A1) and Brown et al. (US 2009/0033653 A1) fail to disclose the rendering each scene element in the first scene element set comprises: determining, according to the index of each scene element in the first scene element set, a color value corresponding to each scene element, the scene elements with different indexes corresponding to different color values; determining, for each scene element, a color value of the pixel in the scene element according to the color value corresponding to the scene element. The examiner respectfully agrees and moots in view of the new grounds of rejections regarding claims 1 and 16, since in Koylazov (US 2017/0109898 A1) teaches (“FIG. 3 is a flow diagram of an example technique 300 for determining whether or not to sample another contribution value for a given render element for a given pixel of an image. … The system determines a neighborhood maximum color contribution value for the render element from the current color data maintained for the render element (step 302). That is, in this example, the current color data that is maintained for a given render element for a given pixel includes the maximum of the color contribution values that have been sampled for the render element for the pixel. The system uses the maintained maximum color contribution values to determine the neighborhood maximum color contribution value for the render element. … The system determines a neighborhood minimum color contribution value for the render element from the current color data maintained for the render element (step 304). That is, in this example, the current color data that is maintained for a given render element for a given pixel includes the minimum of the color contribution values that have been sampled for the render element for the pixel. The system uses the maintained minimum color contribution values to determine the neighborhood maximum color contribution value for the render element. … The system determines a difference between the neighborhood maximum color contribution value and the neighborhood minimum color contribution value (step 306) and determines whether or not to sample another contribution value for the render element from the difference (step 308). … the system computes a ratio between the difference and a value derived from the total number of color contribution values that have been sampled for the render element for the pixel. … when the final color has been determined for each pixel in the image, the system applies a color mapping function to the final color as part of the rendering process.” [0034-0039]) Koylazov teaches the render element for a pixel of an image and determine color contribution values for the render elements, Koylazov also teaches the final color for each pixel in the image from the render element color value. Therefore, Koylazov teaches the arguments of the limitations for claims 1 and 16 as it is recited.
Claim Rejections - 35 USC § 103
5. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
6. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
8. Claim(s) 1, 2, 10-13, 16, 17 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ziegler et al. (US 2021/0082185 A1) in view of Brown et al. (US 2009/0033653 A1) and Koylazov (US 2017/0109898 A1).
9. With reference to claim 1, Ziegler teaches A visible element determination method, performed by at least one processor of an electronic device (“the disclosure proposes a way to realistically integrate a rendered image into a computer generated scene. Embodiments of the present disclosure relate to an apparatus, a method and a computer program for rendering a visual scene.” [0002] “The following description explains the function of the rendering concept based on a single scene element, which serves as an example for the one or more objects 22. For this scene element 22, a proxy geometry, e.g. a geometry representation 40, which may comprise a mesh 340, as well as an LFR, which is an example for the set of images 30, is available; or the preparation procedure (the content creation stage 301 and/or the content preparation stage 302) creates the data that may be used, i.e. the geometry representation 40 and the set of images 30. For example, such a scene element 22 may consist of a single object, a single person, several persons or any other combination.” [0101] “rendering the image, e.g. an image of the visual scene 20, to a final image 90 can be realized by projecting 403 the mesh into the image to determine which polygons are visible, followed by a resampling 404 of the appropriate region of the texture map 444 into the final rendered image.” [0120] “Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit.” [0243]) Ziegler also teaches rendering each scene element in a first scene element set under a target perspective to obtain a target rendering image, the first scene element set comprising to-be-rendered scene elements in a target scene under the target perspective, (“a content visualization stage configured to obtain as a first input a set of images of one or more objects, and to obtain as a second input a geometry representation of the one or more objects in a 3D-space, the geometry representation including a position information of the one or more objects within the visual scene, obtain a final image representing the visual scene from a perspective of a target position, the visual scene including the one or more objects, and consider at least one of a lighting effect and/or an object interaction effect between the one or more objects and one or more further objects contained in the visual scene, wherein the content visualization stage includes: a target view synthesis stage configured to obtain a target view image from the set of images irrespective of the geometry representation, the target view image representing the one or more objects from the perspective of the target position, and a texture mapping block being configured to map the target view image on the geometry representation under consideration of the target position.” [0023] “The content visualization step 316 comprises a target view synthesis step 312, which may also be referred to as novel view synthesis stage 312. Based on the provided input images or videos 330, the novel view synthesis stage 312 computes an image that corresponds to what a camera would have seen at the target viewing position 360. The novel view synthesis stage 312 may be executed by the target view synthesis stage 12. Such a novel view synthesis 312 may, for instance, be performed using image based rendering, DIBR, multi-plane images, or any other method. The result of the target view synthesis 312 is the target view image 32.” [0089] “The following description explains the function of the rendering concept based on a single scene element, which serves as an example for the one or more objects 22. For this scene element 22, a proxy geometry, e.g. a geometry representation 40, which may comprise a mesh 340, as well as an LFR, which is an example for the set of images 30, is available; or the preparation procedure (the content creation stage 301 and/or the content preparation stage 302) creates the data that may be used, i.e. the geometry representation 40 and the set of images 30. For example, such a scene element 22 may consist of a single object, a single person, several persons or any other combination.” [0101]) Ziegler further teaches determining a second scene element set in the first scene element set; and determining scene elements in the second scene element set as visible elements under the target perspective, (“The content visualization stage 16 may further comprise a target view synthesis stage 12 configured to obtain a target view image 32 from the set of images 30 irrespective of the geometry representation 40, wherein the target view image 32 represents the one or more objects 22 from the perspective of the target position 60.” [0071] “The following description explains the function of the rendering concept based on a single scene element, which serves as an example for the one or more objects 22. For this scene element 22, a proxy geometry, e.g. a geometry representation 40, which may comprise a mesh 340, as well as an LFR, which is an example for the set of images 30, is available; or the preparation procedure (the content creation stage 301 and/or the content preparation stage 302) creates the data that may be used, i.e. the geometry representation 40 and the set of images 30.” [0101] “the target view image 32 is projected onto the geometry representation 40, e.g. a mesh 340, which is then after some possible additional processing projected back into an image, e.g. a final image 90.” [0115] “rendering the image, e.g. an image of the visual scene 20, to a final image 90 can be realized by projecting 403 the mesh into the image to determine which polygons are visible, followed by a resampling 404 of the appropriate region of the texture map 444 into the final rendered image.” [0120]) Ziegler teaches the target rendering image comprises pixels in scene elements other than the blocked scene elements in the first scene element set; (“that the mesh 340 occludes the one or more objects 22 from the perspective of a target camera position, e.g. the target position 60, 360. In other words, the mesh 340 may indicate the geometry of the one or more objects 22 and the position of the one or more objects 22 within the visual scene 20. For example, the mesh 340 may comprise one or more planes, wherein a plane has a position in the 3D space and occludes one of the one or more objects 22.” [0083] “The content visualization step 316 comprises a target view synthesis step 312, which may also be referred to as novel view synthesis stage 312. Based on the provided input images or videos 330, the novel view synthesis stage 312 computes an image that corresponds to what a camera would have seen at the target viewing position 360. The novel view synthesis stage 312 may be executed by the target view synthesis stage 12. Such a novel view synthesis 312 may, for instance, be performed using image based rendering, DIBR, multi-plane images, or any other method. The result of the target view synthesis 312 is the target view image 32. The target view image 32 may for example be a pixel image or a pixel array, wherein a pixel of the pixel image or pixel array may comprise a texture information and/or a depth information.” [0089] “The following description explains the function of the rendering concept based on a single scene element, which serves as an example for the one or more objects 22. For this scene element 22, a proxy geometry, e.g. a geometry representation 40, which may comprise a mesh 340, as well as an LFR, which is an example for the set of images 30, is available; or the preparation procedure (the content creation stage 301 and/or the content preparation stage 302) creates the data that may be used, i.e. the geometry representation 40 and the set of images 30. … Such an approach allows considering several interactions between the rendered light-field content, i.e. the one or more objects 22, and the remaining CG objects 26, representing thus immediate advantages of the proposed method: The mapping of the rendered pixel to the proxy geometry allows computing occlusions between CG objects 26 and objects 22 visualized by the light-field.” [0101-0102] “rendering the image, e.g. an image of the visual scene 20, to a final image 90 can be realized by projecting 403 the mesh into the image to determine which polygons are visible, followed by a resampling 404 of the appropriate region of the texture map 444 into the final rendered image.” [0120])
PNG
media_image1.png
747
545
media_image1.png
Greyscale
Ziegler does not explicitly teach each scene element in the first scene element set having a corresponding index, and a color value of each pixel in the target rendering image being rendered according to an index of a scene element to which each pixel belongs; wherein the rendering each scene element in the first scene element set comprises: determining, according to the index of each scene element in the first scene element set, a color value corresponding to each scene element, the scene elements with different indexes corresponding to different color values; determining, for each scene element, a color value of the pixel in the scene element according to the color value corresponding to the scene element, determining the index of the scene element to which each pixel belongs according to the color value of each pixel in the target rendering image, to obtain a target index set; the indexes of the scene elements in the second scene element set being included in indexes in the target index set. This is what Brown teaches. Brown teaches each scene element in the first scene element set having a corresponding index, and a color value of each pixel in the target rendering image being rendered according to an index of a scene element to which each pixel belongs; determining the index of the scene element to which each pixel belongs according to the color value of each pixel in the target rendering image, to obtain a target index set; the indexes of the scene elements in the second scene element set being included in indexes in the target index set. (“an image processing system may use a recursive ray tracing algorithm to render a two dimensional image from a three dimensional scene. The image processing system using a recursive ray tracing algorithm may use a processing element to perform ray tracing. The processor may be used to traverse a ray through a spatial index, and to determine if the ray intersects any objects within a bounding volume of the spatial index. If the ray intersects an object contained within a bounding volume, the image processing system, using the same processor, may issue secondary rays into the three dimensional scene to determine if they intersect any objects and, consequently, contribute color to the object intersected by the original ray. … If the processing element determines that the secondary rays intersect objects within the three dimensional scene the image processing element may issue more secondary rays into the scene to determine if those secondary rays intersect objects and contribute color to the object intersected by the original ray. When performing calculations to determine if the secondary rays intersect objects within the three dimensional scene, the processor may store previous secondary ray information in the processor's memory cache. By issuing more and more secondary rays into the scene, the image processing system may finally determine the total contribution of color from secondary rays to the object intersected by the original ray. From the color of the object intersected by the original ray and the contribution of color due to secondary rays, the color of the pixel through which the original ray passed may be finally determined.” [0086-0087] “at step 610 the image processing system may use a use a workload manager 205 processing element to traverse the spatial index (e.g., kd-Tree). The spatial index may be stored within the shared memory cache 110 of the image processing system. Traversing the kd-Tree may include performing calculations which determine if the original ray intersects bounding volumes which are defined by nodes within the spatial index. Furthermore, traversing the spatial index may include taking branches to nodes which defined bounding volumes intersected by the ray. A workload manager 205 may use the coordinates and trajectory of an issued ray (e.g., the original ray) to determine if the ray intersects bounding volumes defined by the nodes in the spatial index. The workload manager 205 may continue traversing the spatial index until the original ray intersects a bounding volume which contains only primitives (i.e., a leaf node). … The workload manager 205 may send information which defines the original ray and the leaf node (e.g., trajectory of the ray, pixel through which the original ray passed, bounding volume defined by the leaf node, etc.) to the vector throughput engine 210. … By coupling the pixel information with the information which defines the original ray, there is no need to send the original ray back to the workload manager 205 if the vector throughput engine 210 determines that the ray intersected an object and, consequently, determines a color of the pixel. According to one embodiment of the invention, the vector throughput engine 210 may use the pixel information to update the color of the pixel by writing to memory location within a frame buffer (e.g., stored in the shared memory cache 110) which corresponds to the pixel.” [0093-0095]) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Brown into Ziegler, in order to reduce workload experienced while maintaining the quality of the rendered image.
The combination of Ziegler and Brown does not explicitly teach the rendering each scene element in the first scene element set comprises: determining, according to the index of each scene element in the first scene element set, a color value corresponding to each scene element, the scene elements with different indexes corresponding to different color values; determining, for each scene element, a color value of the pixel in the scene element according to the color value corresponding to the scene element. This is what Koylazov teaches (“FIG. 3 is a flow diagram of an example technique 300 for determining whether or not to sample another contribution value for a given render element for a given pixel of an image. … The system determines a neighborhood maximum color contribution value for the render element from the current color data maintained for the render element (step 302). That is, in this example, the current color data that is maintained for a given render element for a given pixel includes the maximum of the color contribution values that have been sampled for the render element for the pixel. The system uses the maintained maximum color contribution values to determine the neighborhood maximum color contribution value for the render element. … The system determines a neighborhood minimum color contribution value for the render element from the current color data maintained for the render element (step 304). That is, in this example, the current color data that is maintained for a given render element for a given pixel includes the minimum of the color contribution values that have been sampled for the render element for the pixel. The system uses the maintained minimum color contribution values to determine the neighborhood maximum color contribution value for the render element. … The system determines a difference between the neighborhood maximum color contribution value and the neighborhood minimum color contribution value (step 306) and determines whether or not to sample another contribution value for the render element from the difference (step 308). … the system computes a ratio between the difference and a value derived from the total number of color contribution values that have been sampled for the render element for the pixel. … when the final color has been determined for each pixel in the image, the system applies a color mapping function to the final color as part of the rendering process.” [0034-0039]) Koylazov teaches the render element for a pixel of an image and determine color contribution values for the render elements, Koylazov also teaches the final color for each pixel in the image from the render element color value. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Koylazov into the combination of Ziegler and Brown, in order to render an image quicker and use fewer computing resources.
10. With reference to claim 2, Ziegler teaches before the rendering each scene element in the first scene element set, the method further comprises: searching for scene elements within a range of the target perspective from the target scene to obtain the first scene element set. (“a content visualization stage configured to obtain as a first input a set of images of one or more objects, and to obtain as a second input a geometry representation of the one or more objects in a 3D-space, the geometry representation including a position information of the one or more objects within the visual scene, obtain a final image representing the visual scene from a perspective of a target position, the visual scene including the one or more objects, and consider at least one of a lighting effect and/or an object interaction effect between the one or more objects and one or more further objects contained in the visual scene, wherein the content visualization stage includes: a target view synthesis stage configured to obtain a target view image from the set of images irrespective of the geometry representation, the target view image representing the one or more objects from the perspective of the target position, and a texture mapping block being configured to map the target view image on the geometry representation under consideration of the target position.” [0023] “The content visualization step 316 comprises a target view synthesis step 312, which may also be referred to as novel view synthesis stage 312. Based on the provided input images or videos 330, the novel view synthesis stage 312 computes an image that corresponds to what a camera would have seen at the target viewing position 360. The novel view synthesis stage 312 may be executed by the target view synthesis stage 12. Such a novel view synthesis 312 may, for instance, be performed using image based rendering, DIBR, multi-plane images, or any other method. The result of the target view synthesis 312 is the target view image 32.” [0089] “The following description explains the function of the rendering concept based on a single scene element, which serves as an example for the one or more objects 22. For this scene element 22, a proxy geometry, e.g. a geometry representation 40, which may comprise a mesh 340, as well as an LFR, which is an example for the set of images 30, is available; or the preparation procedure (the content creation stage 301 and/or the content preparation stage 302) creates the data that may be used, i.e. the geometry representation 40 and the set of images 30.” [0101])
11. With reference to claim 10, Ziegler teaches the rendering each scene element in the first scene element set comprises: determining, based on the first scene element set being a first model set and each scene element in the first scene element set being each model in the first model set, a color value corresponding to each model in the first model set; and determining the target rendering image according to the color value corresponding to each model in the first model set and the position of each model under the target perspective. (“a content visualization stage configured to obtain as a first input a set of images of one or more objects, and to obtain as a second input a geometry representation of the one or more objects in a 3D-space, the geometry representation including a position information of the one or more objects within the visual scene, obtain a final image representing the visual scene from a perspective of a target position, the visual scene including the one or more objects, and consider at least one of a lighting effect and/or an object interaction effect between the one or more objects and one or more further objects contained in the visual scene, wherein the content visualization stage includes: a target view synthesis stage configured to obtain a target view image from the set of images irrespective of the geometry representation, the target view image representing the one or more objects from the perspective of the target position, and a texture mapping block being configured to map the target view image on the geometry representation under consideration of the target position.” [0023] “The input to DIBR is a set of RGB images, each with an associated depth map. The depth map essentially describes for each pixel a location in 3D space. This information allows synthesizing a new view for a virtual camera. By merging several novel views synthesized from different input views, occlusions can be mitigated. Technically, DIBR is often performed by a forward warp of the depth map to the virtual camera. Then a backward warp computes for every target pixel its color [5]. This two-staged approach allows subpixel interpolation of the RGB values and thus avoids image artefacts.” [0058] “The content visualization step 316 comprises a target view synthesis step 312, which may also be referred to as novel view synthesis stage 312. Based on the provided input images or videos 330, the novel view synthesis stage 312 computes an image that corresponds to what a camera would have seen at the target viewing position 360. The novel view synthesis stage 312 may be executed by the target view synthesis stage 12. Such a novel view synthesis 312 may, for instance, be performed using image based rendering, DIBR, multi-plane images, or any other method. The result of the target view synthesis 312 is the target view image 32.” [0089] “The following description explains the function of the rendering concept based on a single scene element, which serves as an example for the one or more objects 22. For this scene element 22, a proxy geometry, e.g. a geometry representation 40, which may comprise a mesh 340, as well as an LFR, which is an example for the set of images 30, is available; or the preparation procedure (the content creation stage 301 and/or the content preparation stage 302) creates the data that may be used, i.e. the geometry representation 40 and the set of images 30. For example, such a scene element 22 may consist of a single object, a single person, several persons or any other combination.” [0101] “Texture mapping 18, assigns the pixels of the target view image 32, which may be a rendered RGB image, to mesh polygons, which may be parts of the geometry representation 40 or the mesh 340. The target view image 32 may comprise a texture map 434, for example a pixel array with color or texture information, and a depth map 436.” [0112] “rendering the image, e.g. an image of the visual scene 20, to a final image 90 can be realized by projecting 403 the mesh into the image to determine which polygons are visible, followed by a resampling 404 of the appropriate region of the texture map 444 into the final rendered image.” [0120])
Ziegler does not explicitly teach a color value corresponding to each model according to an index of each model, the models with different indexes corresponding to different color values; This is what Brown teaches (“at step 610 the image processing system may use a use a workload manager 205 processing element to traverse the spatial index (e.g., kd-Tree). The spatial index may be stored within the shared memory cache 110 of the image processing system. Traversing the kd-Tree may include performing calculations which determine if the original ray intersects bounding volumes which are defined by nodes within the spatial index. Furthermore, traversing the spatial index may include taking branches to nodes which defined bounding volumes intersected by the ray. A workload manager 205 may use the coordinates and trajectory of an issued ray (e.g., the original ray) to determine if the ray intersects bounding volumes defined by the nodes in the spatial index. The workload manager 205 may continue traversing the spatial index until the original ray intersects a bounding volume which contains only primitives (i.e., a leaf node). … The workload manager 205 may send information which defines the original ray and the leaf node (e.g., trajectory of the ray, pixel through which the original ray passed, bounding volume defined by the leaf node, etc.) to the vector throughput engine 210. … By coupling the pixel information with the information which defines the original ray, there is no need to send the original ray back to the workload manager 205 if the vector throughput engine 210 determines that the ray intersected an object and, consequently, determines a color of the pixel. According to one embodiment of the invention, the vector throughput engine 210 may use the pixel information to update the color of the pixel by writing to memory location within a frame buffer (e.g., stored in the shared memory cache 110) which corresponds to the pixel.” [0093-0095]) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Brown into Ziegler, in order to reduce workload experienced while maintaining the quality of the rendered image.
12. With reference to claim 11, Ziegler teaches the rendering each scene element in the first scene element set comprises: determining, based on the first scene element set being a first primitive set and each scene element in the first scene element set being each primitive in the first primitive set, a color value corresponding to each primitive in the first primitive set; and determining the target rendering image according to the color value corresponding to each primitive in the first primitive set and the position of each primitive under the target perspective. (“a content visualization stage configured to obtain as a first input a set of images of one or more objects, and to obtain as a second input a geometry representation of the one or more objects in a 3D-space, the geometry representation including a position information of the one or more objects within the visual scene, obtain a final image representing the visual scene from a perspective of a target position, the visual scene including the one or more objects, and consider at least one of a lighting effect and/or an object interaction effect between the one or more objects and one or more further objects contained in the visual scene, wherein the content visualization stage includes: a target view synthesis stage configured to obtain a target view image from the set of images irrespective of the geometry representation, the target view image representing the one or more objects from the perspective of the target position, and a texture mapping block being configured to map the target view image on the geometry representation under consideration of the target position.” [0023] “The input to DIBR is a set of RGB images, each with an associated depth map. The depth map essentially describes for each pixel a location in 3D space. This information allows synthesizing a new view for a virtual camera. By merging several novel views synthesized from different input views, occlusions can be mitigated. Technically, DIBR is often performed by a forward warp of the depth map to the virtual camera. Then a backward warp computes for every target pixel its color [5]. This two-staged approach allows subpixel interpolation of the RGB values and thus avoids image artefacts.” [0058] “The content visualization step 316 comprises a target view synthesis step 312, which may also be referred to as novel view synthesis stage 312. Based on the provided input images or videos 330, the novel view synthesis stage 312 computes an image that corresponds to what a camera would have seen at the target viewing position 360. The novel view synthesis stage 312 may be executed by the target view synthesis stage 12. Such a novel view synthesis 312 may, for instance, be performed using image based rendering, DIBR, multi-plane images, or any other method. The result of the target view synthesis 312 is the target view image 32.” [0089] “The following description explains the function of the rendering concept based on a single scene element, which serves as an example for the one or more objects 22. For this scene element 22, a proxy geometry, e.g. a geometry representation 40, which may comprise a mesh 340, as well as an LFR, which is an example for the set of images 30, is available; or the preparation procedure (the content creation stage 301 and/or the content preparation stage 302) creates the data that may be used, i.e. the geometry representation 40 and the set of images 30. For example, such a scene element 22 may consist of a single object, a single person, several persons or any other combination.” [0101] “Texture mapping 18, assigns the pixels of the target view image 32, which may be a rendered RGB image, to mesh polygons, which may be parts of the geometry representation 40 or the mesh 340. The target view image 32 may comprise a texture map 434, for example a pixel array with color or texture information, and a depth map 436.” [0112] “rendering the image, e.g. an image of the visual scene 20, to a final image 90 can be realized by projecting 403 the mesh into the image to determine which polygons are visible, followed by a resampling 404 of the appropriate region of the texture map 444 into the final rendered image.” [0120])
Ziegler does not explicitly teach a color value corresponding to each primitive according to an index of each primitive, the primitives with different indexes corresponding to different color values; This is what Brown teaches (“at step 610 the image processing system may use a use a workload manager 205 processing element to traverse the spatial index (e.g., kd-Tree). The spatial index may be stored within the shared memory cache 110 of the image processing system. Traversing the kd-Tree may include performing calculations which determine if the original ray intersects bounding volumes which are defined by nodes within the spatial index. Furthermore, traversing the spatial index may include taking branches to nodes which defined bounding volumes intersected by the ray. A workload manager 205 may use the coordinates and trajectory of an issued ray (e.g., the original ray) to determine if the ray intersects bounding volumes defined by the nodes in the spatial index. The workload manager 205 may continue traversing the spatial index until the original ray intersects a bounding volume which contains only primitives (i.e., a leaf node). … The workload manager 205 may send information which defines the original ray and the leaf node (e.g., trajectory of the ray, pixel through which the original ray passed, bounding volume defined by the leaf node, etc.) to the vector throughput engine 210. … By coupling the pixel information with the information which defines the original ray, there is no need to send the original ray back to the workload manager 205 if the vector throughput engine 210 determines that the ray intersected an object and, consequently, determines a color of the pixel. According to one embodiment of the invention, the vector throughput engine 210 may use the pixel information to update the color of the pixel by writing to memory location within a frame buffer (e.g., stored in the shared memory cache 110) which corresponds to the pixel.” [0093-0095]) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Brown into Ziegler, in order to reduce workload experienced while maintaining the quality of the rendered image.
13. With reference to claim 12, Ziegler teaches the visible element comprises a visible model; and the determining scene elements in the second scene element set comprises: determining, based on the second scene element set being a second model set and each scene element in the second scene element set being each model in the second model set, each model in the second model set as the visible model under the target perspective. (“a content visualization stage configured to obtain as a first input a set of images of one or more objects, and to obtain as a second input a geometry representation of the one or more objects in a 3D-space, the geometry representation including a position information of the one or more objects within the visual scene, obtain a final image representing the visual scene from a perspective of a target position, the visual scene including the one or more objects, and consider at least one of a lighting effect and/or an object interaction effect between the one or more objects and one or more further objects contained in the visual scene, wherein the content visualization stage includes: a target view synthesis stage configured to obtain a target view image from the set of images irrespective of the geometry representation, the target view image representing the one or more objects from the perspective of the target position, and a texture mapping block being configured to map the target view image on the geometry representation under consideration of the target position.” [0023] “The input to DIBR is a set of RGB images, each with an associated depth map. The depth map essentially describes for each pixel a location in 3D space. This information allows synthesizing a new view for a virtual camera. By merging several novel views synthesized from different input views, occlusions can be mitigated. Technically, DIBR is often performed by a forward warp of the depth map to the virtual camera. Then a backward warp computes for every target pixel its color [5]. This two-staged approach allows subpixel interpolation of the RGB values and thus avoids image artefacts.” [0058] “The content visualization stage 16 may further comprise a target view synthesis stage 12 configured to obtain a target view image 32 from the set of images 30 irrespective of the geometry representation 40, wherein the target view image 32 represents the one or more objects 22 from the perspective of the target position 60.” [0071] “The content visualization step 316 comprises a target view synthesis step 312, which may also be referred to as novel view synthesis stage 312. Based on the provided input images or videos 330, the novel view synthesis stage 312 computes an image that corresponds to what a camera would have seen at the target viewing position 360. The novel view synthesis stage 312 may be executed by the target view synthesis stage 12. Such a novel view synthesis 312 may, for instance, be performed using image based rendering, DIBR, multi-plane images, or any other method. The result of the target view synthesis 312 is the target view image 32.” [0089] “The following description explains the function of the rendering concept based on a single scene element, which serves as an example for the one or more objects 22. For this scene element 22, a proxy geometry, e.g. a geometry representation 40, which may comprise a mesh 340, as well as an LFR, which is an example for the set of images 30, is available; or the preparation procedure (the content creation stage 301 and/or the content preparation stage 302) creates the data that may be used, i.e. the geometry representation 40 and the set of images 30. For example, such a scene element 22 may consist of a single object, a single person, several persons or any other combination.” [0101] “Texture mapping 18, assigns the pixels of the target view image 32, which may be a rendered RGB image, to mesh polygons, which may be parts of the geometry representation 40 or the mesh 340. The target view image 32 may comprise a texture map 434, for example a pixel array with color or texture information, and a depth map 436.” [0112] “rendering the image, e.g. an image of the visual scene 20, to a final image 90 can be realized by projecting 403 the mesh into the image to determine which polygons are visible, followed by a resampling 404 of the appropriate region of the texture map 444 into the final rendered image.” [0120])
14. With reference to claim 13, Ziegler teaches the visible element comprises a visible primitive; and the determining scene elements in the second scene element set comprises: determining, based on the second scene element set being a second primitive set and each scene element in the second scene element set being each primitive in the second primitive set, each primitive in the second primitive set as the visible primitive under the target perspective. (“a content visualization stage configured to obtain as a first input a set of images of one or more objects, and to obtain as a second input a geometry representation of the one or more objects in a 3D-space, the geometry representation including a position information of the one or more objects within the visual scene, obtain a final image representing the visual scene from a perspective of a target position, the visual scene including the one or more objects, and consider at least one of a lighting effect and/or an object interaction effect between the one or more objects and one or more further objects contained in the visual scene, wherein the content visualization stage includes: a target view synthesis stage configured to obtain a target view image from the set of images irrespective of the geometry representation, the target view image representing the one or more objects from the perspective of the target position, and a texture mapping block being configured to map the target view image on the geometry representation under consideration of the target position.” [0023] “The input to DIBR is a set of RGB images, each with an associated depth map. The depth map essentially describes for each pixel a location in 3D space. This information allows synthesizing a new view for a virtual camera. By merging several novel views synthesized from different input views, occlusions can be mitigated. Technically, DIBR is often performed by a forward warp of the depth map to the virtual camera. Then a backward warp computes for every target pixel its color [5]. This two-staged approach allows subpixel interpolation of the RGB values and thus avoids image artefacts.” [0058] “The content visualization stage 16 may further comprise a target view synthesis stage 12 configured to obtain a target view image 32 from the set of images 30 irrespective of the geometry representation 40, wherein the target view image 32 represents the one or more objects 22 from the perspective of the target position 60.” [0071] “The content visualization step 316 comprises a target view synthesis step 312, which may also be referred to as novel view synthesis stage 312. Based on the provided input images or videos 330, the novel view synthesis stage 312 computes an image that corresponds to what a camera would have seen at the target viewing position 360. The novel view synthesis stage 312 may be executed by the target view synthesis stage 12. Such a novel view synthesis 312 may, for instance, be performed using image based rendering, DIBR, multi-plane images, or any other method. The result of the target view synthesis 312 is the target view image 32.” [0089] “The following description explains the function of the rendering concept based on a single scene element, which serves as an example for the one or more objects 22. For this scene element 22, a proxy geometry, e.g. a geometry representation 40, which may comprise a mesh 340, as well as an LFR, which is an example for the set of images 30, is available; or the preparation procedure (the content creation stage 301 and/or the content preparation stage 302) creates the data that may be used, i.e. the geometry representation 40 and the set of images 30. For example, such a scene element 22 may consist of a single object, a single person, several persons or any other combination.” [0101] “Texture mapping 18, assigns the pixels of the target view image 32, which may be a rendered RGB image, to mesh polygons, which may be parts of the geometry representation 40 or the mesh 340. The target view image 32 may comprise a texture map 434, for example a pixel array with color or texture information, and a depth map 436.” [0112] “rendering the image, e.g. an image of the visual scene 20, to a final image 90 can be realized by projecting 403 the mesh into the image to determine which polygons are visible, followed by a resampling 404 of the appropriate region of the texture map 444 into the final rendered image.” [0120])
15. Claim 16 is similar in scope to claim 1, and thus is rejected under similar rationale. Ziegler additionally teaches at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, (“computer programs are provided, wherein each of the computer programs is configured to implement the above-described method when being executed on a computer or signal processor, so that the above-described method is implemented by one of the computer programs.” [0028] “embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.” [0246] “A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like.” [0253])
16. Claim 17 is similar in scope to claim 2, and thus is rejected under similar rationale.
17. Claim 20 is similar in scope to claim 1, and thus is rejected under similar rationale. Ziegler additionally teaches A non-transitory computer-readable storage medium, storing a computer program that when executed by at least one processor causes the at least one processor to: (“computer programs are provided, wherein each of the computer programs is configured to implement the above-described method when being executed on a computer or signal processor, so that the above-described method is implemented by one of the computer programs.” [0028] “embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.” [0246] “a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitory.” [0249])
Allowable Subject Matter
18. Claims 4-9, 14, 15 and 19 are objected to being dependent upon rejected base claims. The claims would be allowable if rewritten in independent form including all the limitations of the base claims and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claims 4 and 19, the prior arts of record fails to either individually or in combination teach the claimed feature of: “determining a corresponding storage position of each scene element in a target storage space according to a position of each scene element under the target perspective; and storing the color value of the pixel in each scene element to the corresponding storage position in the target storage space, the color value of the pixel stored in the target storage space being the color value of the pixel in the target rendering image.”
Claims 5-8 are also objected to for depending from claim 4.
Regarding claim 9, the prior arts of record fails to either individually or in combination teach the claimed feature of: “the target rendering image comprising a plurality of image blocks, and each thread in the target thread set being used for reading the color value of the pixel in an image block in the target rendering image every time.”
Regarding claim 14, the prior arts of record fails to either individually or in combination teach the claimed feature of: “setting a value of a unit other than the first unit set in the first array as a second value, a quantity of units in the first array being a quantity of scene elements in the first scene element set, the units in the first array and the scene elements in the first scene element set having a one-to-one correspondence relationship,”
Regarding claim 15, the prior arts of record fails to either individually or in combination teach the claimed feature of: “setting a value of a unit other than the second unit set in the second array as a third value, the second unit set being a unit set corresponding to the first scene element set in the second array, the second unit set comprising the first unit set, a quantity of units in the second array being a quantity of scene elements in the target scene, the units in the second array and the scene elements in the target scene having a one-to-one correspondence relationship,”
Conclusion
19. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michelle Chin whose telephone number is (571)270-3697. The examiner can normally be reached on Monday-Friday 8:00 AM-4:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http:/Awww.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Kent Chang can be reached on (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https:/Awww.uspto.gov/patents/apply/patent- center for more information about Patent Center and https:/Awww.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHELLE CHIN/
Primary Examiner, Art Unit 2614
Prosecution Insights