DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 1 recited the limitation “…and if a tracing distance of the ray is less than a second distance smaller than a first distance….”, however “tracing distance” is not well described in the claims. Therefore, the claims 1-20 are indefinite and are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 6-8, 11-13, and 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lukierski et al. (WO 2018015716) in view of Hughes et al. (GB 2596566), and Woo et al. (KR 102261250).
Regarding claim 1.
Lukierski teaches:
One or more non-transitory computer-readable media having stored therein instructions that, when executed, cause one or more processors of an information processing apparatus to execute image processing comprising (Lukierski [Pg 5 Par 2] In other examples, a non-transitory computer-readable storage medium may comprise computer-executable instructions which, when executed by a processor, cause a computing device, such as an embedded computer in a robotic device or a remote processor in a distributed system, to perform any of the methods discussed above.):
regarding objects in a virtual space, performing a first depth test using a first depth buffer and updating the first depth buffer (Lukierski [Pg 11 Par 4] Depth values are then estimated (630) by evaluating a volumetric function of the sequence of images and the pose data);
performing drawing in a frame buffer based on a result of the first depth test (Lukierski [Pg 22 Par 6 – Pg 22 Par 1] As shown in Figure 12, a mobile computing device 1250 may be configured to access data from the room database 1230 and to use this data to render a room plan 1260 on a display of the device 1250. The room plan may comprise one or more room schematics that are rendered based on the dimensions stored in the room database 1230. In Figure 12, three boxes of different dimensions are shown on the display, indicating room dimensions for three separate rooms in a house or office. A room plan may comprise a plurality of different floors. A room plan may also display a room class label for each room.);
and regarding a first type of object among the objects, further performing a second depth test using a second depth buffer and updating the second depth buffer (Lukierski [Pg 14 Par 2] a first set of depth values (i.e. D) from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three- dimensional volume.),
with respect to each pixel of the frame buffer in which the drawing is performed, using the pixel as a pixel of interest (Lukierski [Pg 25 Par 1] The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values.),
tracing the ray along the ray tracing direction, and if a tracing distance of the ray is less than a second distance smaller than a first distance (Lukierski [Pg 25 Par 1] a first set of depth values from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three-dimensional volume. Ray tracing may be used to determine the second set of depth values. The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values. The function may comprise an asymmetric function, wherein the asymmetric function returns higher values when the first set of depth values are greater than the second set of depth values as compared to when the first set of depth values are less than the second set of depth values.),
and if the tracing distance of the ray is greater than or equal to the second distance, determining the collision position based on a depth of the second depth buffer (Lukierski [Pg 3 Par 5] a first set of depth values from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three-dimensional volume. Ray tracing may be used to determine the second set of depth values. The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values. The function may comprise an asymmetric function, wherein the asymmetric function returns higher values when the first set of depth values are greater than the second set of depth values as compared to when the first set of depth values are less than the second set of depth values.);
Lukierski fails to teach:
based on a depth of the first depth buffer, calculating a direction from a virtual camera to a position in the virtual space relating to the pixel of interest as an incidence direction (Hughes [Pg 2 Par 2] in rendering systems using ray-tracing, the simulation is performed by emitting virtual or simulated "rays" from the viewing position such as a virtual camera and determining what interactions they undergo when they hit an object or a light source (the latter being either directly or after having hit an object.),
and calculating as a ray tracing direction a direction of a ray reflected from the position as a reflection position (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.);
based on the depth of the first depth buffer, determining a collision position where the ray collides with an object in the virtual space (Woo [Pg 7 Par 9] In addition, the ray tracing apparatus 500 may generate a first ray using ray information through the software unit unit 510 and calculate a collision point based on the first ray and collision triangle information (step S650). The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).),
and if the collision position is determined in a range where the tracing distance is less than or equal to the first distance, determining a color based on a color of a pixel in the frame buffer relating to the collision position as a reflected appearance color to be added to a color of the pixel of interest (Woo [Pg 7 Par 9] In addition, the ray tracing apparatus 500 may generate a first ray using ray information through the software unit 510 and calculate a collision point based on the first ray and collision triangle information (step S650). The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
Regarding claim 2:
Lukierski, Hughes, and Woo teaches:
The one or more non-transitory computer-readable media according to claim 1,
wherein the image processing further comprises based on a parameter that differs between the collision position based on the first depth buffer and the collision position based on the second depth buffer, determining the reflected appearance color to be added to the color of the pixel of interest based on the color of the pixel in the frame buffer relating to the collision position (Woo [Pg 7 Par 9] The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
Regarding claim 3:
Lukierski, Hughes, and Woo teaches:
The one or more non-transitory computer-readable media according to claim 1,
wherein the image processing further comprises: regarding a second type of object, placing a flat surface object in which an image rendered without performing drawing in the frame buffer is set as a texture at a position of the second type of object in the virtual space (Hughes [Pg 5 Par 2] At this point on the virtual object 250, the object surface's colour, orientation, reflectivity and other optical properties are taken into account.);
if the ray intersects the flat surface object, determining a reflected appearance color to be further added to the color of the pixel of interest based on a color of the texture at an intersection position of the intersection (Hughes [Pg 5 Par 5] As described, the ray-tracing process can be very computationally expensive and require powerful computing resources to carry out, particularly where the virtual scene includes many virtual objects and virtual light sources. Each virtual ray must be tested as to whether it intersects with each object in the scene.);
reflecting the reflected appearance color on the frame buffer (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.);
and after reflecting the reflected appearance color, drawing the second type of object in the frame buffer (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
Regarding claim 6.
Lukierski teaches:
An information processing system comprising: one or more processors that execute image processing comprising: (Lukierski [Pg 5 Par 2] In other examples, a non-transitory computer-readable storage medium may comprise computer-executable instructions which, when executed by a processor, cause a computing device, such as an embedded computer in a robotic device or a remote processor in a distributed system, to perform any of the methods discussed above.):
regarding objects in a virtual space, performing a first depth test using a first depth buffer and updating the first depth buffer (Lukierski [Pg 11 Par 4] Depth values are then estimated (630) by evaluating a volumetric function of the sequence of images and the pose data);
performing drawing in a frame buffer based on a result of the first depth test (Lukierski [Pg 22 Par 6 – Pg 22 Par 1] As shown in Figure 12, a mobile computing device 1250 may be configured to access data from the room database 1230 and to use this data to render a room plan 1260 on a display of the device 1250. The room plan may comprise one or more room schematics that are rendered based on the dimensions stored in the room database 1230. In Figure 12, three boxes of different dimensions are shown on the display, indicating room dimensions for three separate rooms in a house or office. A room plan may comprise a plurality of different floors. A room plan may also display a room class label for each room.);
and regarding a first type of object among the objects, further performing a second depth test using a second depth buffer and updating the second depth buffer (Lukierski [Pg 14 Par 2] a first set of depth values (i.e. D) from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three- dimensional volume.),
with respect to each pixel of the frame buffer in which the drawing is performed, using the pixel as a pixel of interest (Lukierski [Pg 25 Par 1] The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values.),
tracing the ray along the ray tracing direction, and if a tracing distance of the ray is less than a second distance smaller than a first distance (Lukierski [Pg 25 Par 1] a first set of depth values from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three-dimensional volume. Ray tracing may be used to determine the second set of depth values. The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values. The function may comprise an asymmetric function, wherein the asymmetric function returns higher values when the first set of depth values are greater than the second set of depth values as compared to when the first set of depth values are less than the second set of depth values.),
and if the tracing distance of the ray is greater than or equal to the second distance, determining the collision position based on a depth of the second depth buffer (Lukierski [Pg 3 Par 5] a first set of depth values from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three-dimensional volume. Ray tracing may be used to determine the second set of depth values. The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values. The function may comprise an asymmetric function, wherein the asymmetric function returns higher values when the first set of depth values are greater than the second set of depth values as compared to when the first set of depth values are less than the second set of depth values.);
Lukierski fails to teach:
based on a depth of the first depth buffer, calculating a direction from a virtual camera to a position in the virtual space relating to the pixel of interest as an incidence direction (Hughes [Pg 2 Par 2] in rendering systems using ray-tracing, the simulation is performed by emitting virtual or simulated "rays" from the viewing position such as a virtual camera and determining what interactions they undergo when they hit an object or a light source (the latter being either directly or after having hit an object.),
and calculating as a ray tracing direction a direction of a ray reflected from the position as a reflection position (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.);
based on the depth of the first depth buffer, determining a collision position where the ray collides with an object in the virtual space (Woo [Pg 7 Par 9] In addition, the ray tracing apparatus 500 may generate a first ray using ray information through the software unit 510 and calculate a collision point based on the first ray and collision triangle information (step S650). The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).),
and if the collision position is determined in a range where the tracing distance is less than or equal to the first distance, determining a color based on a color of a pixel in the frame buffer relating to the collision position as a reflected appearance color to be added to a color of the pixel of interest (Woo [Pg 7 Par 9] In addition, the ray tracing apparatus 500 may generate a first ray using ray information through the software unit 510 and calculate a collision point based on the first ray and collision triangle information (step S650). The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
Regarding claim 7:
Lukierski, Hughes, and Woo teaches:
The image processing system according to claim 6,
wherein the image processing further comprises based on a parameter that differs between the collision position based on the first depth buffer and the collision position based on the second depth buffer, determining the reflected appearance color to be added to the color of the pixel of interest based on the color of the pixel in the frame buffer relating to the collision position (Woo [Pg 7 Par 9] The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
Regarding claim 8:
Lukierski, Hughes, and Woo teaches:
The image processing system according to claim 6,
wherein the image processing further comprises: regarding a second type of object, placing a flat surface object in which an image rendered without performing drawing in the frame buffer is set as a texture at a position of the second type of object in the virtual space (Hughes [Pg 5 Par 2] At this point on the virtual object 250, the object surface's colour, orientation, reflectivity and other optical properties are taken into account.);
if the ray intersects the flat surface object, determining a reflected appearance color to be further added to the color of the pixel of interest based on a color of the texture at an intersection position of the intersection (Hughes [Pg 5 Par 5] As described, the ray-tracing process can be very computationally expensive and require powerful computing resources to carry out, particularly where the virtual scene includes many virtual objects and virtual light sources. Each virtual ray must be tested as to whether it intersects with each object in the scene.);
reflecting the reflected appearance color on the frame buffer (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.);
and after reflecting the reflected appearance color, drawing the second type of object in the frame buffer (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
------------------------------------------------------------------------------------------------------------
Regarding claim 11.
Lukierski teaches:
An image processing method comprising: (Lukierski [Pg 5 Par 2] In other examples, a non-transitory computer-readable storage medium may comprise computer-executable instructions which, when executed by a processor, cause a computing device, such as an embedded computer in a robotic device or a remote processor in a distributed system, to perform any of the methods discussed above.):
regarding objects in a virtual space, performing a first depth test using a first depth buffer and updating the first depth buffer (Lukierski [Pg 11 Par 4] Depth values are then estimated (630) by evaluating a volumetric function of the sequence of images and the pose data);
performing drawing in a frame buffer based on a result of the first depth test (Lukierski [Pg 22 Par 6 – Pg 22 Par 1] As shown in Figure 12, a mobile computing device 1250 may be configured to access data from the room database 1230 and to use this data to render a room plan 1260 on a display of the device 1250. The room plan may comprise one or more room schematics that are rendered based on the dimensions stored in the room database 1230. In Figure 12, three boxes of different dimensions are shown on the display, indicating room dimensions for three separate rooms in a house or office. A room plan may comprise a plurality of different floors. A room plan may also display a room class label for each room.);
and regarding a first type of object among the objects, further performing a second depth test using a second depth buffer and updating the second depth buffer (Lukierski [Pg 14 Par 2] a first set of depth values (i.e. D) from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three- dimensional volume.),
with respect to each pixel of the frame buffer in which the drawing is performed, using the pixel as a pixel of interest (Lukierski [Pg 25 Par 1] The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values.),
tracing the ray along the ray tracing direction, and if a tracing distance of the ray is less than a second distance smaller than a first distance (Lukierski [Pg 25 Par 1] a first set of depth values from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three-dimensional volume. Ray tracing may be used to determine the second set of depth values. The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values. The function may comprise an asymmetric function, wherein the asymmetric function returns higher values when the first set of depth values are greater than the second set of depth values as compared to when the first set of depth values are less than the second set of depth values.),
and if the tracing distance of the ray is greater than or equal to the second distance, determining the collision position based on a depth of the second depth buffer (Lukierski [Pg 3 Par 5] a first set of depth values from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three-dimensional volume. Ray tracing may be used to determine the second set of depth values. The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values. The function may comprise an asymmetric function, wherein the asymmetric function returns higher values when the first set of depth values are greater than the second set of depth values as compared to when the first set of depth values are less than the second set of depth values.);
Lukierski fails to teach:
based on a depth of the first depth buffer, calculating a direction from a virtual camera to a position in the virtual space relating to the pixel of interest as an incidence direction (Hughes [Pg 2 Par 2] in rendering systems using ray-tracing, the simulation is performed by emitting virtual or simulated "rays" from the viewing position such as a virtual camera and determining what interactions they undergo when they hit an object or a light source (the latter being either directly or after having hit an object.),
and calculating as a ray tracing direction a direction of a ray reflected from the position as a reflection position (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.);
based on the depth of the first depth buffer, determining a collision position where the ray collides with an object in the virtual space (Woo [Pg 7 Par 9] In addition, the ray tracing apparatus 500 may generate a first ray using ray information through the software unit 510 and calculate a collision point based on the first ray and collision triangle information (step S650). The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).),
and if the collision position is determined in a range where the tracing distance is less than or equal to the first distance, determining a color based on a color of a pixel in the frame buffer relating to the collision position as a reflected appearance color to be added to a color of the pixel of interest (Woo [Pg 7 Par 9] In addition, the ray tracing apparatus 500 may generate a first ray using ray information through the software unit 510 and calculate a collision point based on the first ray and collision triangle information (step S650). The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
Regarding claim 12:
Lukierski, Hughes, and Woo teaches:
The image processing method according to claim 11,
wherein the image processing further comprises based on a parameter that differs between the collision position based on the first depth buffer and the collision position based on the second depth buffer, determining the reflected appearance color to be added to the color of the pixel of interest based on the color of the pixel in the frame buffer relating to the collision position (Woo [Pg 7 Par 9] The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
Regarding claim 13:
Lukierski, Hughes, and Woo teaches:
The image processing method according to claim 11,
wherein the image processing further comprises: regarding a second type of object, placing a flat surface object in which an image rendered without performing drawing in the frame buffer is set as a texture at a position of the second type of object in the virtual space (Hughes [Pg 5 Par 2] At this point on the virtual object 250, the object surface's colour, orientation, reflectivity and other optical properties are taken into account.);
if the ray intersects the flat surface object, determining a reflected appearance color to be further added to the color of the pixel of interest based on a color of the texture at an intersection position of the intersection (Hughes [Pg 5 Par 5] As described, the ray-tracing process can be very computationally expensive and require powerful computing resources to carry out, particularly where the virtual scene includes many virtual objects and virtual light sources. Each virtual ray must be tested as to whether it intersects with each object in the scene.);
reflecting the reflected appearance color on the frame buffer (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.);
and after reflecting the reflected appearance color, drawing the second type of object in the frame buffer (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
------------------------------------------------------------------------------------------------------------
Regarding claim 16.
Lukierski teaches:
An information processing apparatus comprising: one or more processors that execute image processing comprising: (Lukierski [Pg 5 Par 2] In other examples, a non-transitory computer-readable storage medium may comprise computer-executable instructions which, when executed by a processor, cause a computing device, such as an embedded computer in a robotic device or a remote processor in a distributed system, to perform any of the methods discussed above.):
regarding objects in a virtual space, performing a first depth test using a first depth buffer and updating the first depth buffer (Lukierski [Pg 11 Par 4] Depth values are then estimated (630) by evaluating a volumetric function of the sequence of images and the pose data);
performing drawing in a frame buffer based on a result of the first depth test (Lukierski [Pg 22 Par 6 – Pg 22 Par 1] As shown in Figure 12, a mobile computing device 1250 may be configured to access data from the room database 1230 and to use this data to render a room plan 1260 on a display of the device 1250. The room plan may comprise one or more room schematics that are rendered based on the dimensions stored in the room database 1230. In Figure 12, three boxes of different dimensions are shown on the display, indicating room dimensions for three separate rooms in a house or office. A room plan may comprise a plurality of different floors. A room plan may also display a room class label for each room.);
and regarding a first type of object among the objects, further performing a second depth test using a second depth buffer and updating the second depth buffer (Lukierski [Pg 14 Par 2] a first set of depth values (i.e. D) from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three- dimensional volume.),
with respect to each pixel of the frame buffer in which the drawing is performed, using the pixel as a pixel of interest (Lukierski [Pg 25 Par 1] The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values.),
tracing the ray along the ray tracing direction, and if a tracing distance of the ray is less than a second distance smaller than a first distance (Lukierski [Pg 25 Par 1] a first set of depth values from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three-dimensional volume. Ray tracing may be used to determine the second set of depth values. The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values. The function may comprise an asymmetric function, wherein the asymmetric function returns higher values when the first set of depth values are greater than the second set of depth values as compared to when the first set of depth values are less than the second set of depth values.),
and if the tracing distance of the ray is greater than or equal to the second distance, determining the collision position based on a depth of the second depth buffer (Lukierski [Pg 3 Par 5] a first set of depth values from the evaluation of the volumetric function of the image data and the pose data, and a second set of depth values estimated from the reference position to the walls of the three-dimensional volume. Ray tracing may be used to determine the second set of depth values. The function of the error may be evaluated by comparing a depth image with pixel values defining the first set of depth values with a depth image with pixel values defining second set of depth values. The function may comprise an asymmetric function, wherein the asymmetric function returns higher values when the first set of depth values are greater than the second set of depth values as compared to when the first set of depth values are less than the second set of depth values.);
Lukierski fails to teach:
based on a depth of the first depth buffer, calculating a direction from a virtual camera to a position in the virtual space relating to the pixel of interest as an incidence direction (Hughes [Pg 2 Par 2] in rendering systems using ray-tracing, the simulation is performed by emitting virtual or simulated "rays" from the viewing position such as a virtual camera and determining what interactions they undergo when they hit an object or a light source (the latter being either directly or after having hit an object.),
and calculating as a ray tracing direction a direction of a ray reflected from the position as a reflection position (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.);
based on the depth of the first depth buffer, determining a collision position where the ray collides with an object in the virtual space (Woo [Pg 7 Par 9] In addition, the ray tracing apparatus 500 may generate a first ray using ray information through the software unit 510 and calculate a collision point based on the first ray and collision triangle information (step S650). The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).),
and if the collision position is determined in a range where the tracing distance is less than or equal to the first distance, determining a color based on a color of a pixel in the frame buffer relating to the collision position as a reflected appearance color to be added to a color of the pixel of interest (Woo [Pg 7 Par 9] In addition, the ray tracing apparatus 500 may generate a first ray using ray information through the software unit 510 and calculate a collision point based on the first ray and collision triangle information (step S650). The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
Regarding claim 17:
Lukierski, Hughes, and Woo teaches:
The image processing apparatus according to claim 16,
wherein the image processing further comprises based on a parameter that differs between the collision position based on the first depth buffer and the collision position based on the second depth buffer, determining the reflected appearance color to be added to the color of the pixel of interest based on the color of the pixel in the frame buffer relating to the collision position (Woo [Pg 7 Par 9] The ray tracing apparatus 500 may calculate a color value on the collision point through the software unit 510 (step S660). When the color values for all pixels in the frame are determined through the software unit 510, the ray tracing apparatus 500 may output an image related to the frame (step S670).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
Regarding claim 18:
Lukierski, Hughes, and Woo teaches:
The image processing apparatus according to claim 16,
wherein the image processing further comprises: regarding a second type of object, placing a flat surface object in which an image rendered without performing drawing in the frame buffer is set as a texture at a position of the second type of object in the virtual space (Hughes [Pg 5 Par 2] At this point on the virtual object 250, the object surface's colour, orientation, reflectivity and other optical properties are taken into account.);
if the ray intersects the flat surface object, determining a reflected appearance color to be further added to the color of the pixel of interest based on a color of the texture at an intersection position of the intersection (Hughes [Pg 5 Par 5] As described, the ray-tracing process can be very computationally expensive and require powerful computing resources to carry out, particularly where the virtual scene includes many virtual objects and virtual light sources. Each virtual ray must be tested as to whether it intersects with each object in the scene.);
reflecting the reflected appearance color on the frame buffer (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.);
and after reflecting the reflected appearance color, drawing the second type of object in the frame buffer (Hughes [Pg 5 Par 3] In other examples, if the rays 260 impact another object, then depending upon the optical properties of the location 240 and its orientation, a reflection of that other object may be rendered at the surface of the object 250.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Lukierski with Hughes. Calculating ray tracing directions, reflections, and collision points, as in Hughes and Woo, would benefit the Lukierski teachings by allowing a way to use ray tracing to make more realistic rendering. Additionally, this is the application of a known technique, calculating ray tracing directions, reflections, and collision points, to yield predictable results.
Allowable Subject Matter
Claims 4, 5, 9, 10, 14, 15, 19, and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten to overcome the 35 U.S.C. 112(b) rejection of claims 1-20, and if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DENIS VASILIY MINKO whose telephone number is (571)270-5226. The examiner can normally be reached Monday-Thursday 8:30-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DENIS VASILIY MINKO/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights