DETAILED ACTION
Satus of claims:
1, 2, 4-11, 13-20 are pending
18-20 are new
3 and 12 are canceled
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 9, and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Douady et al. (US 20220256076) in view of Wang et al. (CN 111145103) and Zhang et al. (CN 113313808).
Regarding claim 1.
Douady teaches:
determining, based on a high dynamic range (HDR) original image of a target scene and a three-dimensional space model of the target scene, texture index information of the three-dimensional space model, wherein the texture index information is used to index HDR texture of the three-dimensional space model (Douady [0008] In a third aspect, the subject matter described in this specification can be embodied in systems that include an image sensor configured to capture image data using a plurality of selectable exposure times; and a processing apparatus that is configured to: receive a first image from the image sensor, where the first image is captured with a first exposure time; receive a second image from the image sensor, where the second image is captured with a second exposure time that is less than the first exposure time; determine a high dynamic range image based on the first image in a raw format and the second image in a raw format, in which an image portion of the high dynamic range image is based on a corresponding image portion of the second image when a pixel of a corresponding image portion of the first image is saturated; and store, display, or transmit an output image that is based on the high dynamic range image. [0330] For example, the ISP/stitch engine 2122 can perform edge detection, texture analysis, color analysis, depth analysis, and/or any other suitable operations in order to identify the portions of the images representative of the same field of view. [0072] In some implementations, the user interface device 120 may display, or otherwise present, content, such as images or video, acquired by the image capture apparatus 110. For example, a display of the user interface device 120 may be a viewport into the three-dimensional space represented by the panoramic images or video captured or created by the image capture apparatus 110.);
wherein the three-dimensional space model comprises a plurality of surface points (Douady [0051] Three-dimensional noise reduction processing may be implemented to reduce noise levels (e.g., standard deviation, variance, or signal-to-noise-ratio) in pixel values in a sequence of captured images (e.g., frames of video) and improve the quality of the captured images. Three-dimensional noise reduction processing may include temporal noise reduction processing, which combines (e.g., using weighted averages) pixel values for an incoming current image with pixel values for corresponding pixels of a recirculated image that may be based on (e.g., via recursive processing of incoming current images in the sequence of images) one or more previous images in the sequence of images. Whether and/or how significantly an image portion (e.g., a pixel or block of pixels) of the recirculated image is combined with the current image may be determined (e.g., by determining mixing weights for respective image portions) based on an assessment as to how well the image portion corresponds to an image portion of the current image at the same spatial location.),
generating an HDR map of the three-dimensional space model based on the texture index information of the three-dimensional space model (Douady [0222] The technique 1200 includes determining 1210 initial blending ratios for respective image portions of the high dynamic range image (e.g., a current image to be input to a temporal noise reduction module) to obtain an initial blending ratio map.);
to obtain a corresponding noise illuminance value, thereby generating a noise illuminance map corresponding to the plurality of surface points of the three-dimensional space model; (Douady [0062] The image capture apparatus 110 may obtain, or capture, image content, such as images, video, or both, with a 360° field-of-view, which may be referred to herein as panoramic or spherical content. For example, each of the image capture devices 130, 132, 134 may include respective lenses, for receiving and focusing light, and respective image sensors for converting the received and focused light to an image signal, such as by measuring or sampling the light, and the multiple image capture devices 130, 132, 134 may be arranged such that respective image sensors and lenses capture a combined field-of-view characterized by a spherical or near spherical field-of-view. [0054] To accommodate noise reduction processing occurring later in an image processing pipeline, estimates of noise level for respective image portions (e.g., pixels or Bayer blocks of 4 pixels), which are used by a noise reduction module to filter out noise, may be determined and stored in an input noise map for the high dynamic range image that may be passed into a noise reduction module with a corresponding current high dynamic range image. In some implementations, determining an input noise map for the high dynamic range image has small (e.g., marginal) impact on the consumption of image processing resource, and thus the savings due to downstream processing for a single high dynamic ranges images is substantially preserved jeopardized.);
Douady fails to teach:
performing denoising processing on the noise illuminance map of the three-dimensional space model by using a pre-trained Monte-Carlo-denoising neural network to obtain a noise-free illuminance map of the three-dimensional space model (Wang [Pg 1 Par 3] Monte Carlo denoising method, can be divided into Monte Carlo based on Monte Carlo method for de-noising image space and de-noising method based on machine learning.); and
storing the three-dimensional space model, the HDR map, and the noise-free illuminance map as a global illumination representation of the target scene (Wang [Pg 2 Par 4] The purpose of the invention is to provide a detail preservation Monte Carlo-based neural network model de-noising method, by using neural network structure comprises a feature extractor and nucleic predictor, adding the light transmit covariance in the path space in the auxiliary feature input buffer area, and increase the perceived loss in the loss function for training the network, so that the denoising result can keep better geometric details and illumination details.).
Wang teaches:
performing denoising processing on the noise illuminance map of the three-dimensional space model by using a pre-trained Monte-Carlo-denoising neural network to obtain a noise-free illuminance map of the three-dimensional space model (Wang [Pg 1 Par 3] Monte Carlo denoising method, can be divided into Monte Carlo based on Monte Carlo method for de-noising image space and de-noising method based on machine learning.); and
storing the three-dimensional space model, the HDR map, and the noise-free illuminance map as a global illumination representation of the target scene (Wang [Pg 2 Par 4] The purpose of the invention is to provide a detail preservation Monte Carlo-based neural network model de-noising method, by using neural network structure comprises a feature extractor and nucleic predictor, adding the light transmit covariance in the path space in the auxiliary feature input buffer area, and increase the perceived loss in the loss function for training the network, so that the denoising result can keep better geometric details and illumination details.).
wherein the HDR map comprises a plurality of pixel points, with each pixel point of the HDR map comprises a HDR value for a surface point in the three-dimensional space model (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image;);
sampling and integrating, for each surface point in the three-dimensional space model, the HDR map along hemispheric directions of the respective surface point (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image; [Pg 8 Par 5] In the above step 102, the off-line rendering of the virtual bounding volume sampling technology can be realized. if the virtual surrounding body is a sphere, namely using the off-line rendering of hemispherical sampling technology, the lighting information of the light emitting object reflects the illumination information on the inner wall of the virtual surrounding body for sampling.)
Douady and Wang teach:
A method for global illumination representation in an indoor scene (Douady [0189] The high dynamic range images 844 may include image portions captured with multiple different exposure times. Since noise levels for pixels can depend on exposure time, the high dynamic range images 844 may have different estimates of noise levels in different image portions that vary dynamically between successive high dynamic range images 844 based on the brightness patterns in a captured scene.) (Wang [Pg 2 Par 4] The purpose of the invention is to provide a detail preservation Monte Carlo-based neural network model de-noising method, by using neural network structure comprises a feature extractor and nucleic predictor, adding the light transmit covariance in the path space in the auxiliary feature input buffer area, and increase the perceived loss in the loss function for training the network, so that the denoising result can keep better geometric details and illumination details.), comprising:
Zhang teaches:
wherein the HDR map comprises a plurality of pixel points, with each pixel point of the HDR map comprises a HDR value for a surface point in the three-dimensional space model (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image;);
sampling and integrating, for each surface point in the three-dimensional space model, the HDR map along hemispheric directions of the respective surface point (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image; [Pg 8 Par 5] In the above step 102, the off-line rendering of the virtual bounding volume sampling technology can be realized. if the virtual surrounding body is a sphere, namely using the off-line rendering of hemispherical sampling technology, the lighting information of the light emitting object reflects the illumination information on the inner wall of the virtual surrounding body for sampling.)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady with Wang and Zhang. Having a denoising model to get illuminance, and generating HDR images/maps, as in Wang and Zhang, would benefit the Douady teachings by allowing the HDR image to get denoised with it’s illuminance. Additionally, this is the application of a known technique, Having a denoising model to get illuminance, and generating HDR images/maps and using monte-carlo denoising method, to yield predictable results.
Regarding claim 9.
Douady teaches:
a texture indexing module, configured to determine, based on a high dynamic range (HDR) original image of a target scene and a three-dimensional space model of the target scene, texture index information of the three-dimensional space model, wherein the texture index information is used to index HDR texture of the three-dimensional space model (Douady [0008] In a third aspect, the subject matter described in this specification can be embodied in systems that include an image sensor configured to capture image data using a plurality of selectable exposure times; and a processing apparatus that is configured to: receive a first image from the image sensor, where the first image is captured with a first exposure time; receive a second image from the image sensor, where the second image is captured with a second exposure time that is less than the first exposure time; determine a high dynamic range image based on the first image in a raw format and the second image in a raw format, in which an image portion of the high dynamic range image is based on a corresponding image portion of the second image when a pixel of a corresponding image portion of the first image is saturated; and store, display, or transmit an output image that is based on the high dynamic range image. [0330] For example, the ISP/stitch engine 2122 can perform edge detection, texture analysis, color analysis, depth analysis, and/or any other suitable operations in order to identify the portions of the images representative of the same field of view. [0072] In some implementations, the user interface device 120 may display, or otherwise present, content, such as images or video, acquired by the image capture apparatus 110. For example, a display of the user interface device 120 may be a viewport into the three-dimensional space represented by the panoramic images or video captured or created by the image capture apparatus 110.);
wherein the three-dimensional space model comprises a plurality of surface points (Douady [0051] Three-dimensional noise reduction processing may be implemented to reduce noise levels (e.g., standard deviation, variance, or signal-to-noise-ratio) in pixel values in a sequence of captured images (e.g., frames of video) and improve the quality of the captured images. Three-dimensional noise reduction processing may include temporal noise reduction processing, which combines (e.g., using weighted averages) pixel values for an incoming current image with pixel values for corresponding pixels of a recirculated image that may be based on (e.g., via recursive processing of incoming current images in the sequence of images) one or more previous images in the sequence of images. Whether and/or how significantly an image portion (e.g., a pixel or block of pixels) of the recirculated image is combined with the current image may be determined (e.g., by determining mixing weights for respective image portions) based on an assessment as to how well the image portion corresponds to an image portion of the current image at the same spatial location.),
a texture generating module, configured to generate an HDR map of the three-dimensional space model based on the texture index information of the three-dimensional space model (Douady [0222] The technique 1200 includes determining 1210 initial blending ratios for respective image portions of the high dynamic range image (e.g., a current image to be input to a temporal noise reduction module) to obtain an initial blending ratio map.);
to obtain a corresponding noise illuminance value, thereby generating a noise illuminance map corresponding to the plurality of surface points of the three-dimensional space model; (Douady [0062] The image capture apparatus 110 may obtain, or capture, image content, such as images, video, or both, with a 360° field-of-view, which may be referred to herein as panoramic or spherical content. For example, each of the image capture devices 130, 132, 134 may include respective lenses, for receiving and focusing light, and respective image sensors for converting the received and focused light to an image signal, such as by measuring or sampling the light, and the multiple image capture devices 130, 132, 134 may be arranged such that respective image sensors and lenses capture a combined field-of-view characterized by a spherical or near spherical field-of-view. [0054] To accommodate noise reduction processing occurring later in an image processing pipeline, estimates of noise level for respective image portions (e.g., pixels or Bayer blocks of 4 pixels), which are used by a noise reduction module to filter out noise, may be determined and stored in an input noise map for the high dynamic range image that may be passed into a noise reduction module with a corresponding current high dynamic range image. In some implementations, determining an input noise map for the high dynamic range image has small (e.g., marginal) impact on the consumption of image processing resource, and thus the savings due to downstream processing for a single high dynamic ranges images is substantially preserved jeopardized.);
Douady fails to teach:
a denoising module, configured to perform denoising processing on the noise illuminance map of the three-dimensional space model by using a pre-trained Monte-Carlo-denoising neural network to obtain a noise-free illuminance map of the three-dimensional space model (Wang [Pg 1 Par 3] Monte Carlo denoising method, can be divided into Monte Carlo based on Monte Carlo method for de-noising image space and de-noising method based on machine learning.); and
a storing module, configured to store the three-dimensional space model, the HDR map, and the noise-free illuminance map as a global illumination representation of the target scene (Wang [Pg 2 Par 4] The purpose of the invention is to provide a detail preservation Monte Carlo-based neural network model de-noising method, by using neural network structure comprises a feature extractor and nucleic predictor, adding the light transmit covariance in the path space in the auxiliary feature input buffer area, and increase the perceived loss in the loss function for training the network, so that the denoising result can keep better geometric details and illumination details.).
wherein the HDR map comprises a plurality of pixel points, with each pixel point of the HDR map comprises a HDR value for a surface point in the three-dimensional space model (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image;);
sampling and integrating, for each surface point in the three-dimensional space model, the HDR map along hemispheric directions of the respective surface point (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image; [Pg 8 Par 5] In the above step 102, the off-line rendering of the virtual bounding volume sampling technology can be realized. if the virtual surrounding body is a sphere, namely using the off-line rendering of hemispherical sampling technology, the lighting information of the light emitting object reflects the illumination information on the inner wall of the virtual surrounding body for sampling.)
Wang teaches:
a denoising module, configured to perform denoising processing on the noise illuminance map of the three-dimensional space model by using a pre-trained Monte-Carlo-denoising neural network to obtain a noise-free illuminance map of the three-dimensional space model (Wang [Pg 1 Par 3] Monte Carlo denoising method, can be divided into Monte Carlo based on Monte Carlo method for de-noising image space and de-noising method based on machine learning.); and
a storing module, configured to store the three-dimensional space model, the HDR map, and the noise-free illuminance map as a global illumination representation of the target scene (Wang [Pg 2 Par 4] The purpose of the invention is to provide a detail preservation Monte Carlo-based neural network model de-noising method, by using neural network structure comprises a feature extractor and nucleic predictor, adding the light transmit covariance in the path space in the auxiliary feature input buffer area, and increase the perceived loss in the loss function for training the network, so that the denoising result can keep better geometric details and illumination details.).
Douady and Wang teach:
A device for global illumination in an indoor scene, (Douady [0189] The high dynamic range images 844 may include image portions captured with multiple different exposure times. Since noise levels for pixels can depend on exposure time, the high dynamic range images 844 may have different estimates of noise levels in different image portions that vary dynamically between successive high dynamic range images 844 based on the brightness patterns in a captured scene. [0097] In some implementations, the electronic storage unit 224 may include a system memory module that may store executable computer instructions that, when executed by the processor 222, perform various functionalities including those described herein.) (Wang [Pg 2 Par 4] The purpose of the invention is to provide a detail preservation Monte Carlo-based neural network model de-noising method, by using neural network structure comprises a feature extractor and nucleic predictor, adding the light transmit covariance in the path space in the auxiliary feature input buffer area, and increase the perceived loss in the loss function for training the network, so that the denoising result can keep better geometric details and illumination details.), comprising:
Zhang teaches:
wherein the HDR map comprises a plurality of pixel points, with each pixel point of the HDR map comprises a HDR value for a surface point in the three-dimensional space model (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image;);
sampling and integrating, for each surface point in the three-dimensional space model, the HDR map along hemispheric directions of the respective surface point (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image; [Pg 8 Par 5] In the above step 102, the off-line rendering of the virtual bounding volume sampling technology can be realized. if the virtual surrounding body is a sphere, namely using the off-line rendering of hemispherical sampling technology, the lighting information of the light emitting object reflects the illumination information on the inner wall of the virtual surrounding body for sampling.)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady with Wang and Zhang. Having a denoising model to get illuminance, and generating HDR images/maps, as in Wang and Zhang, would benefit the Douady teachings by allowing the HDR image to get denoised with it’s illuminance. Additionally, this is the application of a known technique, Having a denoising model to get illuminance, and generating HDR images/maps and using monte-carlo denoising method, to yield predictable results.
Regarding claim 10.
Douady teaches:
determining, based on a high dynamic range (HDR) original image of a target scene and a three-dimensional space model of the target scene, texture index information of the three-dimensional space model, wherein the texture index information is used to index HDR texture of the three-dimensional space model (Douady [0008] In a third aspect, the subject matter described in this specification can be embodied in systems that include an image sensor configured to capture image data using a plurality of selectable exposure times; and a processing apparatus that is configured to: receive a first image from the image sensor, where the first image is captured with a first exposure time; receive a second image from the image sensor, where the second image is captured with a second exposure time that is less than the first exposure time; determine a high dynamic range image based on the first image in a raw format and the second image in a raw format, in which an image portion of the high dynamic range image is based on a corresponding image portion of the second image when a pixel of a corresponding image portion of the first image is saturated; and store, display, or transmit an output image that is based on the high dynamic range image. [0330] For example, the ISP/stitch engine 2122 can perform edge detection, texture analysis, color analysis, depth analysis, and/or any other suitable operations in order to identify the portions of the images representative of the same field of view. [0072] In some implementations, the user interface device 120 may display, or otherwise present, content, such as images or video, acquired by the image capture apparatus 110. For example, a display of the user interface device 120 may be a viewport into the three-dimensional space represented by the panoramic images or video captured or created by the image capture apparatus 110.);
wherein the three-dimensional space model comprises a plurality of surface points (Douady [0051] Three-dimensional noise reduction processing may be implemented to reduce noise levels (e.g., standard deviation, variance, or signal-to-noise-ratio) in pixel values in a sequence of captured images (e.g., frames of video) and improve the quality of the captured images. Three-dimensional noise reduction processing may include temporal noise reduction processing, which combines (e.g., using weighted averages) pixel values for an incoming current image with pixel values for corresponding pixels of a recirculated image that may be based on (e.g., via recursive processing of incoming current images in the sequence of images) one or more previous images in the sequence of images. Whether and/or how significantly an image portion (e.g., a pixel or block of pixels) of the recirculated image is combined with the current image may be determined (e.g., by determining mixing weights for respective image portions) based on an assessment as to how well the image portion corresponds to an image portion of the current image at the same spatial location.),
generating an HDR map of the three-dimensional space model based on the texture index information of the three-dimensional space model (Douady [0222] The technique 1200 includes determining 1210 initial blending ratios for respective image portions of the high dynamic range image (e.g., a current image to be input to a temporal noise reduction module) to obtain an initial blending ratio map.);
to obtain a corresponding noise illuminance value, thereby generating a noise illuminance map corresponding to the plurality of surface points of the three-dimensional space model; (Douady [0062] The image capture apparatus 110 may obtain, or capture, image content, such as images, video, or both, with a 360° field-of-view, which may be referred to herein as panoramic or spherical content. For example, each of the image capture devices 130, 132, 134 may include respective lenses, for receiving and focusing light, and respective image sensors for converting the received and focused light to an image signal, such as by measuring or sampling the light, and the multiple image capture devices 130, 132, 134 may be arranged such that respective image sensors and lenses capture a combined field-of-view characterized by a spherical or near spherical field-of-view. [0054] To accommodate noise reduction processing occurring later in an image processing pipeline, estimates of noise level for respective image portions (e.g., pixels or Bayer blocks of 4 pixels), which are used by a noise reduction module to filter out noise, may be determined and stored in an input noise map for the high dynamic range image that may be passed into a noise reduction module with a corresponding current high dynamic range image. In some implementations, determining an input noise map for the high dynamic range image has small (e.g., marginal) impact on the consumption of image processing resource, and thus the savings due to downstream processing for a single high dynamic ranges images is substantially preserved jeopardized.);
Douady fails to teach:
performing denoising processing on the noise illuminance map of the three-dimensional space model by using a pre-trained Monte-Carlo-denoising neural network to obtain a noise-free illuminance map of the three-dimensional space model (Wang [Pg 1 Par 3] Monte Carlo denoising method, can be divided into Monte Carlo based on Monte Carlo method for de-noising image space and de-noising method based on machine learning.); and
storing the three-dimensional space model, the HDR map, and the noise-free illuminance map as a global illumination representation of the target scene (Wang [Pg 2 Par 4] The purpose of the invention is to provide a detail preservation Monte Carlo-based neural network model de-noising method, by using neural network structure comprises a feature extractor and nucleic predictor, adding the light transmit covariance in the path space in the auxiliary feature input buffer area, and increase the perceived loss in the loss function for training the network, so that the denoising result can keep better geometric details and illumination details.).
wherein the HDR map comprises a plurality of pixel points, with each pixel point of the HDR map comprises a HDR value for a surface point in the three-dimensional space model (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image;);
sampling and integrating, for each surface point in the three-dimensional space model, the HDR map along hemispheric directions of the respective surface point (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image; [Pg 8 Par 5] In the above step 102, the off-line rendering of the virtual bounding volume sampling technology can be realized. if the virtual surrounding body is a sphere, namely using the off-line rendering of hemispherical sampling technology, the lighting information of the light emitting object reflects the illumination information on the inner wall of the virtual surrounding body for sampling.)
Wang teaches:
performing denoising processing on the noise illuminance map of the three-dimensional space model by using a pre-trained Monte-Carlo-denoising neural network to obtain a noise-free illuminance map of the three-dimensional space model (Wang [Pg 1 Par 3] Monte Carlo denoising method, can be divided into Monte Carlo based on Monte Carlo method for de-noising image space and de-noising method based on machine learning.); and
storing the three-dimensional space model, the HDR map, and the noise-free illuminance map as a global illumination representation of the target scene (Wang [Pg 2 Par 4] The purpose of the invention is to provide a detail preservation Monte Carlo-based neural network model de-noising method, by using neural network structure comprises a feature extractor and nucleic predictor, adding the light transmit covariance in the path space in the auxiliary feature input buffer area, and increase the perceived loss in the loss function for training the network, so that the denoising result can keep better geometric details and illumination details.).
Douady and Wang teach:
An electronic device, comprising: a memory, configured to store computer programs; and a processor, configured to execute a computer program stored in the memory, and when the computer program is executed, perform: (Douady [0189] The high dynamic range images 844 may include image portions captured with multiple different exposure times. Since noise levels for pixels can depend on exposure time, the high dynamic range images 844 may have different estimates of noise levels in different image portions that vary dynamically between successive high dynamic range images 844 based on the brightness patterns in a captured scene. [0097] In some implementations, the electronic storage unit 224 may include a system memory module that may store executable computer instructions that, when executed by the processor 222, perform various functionalities including those described herein.) (Wang [Pg 2 Par 4] The purpose of the invention is to provide a detail preservation Monte Carlo-based neural network model de-noising method, by using neural network structure comprises a feature extractor and nucleic predictor, adding the light transmit covariance in the path space in the auxiliary feature input buffer area, and increase the perceived loss in the loss function for training the network, so that the denoising result can keep better geometric details and illumination details.), comprising:
Zhang teaches:
wherein the HDR map comprises a plurality of pixel points, with each pixel point of the HDR map comprises a HDR value for a surface point in the three-dimensional space model (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image;);
sampling and integrating, for each surface point in the three-dimensional space model, the HDR map along hemispheric directions of the respective surface point (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image; [Pg 8 Par 5] In the above step 102, the off-line rendering of the virtual bounding volume sampling technology can be realized. if the virtual surrounding body is a sphere, namely using the off-line rendering of hemispherical sampling technology, the lighting information of the light emitting object reflects the illumination information on the inner wall of the virtual surrounding body for sampling.)
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady with Wang and Zhang. Having a denoising model to get illuminance, and generating HDR images/maps, as in Wang and Zhang, would benefit the Douady teachings by allowing the HDR image to get denoised with it’s illuminance. Additionally, this is the application of a known technique, Having a denoising model to get illuminance, and generating HDR images/maps and using monte-carlo denoising method, to yield predictable results.
Claim(s) 2, 11, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Douady et al. (US 20220256076) in view of Wang et al. (CN 111145103), Zhang et al. (CN 113313808), and Quan et al. (CN 106296828), and Petkov et al. (US 20180260997).
Regarding claim 2.
Douady, Wang, and Zhang teach:
The method of claim 1,
generating, based on the illuminance of each surface point of the plurality of surface points, a noise illuminance map of the three-dimensional space model (Douady [0062] The image capture apparatus 110 may obtain, or capture, image content, such as images, video, or both, with a 360° field-of-view, which may be referred to herein as panoramic or spherical content. For example, each of the image capture devices 130, 132, 134 may include respective lenses, for receiving and focusing light, and respective image sensors for converting the received and focused light to an image signal, such as by measuring or sampling the light, and the multiple image capture devices 130, 132, 134 may be arranged such that respective image sensors and lenses capture a combined field-of-view characterized by a spherical or near spherical field-of-view. [0054] To accommodate noise reduction processing occurring later in an image processing pipeline, estimates of noise level for respective image portions (e.g., pixels or Bayer blocks of 4 pixels), which are used by a noise reduction module to filter out noise, may be determined and stored in an input noise map for the high dynamic range image that may be passed into a noise reduction module with a corresponding current high dynamic range image. In some implementations, determining an input noise map for the high dynamic range image has small (e.g., marginal) impact on the consumption of image processing resource, and thus the savings due to downstream processing for a single high dynamic ranges images is substantially preserved jeopardized.).
Douady, Wang, and Zhang fail to teach:
wherein sampling and integrating the HDR map in the hemispheric direction of any surface point in the three-dimensional space model to obtain the noise illuminance map of the three-dimensional space model comprises:
determining, based on any pixel point of the HDR map, the spatial coordinates of the corresponding spatial point in the three-dimensional space model (Quan [Pg 3 par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter.);
sampling and integrating, using a Monte-Carlo random sampling algorithm and the spatial coordinates of the corresponding spatial point, the HDR map in the hemispheric direction of the corresponding spatial point to obtain illuminance of the corresponding spatial point (Quan [Pg 3 par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter) (Petkov [[0079] Figure 4 is a flow chart diagram of one embodiment of a method for three-dimensional rendering in a rendering system. Due to the time needed to render (e.g., seconds or minutes) using physically-based rendering (e.g., global illumination volume rendering using Monte Carlo sampling), interactive performance of the rendering is limited. While this computationally expensive rendering may produce high quality images with improved perception of depth cues, the rendering uses a high number of samples per pixel to compute the rendering equation.); and
Quan teaches:
wherein sampling and integrating, for each surface point in the three-dimensional space model, the HDR map along hemispheric directions of the respective surface point to obtain the noise illuminance value, thereby generating the noise illuminance map corresponding to the plurality of surface points of the three-dimensional space model comprises:
determining, the spatial coordinates of the respective surface point in the three-dimensional space model (Quan [Pg 3 Par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter.);
Quan and Petkov teach:
sampling and integrating, using a Monte-Carlo random sampling algorithm and the spatial coordinates of the respective surface, the HDR map in the hemispheric direction of the corresponding spatial point to obtain illuminance of the corresponding spatial point (Quan [Pg 3 Par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter) (Petkov [0079] Figure 4 is a flow chart diagram of one embodiment of a method for three-dimensional rendering in a rendering system. Due to the time needed to render (e.g., seconds or minutes) using physically-based rendering (e.g., global illumination volume rendering using Monte Carlo sampling), interactive performance of the rendering is limited. While this computationally expensive rendering may produce high quality images with improved perception of depth cues, the rendering uses a high number of samples per pixel to compute the rendering equation.); and
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Quan and Petkov. Getting coordinates to get illuminance and using monte-carlo algorithm, as in Quan and Petkov, would benefit the Douady, Wang and Zhang teachings by allowing to get certain parts of a scene while using a monte-carlo algorithm. Additionally this is the application of a known technique, getting coordinates to get illuminance and using monte-carlo algorithm, to yield predictable results.
Regarding claim 11.
Douady, Wang, and Zhang teach:
The electronic device of claim 10,
generating, based on the illuminance of each surface point of the plurality of surface points, a noise illuminance map of the three-dimensional space model (Douady [0062] The image capture apparatus 110 may obtain, or capture, image content, such as images, video, or both, with a 360° field-of-view, which may be referred to herein as panoramic or spherical content. For example, each of the image capture devices 130, 132, 134 may include respective lenses, for receiving and focusing light, and respective image sensors for converting the received and focused light to an image signal, such as by measuring or sampling the light, and the multiple image capture devices 130, 132, 134 may be arranged such that respective image sensors and lenses capture a combined field-of-view characterized by a spherical or near spherical field-of-view. [0054] To accommodate noise reduction processing occurring later in an image processing pipeline, estimates of noise level for respective image portions (e.g., pixels or Bayer blocks of 4 pixels), which are used by a noise reduction module to filter out noise, may be determined and stored in an input noise map for the high dynamic range image that may be passed into a noise reduction module with a corresponding current high dynamic range image. In some implementations, determining an input noise map for the high dynamic range image has small (e.g., marginal) impact on the consumption of image processing resource, and thus the savings due to downstream processing for a single high dynamic ranges images is substantially preserved jeopardized.).
Douady, Wang, and Zhang fail to teach:
wherein sampling and integrating the HDR map in the hemispheric direction of any surface point in the three-dimensional space model to obtain the noise illuminance map of the three-dimensional space model comprises:
determining, based on any pixel point of the HDR map, the spatial coordinates of the corresponding spatial point in the three-dimensional space model (Quan [Pg 3 par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter.);
sampling and integrating, using a Monte-Carlo random sampling algorithm and the spatial coordinates of the corresponding spatial point, the HDR map in the hemispheric direction of the corresponding spatial point to obtain illuminance of the corresponding spatial point (Quan [Pg 3 par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter) (Petkov [[0079] Figure 4 is a flow chart diagram of one embodiment of a method for three-dimensional rendering in a rendering system. Due to the time needed to render (e.g., seconds or minutes) using physically-based rendering (e.g., global illumination volume rendering using Monte Carlo sampling), interactive performance of the rendering is limited. While this computationally expensive rendering may produce high quality images with improved perception of depth cues, the rendering uses a high number of samples per pixel to compute the rendering equation.); and
Quan teaches:
wherein sampling and integrating, for each surface point in the three-dimensional space model, the HDR map along hemispheric directions of the respective surface point to obtain the noise illuminance value, thereby generating the noise illuminance map corresponding to the plurality of surface points of the three-dimensional space model comprises:
determining, the spatial coordinates of the respective surface point in the three-dimensional space model (Quan [Pg 3 Par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter.);
Quan and Petkov teach:
sampling and integrating, using a Monte-Carlo random sampling algorithm and the spatial coordinates of the respective surface, the HDR map in the hemispheric direction of the corresponding spatial point to obtain illuminance of the corresponding spatial point (Quan [Pg 3 Par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter) (Petkov [0079] Figure 4 is a flow chart diagram of one embodiment of a method for three-dimensional rendering in a rendering system. Due to the time needed to render (e.g., seconds or minutes) using physically-based rendering (e.g., global illumination volume rendering using Monte Carlo sampling), interactive performance of the rendering is limited. While this computationally expensive rendering may produce high quality images with improved perception of depth cues, the rendering uses a high number of samples per pixel to compute the rendering equation.); and
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Quan and Petkov. Getting coordinates to get illuminance and using monte-carlo algorithm, as in Quan and Petkov, would benefit the Douady, Wang and Zhang teachings by allowing to get certain parts of a scene while using a monte-carlo algorithm. Additionally this is the application of a known technique, getting coordinates to get illuminance and using monte-carlo algorithm, to yield predictable results.
Regarding claim 18.
Douady, Wang, and Zhang teach:
The electronic device of claim 10,
generating, based on the illuminance of each surface point of the plurality of surface points, a noise illuminance map of the three-dimensional space model (Douady [0062] The image capture apparatus 110 may obtain, or capture, image content, such as images, video, or both, with a 360° field-of-view, which may be referred to herein as panoramic or spherical content. For example, each of the image capture devices 130, 132, 134 may include respective lenses, for receiving and focusing light, and respective image sensors for converting the received and focused light to an image signal, such as by measuring or sampling the light, and the multiple image capture devices 130, 132, 134 may be arranged such that respective image sensors and lenses capture a combined field-of-view characterized by a spherical or near spherical field-of-view. [0054] To accommodate noise reduction processing occurring later in an image processing pipeline, estimates of noise level for respective image portions (e.g., pixels or Bayer blocks of 4 pixels), which are used by a noise reduction module to filter out noise, may be determined and stored in an input noise map for the high dynamic range image that may be passed into a noise reduction module with a corresponding current high dynamic range image. In some implementations, determining an input noise map for the high dynamic range image has small (e.g., marginal) impact on the consumption of image processing resource, and thus the savings due to downstream processing for a single high dynamic ranges images is substantially preserved jeopardized.).
Douady, Wang, and Zhang fail to teach:
wherein sampling and integrating the HDR map in the hemispheric direction of any surface point in the three-dimensional space model to obtain the noise illuminance map of the three-dimensional space model comprises:
determining, based on any pixel point of the HDR map, the spatial coordinates of the corresponding spatial point in the three-dimensional space model (Quan [Pg 3 par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter.);
sampling and integrating, using a Monte-Carlo random sampling algorithm and the spatial coordinates of the corresponding spatial point, the HDR map in the hemispheric direction of the corresponding spatial point to obtain illuminance of the corresponding spatial point (Quan [Pg 3 par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter) (Petkov [[0079] Figure 4 is a flow chart diagram of one embodiment of a method for three-dimensional rendering in a rendering system. Due to the time needed to render (e.g., seconds or minutes) using physically-based rendering (e.g., global illumination volume rendering using Monte Carlo sampling), interactive performance of the rendering is limited. While this computationally expensive rendering may produce high quality images with improved perception of depth cues, the rendering uses a high number of samples per pixel to compute the rendering equation.); and
Quan teaches:
wherein sampling and integrating, for each surface point in the three-dimensional space model, the HDR map along hemispheric directions of the respective surface point to obtain the noise illuminance value, thereby generating the noise illuminance map corresponding to the plurality of surface points of the three-dimensional space model comprises:
determining, the spatial coordinates of the respective surface point in the three-dimensional space model (Quan [Pg 3 Par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter.);
Quan and Petkov teach:
sampling and integrating, using a Monte-Carlo random sampling algorithm and the spatial coordinates of the respective surface, the HDR map in the hemispheric direction of the corresponding spatial point to obtain illuminance of the corresponding spatial point (Quan [Pg 3 Par 7] Preferably, in step S1, extracts the centroid point position coordinate of each scene target external rectangular boundary coordinates, representing a scene target spatial pattern of point-line coordinate, texture mapping relation, illumination intensity and direction as the scene target parameter) (Petkov [0079] Figure 4 is a flow chart diagram of one embodiment of a method for three-dimensional rendering in a rendering system. Due to the time needed to render (e.g., seconds or minutes) using physically-based rendering (e.g., global illumination volume rendering using Monte Carlo sampling), interactive performance of the rendering is limited. While this computationally expensive rendering may produce high quality images with improved perception of depth cues, the rendering uses a high number of samples per pixel to compute the rendering equation.); and
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Quan and Petkov. Getting coordinates to get illuminance and using monte-carlo algorithm, as in Quan and Petkov, would benefit the Douady, Wang and Zhang teachings by allowing to get certain parts of a scene while using a monte-carlo algorithm. Additionally this is the application of a known technique, getting coordinates to get illuminance and using monte-carlo algorithm, to yield predictable results.
Claim(s) 4 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Douady et al. (US 20220256076) in view of Wang et al. (CN 111145103), Zhang et al. (CN 113313808) and Liu et al. (CN 113592999).
Regarding claim 4.
Douady, Wang, and Zhang teach:
The method of claim 1,
Douady, Wang, and Zhang fail to teach:
further comprising: receiving an illuminance query instruction, wherein the illuminance query instruction carries position coordinates of the illuminance point to be queried; and obtaining, based on the position coordinates of the illuminance point to be queried, the illuminance of the illuminance point to be queried from the noise-free illuminance map (Liu [Pg 8 par 10] Specifically, when the virtual illuminant point value determined by each pixel point corresponding to the preset rendering model is reset, the point value exceeding the preset threshold value can be set as the same preset value; For example, when the reset virtual illuminant point value is between 0-1, can be more than or equal to 0.9995 all virtual illuminant point value is set as 1, then there is a plurality of virtual illuminant point value of 1. at this time, the virtual illuminant point value is 1 corresponding to the position of the pixel point is determined as the virtual illuminant corresponding to the pattern core position (through the graphical core information to represent, specifically the coordinate value of the pixel point).).
Liu teaches:
further comprising: receiving an illuminance query instruction, wherein the illuminance query instruction carries position coordinates of the illuminance point to be queried; and obtaining, based on the position coordinates of the illuminance point to be queried, the illuminance of the illuminance point to be queried from the noise-free illuminance map (Liu [Pg 8 par 10] Specifically, when the virtual illuminant point value determined by each pixel point corresponding to the preset rendering model is reset, the point value exceeding the preset threshold value can be set as the same preset value; For example, when the reset virtual illuminant point value is between 0-1, can be more than or equal to 0.9995 all virtual illuminant point value is set as 1, then there is a plurality of virtual illuminant point value of 1. at this time, the virtual illuminant point value is 1 corresponding to the position of the pixel point is determined as the virtual illuminant corresponding to the pattern core position (through the graphical core information to represent, specifically the coordinate value of the pixel point).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Liu. Receiving an illuminance point, as in Liu, would benefit the Douady, Wang and Zhang teachings by to get coordinates of a point. Additionally this is the application of a known technique, receiving an illuminance point, to yield predictable results.
Regarding claim 13.
Douady, Wang, and Zhang teach:
The electronic device of claim 10,
Douady, Wang, and Zhang fail to teach:
further comprising: receiving an illuminance query instruction, wherein the illuminance query instruction carries position coordinates of the illuminance point to be queried; and obtaining, based on the position coordinates of the illuminance point to be queried, the illuminance of the illuminance point to be queried from the noise-free illuminance map (Liu [Pg 8 par 10] Specifically, when the virtual illuminant point value determined by each pixel point corresponding to the preset rendering model is reset, the point value exceeding the preset threshold value can be set as the same preset value; For example, when the reset virtual illuminant point value is between 0-1, can be more than or equal to 0.9995 all virtual illuminant point value is set as 1, then there is a plurality of virtual illuminant point value of 1. at this time, the virtual illuminant point value is 1 corresponding to the position of the pixel point is determined as the virtual illuminant corresponding to the pattern core position (through the graphical core information to represent, specifically the coordinate value of the pixel point).).
Liu teaches:
further comprising: receiving an illuminance query instruction, wherein the illuminance query instruction carries position coordinates of the illuminance point to be queried; and obtaining, based on the position coordinates of the illuminance point to be queried, the illuminance of the illuminance point to be queried from the noise-free illuminance map (Liu [Pg 8 par 10] Specifically, when the virtual illuminant point value determined by each pixel point corresponding to the preset rendering model is reset, the point value exceeding the preset threshold value can be set as the same preset value; For example, when the reset virtual illuminant point value is between 0-1, can be more than or equal to 0.9995 all virtual illuminant point value is set as 1, then there is a plurality of virtual illuminant point value of 1. at this time, the virtual illuminant point value is 1 corresponding to the position of the pixel point is determined as the virtual illuminant corresponding to the pattern core position (through the graphical core information to represent, specifically the coordinate value of the pixel point).).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Liu. Receiving an illuminance point, as in Liu, would benefit the Douady, Wang and Zhang teachings by to get coordinates of a point. Additionally this is the application of a known technique, receiving an illuminance point, to yield predictable results.
Claim(s) 5 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Douady et al. (US 20220256076) in view of Wang et al. (CN 111145103), Zhang et al. (CN 113313808), Liu et al. (CN 113592999), Johannesson et al. (CN 113924596), and Chen et al. (US 8340453).
Regarding claim 5.
Douady, Wang, and Zhang teach:
The method of claim 1,
Douady, Wang, and Zhang fail to teach:
further comprising: receiving an illuminance query instruction, wherein the illuminance query instruction carries position coordinates of the illuminance point to be queried and the querying method (Liu [Pg 8 par 10] Specifically, when the virtual illuminant point value determined by each pixel point corresponding to the preset rendering model is reset, the point value exceeding the preset threshold value can be set as the same preset value; For example, when the reset virtual illuminant point value is between 0-1, can be more than or equal to 0.9995 all virtual illuminant point value is set as 1, then there is a plurality of virtual illuminant point value of 1. at this time, the virtual illuminant point value is 1 corresponding to the position of the pixel point is determined as the virtual illuminant corresponding to the pattern core position (through the graphical core information to represent, specifically the coordinate value of the pixel point).);
obtaining, using a light diffraction algorithm, an intersection point of the illuminance point to be queried with the three-dimensional space model in the query direction (Johannesson [Pg 5 Par 8] As used herein, " reproduction (") " is generated to represent a virtual model or a part of the real (photocopy) or non-real (non-phototime) image. can be through conventional light projection or light tracking and also considering the rendering technology of diffraction, such as wave optics, GTD algorithm (diffraction geometry theory), PTD algorithm (diffraction physical theory), physical optical (PO), boundary element method (BEM) and so on, to generate reproduction. The reproduction may be two-dimensional or three-dimensional.); and
querying a HDR floating point value of the intersection point in the HDR map to obtain the HDR illumination of the illuminance point to be queried (Chen [0102] Since a floating-point number is used to represent the pixel values in an HDR image, an HDR image can represent much more detail across a wide range of illumination in a scene than can conventional 8-bit images.).
Liu teaches:
further comprising: receiving an illuminance query instruction, wherein the illuminance query instruction carries position coordinates of the illuminance point to be queried and the querying method (Liu [Pg 8 Par 10] Specifically, when the virtual illuminant point value determined by each pixel point corresponding to the preset rendering model is reset, the point value exceeding the preset threshold value can be set as the same preset value; For example, when the reset virtual illuminant point value is between 0-1, can be more than or equal to 0.9995 all virtual illuminant point value is set as 1, then there is a plurality of virtual illuminant point value of 1. at this time, the virtual illuminant point value is 1 corresponding to the position of the pixel point is determined as the virtual illuminant corresponding to the pattern core position (through the graphical core information to represent, specifically the coordinate value of the pixel point).);
Johannesson teaches:
obtaining, using a light diffraction algorithm, an intersection point of the illuminance point to be queried with the three-dimensional space model in the query direction (Johannesson [Pg 5 Par 8] As used herein, " reproduction (") " is generated to represent a virtual model or a part of the real (photocopy) or non-real (non-phototime) image. can be through conventional light projection or light tracking and also considering the rendering technology of diffraction, such as wave optics, GTD algorithm (diffraction geometry theory), PTD algorithm (diffraction physical theory), physical optical (PO), boundary element method (BEM) and so on, to generate reproduction. The reproduction may be two-dimensional or three-dimensional.); and
Chen teaches:
querying a HDR floating point value of the intersection point in the HDR map to obtain the HDR illumination of the illuminance point to be queried (Chen [0102] Since a floating-point number is used to represent the pixel values in an HDR image, an HDR image can represent much more detail across a wide range of illumination in a scene than can conventional 8-bit images.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Liu and Johannesson and Chen. Using a light diffraction algorithm, as in Liu and Johannesson and Chen, would benefit the Douady, Wang and Zhang teachings by allowing to get certain parts of using an algorithm. Additionally, this is the application of a known technique, using a light diffraction algorithm, to yield predictable results.
Regarding claim 14.
Douady, Wang, and Zhang teach:
The electronic device of claim 10,
Douady, Wang, and Zhang fail to teach:
further comprising: receiving an illuminance query instruction, wherein the illuminance query instruction carries position coordinates of the illuminance point to be queried and the querying method (Liu [Pg 8 par 10] Specifically, when the virtual illuminant point value determined by each pixel point corresponding to the preset rendering model is reset, the point value exceeding the preset threshold value can be set as the same preset value; For example, when the reset virtual illuminant point value is between 0-1, can be more than or equal to 0.9995 all virtual illuminant point value is set as 1, then there is a plurality of virtual illuminant point value of 1. at this time, the virtual illuminant point value is 1 corresponding to the position of the pixel point is determined as the virtual illuminant corresponding to the pattern core position (through the graphical core information to represent, specifically the coordinate value of the pixel point).);
obtaining, using a light diffraction algorithm, an intersection point of the illuminance point to be queried with the three-dimensional space model in the query direction (Johannesson [Pg 5 Par 8] As used herein, " reproduction (") " is generated to represent a virtual model or a part of the real (photocopy) or non-real (non-phototime) image. can be through conventional light projection or light tracking and also considering the rendering technology of diffraction, such as wave optics, GTD algorithm (diffraction geometry theory), PTD algorithm (diffraction physical theory), physical optical (PO), boundary element method (BEM) and so on, to generate reproduction. The reproduction may be two-dimensional or three-dimensional.); and
querying a HDR floating point value of the intersection point in the HDR map to obtain the HDR illumination of the illuminance point to be queried (Chen [0102] Since a floating-point number is used to represent the pixel values in an HDR image, an HDR image can represent much more detail across a wide range of illumination in a scene than can conventional 8-bit images.).
Liu teaches:
further comprising: receiving an illuminance query instruction, wherein the illuminance query instruction carries position coordinates of the illuminance point to be queried and the querying method (Liu [Pg 8 Par 10] Specifically, when the virtual illuminant point value determined by each pixel point corresponding to the preset rendering model is reset, the point value exceeding the preset threshold value can be set as the same preset value; For example, when the reset virtual illuminant point value is between 0-1, can be more than or equal to 0.9995 all virtual illuminant point value is set as 1, then there is a plurality of virtual illuminant point value of 1. at this time, the virtual illuminant point value is 1 corresponding to the position of the pixel point is determined as the virtual illuminant corresponding to the pattern core position (through the graphical core information to represent, specifically the coordinate value of the pixel point).);
Johannesson teaches:
obtaining, using a light diffraction algorithm, an intersection point of the illuminance point to be queried with the three-dimensional space model in the query direction (Johannesson [Pg 5 Par 8] As used herein, " reproduction (") " is generated to represent a virtual model or a part of the real (photocopy) or non-real (non-phototime) image. can be through conventional light projection or light tracking and also considering the rendering technology of diffraction, such as wave optics, GTD algorithm (diffraction geometry theory), PTD algorithm (diffraction physical theory), physical optical (PO), boundary element method (BEM) and so on, to generate reproduction. The reproduction may be two-dimensional or three-dimensional.); and
Chen teaches:
querying a HDR floating point value of the intersection point in the HDR map to obtain the HDR illumination of the illuminance point to be queried (Chen [0102] Since a floating-point number is used to represent the pixel values in an HDR image, an HDR image can represent much more detail across a wide range of illumination in a scene than can conventional 8-bit images.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Liu and Johannesson and Chen. Using a light diffraction algorithm, as in Liu and Johannesson and Chen, would benefit the Douady, Wang and Zhang teachings by allowing to get certain parts of using an algorithm. Additionally, this is the application of a known technique, using a light diffraction algorithm, to yield predictable results.
Claim(s) 6 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Douady et al. (US 20220256076) in view of Wang et al. (CN 111145103), Zhang et al. (CN 113313808), and Zi-Ping et al. (CN 111882498).
Regarding claim 6.
Douady, Wang, and Zhang teach:
The method of claim 1,
Douady, Wang, and Zhang fail to teach:
before the determining the texture index information of the three-dimensional space model, the method further comprising: on at least one collection point, collecting the HDR original image of the target scene (Zi-Ping [Pg 2 Par 7] mapping each pixel in the high dynamic range HDR image to be processed to the LDR space with low dynamic range; obtaining the segment colour parameter of the HDR image to be processed under the LDR space;);
converting the HDR original image of the target scene into a low dynamic range (LDR) original image (Zi-Ping [Pg 2 Par 7] wherein, after obtaining the processed HDR image, if it is necessary to display the image, making the human eye see more comfortable; better expressing the information and features in the original image, then the HDR image after the processing is mapped to the LDR space, obtaining the processed HDR image segment colour parameter in the LDR space; namely, the colour value of the HDR image after the processing is mapped to the low dynamic range (LDR) from the high dynamic range (HDR), so that when the LDR space displays the processed HDR image, The scene brightness of the processed HDR image can be converted to the range capable of being displayed, and the image details are ensured not to be lost and distorted. and based on the image processing method provided by the invention, when the LDR space displays the processed HDR image, the edge blurring effect of the image is good, reducing the sawtooth effect, which can reach the good anti-aliasing effect.); and
based on the LDR original image and the corresponding at least one collection point, generating the three-dimensional space model of the target scene (Zi-Ping [Pg 9 Par 5] wherein, after obtaining the processed HDR image, if it is necessary to display the image, making the human eye see more comfortable; better expressing the information and features in the original image, then the HDR image after the processing is mapped to the LDR space, obtaining the processed HDR image segment colour parameter in the LDR space; namely, the colour value of the HDR image after the processing is mapped to the low dynamic range (LDR) from the high dynamic range (HDR), so that when the LDR space displays the processed HDR image, The scene brightness of the processed HDR image can be converted to the range capable of being displayed, and the image details are ensured not to be lost and distorted. and based on the image processing method provided by the invention, when the LDR space displays the processed HDR image, the edge blurring effect of the image is good, reducing the sawtooth effect, which can reach the good anti-aliasing effect.) (Douady [0008] In a third aspect, the subject matter described in this specification can be embodied in systems that include an image sensor configured to capture image data using a plurality of selectable exposure times; and a processing apparatus that is configured to: receive a first image from the image sensor, where the first image is captured with a first exposure time; receive a second image from the image sensor, where the second image is captured with a second exposure time that is less than the first exposure time; determine a high dynamic range image based on the first image in a raw format and the second image in a raw format, in which an image portion of the high dynamic range image is based on a corresponding image portion of the second image when a pixel of a corresponding image portion of the first image is saturated; and store, display, or transmit an output image that is based on the high dynamic range image. [0330] For example, the ISP/stitch engine 2122 can perform edge detection, texture analysis, color analysis, depth analysis, and/or any other suitable operations in order to identify the portions of the images representative of the same field of view. [0072] In some implementations, the user interface device 120 may display, or otherwise present, content, such as images or video, acquired by the image capture apparatus 110. For example, a display of the user interface device 120 may be a viewport into the three-dimensional space represented by the panoramic images or video captured or created by the image capture apparatus 110.).
Zi-Ping teaches:
before the determining the texture index information of the three-dimensional space model, the method further comprising: on at least one collection point, collecting the HDR original image of the target scene (Zi-Ping [Pg 2 Par 7] mapping each pixel in the high dynamic range HDR image to be processed to the LDR space with low dynamic range; obtaining the segment colour parameter of the HDR image to be processed under the LDR space;);
converting the HDR original image of the target scene into a low dynamic range (LDR) original image (Zi-Ping [Pg 9 Par 5] wherein, after obtaining the processed HDR image, if it is necessary to display the image, making the human eye see more comfortable; better expressing the information and features in the original image, then the HDR image after the processing is mapped to the LDR space, obtaining the processed HDR image segment colour parameter in the LDR space; namely, the colour value of the HDR image after the processing is mapped to the low dynamic range (LDR) from the high dynamic range (HDR), so that when the LDR space displays the processed HDR image, The scene brightness of the processed HDR image can be converted to the range capable of being displayed, and the image details are ensured not to be lost and distorted. and based on the image processing method provided by the invention, when the LDR space displays the processed HDR image, the edge blurring effect of the image is good, reducing the sawtooth effect, which can reach the good anti-aliasing effect.); and
based on the LDR original image and the corresponding at least one collection point, generating the three-dimensional space model of the target scene (Zi-Ping [Pg 9 Par 5] wherein, after obtaining the processed HDR image, if it is necessary to display the image, making the human eye see more comfortable; better expressing the information and features in the original image, then the HDR image after the processing is mapped to the LDR space, obtaining the processed HDR image segment colour parameter in the LDR space; namely, the colour value of the HDR image after the processing is mapped to the low dynamic range (LDR) from the high dynamic range (HDR), so that when the LDR space displays the processed HDR image, The scene brightness of the processed HDR image can be converted to the range capable of being displayed, and the image details are ensured not to be lost and distorted. and based on the image processing method provided by the invention, when the LDR space displays the processed HDR image, the edge blurring effect of the image is good, reducing the sawtooth effect, which can reach the good anti-aliasing effect.) (Douady [0008] In a third aspect, the subject matter described in this specification can be embodied in systems that include an image sensor configured to capture image data using a plurality of selectable exposure times; and a processing apparatus that is configured to: receive a first image from the image sensor, where the first image is captured with a first exposure time; receive a second image from the image sensor, where the second image is captured with a second exposure time that is less than the first exposure time; determine a high dynamic range image based on the first image in a raw format and the second image in a raw format, in which an image portion of the high dynamic range image is based on a corresponding image portion of the second image when a pixel of a corresponding image portion of the first image is saturated; and store, display, or transmit an output image that is based on the high dynamic range image. [0330] For example, the ISP/stitch engine 2122 can perform edge detection, texture analysis, color analysis, depth analysis, and/or any other suitable operations in order to identify the portions of the images representative of the same field of view. [0072] In some implementations, the user interface device 120 may display, or otherwise present, content, such as images or video, acquired by the image capture apparatus 110. For example, a display of the user interface device 120 may be a viewport into the three-dimensional space represented by the panoramic images or video captured or created by the image capture apparatus 110.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Zi-Ping. Being able to switch between LDR and HDR, as in Zi-Ping, would benefit the Douady, Wang and Zhang teachings by allowing be compatible with many different types of scenes. Additionally, this is the application of a known technique, being able to switch between LDR and HDR, to yield predictable results.
Regarding claim 15.
Douady, Wang, and Zhang teach:
The electronic device of claim 10,
Douady, Wang, and Zhang fail to teach:
before the determining the texture index information of the three-dimensional space model, the method further comprising: on at least one collection point, collecting the HDR original image of the target scene (Zi-Ping [Pg 2 Par 7] mapping each pixel in the high dynamic range HDR image to be processed to the LDR space with low dynamic range; obtaining the segment colour parameter of the HDR image to be processed under the LDR space;);
converting the HDR original image of the target scene into a low dynamic range (LDR) original image (Zi-Ping [Pg 2 Par 7] wherein, after obtaining the processed HDR image, if it is necessary to display the image, making the human eye see more comfortable; better expressing the information and features in the original image, then the HDR image after the processing is mapped to the LDR space, obtaining the processed HDR image segment colour parameter in the LDR space; namely, the colour value of the HDR image after the processing is mapped to the low dynamic range (LDR) from the high dynamic range (HDR), so that when the LDR space displays the processed HDR image, The scene brightness of the processed HDR image can be converted to the range capable of being displayed, and the image details are ensured not to be lost and distorted. and based on the image processing method provided by the invention, when the LDR space displays the processed HDR image, the edge blurring effect of the image is good, reducing the sawtooth effect, which can reach the good anti-aliasing effect.); and
based on the LDR original image and the corresponding at least one collection point, generating the three-dimensional space model of the target scene (Zi-Ping [Pg 9 Par 5] wherein, after obtaining the processed HDR image, if it is necessary to display the image, making the human eye see more comfortable; better expressing the information and features in the original image, then the HDR image after the processing is mapped to the LDR space, obtaining the processed HDR image segment colour parameter in the LDR space; namely, the colour value of the HDR image after the processing is mapped to the low dynamic range (LDR) from the high dynamic range (HDR), so that when the LDR space displays the processed HDR image, The scene brightness of the processed HDR image can be converted to the range capable of being displayed, and the image details are ensured not to be lost and distorted. and based on the image processing method provided by the invention, when the LDR space displays the processed HDR image, the edge blurring effect of the image is good, reducing the sawtooth effect, which can reach the good anti-aliasing effect.) (Douady [0008] In a third aspect, the subject matter described in this specification can be embodied in systems that include an image sensor configured to capture image data using a plurality of selectable exposure times; and a processing apparatus that is configured to: receive a first image from the image sensor, where the first image is captured with a first exposure time; receive a second image from the image sensor, where the second image is captured with a second exposure time that is less than the first exposure time; determine a high dynamic range image based on the first image in a raw format and the second image in a raw format, in which an image portion of the high dynamic range image is based on a corresponding image portion of the second image when a pixel of a corresponding image portion of the first image is saturated; and store, display, or transmit an output image that is based on the high dynamic range image. [0330] For example, the ISP/stitch engine 2122 can perform edge detection, texture analysis, color analysis, depth analysis, and/or any other suitable operations in order to identify the portions of the images representative of the same field of view. [0072] In some implementations, the user interface device 120 may display, or otherwise present, content, such as images or video, acquired by the image capture apparatus 110. For example, a display of the user interface device 120 may be a viewport into the three-dimensional space represented by the panoramic images or video captured or created by the image capture apparatus 110.).
Zi-Ping teaches:
before the determining the texture index information of the three-dimensional space model, the method further comprising: on at least one collection point, collecting the HDR original image of the target scene (Zi-Ping [Pg 2 Par 7] mapping each pixel in the high dynamic range HDR image to be processed to the LDR space with low dynamic range; obtaining the segment colour parameter of the HDR image to be processed under the LDR space;);
converting the HDR original image of the target scene into a low dynamic range (LDR) original image (Zi-Ping [Pg 9 Par 5] wherein, after obtaining the processed HDR image, if it is necessary to display the image, making the human eye see more comfortable; better expressing the information and features in the original image, then the HDR image after the processing is mapped to the LDR space, obtaining the processed HDR image segment colour parameter in the LDR space; namely, the colour value of the HDR image after the processing is mapped to the low dynamic range (LDR) from the high dynamic range (HDR), so that when the LDR space displays the processed HDR image, The scene brightness of the processed HDR image can be converted to the range capable of being displayed, and the image details are ensured not to be lost and distorted. and based on the image processing method provided by the invention, when the LDR space displays the processed HDR image, the edge blurring effect of the image is good, reducing the sawtooth effect, which can reach the good anti-aliasing effect.); and
based on the LDR original image and the corresponding at least one collection point, generating the three-dimensional space model of the target scene (Zi-Ping [Pg 9 Par 5] wherein, after obtaining the processed HDR image, if it is necessary to display the image, making the human eye see more comfortable; better expressing the information and features in the original image, then the HDR image after the processing is mapped to the LDR space, obtaining the processed HDR image segment colour parameter in the LDR space; namely, the colour value of the HDR image after the processing is mapped to the low dynamic range (LDR) from the high dynamic range (HDR), so that when the LDR space displays the processed HDR image, The scene brightness of the processed HDR image can be converted to the range capable of being displayed, and the image details are ensured not to be lost and distorted. and based on the image processing method provided by the invention, when the LDR space displays the processed HDR image, the edge blurring effect of the image is good, reducing the sawtooth effect, which can reach the good anti-aliasing effect.) (Douady [0008] In a third aspect, the subject matter described in this specification can be embodied in systems that include an image sensor configured to capture image data using a plurality of selectable exposure times; and a processing apparatus that is configured to: receive a first image from the image sensor, where the first image is captured with a first exposure time; receive a second image from the image sensor, where the second image is captured with a second exposure time that is less than the first exposure time; determine a high dynamic range image based on the first image in a raw format and the second image in a raw format, in which an image portion of the high dynamic range image is based on a corresponding image portion of the second image when a pixel of a corresponding image portion of the first image is saturated; and store, display, or transmit an output image that is based on the high dynamic range image. [0330] For example, the ISP/stitch engine 2122 can perform edge detection, texture analysis, color analysis, depth analysis, and/or any other suitable operations in order to identify the portions of the images representative of the same field of view. [0072] In some implementations, the user interface device 120 may display, or otherwise present, content, such as images or video, acquired by the image capture apparatus 110. For example, a display of the user interface device 120 may be a viewport into the three-dimensional space represented by the panoramic images or video captured or created by the image capture apparatus 110.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Zi-Ping. Being able to switch between LDR and HDR, as in Zi-Ping, would benefit the Douady, Wang and Zhang teachings by allowing be compatible with many different types of scenes. Additionally, this is the application of a known technique, being able to switch between LDR and HDR, to yield predictable results.
Claim(s) 7-8, 16-17, 19, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Douady et al. (US 20220256076) in view of Wang et al. (CN 111145103), Zhang et al. (CN 113313808), and Avila et al. (US 20210109987).
Regarding claim 7.
Douady, Wang, and Zhang teach:
The method of claim 1,
Douady, Wang, and Zhang fail to teach:
wherein the texture index information of the three-dimensional space model comprises: picture identification information of the input HDR original image corresponding to the texture of any surface point of the three-dimensional space model (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
a HDR floating point value corresponding to any pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Avila teaches:
wherein the texture index information of the three-dimensional space model comprises: picture identification information of the input HDR original image corresponding to the texture of any surface point of the three-dimensional space model (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
a HDR floating point value corresponding to any pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Avila. Getting an HDR floating point, as in Avila, would benefit the Douady, Wang and Zhang teachings by allowing a way to get different values. Additionally, this is the application of a known technique, Getting an HDR floating point, to yield predictable results.
Regarding claim 8.
Douady, Wang, and Zhang teach:
The method of claim 1,
Douady, Wang, and Zhang fail to teach:
wherein generating the HDR map of the three-dimensional space model based on the texture index information of the three-dimensional space model comprises: based on the texture index information of the three-dimensional space model, indexing the input HDR original image corresponding to the texture of any surface point and the HDR floating point value of the corresponding pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
generating the HDR floating point values of the textures of all surface points of the three-dimensional space model as the HDR textures wherein the HDR floating point values serve as the HDR values of the plurality of pixel points of the HDR map. (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Avila teaches:
wherein generating the HDR map of the three-dimensional space model based on the texture index information of the three-dimensional space model comprises: based on the texture index information of the three-dimensional space model, indexing the input HDR original image corresponding to the texture of any surface point and the HDR floating point value of the corresponding pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
generating the HDR floating point values of the textures of all surface points of the three-dimensional space model as the HDR textures wherein the HDR floating point values serve as the HDR values of the plurality of pixel points of the HDR map. (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Avila. Getting an HDR floating point, as in Avila, would benefit the Douady, Wang and Zhang teachings by allowing a way to get different values. Additionally, this is the application of a known technique, Getting an HDR floating point, to yield predictable results.
Regarding claim 16.
Douady, Wang, and Zhang teach:
The electronic device of claim 10,
Douady, Wang, and Zhang fail to teach:
wherein the texture index information of the three-dimensional space model comprises: picture identification information of the input HDR original image corresponding to the texture of any surface point of the three-dimensional space model (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
a HDR floating point value corresponding to any pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Avila teaches:
wherein the texture index information of the three-dimensional space model comprises: picture identification information of the input HDR original image corresponding to the texture of any surface point of the three-dimensional space model (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
a HDR floating point value corresponding to any pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Avila. Getting an HDR floating point, as in Avila, would benefit the Douady, Wang and Zhang teachings by allowing a way to get different values. Additionally, this is the application of a known technique, Getting an HDR floating point, to yield predictable results.
Regarding claim 17.
Douady, Wang, and Zhang teach:
The electronic device of claim 10,
Douady, Wang, and Zhang fail to teach:
wherein generating the HDR map of the three-dimensional space model based on the texture index information of the three-dimensional space model comprises: based on the texture index information of the three-dimensional space model, indexing the input HDR original image corresponding to the texture of any surface point and the HDR floating point value of the corresponding pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
generating the HDR floating point values of the textures of all surface points of the three-dimensional space model as the HDR textures wherein the HDR floating point values serve as the HDR values of the plurality of pixel points of the HDR map. (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Avila teaches:
wherein generating the HDR map of the three-dimensional space model based on the texture index information of the three-dimensional space model comprises: based on the texture index information of the three-dimensional space model, indexing the input HDR original image corresponding to the texture of any surface point and the HDR floating point value of the corresponding pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
generating the HDR floating point values of the textures of all surface points of the three-dimensional space model as the HDR textures wherein the HDR floating point values serve as the HDR values of the plurality of pixel points of the HDR map. (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Avila. Getting an HDR floating point, as in Avila, would benefit the Douady, Wang and Zhang teachings by allowing a way to get different values. Additionally, this is the application of a known technique, Getting an HDR floating point, to yield predictable results.
Regarding claim 19.
Douady, Wang, and Zhang teach:
The device of claim 9,
Douady, Wang, and Zhang fail to teach:
wherein the texture index information of the three-dimensional space model comprises: picture identification information of the input HDR original image corresponding to the texture of any surface point of the three-dimensional space model (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
a HDR floating point value corresponding to any pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Avila teaches:
wherein the texture index information of the three-dimensional space model comprises: picture identification information of the input HDR original image corresponding to the texture of any surface point of the three-dimensional space model (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
a HDR floating point value corresponding to any pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Avila. Getting an HDR floating point, as in Avila, would benefit the Douady, Wang and Zhang teachings by allowing a way to get different values. Additionally, this is the application of a known technique, Getting an HDR floating point, to yield predictable results.
Regarding claim 20.
Douady, Wang, and Zhang teach:
The device of claim 9,
Douady, Wang, and Zhang fail to teach:
wherein generating the HDR map of the three-dimensional space model based on the texture index information of the three-dimensional space model comprises: based on the texture index information of the three-dimensional space model, indexing the input HDR original image corresponding to the texture of any surface point and the HDR floating point value of the corresponding pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
generating the HDR floating point values of the textures of all surface points of the three-dimensional space model as the HDR textures wherein the HDR floating point values serve as the HDR values of the plurality of pixel points of the HDR map. (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Avila teaches:
wherein generating the HDR map of the three-dimensional space model based on the texture index information of the three-dimensional space model comprises: based on the texture index information of the three-dimensional space model, indexing the input HDR original image corresponding to the texture of any surface point and the HDR floating point value of the corresponding pixel point in the input HDR original image (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.); and
generating the HDR floating point values of the textures of all surface points of the three-dimensional space model as the HDR textures wherein the HDR floating point values serve as the HDR values of the plurality of pixel points of the HDR map. (Avila [0118] As described above, in some implementations, each sample is rendered to an internal buffer. Subsequent operations store to and retrieve from the internal buffer. Some implementations store resulting render in a floating-point HDR texture in order to realistically represent the widely varying levels of reflected light present in a scene. [0027] In some implementations, determining if the surface is visible includes: (i) calculating a surface position of the pixel; (ii) projecting the surface position to coordinates in the prior frame; (iii) determining if a first mesh identifier for the surface position at the coordinates for the prior frame matches a second mesh identifier for the current frame; and (iv) in accordance with a determination that the first mesh identifier and the second mesh identifier match, determining that the surface is visible in the prior frame.).
Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to combine the teachings of Douady, Wang and Zhang with Avila. Getting an HDR floating point, as in Avila, would benefit the Douady, Wang and Zhang teachings by allowing a way to get different values. Additionally, this is the application of a known technique, Getting an HDR floating point, to yield predictable results.
Response to Arguments
Applicant's arguments filed 1/12/2026 have been fully considered but they are not persuasive.
Applicant alleges:
“Aamended claim 1 specifies that each pixel in the HDR map corresponds to a surface point in the three-dimensional (3D) space model, and that sampling and integrating the HDR map include obtaining HDR values for a plurality of surface points along hemispherical directions relative to a given surface point. In this manner, the luminance of a given surface point in the 3D space model is computed using multiple HDR values surrounding that surface point. Support for these amendments can be found for example in paragraphs [0107], [0109] and FIG. 2B of the published application.
The Office alleges that Douady's initial blending ratio map teaches the claimed HDR map of claim 1 (Office action at pages 4-5). However, according to Douady, the initial blending ratios relate to combining overlapping portions of multiple images for the purpose of combining those images (e.g., Douady in paragraph [0222]).
By contrast, amended claim 1 now specifies "wherein the HDR map comprises a plurality of pixel points, with each pixel point of the HDR map comprises a HDR value for a surface point in the three-dimensional space model." In other words, the HDR map carries HDR values for the surface points in the 3D space model. As such, Douady's initial blending ratio map differs from the claimed HDR map.
The Office further alleges that Douady teaches the sampling and integrating process as claimed (Office action at pages 5-6). Douady is generally directed to combining multiple images (e.g., from different image capture devices), and, as discussed above, the initial blending ratios relate to combining overlapping portions of multiple images.
By contrast, claim 1, as amended, recites "sampling and integrating, for each surface point in the three-dimensional space model, the HDR map along hemispheric directions of the respective surface point to obtain a corresponding noise illuminance value, thereby generating a noise illuminance map corresponding to the plurality of surface points of the three- dimensional space model" (emphasis added). In other words, amended claim 1 requires sampling and integrating a portion of the HDR map along hemispherical directions associated
with a surface point in the 3D space model to compute a noise illuminance value, which is different from Douady's image-based blending of overlapping image portions.
The remaining references cannot cure the deficiencies of Douady with respect to the distinct features discussed above.
Because the cited references do not disclose or suggest at least the above-identified distinguishing features of amended independent claim 1, any combination of the cited references, to the extent proper, do not render claim 1 obvious, and claim 1 is patentable over the cited references. Amended independent claims 9 and 10, although different in scope from claim 1, are also patentable over the cited references at least for similar reasons as discussed above with respect to claim 1. Dependent claims 2, 4-8, 11 and 13-17, which depends from claims 1, or 10, are also patentable over the cited references for at least these same reasons, and further due to the additional features recited therein.”
Examiner responds:
An HDR map is commonly defined in the art as an HDRI or High Dynamic Range Image, therefore the HDR Image as referred to in Douady can be considered an HDR Map. The specification does not define as HDR map to be anything different.
In regards to the amendments,
Zhang has been added to teach the features of the amendments.
Zhang teaches the following:
“wherein the HDR map comprises a plurality of pixel points, with each pixel point of the HDR map comprises a HDR value for a surface point in the three-dimensional space model (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image;);
sampling and integrating, for each surface point in the three-dimensional space model, the HDR map along hemispheric directions of the respective surface point (Zhang [Pg 3 Par 13] a processor for obtaining space three-dimensional image; aiming at the luminous object in the space three-dimensional image, configuring a virtual enclosure body; according to the light emitting information of the light emitting object, determining the illumination information corresponding to a plurality of sampling points on the virtual surrounding body; according to the illumination information of multiple sampling points on the virtual surrounding body, generating high dynamic range image; [Pg 8 Par 5] In the above step 102, the off-line rendering of the virtual bounding volume sampling technology can be realized. if the virtual surrounding body is a sphere, namely using the off-line rendering of hemispherical sampling technology, the lighting information of the light emitting object reflects the illumination information on the inner wall of the virtual surrounding body for sampling.)”
Therefore claims 1, 2, 4-11, 13-20 are rejected under 35 U.S.C. 103.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DENIS VASILIY MINKO whose telephone number is (571)270-5226. The examiner can normally be reached Monday-Thursday 8:30-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DENIS VASILIY MINKO/Examiner, Art Unit 2612
/Said Broome/Supervisory Patent Examiner, Art Unit 2612
Prosecution Insights