DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-4, 10-20 are rejected under 35 USC 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claim 1 is a directed to a system, which is one of the statutory categories of invention. The claim recites a method of “generating a plurality of ground truth outputs from three-dimensional (3D) models, light conditions, and color conditions; and generating a plurality of ground truth inputs from the 3D models, the light conditions, and the color conditions”. The limitations merely employ mathematical calculation to generate “ground truth outputs” and “ground truth inputs” from “three-dimensional (3D) models, light conditions, and color conditions” (MPEP 2106.04 (a)(2). Another example is Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 111 USPQ2d 1717 (Fed. Cir. 2014). The patentee in Digitech claimed methods of generating first and second data by taking existing information, manipulating the data using mathematical formulas, and organizing this information into a new form. The court explained that such claims were directed to an abstract idea because they described a process of organizing information through mathematical correlations, like Flook's method of calculating using a mathematical formula. 758 F.3d at 1350, 111 USPQ2d at 1721). The grouping of “mathematical concepts” in the 2019 PEG is not limited to formulas or equations, and in fact specifically includes “mathematical calculations” as an exemplar of a mathematical concept. 2019 PEG Section I, 84 Fed. Reg. at 52. Thus, the limitations recites a concept that falls into the “mathematical concept” group of abstract ideas.
Next, the claim recites the additional limitations of “wherein a ground truth input of the plurality of ground truth inputs is a modified corresponding ground truth output”. The additional limitation is simple mathematical calculations being performed on “ground truth inputs”. This concept is similar to the other types of basic concepts that have been found by the courts to be abstract. In one example, the courts have found mathematical algorithms to be abstract ideas (e.g., a mathematical procedure for converting one form of numerical representation to another in Benson, or an algorithm for calculating parameters indicating an abnormal condition in Grams). The limitation thus describes a “mathematical relationship,” which is specifically identified in the 2019 PEG as an exemplar in the “mathematical concepts” grouping of abstract ideas. 2019 PEG Section I, 84 Fed. Reg. at 52. In addition, because the BRI of limitation (c) requires the performance of an arithmetic operation (convolution operations), this limitation also describes a “mathematical calculation,” which is also specifically identified in the 2019 PEG as an exemplar in the “mathematical concepts” grouping of abstract ideas. Therefore, the claim does not include additional elements providing meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself. Accordingly, the claim is not patent eligible.
Claim 2 depends from claim 1 and recites additional limitations of “wherein the generating the plurality of ground truth inputs generates a plurality of first images and wherein the generating the plurality of ground truth outputs generates a plurality of second images”. Thus, the limitations perform mathematical calculation for generating first images and second images during the ground truth generation. For the same reasons stated previously with respect to claim 1, the limitations do not integrate the recited judicial exception into a practical application. Therefore, the claim does not amount to significantly more than the abstract idea itself. The claim is not patent eligible.
Claim 3 depends from claim 2 and recites additional limitations of “wherein each image of the plurality of first images and the plurality of second images comprises a plurality of pixels”. Thus, the limitations just provide information of the image. For the same reasons stated previously with respect to claims 1 and 2, the limitations do not integrate the recited judicial exception into a practical application. Therefore, the claim does not amount to significantly more than the abstract idea itself. The claim is not patent eligible.
Claim 4 depends from claim 3 and recites additional limitations of “wherein the generating the plurality of ground truth outputs further comprises: increasing a brightness of pixels of a corresponding plurality of pixels”. Thus, the limitations perform mathematical calculation by increasing a brightness of pixels. For the same reasons stated previously with respect to claims 1 , 2 and 3, the limitations do not integrate the recited judicial exception into a practical application. Therefore, the claim does not amount to significantly more than the abstract idea itself. The claim is not patent eligible.
Claim 10 depends from claim 1 and recites additional limitations of “wherein the light conditions indicate a plurality of light sources, each light source of the plurality of light sources comprising a direction, a hue value, a saturation value, and a brightness value”. Thus, the limitations just provide information for light conditions. For the same reasons stated previously with respect to claim 1, the limitations do not integrate the recited judicial exception into a practical application. Therefore, the claim does not amount to significantly more than the abstract idea itself. The claim is not patent eligible.
Claim 11 depends from claim 1 and recites additional limitations of “wherein the generating the ground truth outputs generates ground truth output images, the generating the ground truth inputs generates ground truth input images, and wherein a ground truth output image of the ground truth output image comprises lighting properties of a corresponding ground truth input image”. Thus, the limitations perform mathematical calculation for generating first images and second images during the ground truth generation. For the same reasons stated previously with respect to claim 1, the limitations do not integrate the recited judicial exception into a practical application. Therefore, the claim does not amount to significantly more than the abstract idea itself. The claim is not patent eligible.
Claim 12 depends from claim 11 and recites additional limitations of “wherein the lighting properties comprise at least one of: hue values, saturation values, or brightness values for each of a plurality of pixels of the corresponding ground truth input image”. Thus, the limitations just provide information for light conditions. For the same reasons stated previously with respect to claims 1 and 11, the limitations do not integrate the recited judicial exception into a practical application. Therefore, the claim does not amount to significantly more than the abstract idea itself. The claim is not patent eligible.
Claim 13 is a directed to a non-transitory computer-readable storage medium, which is one of the statutory categories of invention. The claim recites limitations similar to independent claim 1. Accordingly, claim 13 is rejected for the same rationale as claim 1.
Claims 14-16 depend from claim 13 and recite the limitations similar to the claims 2, 11 and 12 do not remedy the problem. They are rejected for the same rationale.
Claim 17 is a directed to a method, which is one of the statutory categories of invention. The claim recites limitations similar to independent claim 1. Accordingly, claim 17 is rejected for the same rationale as claim 1.
Claims 18-20 depend from claim 17 and recite the limitations similar to the claims 2, 11 and 12 do not remedy the problem. They are rejected for the same rationale.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5-9 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Dependent claim 5 depends upon independent claim 1 and recites “rendering a 3D model of the 3D models ...”. Independent claim 1 recites “generating a plurality of ground truth outputs from three-dimensional (3D) models ...”. Thus, “three-dimensional (3D) models” ready exist before generating “ground truth outputs” and “ground truth inputs”. The issue is persons of ordinary skill in the art reading the specification is not able to understand why and how to render “a 3D model of the 3D models”. Therefore, the examiner deems the claim indefinite as it fail to particularly point out and distinctly claim what Applicant regards as the invention. Accordingly, the claim is rejected under U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
Dependent claims 6-9 are rejected because they depend upon independent claim 5.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 13 and 17 are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Wang et al (U.S. Patent Application Publication 2017/0243083 A1).
Regarding claim 1, Wang discloses a system comprising:
one or more computer processors (FIG. 2; paragraph [0042], the system 10 includes a processor device 46); and
one or more computer-readable mediums storing instructions (Paragraph [0042], memory 42, which stores instructions 44) that, when executed by the one or more computer processors (Paragraph [0042], the system 10 includes memory 42, which stores instructions 44 for performing the method described with reference to FIG. 3, and a processor device 46 in communication with the memory for executing the instructions), cause the system to perform operations comprising:
generating a plurality of ground truth outputs (Paragraph [0041], ground truth annotations 32 are automatically generated for the virtual objects 24 of interest from the corresponding real-world annotations 22) from three-dimensional (3D) models (Paragraph [0041], the relevant objects 18 may include, for example, one or more of cars, pedestrians, roads, buildings, and the camera itself ...; paragraph [0048], the graphics engine component 58 takes as input the real-word data 12, which can be a stream of frames in a video sequence 14, and sensor data 16. The graphics engine 58 clones the real-world data into a virtual world by first decomposing the real-world data into respective visual components 20 and the objects 18, which may belong to selected object categories. The graphics engine 58 then generates script 28 (software instructions) for automating or controlling the visual components and objects in the virtual world. Each virtual world 29 that is generated by the script is a photo-realistic, dynamic 3D graphical representation of the real world location and relevant objects in it), light conditions (Paragraph [0041], visual components 20 may include one or more of lighting), and color conditions (Paragraph [0042], in general, each frame of the input digital video sequence includes image data for an array of pixels forming the frame. The image data may include colorant values ...); and
generating a plurality of ground truth inputs (Paragraph [0041], modified ground truth annotations 40 are also generated for the virtual objects 24 of interest from the corresponding real-world annotations 22 ...) from the 3D models (Paragraph [0041], the relevant objects 18 may include, for example, one or more of cars, pedestrians, roads, buildings, and the camera itself ... A modified script 34 is generated in response to a changed condition 36 in the script, which may be an on-demand changed condition or a randomly-generated one. The changed condition 36 can relate to any one of the virtual objects 24 or virtual visual components 26, or a combination thereof. The modified script 34 is able to automatically generate a modified synthetic or virtual video 38 based on the changed condition 36; paragraph [0048], the graphics engine component 58 takes as input the real-word data 12, which can be a stream of frames in a video sequence 14, and sensor data 16. The graphics engine 58 clones the real-world data into a virtual world by first decomposing the real-world data into respective visual components 20 and the objects 18, which may belong to selected object categories. The graphics engine 58 then generates script 28 (software instructions) for automating or controlling the visual components and objects in the virtual world. Each virtual world 29 that is generated by the script is a photo-realistic, dynamic 3D graphical representation of the real world location and relevant objects in it), the light conditions (Paragraph [0041], visual components 20 may include one or more of lighting), and the color conditions (Paragraph [0042], in general, each frame of the input digital video sequence includes image data for an array of pixels forming the frame. The image data may include colorant values ...), wherein a ground truth input of the plurality of ground truth inputs is a modified corresponding ground truth output (Paragraph [0041], the objects and visual components, along with the annotations 22, are cloned into a virtual world, as shown by virtual objects 24 and virtual visual components 26 ... A modified script 34 is generated in response to a changed condition 36 in the script, which may be an on-demand changed condition or a randomly-generated one. The changed condition 36 can relate to any one of the virtual objects 24 or virtual visual components 26, or a combination thereof. The modified script 34 is able to automatically generate a modified synthetic or virtual video 38 based on the changed condition 36. Modified annotations 40 are also generated).
Regarding claim 13, Wang discloses a non-transitory computer-readable storage medium including instructions (FIG. 2; paragraph [0042], the system 10 includes memory 42, which stores instructions 44) that, when processed by at least one processor, configure the at least one processor to perform operations (Paragraph [0042], the system 10 includes memory 42, which stores instructions 44 for performing the method described with reference to FIG. 3, and a processor device 46 in communication with the memory for executing the instructions) comprising:
generating a plurality of ground truth outputs (Paragraph [0041], ground truth annotations 32 are automatically generated for the virtual objects 24 of interest from the corresponding real-world annotations 22) from three-dimensional (3D) models (Paragraph [0041], the relevant objects 18 may include, for example, one or more of cars, pedestrians, roads, buildings, and the camera itself ...; paragraph [0048], the graphics engine component 58 takes as input the real-word data 12, which can be a stream of frames in a video sequence 14, and sensor data 16. The graphics engine 58 clones the real-world data into a virtual world by first decomposing the real-world data into respective visual components 20 and the objects 18, which may belong to selected object categories. The graphics engine 58 then generates script 28 (software instructions) for automating or controlling the visual components and objects in the virtual world. Each virtual world 29 that is generated by the script is a photo-realistic, dynamic 3D graphical representation of the real world location and relevant objects in it), light conditions (Paragraph [0041], visual components 20 may include one or more of lighting), and color conditions (Paragraph [0042], in general, each frame of the input digital video sequence includes image data for an array of pixels forming the frame. The image data may include colorant values ...); and
generating a plurality of ground truth inputs (Paragraph [0041], modified ground truth annotations 40 are also generated for the virtual objects 24 of interest from the corresponding real-world annotations 22 ...) from the 3D models (Paragraph [0041], the relevant objects 18 may include, for example, one or more of cars, pedestrians, roads, buildings, and the camera itself ... A modified script 34 is generated in response to a changed condition 36 in the script, which may be an on-demand changed condition or a randomly-generated one. The changed condition 36 can relate to any one of the virtual objects 24 or virtual visual components 26, or a combination thereof. The modified script 34 is able to automatically generate a modified synthetic or virtual video 38 based on the changed condition 36; paragraph [0048], the graphics engine component 58 takes as input the real-word data 12, which can be a stream of frames in a video sequence 14, and sensor data 16. The graphics engine 58 clones the real-world data into a virtual world by first decomposing the real-world data into respective visual components 20 and the objects 18, which may belong to selected object categories. The graphics engine 58 then generates script 28 (software instructions) for automating or controlling the visual components and objects in the virtual world. Each virtual world 29 that is generated by the script is a photo-realistic, dynamic 3D graphical representation of the real world location and relevant objects in it), the light conditions (Paragraph [0041], visual components 20 may include one or more of lighting), and the color conditions (Paragraph [0042], in general, each frame of the input digital video sequence includes image data for an array of pixels forming the frame. The image data may include colorant values ...), wherein a ground truth input of the plurality of ground truth inputs is a modified corresponding ground truth output (Paragraph [0041], the objects and visual components, along with the annotations 22, are cloned into a virtual world, as shown by virtual objects 24 and virtual visual components 26 ... A modified script 34 is generated in response to a changed condition 36 in the script, which may be an on-demand changed condition or a randomly-generated one. The changed condition 36 can relate to any one of the virtual objects 24 or virtual visual components 26, or a combination thereof. The modified script 34 is able to automatically generate a modified synthetic or virtual video 38 based on the changed condition 36. Modified annotations 40 are also generated).
Regarding claim 17, Wang discloses a method comprising:
generating a plurality of ground truth outputs (FIG. 2; paragraph [0041], ground truth annotations 32 are automatically generated for the virtual objects 24 of interest from the corresponding real-world annotations 22) from three-dimensional (3D) models (Paragraph [0041], the relevant objects 18 may include, for example, one or more of cars, pedestrians, roads, buildings, and the camera itself ...; paragraph [0048], the graphics engine component 58 takes as input the real-word data 12, which can be a stream of frames in a video sequence 14, and sensor data 16. The graphics engine 58 clones the real-world data into a virtual world by first decomposing the real-world data into respective visual components 20 and the objects 18, which may belong to selected object categories. The graphics engine 58 then generates script 28 (software instructions) for automating or controlling the visual components and objects in the virtual world. Each virtual world 29 that is generated by the script is a photo-realistic, dynamic 3D graphical representation of the real world location and relevant objects in it), light conditions (Paragraph [0041], visual components 20 may include one or more of lighting), and color conditions (Paragraph [0042], in general, each frame of the input digital video sequence includes image data for an array of pixels forming the frame. The image data may include colorant values ...); and
generating a plurality of ground truth inputs (Paragraph [0041], modified ground truth annotations 40 are also generated for the virtual objects 24 of interest from the corresponding real-world annotations 22 ...) from the 3D models (Paragraph [0041], the relevant objects 18 may include, for example, one or more of cars, pedestrians, roads, buildings, and the camera itself ... A modified script 34 is generated in response to a changed condition 36 in the script, which may be an on-demand changed condition or a randomly-generated one. The changed condition 36 can relate to any one of the virtual objects 24 or virtual visual components 26, or a combination thereof. The modified script 34 is able to automatically generate a modified synthetic or virtual video 38 based on the changed condition 36; paragraph [0048], the graphics engine component 58 takes as input the real-word data 12, which can be a stream of frames in a video sequence 14, and sensor data 16. The graphics engine 58 clones the real-world data into a virtual world by first decomposing the real-world data into respective visual components 20 and the objects 18, which may belong to selected object categories. The graphics engine 58 then generates script 28 (software instructions) for automating or controlling the visual components and objects in the virtual world. Each virtual world 29 that is generated by the script is a photo-realistic, dynamic 3D graphical representation of the real world location and relevant objects in it), the light conditions (Paragraph [0041], visual components 20 may include one or more of lighting), and the color conditions (Paragraph [0042], in general, each frame of the input digital video sequence includes image data for an array of pixels forming the frame. The image data may include colorant values ...), wherein a ground truth input of the plurality of ground truth inputs is a modified corresponding ground truth output (Paragraph [0041], the objects and visual components, along with the annotations 22, are cloned into a virtual world, as shown by virtual objects 24 and virtual visual components 26 ... A modified script 34 is generated in response to a changed condition 36 in the script, which may be an on-demand changed condition or a randomly-generated one. The changed condition 36 can relate to any one of the virtual objects 24 or virtual visual components 26, or a combination thereof. The modified script 34 is able to automatically generate a modified synthetic or virtual video 38 based on the changed condition 36. Modified annotations 40 are also generated).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-12, 14-16 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al (U.S. Patent Application Publication 2017/0243083 A1) in view of Sunkavalli et al (U.S. Patent Application Publication 2021/0065440 A1).
Regarding claim 2, Wang discloses everything claimed as applied above (see claim 1).
However, Wang fails to disclose wherein the generating the plurality of ground truth inputs generates a plurality of first images and wherein the generating the plurality of ground truth outputs generates a plurality of second images.
In additional, Sunkavalli discloses (Paragraph [0006], the disclosed systems identify a request to render a virtual object at a designated position within a digital image ... the systems further generate 3D-source-specific-lighting parameters utilizing parametric-specific-network layers of the source-specific-lighting-estimation-neural network. In response to the request to render, the systems accordingly render a modified digital image comprising the virtual object at the designated position illuminated according to the 3D-source-specific-lighting parameters) wherein the generating the plurality of ground truth inputs (FIGS. 1 and 2; paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating ground-truth-source-specific-lighting parameters 208a-208n corresponding to the digital training image 204) generates a plurality of first images (FIG. 9; paragraph [0156], the lighting estimation system 108 applies a source-specific-lighting-estimation-neural network to a digital image 900 to generate 3D-source-specific-lighting parameters ... By contrast, researchers apply Gardner's neural network to the digital image 900 to generate non-parametric lighting representations of the digital image 900. Based on Gardner's non-parametric lighting representations, the researchers reconstruct a lighting environment map 904. Gardner's system further generates a lighting-based-object rendering 910 comprising the virtual objects based on the non-parametric lighting representations illuminating a center position of the digital image 900) and wherein the generating the plurality of ground truth outputs (Paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating a ground-truth-environment map 206) generates a plurality of second images (Paragraph [0156], FIG. 9 further depicts a ground-truth-environment map 902 corresponding to the digital image 900 and a ground-truth-object rendering 908 based on lighting conditions from the ground-truth-environment map 902).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 3, the combination of Wang om view of Sunkavalli discloses everything claimed as applied above (see claim 2).
In additional, Sunkavalli discloses wherein each image of the plurality of first images and the plurality of second images comprises a plurality of pixels (FIG. 1; paragraph [0048], 3D-source-specific-lighting parameters define, specify, or otherwise indicate coloring, lighting, or shading of pixels based on a light source illuminating a digital image. Such 3D-source-specific-lighting parameters may, for example, define the shade or hue of pixels for a virtual object at a designated position).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to form each image with a plurality of pixels. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 4, the combination of Wang om view of Sunkavalli discloses everything claimed as applied above (see claim 3), and Wang further disclose wherein the generating the plurality of ground truth outputs further comprises:
increasing a brightness of pixels of a corresponding plurality of pixels (FIG. 2; paragraph [0090], a number of different visual changes can be achieved with the modification component 64. For example, the lighting may be altered by changing the direction and intensity of the directional light source and/or by reducing or increasing the ambient light intensity; paragraph [0093], the generation of ground truth annotations 32, 40 on the synthetic videos 30, 38 may be achieved automatically using an algorithm-based approach which allows richer (i.e., pixel-level)).
Regarding claim 5, Sunkavalli discloses everything claimed as applied above (see claim 1).
However, Wang fails to disclose wherein the generating the plurality of ground truth inputs further comprises:
rendering a 3D model of the 3D models and determining a hue value, a saturation value, and a brightness value for each pixel of a corresponding plurality of pixels to generate a ground truth output image.
In additional, Sunkavalli discloses (Paragraph [0006], the disclosed systems identify a request to render a virtual object at a designated position within a digital image ... the systems further generate 3D-source-specific-lighting parameters utilizing parametric-specific-network layers of the source-specific-lighting-estimation-neural network. In response to the request to render, the systems accordingly render a modified digital image comprising the virtual object at the designated position illuminated according to the 3D-source-specific-lighting parameters) wherein the generating the plurality of ground truth inputs (FIGS. 1 and 2; paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating ground-truth-source-specific-lighting parameters 208a-208n corresponding to the digital training image 204) further comprises:
rendering a 3D model of the 3D models (FIG. 9; paragraph [0156], the lighting estimation system 108 applies a source-specific-lighting-estimation-neural network to a digital image 900 to generate 3D-source-specific-lighting parameters. Based on the 3D-source-specific-lighting parameters, the lighting estimation system 108 projects a predicted environment map 906) and determining a hue value, a saturation value, and a brightness value for each pixel of a corresponding plurality of pixels (FIG. 5D; paragraph [0130], a digital image 524 and corresponding lighting parameter controls within a screen 502 … the computing device 500 generates (or renders) a parameter-based-rendering 528 of the virtual object; paragraph [0131], in addition to the digital image 524 and corresponding imagery, the computing device 500 further presents source-specific-lighting-parameter controls 530a-530c and ambient lighting controls 532 …; paragraph [0133], the source-specific-lighting-parameters controls 530a further include a light-source-direction control 536, a light-source-size control 540, and a light-source-intensity control 542; paragraph [0134], as further shown in FIG. 5D, the source-specific-lighting-parameter controls 530a also include light-source-color controls 544. The light-source-color controls 544 include controls for various color metrics. For example, the light-source-color controls 544 include a control for a red code, a green code, and a blue code and (additionally or alternatively) a hue metric, a saturation metric, and a color-value metric) to generate a ground truth output image (Paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating a ground-truth-environment map 206; paragraph [0156], FIG. 9 further depicts a ground-truth-environment map 902 corresponding to the digital image 900 and a ground-truth-object rendering 908 based on lighting conditions from the ground-truth-environment map 902).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 6, the combination of Wang in view of Sunkavalli discloses everything claimed as applied above (see claim 5).
However, Wang fails to disclose wherein the generating the plurality of ground truth outputs further comprises:
modifying one or more of the hue value, the saturation value, or the brightness value of the ground truth output image to generate a ground truth input image.
In additional, Sunkavalli discloses wherein the generating the plurality of ground truth outputs further comprises:
modifying one or more of the hue value, the saturation value, or the brightness value of the ground truth output image (Paragraph [0134], as further shown in FIG. 5D, the source-specific-lighting-parameter controls 530a also include light-source-color controls 544 ... Based on detecting a user interaction with one of the light-source-size controls 540, the computing device 500 adjusts values from 3D-source-specific-color parameters to alter a lighting color of a predicted lighting source illuminating the parameter-based-rendering 528 in terms of a red code, a green code, a blue code, a hue metric, a saturation metric, or a color-value metric; paragraph [0137], based on adjustments to 3D-source-specific-light parameters or an ambient parameter, the lighting estimation system 108 applies a source-specific-lighting-estimation-neural network to the digital image 524 to generate one or both of adjusted 3D-source-specific-lighting parameters and an ambient parameter. The computing device 500 further generates the adjusted environment map 546 based on a projection of the adjusted 3D-source-specific-lighting parameters ...) to generate a ground truth input image (FIG. 9; paragraph [0156], the lighting estimation system 108 applies a source-specific-lighting-estimation-neural network to a digital image 900 to generate 3D-source-specific-lighting parameters ... By contrast, researchers apply Gardner's neural network to the digital image 900 to generate non-parametric lighting representations of the digital image 900. Based on Gardner's non-parametric lighting representations, the researchers reconstruct a lighting environment map 904. Gardner's system further generates a lighting-based-object rendering 910 comprising the virtual objects based on the non-parametric lighting representations illuminating a center position of the digital image 900).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 7, the combination of Wang in view of Sunkavalli discloses everything claimed as applied above (see claim 5).
However, Wang fails to disclose wherein the operations further comprise:
modifying one or more of the hue value, the saturation value, and the brightness value for each pixel of the corresponding plurality of pixels to whiten the generated ground truth output image.
In additional, Sunkavalli discloses wherein the operations further comprise:
modifying one or more of the hue value, the saturation value, and the brightness value for each pixel of the corresponding plurality of pixels to whiten (Paragraph [0134], as further shown in FIG. 5D, the source-specific-lighting-parameter controls 530a also include light-source-color controls 544 ... Based on detecting a user interaction with one of the light-source-size controls 540, the computing device 500 adjusts values from 3D-source-specific-color parameters to alter a lighting color of a predicted lighting source illuminating the parameter-based-rendering 528 in terms of a red code, a green code, a blue code, a hue metric, a saturation metric, or a color-value metric. It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to understand that the user can use “the light-source-color controls” to modify one or more of the hue value, the saturation value, and the brightness value for each pixel to whiten) the generated ground truth output image (Paragraph [0136], when the lighting estimation system 108 adjusts 3D-source-specific-light parameters based on user interactions with such lighting parameter controls, in some embodiments, the lighting estimation system 108 provides real time (or near real-time) renderings depicted such adjustments. FIG. 5E depicts an example of the lighting estimation system 108 adjusting 3D-source-specific-light parameters in a rendering of a virtual object within a digital image).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 8, the combination of Wang in view of Sunkavalli discloses everything claimed as applied above (see claim 7).
However, Wang fails to disclose wherein the generating the plurality of ground truth inputs further comprises:
using the ground truth output image without the modifying as a corresponding ground truth input image.
In additional, Sunkavalli discloses wherein the generating the plurality of ground truth inputs further comprises:
using the ground truth output image without the modifying as a corresponding ground truth input image (Paragraph [0157], FIG. 9 further depicts a ground-truth-environment map 902 corresponding to the digital image 900 and a ground-truth-object rendering 908 based on lighting conditions from the ground-truth-environment map 902 ...; paragraph [0156], gardner's system further generates a lighting-based-object rendering 910 comprising the virtual objects based on the non-parametric lighting representations illuminating a center position of the digital image 900).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 9, the combination of Wang in view of Sunkavalli discloses everything claimed as applied above (see claim 8).
However, Wang fails to disclose wherein the corresponding ground truth input is less white than a ground truth output.
In additional, Sunkavalli discloses wherein the corresponding ground truth input is less white than a ground truth output (Paragraph [0134], as further shown in FIG. 5D, the source-specific-lighting-parameter controls 530a also include light-source-color controls 544 ... Based on detecting a user interaction with one of the light-source-size controls 540, the computing device 500 adjusts values from 3D-source-specific-color parameters to alter a lighting color of a predicted lighting source illuminating the parameter-based-rendering 528 in terms of a red code, a green code, a blue code, a hue metric, a saturation metric, or a color-value metric. It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to understand that the user can use “the light-source-color controls” to modify one or more of the hue value, the saturation value, and the brightness value for each pixel in order to control “ground truth input is less white than a ground truth output”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 10, Wang discloses everything claimed as applied above (see claim 1).
However, Wang fails to disclose wherein the light conditions indicate a plurality of light sources, each light source of the plurality of light sources comprising a direction, a hue value, a saturation value, and a brightness value.
In additional, Sunkavalli discloses (Paragraph [0006], the disclosed systems identify a request to render a virtual object at a designated position within a digital image ... the systems further generate 3D-source-specific-lighting parameters utilizing parametric-specific-network layers of the source-specific-lighting-estimation-neural network. In response to the request to render, the systems accordingly render a modified digital image comprising the virtual object at the designated position illuminated according to the 3D-source-specific-lighting parameters) wherein the light conditions indicate a plurality of light sources, each light source of the plurality of light sources comprising a direction, a hue value, a saturation value, and a brightness value (Paragraphs [0132]-[0134], As shown in FIG. 5D, for example, the source-specific-lighting-parameters controls 530a include a couple of positioning controls ... the source-specific-lighting-parameters controls 530a further include a light-source-direction control 536 ... the source-specific-lighting-parameter controls 530a also include light-source-color controls 544. The light-source-color controls 544 include controls for various color metrics. For example, the light-source-color controls 544 include a control for a red code, a green code, and a blue code and (additionally or alternatively) a hue metric, a saturation metric, and a color-value metric).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 11, Wang discloses everything claimed as applied above (see claim 1).
However, Wang fails to disclose wherein the generating the ground truth outputs generates ground truth output images, the generating the ground truth inputs generates ground truth input images, and wherein a ground truth output image of the ground truth output image comprises lighting properties of a corresponding ground truth input image.
In additional, Sunkavalli discloses (Paragraph [0006], the disclosed systems identify a request to render a virtual object at a designated position within a digital image ... the systems further generate 3D-source-specific-lighting parameters utilizing parametric-specific-network layers of the source-specific-lighting-estimation-neural network. In response to the request to render, the systems accordingly render a modified digital image comprising the virtual object at the designated position illuminated according to the 3D-source-specific-lighting parameters) wherein the generating the ground truth outputs (FIGS. 1 and 2; paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating a ground-truth-environment map 206) generates ground truth output images (Paragraph [0156], FIG. 9 further depicts a ground-truth-environment map 902 corresponding to the digital image 900 and a ground-truth-object rendering 908 based on lighting conditions from the ground-truth-environment map 902), the generating the ground truth inputs (Paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating ground-truth-source-specific-lighting parameters 208a-208n corresponding to the digital training image 204) generates ground truth input images (FIG. 9; paragraph [0156], the lighting estimation system 108 applies a source-specific-lighting-estimation-neural network to a digital image 900 to generate 3D-source-specific-lighting parameters ... By contrast, researchers apply Gardner's neural network to the digital image 900 to generate non-parametric lighting representations of the digital image 900. Based on Gardner's non-parametric lighting representations, the researchers reconstruct a lighting environment map 904. Gardner's system further generates a lighting-based-object rendering 910 comprising the virtual objects based on the non-parametric lighting representations illuminating a center position of the digital image 900), and wherein a ground truth output image of the ground truth output image comprises lighting properties of a corresponding ground truth input image (Paragraph [0134], as further shown in FIG. 5D, the source-specific-lighting-parameter controls 530a also include light-source-color controls 544. The light-source-color controls 544 include controls for various color metrics. For example, the light-source-color controls 544 include a control for a red code, a green code, and a blue code and (additionally or alternatively) a hue metric, a saturation metric, and a color-value metric).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 12, the combination of Wang in view of Sunkavalli discloses everything claimed as applied above (see claim 11). As discloses in claim 11, Sunkavalli discloses wherein the lighting properties comprise at least one of: hue values, saturation values, or brightness values for each of a plurality of pixels of the corresponding ground truth input image (Paragraph [0134], as further shown in FIG. 5D, the source-specific-lighting-parameter controls 530a also include light-source-color controls 544. The light-source-color controls 544 include controls for various color metrics. For example, the light-source-color controls 544 include a control for a red code, a green code, and a blue code and (additionally or alternatively) a hue metric, a saturation metric, and a color-value metric).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 14, Wang discloses everything claimed as applied above (see claim 13).
However, Wang fails to disclose wherein the generating the plurality of ground truth inputs generates a plurality of first images and wherein the generating the plurality of ground truth outputs generates a plurality of second images.
In additional, Sunkavalli discloses (Paragraph [0006], the disclosed systems identify a request to render a virtual object at a designated position within a digital image ... the systems further generate 3D-source-specific-lighting parameters utilizing parametric-specific-network layers of the source-specific-lighting-estimation-neural network. In response to the request to render, the systems accordingly render a modified digital image comprising the virtual object at the designated position illuminated according to the 3D-source-specific-lighting parameters) wherein the generating the plurality of ground truth inputs (FIGS. 1 and 2; paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating ground-truth-source-specific-lighting parameters 208a-208n corresponding to the digital training image 204) generates a plurality of first images (FIG. 9; paragraph [0156], the lighting estimation system 108 applies a source-specific-lighting-estimation-neural network to a digital image 900 to generate 3D-source-specific-lighting parameters ... By contrast, researchers apply Gardner's neural network to the digital image 900 to generate non-parametric lighting representations of the digital image 900. Based on Gardner's non-parametric lighting representations, the researchers reconstruct a lighting environment map 904. Gardner's system further generates a lighting-based-object rendering 910 comprising the virtual objects based on the non-parametric lighting representations illuminating a center position of the digital image 900) and wherein the generating the plurality of ground truth outputs (Paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating a ground-truth-environment map 206) generates a plurality of second images (Paragraph [0156], FIG. 9 further depicts a ground-truth-environment map 902 corresponding to the digital image 900 and a ground-truth-object rendering 908 based on lighting conditions from the ground-truth-environment map 902).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 15, Wang discloses everything claimed as applied above (see claim 13).
However, Wang fails to disclose wherein the generating the ground truth outputs generates ground truth output images, the generating the ground truth inputs generates ground truth input images, and wherein a ground truth output image of the ground truth output image comprises lighting properties of a corresponding ground truth input image.
In additional, Sunkavalli discloses (Paragraph [0006], the disclosed systems identify a request to render a virtual object at a designated position within a digital image ... the systems further generate 3D-source-specific-lighting parameters utilizing parametric-specific-network layers of the source-specific-lighting-estimation-neural network. In response to the request to render, the systems accordingly render a modified digital image comprising the virtual object at the designated position illuminated according to the 3D-source-specific-lighting parameters) wherein the generating the ground truth outputs (FIGS. 1 and 2; paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating ground-truth-source-specific-lighting parameters 208a-208n corresponding to the digital training image 204) generates ground truth output images (FIG. 9; paragraph [0156], the lighting estimation system 108 applies a source-specific-lighting-estimation-neural network to a digital image 900 to generate 3D-source-specific-lighting parameters ... By contrast, researchers apply Gardner's neural network to the digital image 900 to generate non-parametric lighting representations of the digital image 900. Based on Gardner's non-parametric lighting representations, the researchers reconstruct a lighting environment map 904. Gardner's system further generates a lighting-based-object rendering 910 comprising the virtual objects based on the non-parametric lighting representations illuminating a center position of the digital image 900), the generating the ground truth inputs (Paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating a ground-truth-environment map 206) generates ground truth input images (Paragraph [0156], FIG. 9 further depicts a ground-truth-environment map 902 corresponding to the digital image 900 and a ground-truth-object rendering 908 based on lighting conditions from the ground-truth-environment map 902), and wherein a ground truth output image of the ground truth output image comprises lighting properties of a corresponding ground truth input image (Paragraph [0134], as further shown in FIG. 5D, the source-specific-lighting-parameter controls 530a also include light-source-color controls 544. The light-source-color controls 544 include controls for various color metrics. For example, the light-source-color controls 544 include a control for a red code, a green code, and a blue code and (additionally or alternatively) a hue metric, a saturation metric, and a color-value metric).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 16, the combination of Wang in view of Sunkavalli discloses everything claimed as applied above (see claim 15). As discloses in claim 15, Sunkavalli discloses wherein the lighting properties comprise estimates of at least one of: hue values, saturation values, and brightness values for each of a plurality of pixels of the corresponding ground truth input image (Paragraph [0134], as further shown in FIG. 5D, the source-specific-lighting-parameter controls 530a also include light-source-color controls 544. The light-source-color controls 544 include controls for various color metrics. For example, the light-source-color controls 544 include a control for a red code, a green code, and a blue code and (additionally or alternatively) a hue metric, a saturation metric, and a color-value metric).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 18, Wang discloses everything claimed as applied above (see claim 17).
However, Wang fails to disclose wherein the generating the plurality of ground truth inputs generates a plurality of first images and wherein the generating the plurality of ground truth outputs generates a plurality of second images.
In additional, Sunkavalli discloses (Paragraph [0006], the disclosed systems identify a request to render a virtual object at a designated position within a digital image ... the systems further generate 3D-source-specific-lighting parameters utilizing parametric-specific-network layers of the source-specific-lighting-estimation-neural network. In response to the request to render, the systems accordingly render a modified digital image comprising the virtual object at the designated position illuminated according to the 3D-source-specific-lighting parameters) wherein the generating the plurality of ground truth inputs (FIGS. 1 and 2; paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating ground-truth-source-specific-lighting parameters 208a-208n corresponding to the digital training image 204) generates a plurality of first images (FIG. 9; paragraph [0156], the lighting estimation system 108 applies a source-specific-lighting-estimation-neural network to a digital image 900 to generate 3D-source-specific-lighting parameters ... By contrast, researchers apply Gardner's neural network to the digital image 900 to generate non-parametric lighting representations of the digital image 900. Based on Gardner's non-parametric lighting representations, the researchers reconstruct a lighting environment map 904. Gardner's system further generates a lighting-based-object rendering 910 comprising the virtual objects based on the non-parametric lighting representations illuminating a center position of the digital image 900) and wherein the generating the plurality of ground truth outputs (Paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating a ground-truth-environment map 206) generates a plurality of second images (Paragraph [0156], FIG. 9 further depicts a ground-truth-environment map 902 corresponding to the digital image 900 and a ground-truth-object rendering 908 based on lighting conditions from the ground-truth-environment map 902).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 19, Wang discloses everything claimed as applied above (see claim 17).
However, Wang fails to disclose wherein the generating the ground truth outputs generates ground truth output images, the generating the ground truth inputs generates ground truth input images, and wherein a ground truth output image of the ground truth output image comprises lighting properties of a corresponding ground truth input image.
In additional, Sunkavalli discloses (Paragraph [0006], the disclosed systems identify a request to render a virtual object at a designated position within a digital image ... the systems further generate 3D-source-specific-lighting parameters utilizing parametric-specific-network layers of the source-specific-lighting-estimation-neural network. In response to the request to render, the systems accordingly render a modified digital image comprising the virtual object at the designated position illuminated according to the 3D-source-specific-lighting parameters) wherein the generating the ground truth outputs (FIGS. 1 and 2; paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating a ground-truth-environment map 206) generates ground truth output images (Paragraph [0156], FIG. 9 further depicts a ground-truth-environment map 902 corresponding to the digital image 900 and a ground-truth-object rendering 908 based on lighting conditions from the ground-truth-environment map 902), the generating the ground truth inputs (Paragraph [0052], the digital imagery system 106 optionally uses an HDR-panoramic image 202 (sometimes referred to as an HDR-environment map) as a basis or a precursor image for extracting a digital training image 204 and generating ground-truth-source-specific-lighting parameters 208a-208n corresponding to the digital training image 204) generates ground truth input images (FIG. 9; paragraph [0156], the lighting estimation system 108 applies a source-specific-lighting-estimation-neural network to a digital image 900 to generate 3D-source-specific-lighting parameters ... By contrast, researchers apply Gardner's neural network to the digital image 900 to generate non-parametric lighting representations of the digital image 900. Based on Gardner's non-parametric lighting representations, the researchers reconstruct a lighting environment map 904. Gardner's system further generates a lighting-based-object rendering 910 comprising the virtual objects based on the non-parametric lighting representations illuminating a center position of the digital image 900), and wherein a ground truth output image of the ground truth output image comprises lighting properties of a corresponding ground truth input image (Paragraph [0134], as further shown in FIG. 5D, the source-specific-lighting-parameter controls 530a also include light-source-color controls 544. The light-source-color controls 544 include controls for various color metrics. For example, the light-source-color controls 544 include a control for a red code, a green code, and a blue code and (additionally or alternatively) a hue metric, a saturation metric, and a color-value metric).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Regarding claim 20, the combination of Wang in view of Sunkavalli discloses everything claimed as applied above (see claim 19). As discloses in claim 19, Sunkavalli discloses wherein the lighting properties comprise estimates of at least one of:
hue values, saturation values, and brightness values for each of a plurality of pixels of the corresponding ground truth input image (Paragraph [0134], as further shown in FIG. 5D, the source-specific-lighting-parameter controls 530a also include light-source-color controls 544. The light-source-color controls 544 include controls for various color metrics. For example, the light-source-color controls 544 include a control for a red code, a green code, and a blue code and (additionally or alternatively) a hue metric, a saturation metric, and a color-value metric).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system for assessing video performance analysis taught by Wang incorporate the teachings of Sunkavalli, and applying the digital imagery system taught by Sunkavalli to implement the digital imagery system with the details of the rendering steps, and use the generated ground truth to render images. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify Wang according to the relied-upon teachings of Sunkavalli to obtain the invention as specified in claim.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Xilin Guo whose telephone number is (571)272-5786. The examiner can normally be reached Monday - Friday 9:00 AM-5:30 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XILIN GUO/Primary Examiner, Art Unit 2616