Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
The term “second intermediate value X and a second intermediate value Y based on the second intermediate value Y.” in claim 18 does not make sense, it would be a typo. The term “second intermediate value X and a second intermediate value Y based on the second intermediate value Y.” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The specifications heavily imply that the claim should read “second intermediate value X and second intermediate value Z…” as the three tristimulus values the invention uses are XYZ. The claim is rejected as such in the 103 rejection below. Nonetheless, as it reads the claim is either circularly defining value Y as being based on itself or is defining value Y as being based on another value Y, neither of the situation is made clear by the specifications.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7, 8, are rejected under 35 U.S.C. 103 as being unpatentable over Greenebaum (US 8704859 B2) in view of Jeffrey (US 20060274937 A1)
Regarding claim 1,
Greenebaum:
A true color image enhancement compensation system comprising: an ambient light sensor operational to measure an ambient light level;(Greenebaum Col. 3 Line 30 “In one embodiment disclosed herein, information is received from one or more optical sensors, e.g., an ambient light sensor, an image sensor, or a video camera, and the display device's characteristics are determined using sources such as the display device's ICC profile.”) receive a gray shade look up table;(Greenebaum Col. 3 Line 4 “When an author creates graphical content (e.g., video, image, painting, etc.) on a given display device, they pick colors as appropriate and may fine tune characteristics such as hue, tone, contrast until they achieve the desired result.” Col. 3 Line 30 “In one embodiment disclosed herein, information is received from one or more optical sensors, e.g., an ambient light sensor … Next, an ambient model predicts the effect on a viewer's perception due to ambient environmental conditions. In one embodiment, the ambient model may then be used to determine how the values stored in a LUT should be modified to account for the effect that the environment has on the viewer's perception. For example, the modifications to the LUT may add or remove gamma or modify the blackpoint or white point of the display devices tone response curve, or perform some combination thereof, before sending the image data to the display.” Col. 9 Line 47 “The modifications to the LUT may comprise modifications to add or remove gamma from the system or to modify the black point or white point of the system. “Black point” may be defined as the level of light intensity below which no further detail may be perceived by a viewer, “White point” may be defined as the set of values that serve to define the color “white” in the color space.” Col. 9 Line 60 “Once this level of diffuse reflection is determined, the black point may be adjusted accordingly. For example, if all luminance values below an 8-bit value of 40 would be imperceptible over the level of diffuse reflection (though this is likely an extreme example), the system 600 may set the black point to be 40, thus compressing the pixel luminance values into the range of 41-255.” Note: Greenebaum teaches a LUT (look up table) for an input video or other graphical content that can store black/white point values. The black/white point of data is directly analogous to the claims grey shade values, as seen from Col. 9 Line 60 if expressed through an 8-bit value then 0 is black, and 255 is white, making all values in between some level of grey. As Greenebaum teaches a look up table of these values, Greenebaum teaches a look up table of grey shades.) and generate an output video signal by converting a plurality of gray shades in the input video signal based on a plurality of input video luminance ratios and the gray shade look up table to determine a plurality of new output gray shade values;(Greenebaum Col. 3 Line 34 “Next, an ambient model predicts the effect on a viewer's perception due to ambient environmental conditions. In one embodiment, the ambient model may then be used to determine how the values stored in a LUT should be modified to account for the effect that the environment has on the viewer's perception. For example, the modifications to the LUT may add or remove gamma or modify the black point or white point of the display device's tone response curve, or perform some combination thereof, before sending the image data to the display.” Col. 9 Line 40 “One embodiment of an ambient-aware model for dynamically adjusting a display's characteristic disclosed herein takes information from one or more optical sensors 404 and display profile 104 and makes a prediction of the effect on viewing conditions and the viewer's perception due to ambient conditions. The result of that prediction is used to determine how system 600 modifies the LUT, such that it now serves as an “ambient-aware” LUT 602. The modifications to the LUT may comprise modifications to add or remove gamma from the system or to modify the black point or white point of the system. “Black point” may be defined as the level of light intensity below which no further detail may be perceived by a viewer, “White point” may be defined as the set of values that serve to define the color “white” in the color space.” Note: A “luminance ratio” is a known, industry standard term that is maximum luminance over minimum luminance. Maximum luminance is the display’s max brightness/white with the ambient light, and the minimum luminance is the maximum (darkest) black value plus the ambient light. Thus, a luminance ratio conveys info on the actual perceivable darkest and lightest values a display can output given the ambient light. Greenebaum Col. 9 Line 47, cited previously, provides an example where multiple luminance values in a LUT are below a value of 40 on the 0-255 8-bit greyscale value range. Due to the ambient light in the room causing reflections it is known that values below 40 will look the same. Thus, the luminance ratio is altered as the minimum luminance, or darkest black, is increased to 40. As clarified by Greenebaum Col. 3 line 34, these modifications made to the grey shade values in the LUT are done before the image content is output to the display, thus teaching that these modifications are made to generate a corrected output video signal) to maintain input video color coordinates; (Greenebaum Col. 1 Line 43 “The ICC profile is a set of data that characterizes a color input or output device, or a color space, according to standards promulgated by the International Color Consortium (ICC). ICC profiles may describe the color attributes of a particular device or viewing requirement by defining a mapping between the device source or target color space and a profile connection space (PCS), usually the CIE XYZ color space.” Col. 14 line 3 “The overall goal of some color adaptation models may be to understand how the source material is ideally intended to “look” on a viewer's display. In a typical scenario for video, the ideal viewing conditions may be modeled as a broadcast monitor, in a dim broadcast studio environment lit by 16 lux of CIE Standard Illuminant D65 light. This source rendering intent may be modeled, e.g., by attaching an ICC profile to the source. The attachment of a profile to the source may allow the display device to interpret and render the content according to the source creator's “rendering intent.” Once the rendering intent has been determined, the display device may then determine how to transformation the source content to make it match the ideal appearance on display device” Note: The specifications define ‘color coordinates’ as ¶47 “The tristimulus values determined in Equation 13 may then be used to calculate the 1931 CIE x,y color coordinates per Equations 14 and 15, as follows:”
PNG
media_image1.png
388
736
media_image1.png
Greyscale
It can be seen that the color coordinates are defined as x,y, and z, and originate from the CIE color coordinates. These same CIE color coordinates are used by Greenebaum where it is taught that the color coordinates are attempted to be preserved by a color adaptation model that aims to preserve the intended color of a given display with its knowledge of the display and the CIE coordinates.) and a processor operational to: develop the gray shade look up table based on the video data and the ambient light level; (Greenebaum Col 9. Line 47, cited above, teaches adjusting multiple greyscale values contained in a look up table based on ambient light levels.) dynamically remap the plurality of gray shades in the gray shade look up table in response to the ambient light signal (Greenebaum Col. 9 Line 40, cited above, specifically states the values of a gray shade values of an LUT that are modified based on the ambient light are adjusted dynamically.) to compress a first region of the plurality of gray shades and stretch a second region of the plurality of gray shades;(Col. 9 Line 60 “Once this level of diffuse reflection is determined, the black point may be adjusted accordingly. For example, if all luminance values below an 8-bit value of 40 would be imperceptible over the level of diffuse reflection (though this is likely an extreme example), the system 600 may set the black point to be 40, thus compressing the pixel luminance values into the range of 41-255. In one particular embodiment, this “black point compensation' is performed by “stretching or otherwise modifying the values in the LUT, as is discussed further below in reference to FIG. 9.” Note: Greenebaum teaches an example where because of the ambient lighting present black values below 40 are not perceptibly different. To correct this, the region from 0-40 is compressed and the new minimum luminance is 41, this is the claims “first region” that is compressed. The new remaining range 41-255 is the claims “second region” and must now be stretched to convey the same range previously conveyed by a wider range, 0-255.) dynamically remap the plurality of gray shades in the gray shade look up table to correct for ambient lighting conditions; (Greenebaum Col. 9 Line 40, cited above, clarifies that these corrections made for ambient lighting conditions performed by modifying/remapping values in the LUT like grey shade values is done dynamically.) and transfer the gray shade look up table to the circuit.(
PNG
media_image2.png
472
666
media_image2.png
Greyscale
Greenebaum Col. 3 Line 30, cited previously, teaches that once the input video data is processed into the output video stream it will be sent to the display. As seen in Fig. 6, the LUT after it has been adjusted to be ambient aware will be sent to the display, which contains its own onboard circuit, thus teaching transferring the LUT to the circuit. )
While Greenebaum teaches generating an output video signal by adjusting the grey scale values in the image data stored in a look up table to account for ambient lighting conditions by contrasting and stretching different grey scale regions it does not teach that the grey scale/shade data is placed into a histogram. This is taught by Jeffery which teaches a circuit operational to: generate a histogram based on an input video signal;(Jeffrey ¶24 “Image capture device 212 (e.g., digital video camera, digital camera, etc.) records photographic images as image data and outputs the raw image data to graphics controller 206 for display on display panel 208.” ¶3 “One method for evaluating the contrast of a digital image includes the use of a histogram table. The histogram table represents a frequency of occurrence of each pixel value within the digital image. Therefore, evaluation of the histogram table can identify similar pixel values that occur with sufficient frequency in the image such that contrast is adversely affected.” Note: Jeffery teaches a histogram is made from the digital image, where the digital image originates from recorded data making the images frames of a video. While Jeffery teaches that a histogram table is made as opposed to a histogram the two are functionally identical, with one simply displaying the data in a table as opposed to the bars in a traditional histogram. From the perspective of the computer the two are identical and contain the exact same data, the difference in how the data is visualized is only relevant for a human observer.) export the histogram; and a processor operational to: receive the histogram from the circuit; (Jeffrey ¶33 “After the cumulative histogram value is calculated, cumulative histogram calculation circuitry 410 sends the cumulative histogram value to cumulative histogram normalization circuitry 412 for normalization.” Note: Jeffery teaches the histogram is sent from the circuit it is made on to another circuit capable of processing the histogram as it will normalize it, thus teaching that a circuit can export the histogram to a processor.)
A portion of claim 1 recites to develop the gray shade look up table based on the histogram that is not addressed as it introduces no new content. Jeffery teaches making a histogram from video data and Greenebaum teaches leveraging the video data to develop the grey shade look up table. A histogram does not transform or alter data in any way and is simply a way to visualize data for a human observer. The only new info introduced is the division of the data into bins/groups to form the bars of a histogram. The specifications or claims make no reference to the bins or groups of a histogram being relevant, thus, whenever the claims reference using the histogram, or having something “based on the histogram” it is simply stating that it is using the input video data, which Greenebaum has clearly been shown to use, is used..
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Jeffrey where a system that accepts input video data, creates a look up table of grey shade values to adjust the grey shade values with respect to sensed ambient light by stretching and compressing different regions of the grey shades to create an output video signal will create a histogram based on the input video data on a circuit and export the histogram to a processor.
There are several reasons that would motivate one to do so, a histogram is a common means for displaying, storing, and visualizing data by bins/groups. If one wished for a human observer to be able to understand part of the process to have the effect of making the overall system more comprehensible a histogram which provides a means to observe and understand the data in a visual format could be leveraged.
Regarding claim 2,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the processor includes: a constant contrast table (Note:
PNG
media_image3.png
133
710
media_image3.png
Greyscale
Regardless of the typo where “contrast” is listed twice the specifications seem to define a contrast table as simply applying the contrast ratio change to the gray shade values which are stored in the LUT, and not as an entirely different table or set of data. As Greenebaum Col. 9 Line 40 teaches adjusting grey shade values to a contrast ratio Greenebaum teaches using a contrast table) operational to adjust each successive gray shade of the plurality of gray shades to a constant contrast ratio. (Greenebaum Col. 9 Line 40 “One embodiment of an ambient-aware model for dynamically adjusting a display's characteristic disclosed herein takes information from one or more optical sensors 404 and display profile 104 and makes a prediction of the effect on viewing conditions and the viewer's perception due to ambient conditions. The result of that prediction is used to determine how system 600 modifies the LUT, such that it now serves as an “ambient-aware” LUT 602. The modifications to the LUT may comprise modifications to add or remove gamma from the system or to modify the black point or white point of the system. Col. 9 Line 60 “Once this level of diffuse reflection is determined, the black point may be adjusted accordingly. For example, if all luminance values below an 8-bit value of 40 would be imperceptible over the level of diffuse reflection (though this is likely an extreme example), the system 600 may set the black point to be 40, thus compressing the pixel luminance values into the range of 41-255. In one particular embodiment, this “black point compensation' is performed by “stretching or otherwise modifying the values in the LUT, as is discussed further below in reference to FIG. 9.” Note: In claim 1 a luminance ratio was discussed and represents max luminance over minimum luminance, where the max/min luminance is the brightest white/darkest black that the display can obtain plus the ambient light. The contrast ratio is the same but without the ambient light representing the theoretical brightest white/darkest black. Thus, when Greenebaum in Col. 9 line 60 teaches increasing the lowest black value a grey shade value can be from 0 to 41 it teaches adjusting the pixel values to fit a contrast ratio as the minimum contrast has been changed. As clarified by Greenebaum Col. 9 line 40 this changes the unmodified LUT so “that it now serves as an ‘ambient-aware’ LUT”, thus all future pixel values input will be fit to the new contrast ratio which has a changed minimum contrast. This teaches the claims language that each successive gray shade will be fit to the contrast ratio.)
Regarding claim 3,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the processor includes: a gamma table (Note: The specifications define a gamma table as
PNG
media_image4.png
132
700
media_image4.png
Greyscale
As seen from the definition the gamma assignment table is simply the process of converting the input gray shade values to output gray shade values via a gamma offset. While the particular equation 110 that is later listed is not taught the gamma table itself is taught as Greenebaum Col. 3 Line 30, cited below, teaches adding or removing gamma to change the gray shade values of an image to be sent to the display.) operational to adjust each successive gray shade of the plurality of gray shades to follow an offset gamma function. (Greenebaum Col. 3 Line 30 “In one embodiment disclosed herein, information is received from one or more optical sensors, e.g., an ambient light sensor … Next, an ambient model predicts the effect on a viewer's perception due to ambient environmental conditions. In one embodiment, the ambient model may then be used to determine how the values stored in a LUT should be modified to account for the effect that the environment has on the viewer's perception. For example, the modifications to the LUT may add or remove gamma or modify the blackpoint or white point of the display devices tone response curve, or perform some combination thereof, before sending the image data to the display.” Col. 6 Line 29 “System 112 may then utilize a LUT 110 to perform a so-called “gamma adjustment process.” LUT 110 may comprise a two-column table of positive, real values spanning a particular range, e.g., from zero to one. The first column values may correspond to an input image value, whereas the second column value in the corresponding row of the LUT 110 may correspond to an output image value that the input image value will be “transformed” into before being ultimately being displayed on display 114. LUT 110 may be used to account for the imperfections in the display 114's luminance response curve, also known as a transfer function.” Note: Aside from teaching that gray shade values in the LUT can be adjusted by contrasting and stretching a region of the shades Greenebaum also directly states that changing the gamma can change the gray shade values. Once the LUT is updated all future input values will be output in the same manner with the gamma adjustment or other change, thus teaching that the successive grey shades in future image frames will also be modified by the LUT. A “gamma offset function” as referenced by the claims simply refers to a function which will change, or offset, the gamma in the entire display rather than only certain areas. This is taught, as Greenebaum teaches that the entire output image will be transformed with the adjusted gamma.)
Regarding claim 4,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the processor includes: a plurality of table modifiers operational (Greenebaum Col. 12 Line 11, cited below, teaches shaping functions that can modify the plurality of gray shade values. Note: A “table modifier” is defined by the specifications as ¶350 “Assignment Table Modifiers block 268: [0351] Block 268 provides an assignment table developed per Block 262 and is modified by multiplying other shaping functions” As a table modifier is simply applying a shaping function, and Greenebaum teaches applying shaping functions, Greenebaum teaches the use of table modifiers.) to modify one of a plurality of shaping functions for the plurality of gray shades. (Greenebaum Col. 12 Line 11 “Referring now to FIG. 9, a graph 900 representative of a LUT transformation and a graph 906 representative of a reshaped display response curve are shown, in accordance with one embodiment. As mentioned above in reference to FIG. 3, the x-axis of LUT transformation graph 900 represents input image values spanning a particular range, e.g., from zero to one. The y-axis of LUT transformation graph 900 represents output image values spanning a particular range, e.g., from zero to one. As mentioned above in reference to FIG. 8, it may be beneficial to adjust the black point of the system, such that the lowest input value sent to the LUT will be translated into an image value capable of eliciting an illuminance response in the display device that is sufficient to overcome the diffuse reflection levels.”
PNG
media_image5.png
478
684
media_image5.png
Greyscale
Note: While a table modifier, or plurality of table modifiers is indefinite as established by the previous 122(b) rejection, Greenebaum is shown to teach a shaping function. A shaping function in this context accepts a value relating to display such as color, brightness, opacity, etc... Fig. 9 shows an example where the look up table gray shade and their default output are in graph 900, graph 906 shows the range of input values possible from the look up table after they been ‘reshaped’. The reshaped display response curve that maps the same input x values, the look up table inputs, to different y values than the default graph is the curve of the shaping function that has been applied to the look up table inputs. Thus, Greenebaum teaches a shaping function to modify the plurality of grey shades.)
Regarding claim 7,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the circuit includes: a frame rate controller operational to convert high-dynamic range video data to dithered video. (Col. 15 Line 34 “Performing black point compensation at this stage of the process may also advantageously allow for the application of dithering to mitigate banding problems in the resultant display caused by, e.g., the compression of the source material into fewer, visible levels.” Note: Greenebaum teaches that dithering can be performed on the input video. Dithering is simply the introduction of noise to an image often to randomize reduce visual errors like banding as also taught by Greenebaum, thus when dithering is applied to the normal high-dynamic range video input it will become a dithered video.)
Regarding claim 8,
Greenebaum teaches:
A true color image enhancement compensation system comprising: an ambient light sensor operational to measure an ambient light level;(Greenebaum Col. 3 Line 30, cited in claim 1, teaches an ambient light sensor to measure ambient light.) a circuit operational to: receive a gray shade look up table;( (Greenebaum Col. 3 Line 4, cited in claim 1, teaches a gray shade look up table.) dynamically remap a plurality of gray shades in the gray shade look up table using a highest remapped color to determine and maintain a plurality of input luminance ratios of a plurality of lower gray shade colors; (Greenebaum Col. 3 Line 30 “In one embodiment disclosed herein, information is received from one or more optical sensors, e.g., an ambient light sensor … Next, an ambient model predicts the effect on a viewer's perception due to ambient environmental conditions. In one embodiment, the ambient model may then be used to determine how the values stored in a LUT should be modified to account for the effect that the environment has on the viewer's perception. For example, the modifications to the LUT may add or remove gamma or modify the blackpoint or white point of the display devices tone response curve, or perform some combination thereof, before sending the image data to the display.” Col. 9 Line 47 “The modifications to the LUT may comprise modifications to add or remove gamma from the system or to modify the black point or white point of the system. “Black point” may be defined as the level of light intensity below which no further detail may be perceived by a viewer, “White point” may be defined as the set of values that serve to define the color “white” in the color space.” Col. 9 Line 60 “Once this level of diffuse reflection is determined, the black point may be adjusted accordingly. For example, if all luminance values below an 8-bit value of 40 would be imperceptible over the level of diffuse reflection (though this is likely an extreme example), the system 600 may set the black point to be 40, thus compressing the pixel luminance values into the range of 41-255.” Note: As established previously a luminance ratio is the max luminance, the ‘highest’ gray shade value or the whitest white plus the ambient light, over the min luminance, the ‘lowest’ gray shade value or the darkest black plus the ambient light. The claims teach that the gray shade values are remapped using the ‘highest remapped color’. Greenebaum Col. 9 Line 60 teaches this exactly as it is taught that either the “black point”, the “white point” which are the highest and lowest colors can be changed in order to remap the gray shade values. An example is given where the black point, or lowest color, is changed from 0 to 41 which contrasts and stretches different regions of the gray shade range.) and generate an output video signal by converting a plurality of gray shades in the input video signal based on a plurality of input video luminance ratios and the gray shade look up table to determine a plurality of new output gray shade values (Greenebaum Col. 9 line 60 and Col. 3 Line 30, cited above, teach generating an output video with gray shade values adjusted to compensate for the ambient light by adjusting the white/black point, which implicitly alters the luminance ratios. As this is done for each image which each contain a plurality of gray shades in their LUT Greenebaum teaches generating an output video from a plurality of gray shades based on the plurality of input luminance ratios, where each individual frame will have its own luminance ratio.) to maintain input video color coordinates (Col. 1 Line 43 “The ICC profile is a set of data that characterizes a color input or output device, or a color space, according to standards promulgated by the International Color Consortium (ICC). ICC profiles may describe the color attributes of a particular device or viewing requirement by defining a mapping between the device source or target color space and a profile connection space (PCS), usually the CIE XYZ color space.” Col. 14 Line 3 “The overall goal of some color adaptation models may be to understand how the source material is ideally intended to “look” on a viewer's display. In a typical scenario for video, the ideal viewing conditions may be modeled as a broadcast monitor, in a dim broadcast studio environment lit by 16 lux of CIE Standard Illuminant D65 light. This source rendering intent may be modeled, e.g., by attaching an ICC profile to the source. The attachment of a profile to the source may allow the display device to interpret and render the content according to the source creator's “rendering intent.” Once the rendering intent has been determined, the display device may then determine how to transformation the source content to make it match the ideal appearance on display device” Note: The specifications define ‘color coordinates’ as ¶47 “The tristimulus values determined in Equation 13 may then be used to calculate the 1931 CIE x,y color coordinates per Equations 14 and 15, as follows:”
PNG
media_image1.png
388
736
media_image1.png
Greyscale
It can be seen that the color coordinates are defined as x,y, and z, and originate from the CIE color coordinates. These same CIE color coordinates are used by Greenebaum where it is taught that the color coordinates are attempted to be preserved by a color adaptation model that aims to preserve the intended color of a given display with its knowledge of the display and the CIE coordinates.) and transfer the gray shade look up table to the circuit. (Greenebaum Col. 3 Line 30 and Fig. 6, cited in claim 1, teach the transfer of the grey shade look up table to the circuit )
While Greenebaum teaches generating an output video signal by adjusting the grey scale values in the image data stored in a look up table to account for ambient lighting conditions by contrasting and stretching different grey scale regions it does not teach that the grey scale/shade data is placed into a histogram. This is taught by Jeffery which teaches a circuit operational to: generate a histogram based on an input video signal;(Jeffrey ¶24 “Image capture device 212 (e.g., digital video camera, digital camera, etc.) records photographic images as image data and outputs the raw image data to graphics controller 206 for display on display panel 208.” ¶3 “One method for evaluating the contrast of a digital image includes the use of a histogram table. The histogram table represents a frequency of occurrence of each pixel value within the digital image. Therefore, evaluation of the histogram table can identify similar pixel values that occur with sufficient frequency in the image such that contrast is adversely affected.” Note: Jeffery teaches a histogram is made from the digital image, where the digital image originates from recorded data making the images frames of a video. While Jeffery teaches that a histogram table is made as opposed to a histogram the two are functionally identical, with one simply displaying the data in a table as opposed to the bars in a traditional histogram. From the perspective of the computer the two are identical and contain the exact same data, the difference in how the data is visualized is only relevant for a human observer.) export the histogram; and a processor operational to: receive the histogram from the circuit; (Jeffrey ¶33 “After the cumulative histogram value is calculated, cumulative histogram calculation circuitry 410 sends the cumulative histogram value to cumulative histogram normalization circuitry 412 for normalization.” Note: Jeffery teaches the histogram is sent from the circuit it is made on to another circuit capable of processing the histogram as it will normalize it, thus teaching that a circuit can export the histogram to a processor.)
A portion of claim 1 recites to develop the gray shade look up table based on the histogram that is not addressed as it introduces no new content. Jeffery teaches making a histogram from video data and Greenebaum teaches leveraging the video data to develop the grey shade look up table. A histogram does not transform or alter data in any way and is simply a way to visualize data for a human observer. The only new info introduced is the division of the data into bins/groups to form the bars of a histogram. The specifications or claims make no reference to the bins or groups of a histogram being relevant, thus, whenever the claims reference using the histogram, or having something “based on the histogram” it is simply stating that it is using the input video data, which Greenebaum has clearly been shown to use.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Jeffrey where a system that accepts input video data, creates a look up table of grey shade values to adjust the grey shade values with respect to sensed ambient light by stretching and compressing different regions of the grey shades to create an output video signal will create a histogram based on the input video data on a circuit and export the histogram to a processor.
There are several reasons that would motivate one to do so, a histogram is a common means for displaying, storing, and visualizing data by bins/groups. If one wished for a human observer to be able to understand part of the process to have the effect of making the overall system more comprehensible a histogram which provides a means to observe and understand the data in a visual format could be leveraged.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Greenebaum (US 8704859 B2) in view of Jeffrey (US 20060274937 A1) in view of Kurokawa (US 8552946 B2)
Regarding claim 5,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the processor includes:
Greenebaum does not however teach a gray shade detection threshold such that a specific percent of pixels are above the threshold, this is taught in Kurokawa which teaches a gray shade detection threshold operational to determine a maximum gray scale number such that a specific percentage of a plurality of pixels are above a threshold percentage.(Kurokawa Col. 12 Line 17 “As shown in FIG. 1B, in the display image, the t grayscale 301 at which the number of pixels of the t grayscale 301 or higher and the maximum grayscale (255 grayscale) or lower is p % 312 of the total number of pixels is referred to as a threshold grayscale t 301” Col. 44 Line 23 “The threshold value is a grayscale value corresponding to the position of upper several % in the histogram of the display screen.” Note: Kurokawa teaches that a specific threshold t of grayscale values of a display image is the greyscale value where a certain percent p of the pixels are above said threshold. This teaches the claims language of a “gray shade detection threshold” that is determined such that a specific percent of pixels are above the threshold.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Kurokawa where the pixels in the input video stream with grey shade values have a threshold determined such that a certain percent of the pixels have a gray shade value above the threshold.
There are several reasons that would motivate one to do so, when changing grey shade values in response to ambient light changes one could easily run into an issue where grey shade values are boosted to their maximum brightness and are unable to be boosted further. In such a situation, pixels that should be visually distinct with different grey shade values may look the same as they have been both over adjusted to the highest brightness. To allow for increasing in luminance/contrast without losing visual information in the image a threshold could be enforced so that the bright parts of an image remain bright relative to other pixel values.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Greenebaum (US 8704859 B2) in view of Jeffrey (US 20060274937 A1) in view of Greenebaum2 (US 20200380938 A1)
Regarding claim 6,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the processor includes:
Greenebaum does not however teach that the rate at which the gray shade values can be changed at is limited, this is taught by Greenebaum2 which teaches a maximum gray shade rate limiter operational to control a rate at which a breakpoint value (Greenebaum2 ¶3 “Potential changes in a viewer's perception of the displayed content include tonality changes (which may be modeled using a gamma function), as well as changes to white point (the absolute color perceived as being white) and black point (the highest brightness level indifferentiable from true black).” ¶7 “The output of the ambient conditions model may comprise modifications to the display's transfer function, gamma boost, tone mapping, re-saturation, black point, white point, or a combination thereof.” Note: The specifications provide no explanation as to what a “breakpoint” value is, going off of the broadest reasonable interpretation, in the context of image processing a breakpoint is a threshold where values above the breakpoint become rounded to being white, 1, and values below the breakpoint are rounded down to black, 0. This describes Greenbaum2’s black and white point, which are the levels where the tone of a pixel becomes viewed as simply black/white with no finer distinction. What this value will be is relative to the gray shaed values, the brightness, contrast, etc… Greenebaum2 teaches that this value is changed when the ambient conditions change, thus teaching a changing breakpoint value.) in adjusted to a single gray shade value each time increment. (Greenebaum2 ¶37 “Thus, if necessary, system 212 may then utilize LUT 210 to perform a so-called “gamma adjustment process.” LUT 210 may comprise a two-column table of positive, real values spanning a particular range, e.g., from zero to one. The first column values may correspond to an input image value, whereas the second column value in the corresponding row of the LUT 210 may correspond to an output image value that the input image value will be “transformed” into before being ultimately being displayed on display 102. ” ¶61 “As alluded to above, in some embodiments, the modifications to LUTs 560 may be implemented gradually (e.g., over a determined interval of time), via animation engine 555. According to some such embodiments, animation engine 555 may be configured to adjust the LUTs 560 based on the rate at which it is predicted the viewer's vision will adapt to the changes.” Note: Greenebaum2 teaches that the rate at which modifications to the LUT, which stores gray shade values, is limited based on a predicted rate that the viewer’s vision would adapt to the visual changes. This shows Greenebaum2 teaches a shade rate limiter that limits the rate at which gray shade values are changed by. Greenbaum2 has also been shown to teach breakpoint values are used and are determined based on ambient lighting conditions, as Greenebaum2 teaches that the grey shade values themselves will be adjusted with a limited rate then the breakpoint values, inherently defined by the gray shade values, will change at that rate also.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Greenebaum2 where a system that can update gray shade values based on ambient light can limit the rate at which gray shade values are updated, also meaning the rate at which the breakpoint is updated.
There are several reasons that would motivate one to do so, as gray shade values are already being adjusted in response to ambient light to create a clearer viewing experience the viewing experience of the user could be further enhanced by limiting the rate at which gray shade values can be updated as to allow the users eyes to adjust to the shift in brightness more easily.
Claims 9-13 are rejected under 35 U.S.C. 103 as being unpatentable over Greenebaum (US 8704859 B2) in view of Jeffrey (US 20060274937 A1) and further in view of Vo (US 11398017 B2)
Regarding claim 9,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the circuit is further configured to: find a gray shade
While Greenebaum teaches finding gray shade values and adjusting them to preserve color coordinates it does not teach finding a gray shade maximum input value among the three-color values of a pixel. This is taught by Vo which teaches find a gray shade maximum input value among three color values of an incoming pixel in the input video signal. (Vo
PNG
media_image6.png
284
252
media_image6.png
Greyscale
Col. 14 Line 17 “The maximum function unit 281 is configured to: (1) receive a mastered video (e.g., from the de-quantizer 220) including a red (R) code value, a green (G) code value, and a blue (B) code value, and (2) determine a maximum code value maxRGB of the R, G, and B code values by applying a maximum (i.e., max) function to the R, G, and B code values (e.g., max(R, G, B))” Col. 14 Line 24 “Electro-Optical Transfer Function (EOTF) unit 282. The EOTF unit 282 is configured to: (1) receive a maximum code value maxRGB (e.g., from the maximum function unit 281), and (2) determine a linear luminance value X by applying an EOTF to the maximum code value maxRGB.” Col. 14 Line 32 “The tone mapping unit 283 is configured to: (1) receive a linear luminance value X (e.g., from the EOTF unit 282), (2) receive a modified tone mapping function (i.e., modified tone mapping curve) (e.g., from the ambient light compensation system 270), and (3) determine (i.e., lookup) a tone mapping value Y that the modified tone mapping function maps the linear luminance value X to (e.g., tmLUT(X)).” Note: Vo teaches a linear luminance value X is obtained by an electrico optical transfer function (EOTF). The EOTF function accepts the output of max(R,G,B) as input, where the max function returns the maximum code value for the three color values. As defined by the specifications ¶141 “Step 1: Find the Gray Shade maximum input value, GSmaxin, from Input R1, G1, B1 Values and Lookup Multiplier Mmaxin using GSmaxin. [0142] The first step consists of two parts. First, the maximum value of the red, green, and blue gray shades of the incoming pixel is determined as a gray scale maximum input value GSmaxin.”, the output of Vo’s max function is the directly the same as the specifications ‘gray shade maximum input value’)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Vo where a gray shade max input value is obtained from the incoming color values of an incoming pixel.
There are several reasons that would motivate one to do so, when adjusting gray shade values in response to ambient light it is possible that the original tone of the image may represented incorrectly, to help preserve the color in the image it may be useful to collect relevant color data such as the most prominent color value of each given pixel aka the claims ‘gray shade maximum input value’.
Regarding claim 10,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the circuit is further configured to: lookup a multiplier value using a lookup table based on the gray shade maximum input value. (
PNG
media_image6.png
284
252
media_image6.png
Greyscale
Col. 14 Line 32 ”The tone mapping unit 283 is configured to: (1) receive a linear luminance value X (e.g., from the EOTF unit 282), (2) receive a modified tone mapping function (i.e., modified tone mapping curve) (e.g., from the ambient light compensation system 270), and (3) determine (i.e., lookup) a tone mapping value Y that the modified tone mapping function maps the linear luminance value X to (e.g., tmLUT(X)).” Note: It was previously shown how the ‘gray shade maximum input value’ is the output of a max(R,G,B) function. Here, it can be seen that that same ‘gray shade maximum input value’ is made into value X from another function, the value X is then input to a tone mapping look up table, tmLUT, which will map the linear luminance X to a tone mapping value Y. Y, the output of the LUT, provides the needed tone-mapping information to remap the color values but is viewed with X to produce the final multiplier r. As seen from table 1 the remapped color values R’, G’, and B’ are obtained by taking the original color values and applying the multiplier r. As r is simply Y/X, where Y is the value obtained from a look up table based on the gray shade max input, to multiply by r is also to multiply with Y. Thus, Vo teaches a multiplier that is obtained via a look up table that accepts the gray shade max as input.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Vo where a multiplier value is obtained from a look up table that accepts the gray shade max value as an input.
There are several reasons that would motivate one to do so, as the multiplier is what will eventually be applied to the color values to correct the ambient light’s influence the correction the multiplier applies must be specific to each pixel. This requires looking up each tone correction info per pixel, a process that could be made more efficient by finding it with a look up table.
Regarding claim 11,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the circuit is further configured to: lookup the highest remapped color using a transfer function based on the gray shade maximum input value. (
PNG
media_image6.png
284
252
media_image6.png
Greyscale
Col. 14 Line 24 “Electro-Optical Transfer Function (EOTF) unit 282. The EOTF unit 282 is configured to: (1) receive a maximum code value maxRGB (e.g., from the maximum function unit 281), and (2) determine a linear luminance value X by applying an EOTF to the maximum code value maxRGB.” Note: It has been established in claims 9 and 10 that the claims ‘gray shade maximum input value’ is Vo’s output of the max function, which is made into X. The final ‘highest remapped color’ is obtained by multiplying the color value by ‘r’, where r is determined by Y. Y was previously shown in claim 10 to be the output of a look up table that accepts the ‘gray shade max input value’ and is a tone mapping value specific to a pixel’s color value. Here, Vo teaches that the ‘highest remapped color’, is obtained or ‘looked up’ via a transfer function. An example can be seen where the remapped R’ that leverages the original color value R and the multiplier ‘r’ obtained from tone mapping info Y is input to a transfer function, OETF or optical-electical transfer function, to produce the final ‘highest remapped color’ R’.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Vo where the highest remapped color is obtained via a transfer function.
There are several reasons that would motivate one to do so, the highest remapped color value is part of pixel data that must be sent to the display, meaning it must be transferred from the device’s onboard processor/memory to the display. This final step which can consume time and processing power can be handled automatically if the highest remapped color value is always set to be the output of a transfer function to handle the transferring.
Regarding claim 12,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the circuit is further configured to: remap the plurality of gray shades
Greenebaum has already been shown that the plurality of gray shades can be remapped in claim 1. Greenebaum does not teach remapping based on a multiplier and a ‘gray shade maximum input value’, this is taught by Vo which teaches remapping based on the multiplier value and the gray shade maximum input value. (
PNG
media_image6.png
284
252
media_image6.png
Greyscale
Note: In claim 11 it was established that Vo’s R’, B’, and G’ values are the remapped color values. Here it can be seen that the remapping is done via the ‘gray shade maximum input value’ which has been established in claims 9 and 10 to be the output of the max(R,G,B) function. The output of the max() function, the gray shade mad input value, is used to make X, which is used to obtain a multiplier r. As the remapped color values can be seen to be obtained from multiplier r, which uses X which was obtained from the ‘gray shade maximum input value’, Vo teaches using the gray shade max input value and a multiplier to perform remapping.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Vo where the remapping that is performed via the gray shade max input value and a multiplier is remapping for the gray shade values.
There are several reasons that would motivate one to do so, Greenebaum already teaches correcting gray shade values based on ambient lighting, Vo teaches using gray shade values to similarly correct color values based on ambient lighting that involves a ‘gray shade max input value’ (the highest color value in a pixel) and a multiplier (a value that helps correct tone. One could enhance the gray shade remapping by leveraging the two described components in Vo that consider more information like color tone when remapping.
Regarding claim 13,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to
Greenebuam does not however detail a transfer function loadable from an external source to the circuit. This is taught by Vo which teaches wherein the transfer function is loadable from a source external to the circuit. (Vo Table 1, Col. 14 Line 24, teach a transfer function that transfers data from optical to electrical. This implicitly teaches the transfer function is loadable from an external source, as opposed to one internal circuit to another as to transfer from optical to electrical, or vice versa, requires a translation from an optical circuit to an electrical circuit which are two independent circuit types that cannot work together and require a light to electricity conversion.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Vo where the transfer functions is loadable from an external source to the circuit.
There are several reasons that would motivate one to do so, in a display context data will need to be transferred from a device that can produce image content to the device which can display the content, thus if a transfer function cannot load from an external source to a circuit there can be no back and forth communication between the devices which must happen to properly display content.
Claims 14 are rejected under 35 U.S.C. 103 as being unpatentable over Greenebaum (US 8704859 B2) in view of Jeffrey (US 20060274937 A1) in view of Vo (US 11398017 B2) in view of Kurokawa (US 8552946 B2) and further in view of Tripathi (US 20130223733 A1).
Regarding claim 14,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the plurality of gray shades as remapped
While Greenebaum teaches remapping the plurality of gray shade values it does not specify the means through which division can be performed. This is found in Vo which teaches wherein a plurality of numerators of the plurality of gray shades as remapped may be calculated prior to a right shift division to maintain an accuracy of results. (Kurokawa Col. 19 Line 1
PNG
media_image7.png
350
590
media_image7.png
Greyscale
Col. 22 Line 8
PNG
media_image8.png
444
638
media_image8.png
Greyscale
Note: Kurokawa teaches multiple instances in which gray shade values are used as numerators in an equation, thus teaching a plurality of numerators of the plurality of gray shades that have already been obtained/calculated prior to the division operation.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Kurokawa where the plurality of remapped gray shades that have already been calculated are used as a plurality of numerators in a division operation.
There are several reasons that would motivate one to do so, normalizing gray shade values by adjusting their range and another of other image processing operations that may better fit the values to the monitor or produce a better picture require a division operation. If one wished to perform such operations it would be obvious that the gray shade values would need to already be calculated before adjusting.
Kurokawa does not however teach that the division operation performed involves a right shift division. This is taught by Tripathi which teaches wherein the plurality of pixel values may be calculated prior to a right shift division (Tripathi ¶56 “For the implementation illustrated in FIG. 5, the input pixel component may be represented by ‘N’ bits. The N-bit input pixel component may be received by divider 42 and then may be shifted left ‘N’ bits in unit … The output from adder 52 may be shifted right by 2*N bits in unit 54. For example, if the input pixel component is an 8-bit number, than unit 54 may right-shift the sum produced by adder 52 by 16 bits. Instead of performing a conventional right-shift operation which could result in lost bits, unit 54 may instead be seen to shift the decimal point (i.e., radix point) left by 2*N bits to retain the precision of the original value.” Note: Tripathi teaches pixel values are divided via right shift division specifically to retain the precision)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Tripathi where the remapped already calculated gray shade values are numerators in a division operation where the division operation is a right shift division.
There are several reasons that would motivate one to do so, when adjusting pixel values such as gray shade values small adjustments may have a noticeable change in how the pixel looks. To prevent erroneous pixel value changes when dividing one could use right shift division, a method that retains the precision and accuracy of its inputs.
Claims 15 is rejected under 35 U.S.C. 103 as being unpatentable over Greenebaum (US 8704859 B2) in view of Jeffrey (US 20060274937 A1) in view of Yu (US 20070297690 A1) and further in view of Dutta (US 10200571 B2).
Regarding claim 15,
Greenebaum teaches:
A true color image enhancement compensation system comprising: an ambient light sensor operational to measure an ambient light level; a circuit operational to: generate a histogram based on an input video signal; export the histogram; receive a gray shade look up table; dynamically remap a plurality of gray shades in the gray shade look up table and generate an output video signal by converting a plurality of gray shades in the input video signal based on a plurality of input video luminance ratios and the gray shade look up table to determine a plurality of new output gray shade values to maintain input video color coordinates and determine a plurality of normalized gray shades based on a normalization of a highest color when an upper limit is reached; and a processor operational to: receive the histogram from the circuit; develop the gray shade look up table based on the histogram and the ambient light level; and transfer the gray shade look up table to the circuit.
As all content listed above is encompassed in claims 8 and 1 it is rejected under the same rationale. Greenebaum has not however been shown to teach dynamically remapping gray shades based on a tristimulus conversion matrix and a normalization step. This is taught by Dutta which teaches to dynamically remap a plurality of gray shades in the gray shade look up table using a tristimulus conversion matrix (Col. 5 Line 60
PNG
media_image9.png
218
606
media_image9.png
Greyscale
Col. 6 Line 26
PNG
media_image10.png
206
600
media_image10.png
Greyscale
Note: Dutta teaches that RGB values are converted to XYZ values, known tristimulus values, via an M matrix. To convert from RGB into XYZ tristimulus values implicitly requires a tristimulus matrix, thus the matrix M is a tristimulus matrix. It is specifically taught that the white point values of the coordinate are converted. The “whitepoint” is analogous to the claims ‘gray shade’ more commonly referred to as greyscale values. Thus, the conversion of gray shades is taught.) to determine a plurality of normalized gray shades (Dutta
Col. 6 Line 26
PNG
media_image11.png
640
498
media_image11.png
Greyscale
Note: In the continuation of the previously provided citation it can be seen that Dutta teaches the normalization of the color values where their range is readjusted with respect to a gamma encoding. As the values are normalized to the norm of the gamma encoding, Dutta teaches normalizing a plurality of gray shades.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Dutta where gray shade values that are remapped are then converted with a tristimulus matrix and normalized.
There are several reasons that would motivate one to do so, the converted XYZ tristimulus values will separate chromaticity from luminance/brightness which is useful for properly calculating how to correct for things that effect luminance like ambient light. Similarly, normalization helps preserve the tone and color in an image when changes are being made, thus both could be implemented to enhance the final quality of image produced.
Dutta does not however teach that its normalization involves a highest color checked to be at its upper limit. This is taught by Yu which teaches to determine a plurality of normalized gray shades based on a normalization of a highest color when an upper limit is reached (Yu ¶43 “Thus, step 122 must be performed first to find the maximum value P (P=MAX (R, G, B) among the RGB colors. Then, step 124 is performed to judge if P is overflowed (exceeding 10 bits). If P is not overflowed, the gray scale value of the pixel remains unchanged and is directly outputted through the signaling route 126. If P is greater or equal to the gray scale 1024 (10000000000), it indicates that the gray scale value of the pixel is overflowed to greater than 10 bits, and then the overflow compensation look-up table 32 is used for calculating the compensation.” ¶44 “Only when the pixel of the image signal is overflowed does the gray scale value of the overflowed image is lowered by using the method according to the invention … when the gray scale value of one of the basic colors of the displayed pixel is overflowed, the maximum gray scale value P of all basic colors is used to look up the corresponding overflow compensation gain value from the look-up table and to multiply the RGB gray scale value of the image signal by this overflow compensation gain value.” Note: Yu teaches a maximum value between the R, G, and B values is found teaching a highest color. This highest color P is checked to see if it is at the upper limit which in this case will happen if it overflows its maximum 10 bit size. To normalize the gray scale values, or to rescale them to fit within an established range they are lowered so that they fit within the range, which only happens if a highest color reaches an upper limit.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Yu where gray shade values are normalized if a highest color reaches a limit.
There are several reasons that would motivate one to do so, when adjusting luminance in tristimulus values to correct for ambient light it is possible that when we convert back to RGB our values are not scaled properly. Specifically, if two colors with different color values intended to be visually distinct are both above the upper limit they will be handled the same upper limit color. To prevent this and maintain the visual distinction normalization to an upper limit when a value exceeds it can be performed.
Claims 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Greenebaum (US 8704859 B2) in view of Jeffrey (US 20060274937 A1) in view of Yu (US 20070297690 A1) in view of Dutta (US 10200571 B2) and further in view of Kang (US 20130155121 A1).
Regarding claim 16,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the circuit is further operational to perform a tristimulus calculation that includes: convert the plurality of gray shades to a plurality of different grey shades using a lookup table.
Greenebaum has been established to teach converting a plurality of gray shades using a look up table in multiple ways. Greenebaum does not however teach converting gray shades to a new set of gray shades to prepare them to be converted to first intermediate gray shades as part of a conversion method. This is taught by Kang which teaches first convert the plurality of gray shades to a plurality of new gray shades using a first transfer table; (Kang ¶7 “The present invention relates to a white balance adjusting method, which [0008] comprises steps of: A. obtaining the maximum spectral tristimulus values Xmax, Ymax and Zmax in CIE1931 chromaticity coordinate system for a panel to be tested, the minimum spectral tristimulus values X0, Y0 and Z0 in CIE1931 chromaticity coordinate system for the panel to be tested, spectral tristimulus values RXm, RYm, RZm of the respective gray levels of Red for the panel to be tested, spectral tristimulus values GXm, GYm, GZm of the respective gray levels of Green for the panel to be tested, and spectral tristimulus values BXm, BYm, BZm of the respective gray levels of Blue for the panel to be tested, where m is the gray level number, X, Y, Z are the spectral tristimulus values of the panel to be tested” Note: Kang teaches that the gray levels of an input image’s color values are read. Specifically, the color values/channels R, G, and B each have an associated ‘m’ value denoting their gray levels. From this initial gray shade information tristimulus values X, Y, and Z coordinates for each color value (meaning R has an RX, RY, RZ, as seen above) where each has its own associated gray level m. Thus, through conversion to the CIE coordinate system Kang teaches new gray shades are obtained. To convert into tristimulus values from RGB implicitly requires a tristimulus matrix, analogous to a table, teaching a transfer table is used.) and second convert the plurality of new gray shades to a plurality of first intermediate gray shades using a lookup table. (Kang ¶40 “Step 102. converting the maximum spectral tristimulus values Xmax, Ymax and Zmax into the maximum color stimulus values Lmax, amax and bmax in CIE1931 chromaticity coordinate system according to a conversion formula of CIE1931 chromaticity coordinate system, where L is psychological lightness, a, b indicate psychological chroma. [0041] Step 103. converting the maximum color stimulus values Lmax, amax and bmax into a hue H and a chroma C of the panel to be tested. [0042] Step 104. computing spectral stimulus values Y1 to Ymax-1 for intermediate gray levels based on Ymax and Y0 according to the principle that brightness variance matches gray level index variance” Note: Kang teaches that from the first intermediate values X, Y, and Z are converted to intermediate values L, a, and b values before being converted into another intermediate gray level, Y. Thus teaching converting the new gray shades into a first intermediate gray shades as the XYZ values are converted into L, a, and b values.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Kang where grey shades that are remapped to new gray shade values are made into first intermediate values via a look up table.
There are several reasons that would motivate one to do so, to adjust pixel values such as grey shade values to correct for ambient light, as the current invention is attempting to do, it is useful to convert temporarily into intermediate values such as tristimulus values as taught by Kang and the present invention. While the tristimulus values will eventually have to be converted back to RGB so the display can accept its format, tristimulus values separate luminance from chromaticity allowing for things which interfere with luminance, like ambient light, to be compensated for to produce a better image.
Regarding claim 17,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the tristimulus calculation further includes: convert the plurality of first intermediate gray shades using a lookup table
Greenebaum has been shown to teach converting a plurality of gray shades via a look up table. Greenebaum does not however teach converting first intermediate gray shades to a second intermediate value Y using a tristimulus conversing matrix. This is taught in Kang which teaches to convert the plurality of first intermediate gray shades to a second intermediate value Y using the tristimulus conversion matrix. (Kang ¶40 “Step 102. converting the maximum spectral tristimulus values Xmax, Ymax and Zmax into the maximum color stimulus values Lmax, amax and bmax in CIE1931 chromaticity coordinate system according to a conversion formula of CIE1931 chromaticity coordinate system, where L is psychological lightness, a, b indicate psychological chroma. [0041] Step 103. converting the maximum color stimulus values Lmax, amax and bmax into a hue H and a chroma C of the panel to be tested. [0042] Step 104. computing spectral stimulus values Y1 to Ymax-1 for intermediate gray levels based on Ymax and Y0 according to the principle that brightness variance matches gray level index variance” Note: As established previously in claim 16, the new gray shade values are converted to a first intermediate values L, a, and b. These values are the converted into another other intermediate gray shade values Y1 to Ymax-1, thus teaching an intermediate value Y is obtained. It has been clarified that the XYZ values are tristimulus values, to convert to tristimulus values implicitly requires the use of a tristimulus conversion matrix, which describes the matrix/mathematical step needed to convert into XYZ CIE coordinates. Thus, converting via a tristimulus matrix is implicitly taught.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Kang where: obtaining remapped gray shades via a look up table also involves converting to a Y value obtained via a tristimulus conversion matrix.
There are several reasons that would motivate one to do so, Y in the XYZ tristimulus values represents luminance. As we seeking to correct for ambient light’s which impacts luminance it can be better handled if we understand the luminance of the pixels in an image which can be accomplished by converting input values to Y via a tristimulus conversion matrix.
Regarding claim 18,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the tristimulus calculation further includes: convert the plurality of first intermediate gray shades
Greenebaum does not however teach converting intermediate gray shades to a second intermediate values X and Z.
convert the plurality of first intermediate gray shades to a second intermediate value X and a second intermediate value Y based on the second intermediate value Y. (Kang ¶43 “Step 105. computing spectral stimulus values X1 to Xmax-1 and Z1 to Zmax-1 for intermediate gray levels according to the hue H, the chroma C and the spectral stimulus values Y1 to Ymax-1 of the respective gray levels;” ¶52 “Step 104 particularly comprises computing spectral stimulus values Y1 to Y254 for intermediate gray levels based on Y255 and Y0 according to a formula which indicates that brightness variance matching gray level index variance Y t=[(t/255)E*(Y 255 −Y 0)]+Y 0, [0053] where E is the gamma value, the range thereof is 2.0˜2.4, preferably 2.2, and a range of t is 1˜254.” Note: As addressed above, rather than obtaining second intermediate X and a “second intermediate value Y based on the second intermediate value Y” which has no meaning, the claim intends to say obtaining a second intermediate X and “obtaining a second intermediate value Z”. In claims 16 and 17 it was established that the values Y1 to Ymax-1 are second intermediate values obtained from first intermediate values. Here, it is taught that values X1 to Xmax-1 and Z1 to Zmax-1 are obtained from the Y values. Thus, obtaining a second intermediate X and Z from previous intermediate Y is taught.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Kang where a second intermediate value X and Z based on Y are obtained.
There are several reasons that would motivate one to do so, when converting into tristimulus values X and Z denote the color value information for red, green, and blue, where Y denotes the luminance/brightness. When correcting for ambient light luminance is directly impacted, as a result the luminance changes how the color is perceived by the user even if the color value remains the same. Thus, obtaining the color values in the same space to potentially correct them if needed would be useful to make a higher quality output image.
Note: The below rejection of claims 19-20 does not directly rely directly on citations from Kang to teach them, only relying on Greenebaum and Dutta references. However, references to intermediate values are made which have previously been shown to be taught by Greenebaum combined with Kang. To maintain the sequential ordering of claims 16-20 which describes a series of ordered steps that make components used in subsequent claims and more clearly establish how the series of steps is taught claims 19-20 are provided here, though it is acknowledged they only rely on Greenebaum and Dutta.
Regarding claim 19,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the tristimulus calculation further includes:
Greenebaum does not however teach the use of an inverse matrix, this is taught in Dutta which teaches to convert the second intermediate values into a plurality of third intermediate values using an inverse matrix.( Col. 5 Line 60
PNG
media_image9.png
218
606
media_image9.png
Greyscale
Col. 6 Line 26
PNG
media_image10.png
206
600
media_image10.png
Greyscale
Note: Dutta teaches that once RGB have been converted to XYZ tristimulus values with an M matrix they are converted in the opposite direction back to RGB from XYZ with the inverse matrix N-1. Thus, Dutta teaches the use of an inverse matrix to convert the second intermediate values, here the XYZ values, to a third intermediate values RGB. It is known that they are intermediate values as seen below in claim 20 where the reconverted RGB values are normalized, making them intermediate values.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Dutta where the second intermediate values are normalized via an inverse matrix.
There are several reasons that would motivate one to do so, in the process of making the input pixel values ‘intermediate values’ they are converted to tristimulus XYZ values with a tristimulus conversion matrix. To provide the image to the display to be properly read it should be put back into its RGB format which can be accomplished by applying an inverse matrix.
Regarding claim 20,
Greenebaum teaches:
The true color image enhancement compensation system according to The true color image enhancement compensation system according to wherein the tristimulus calculation further includes:
Greenebaum does not however directly teach normalizing the third intermediate values. This is taught by Dutta which teaches to normalize the plurality of third intermediate values. (Dutta Col. 6 Line 26
PNG
media_image11.png
640
498
media_image11.png
Greyscale
Note: Normalization involves the adjusting of the range of pixel values, here Dutta teaches that when tristimulus values are converted back to RGB via an inverse matrix they must be normalized to with the display’s gamma encoding. As seen from the screenshot above, the range of the modified R’G’B’ pixel values are normalized with the gamma value equal to 0.45 where values are adjusted based on whether or not they are above/below a threshold and are adjusted with the gamma encoding.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Greenebaum with Dutta where the third intermediate values are normalized.
There are several reasons that would motivate one to do so, as established in claim 19 the third intermediate values are the RGB values reconverted back from XYZ tristimulus values. In the process of adjusting the values when they were tristimulus values it is possible the RGB color values will now fall outside an acceptable range, a problem that could be solved with normalization.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALAN GREGORY HAKALA whose telephone number is (571)272-7863. The examiner can normally be reached 8:00am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at (571) 270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALAN GREGORY HAKALA/
Examiner, Art Unit 2617
/KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights