DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Uchidate (US 20130027751 A1) and Price (US 20230117006 A1).
Regarding Claim 1, Sun teaches an image capture system (Detailed Ways, pg. 3: “the electronic device includes a processor, a memory, a display, and a camera connected by a system bus”), comprising:
a camera system that captures image data of a local environment and a measurement of brightness of the local environment at a time of capture (Detailed Ways, pg. 3: “the electronic device 110 can call the camera on the camera to perform shooting, such as real-time scanning of the object 120 in the environment to obtain a frame image, and generate a captured image according to the frame image. Optionally, the camera may be a dual camera, including a main camera and a sub camera, and the image is generated according to the main camera and the sub camera. The electronic device may use the frame image or the generated image as an image to be processed, and acquire environment information of the image to be processed obtained by moving the camera in advance; determine a corresponding target brightness parameter according to the environment information; and process the image according to the target brightness parameter”), and
a processor that generates an image file having pixels representing the image data at the time of capture (Detailed Ways, pg. 3: “the electronic device includes a processor, a memory, a display, and a camera connected by a system bus”; Detailed Ways, pg. 3: “The processor is used to provide computing and control capabilities to support the operation of the entire electronic device. The memory is used to store data, programs, etc., and at least one computer program is stored on the memory, and the computer program can be executed by the processor to implement an image brightness processing method suitable for an electronic device provided in the embodiments of the present application”),
Sun does not teach the pixel values having value distributions mapped according to predetermined maximum values associated with the image file.
However, Uchidate teaches pixel values having value distributions mapped according to predetermined maximum values associated with the image file (Paragraph [0186]: In this comparative example, the first halftone image data is data of three gradations (minimum: 0, maximum 2), and the quantization error data used as correction data is data of 256 gradations (minimum: 0, maximum 255). Since the maximum value of each pixel value is not equal between the first halftone image data and the correction data, normalization processing needs to be executed to make the maximum values equal to each other. In this comparative example, before correction processing, the output calculation unit 5032 multiplies each pixel value contained in the first halftone image data by 255/2 to normalize the maximum value of each pixel value to 255, making the maximum value equal to the maximum value of a pixel value contained in the correction data”. Notes: value distribution, in its broadest reasonable interpretation, describes pixel intensity distribution ex. RGB or another format).
Sun and Uchidate are considered analogous in the art with respect to correcting images based on pixel values. A common motivation in the art is to normalize pixel values from an image according to a maximum value associated with the image, as doing so enables ease in comparison and adjustment, as is evident in Uchidate.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the processing of pixel values of Sun with the mapping of pixel values to predetermined maximum values associated with the image file of Uchidate; Doing so would yield the predictable result of enabling ease of comparison and adjustment in images.
Sun as modified does not explicitly teach the image file also containing metadata representing the measurement of brightness of the local environment at a time of capture.
However, Price teaches an image file also containing metadata representing the measurement of brightness of the local environment at a time of capture (Paragraph [0078]: “image metadata (e.g., motion data, illuminance, dynamic range, temperature, temporal recency, etc.)”. Notes: metadata such as illuminance (brightness), temperature, etc. are descriptors for a local environment generated with an image capture).
Sun as modified and Price are considered analogous in the art with respect to the retrieval of brightness information for an obtained image. A person having ordinary skill in the art would appreciate that image metadata stores environmental information such as brightness, and that using metadata to adjust the display of an image is well known in the art, as is evident in Price.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the use of brightness information obtained from the environment at the capture of an image of Sun as modified with the use of metadata including brightness information regarding an environment at the capture of an image of Price; Doing so would yield the predictable result of using metadata including brightness information for image display adjustment.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Uchidate (US 20130027751 A1) and Price (US 20230117006 A1), and in further view of Herman (US 20200232895 A1).
Regarding Claim 2, the system of Claim 1 is rejected over Sun as modified.
Sun as modified teaches the system of Claim 1, wherein the measurement of brightness is a scene parameter representing average brightness of the captured image data (Sun, Detailed Ways, pg. 6-7: “The electronic device can acquire the Y data at each pixel in the environmental information to read the brightness information of each pixel, and calculate the ambient illuminance according to the brightness information. Alternatively, the Y data at each pixel point may be superimposed and averaged, and the calculated average value is taken as the ambient illuminance”).
Sun as modified does not explicitly teach a lux parameter.
However, Herman teaches a lux parameter (Paragraph [0072]: “an average brightness (e.g., in lux) of the pixels identified as the liquid droplet 24 with an average brightness of a remainder of the pixel of the image data”).
Sun as modified and Herman are considered analogous in the art with respect to determining an average brightness of captured image data. A common parameter for brightness is lux, as is evident in Herman. One would be motivated to utilize lux as a brightness parameter to conform with known standards.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the brightness parameter of Sun as modified with the lux parameter of Herman; Doing so would yield the predictable result of the brightness of the captured image data being represented in a known brightness parameter.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Uchidate (US 20130027751 A1) and Price (US 20230117006 A1), and in further view of Cho (US 20140307007 A1).
Regarding Claim 3, the system of Claim 1 is rejected over Sun as modified.
Sun as modified does not teach that the measurement of brightness is a clipping point lux parameter representing maximum brightness of the captured image data.
However, Cho teaches a measurement of brightness is a clipping point lux parameter representing maximum brightness of a captured image data (Paragraph [0065]: “An electronic device 100 may receive ambient light information to determine one of the image characteristics of the paper mode. FIG. 7 shows target luminance based on illumination of ambient light in the paper mode when reflectance of paper is 80% as an example. The target brightness value is the maximum brightness value of the converted input image to be displayed on the display apparatus 120, and the target brightness value is determined by the illumination (lux) level of the ambient light. Referring to FIG. 7, an x-coordinate indicates the illumination level of the ambient light, and a y-coordinate indicates maximum brightness value of the converted input image to be displayed on the display apparatus 120”).
Sun as modified and Cho are considered analogous in the art with respect to the display of images factoring in a measurement of brightness related to ambient lighting in an environment corresponding with a captured image. Lux as a measure of brightness is well known in the art, and a maximum measure of brightness is frequently utilized to adjust the display of an image for viewing the image more effectively, as is evident in Cho.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the adjustment of a display of an image in accordance with a measure of brightness of Sun as modified with the use of a maximum brightness in lux of a captured image of Cho; Doing so would yield the predictable result of being able to display the image more effectively in accordance with a measure of brightness.
Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1).
Regarding Claim 4, Sun teaches an image processing method (Summary of the Inventions, pg. 1: “Various embodiments provided in accordance with the present application provide an image brightness processing method, a non-transitory computer readable storage medium, and an electronic device. An image brightness processing method includes: Get the image to be processed; Obtaining environmental information of the image to be processed obtained by moving the camera in advance; Determining a corresponding target brightness parameter according to the environmental information”), comprising:
responsive to image information having pixel values of an image and information representing measured brightness of the image at a time of image capture (Summary of the Inventions, pg. 1: “Get the image to be processed; Obtaining environmental information of the image to be processed obtained by moving the camera in advance; Determining a corresponding target brightness parameter according to the environmental information”):
generating an adjustment function based on statistics of the pixel values and the measured brightness (Detailed Ways, pg. 6-7: “The electronic device can acquire the Y data at each pixel in the environmental information to read the brightness information of each pixel, and calculate the ambient illuminance according to the brightness information. Alternatively, the Y data at each pixel point may be superimposed and averaged, and the calculated average value is taken as the ambient illuminance. The target brightness parameter can be determined based on the ambient illumination”; Detailed Ways, pg. 9: “Operation 712, performing brightness processing on the image to be processed according to the target brightness parameter. Taking the brightness parameter as the exposure degree, the values corresponding to the preset current brightness parameter and the reference brightness parameter are respectively 0.6 and 0.4, and the current brightness parameter of the image to be processed is: the current exposure degree is 200 as an example for description. The electronic device calculates the average value of the Y channel on all the pixels in the panoramic image, and obtains an ambient illuminance of 2000 Lux. From the relationship table, it is found that the sensitivity of the ambient illumination of 2000 Lux is 230. Then the target brightness parameter is: target sensitivity = 200 × 0.6 + 230 × 0.4 = 212. That is, the electronic device can perform brightness processing on the image to be processed according to the sensitivity of 212”),
applying the adjustment function to the pixel values, and outputting adjusted pixel values to a display (Detailed Ways, pg. 9: “Operation 712, performing brightness processing on the image to be processed according to the target brightness parameter. Taking the brightness parameter as the exposure degree, the values corresponding to the preset current brightness parameter and the reference brightness parameter are respectively 0.6 and 0.4, and the current brightness parameter of the image to be processed is: the current exposure degree is 200 as an example for description. The electronic device calculates the average value of the Y channel on all the pixels in the panoramic image, and obtains an ambient illuminance of 2000 Lux. From the relationship table, it is found that the sensitivity of the ambient illumination of 2000 Lux is 230. Then the target brightness parameter is: target sensitivity = 200 × 0.6 + 230 × 0.4 = 212. That is, the electronic device can perform brightness processing on the image to be processed according to the sensitivity of 212”; Detail Ways, pg 11: “The processed image data can be output to display 980 for viewing by a user and/or further processed by a graphics engine or a GPU (Graphics Processing Unit)”. Notes: performing processing on the image to be processed inherently processes the pixel values).
Sun as modified does not explicitly teach the image file also containing metadata representing the measurement of brightness of the local environment at a time of capture.
However, Price teaches an image file also containing metadata representing the measurement of brightness of the local environment at a time of capture (Paragraph [0078]: “image metadata (e.g., motion data, illuminance, dynamic range, temperature, temporal recency, etc.)”. Notes: metadata such as illuminance (brightness), temperature, etc. are descriptors for a local environment generated with an image capture).
Sun and Price are considered analogous in the art with respect to the retrieval of brightness information for an obtained image. A person having ordinary skill in the art would appreciate that image metadata stores environmental information such as brightness, and that using metadata to adjust the display of an image is well known in the art, as is evident in Price.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the use of brightness information obtained from the environment at the capture of an image of Sun with the use of metadata including brightness information regarding an environment at the capture of an image of Price; Doing so would yield the predictable result of using metadata including brightness information for image display adjustment.
Regarding Claim 15, Sun teaches A method comprising:
converting pixel values of a source image from a domain of the source image to a physical domain according to data, stored with the source image (Detailed ways, pg 3: “the electronic device 110 can call the camera on the camera to perform shooting, such as real-time scanning of the object 120 in the environment to obtain a frame image, and generate a captured image according to the frame image. Optionally, the camera may be a dual camera, including a main camera and a sub camera, and the image is generated according to the main camera and the sub camera. The electronic device may use the frame image or the generated image as an image to be processed, and acquire environment information of the image to be processed obtained by moving the camera in advance; determine a corresponding target brightness parameter according to the environment information; and process the image according to the target brightness parameter Perform brightness processing... the electronic device includes a processor, a memory, a display, and a camera connected by a system bus”; Detailed Ways, Pg 4: “Operation 308, performing brightness processing on the image to be processed according to the target brightness parameter. The image to be processed is composed of a plurality of pixels, each of which may be composed of a plurality of color channels, each color channel representing a color component. For example, an image can be composed of three channels of RGB (red, green, and blue), or it can be composed of three channels of HSV (hue, saturation, and lightness), or it can be composed of CMY (cyan, magenta, or magenta and Yellow three colors) three channel composition, or YUV (also known as YCrCb, is a color coding method used by European television systems) data format. Each format can be converted to each other, for example, each pixel can be converted from YUV format to RGB format”),
representing a brightness of the source image measured at a time of capture of the source image, performing a grading operation on the physical-domain pixel values (Detailed Ways, pg 6-7: “The electronic device can acquire the Y data at each pixel in the environmental information to read the brightness information of each pixel, and calculate the ambient illuminance according to the brightness information. Alternatively, the Y data at each pixel point may be superimposed and averaged, and the calculated average value is taken as the ambient illuminance. The target brightness parameter can be determined based on the ambient illumination. The electronic device can establish a correspondence between the ambient illuminance and the target brightness parameter, and the correspondence can be embodied by a comparison table between the environment parameter and the target brightness parameter. The electronic device can query the brightness parameter corresponding to the calculated ambient illuminance from the comparison table, and determine the target brightness parameter according to the brightness parameter. For example, the brightness parameter of the query may be directly used as the target brightness parameter, or the target brightness parameter may be multiplied by a corresponding coefficient, and the obtained product is used as the target brightness parameter. The coefficient may be a fixed coefficient and may also be a coefficient determined according to an area occupied by the image to be processed in the environmental information”; Detailed Ways, pg 9: “Operation 712, performing brightness processing on the image to be processed according to the target brightness parameter. Taking the brightness parameter as the exposure degree, the values corresponding to the preset current brightness parameter and the reference brightness parameter are respectively 0.6 and 0.4, and the current brightness parameter of the image to be processed is: the current exposure degree is 200 as an example for description. The electronic device calculates the average value of the Y channel on all the pixels in the panoramic image, and obtains an ambient illuminance of 2000 Lux. From the relationship table, it is found that the sensitivity of the ambient illumination of 2000 Lux is 230. Then the target brightness parameter is: target sensitivity = 200 × 0.6 + 230 × 0.4 = 212. That is, the electronic device can perform brightness processing on the image to be processed according to the sensitivity of 212”).
Sun does not explicitly teach metadata is captured and used to define brightness at time of capture for use in pixel value conversion.
However, Price teaches metadata is captured and used to define brightness at time of capture for use in pixel value conversion (Paragraph [0078]: “image metadata (e.g., motion data, illuminance, dynamic range, temperature, temporal recency, etc.)”).
Sun and Price are considered analogous in the art with respect to the retrieval of brightness information for an obtained image. A person having ordinary skill in the art would appreciate that image metadata stores environmental information such as brightness, and that using metadata to adjust the display of an image is well known in the art, as is evident in Price.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the use of brightness information obtained from the environment at the capture of an image of Sun with the use of metadata including brightness information regarding an environment at the capture of an image of Price; Doing so would yield the predictable result of using metadata including brightness information for image display adjustment.
Claims 5, 11 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1), and in further view of Park S (US 20220303455 A1).
Regarding Claim 5, the image processing method of Claim 4 is rejected over Sun as modified.
Sun as modified does not teach that the adjustment function constrains maximum brightness of the adjusted pixel values.
However, Park S teaches an adjustment function constraining maximum brightness of the adjusted pixel values (Paragraph [0086]: “For example, in a case of low illumination of the target portion 320, the image sensor may adjust the brightness of the target portion 320 through the brightness correction operation on the target portion 320. The brightness correction operation may be performed by, for example, the contrast stretching technique, the histogram equalization technique” Paragraph [0106]: “The y-histogram module 530 may calculate a histogram obtained by normalizing pixel values (e.g., color values and brightness values of pixels) of an image frame to be between values 0 to 255”. Notes: normalizing pixel values to be between 0 to 255 constrains max pixel values to a maximum of 255).
Sun as modified and Park S are considered analogous in the art with respect brightness adjustment of images. A common motivation in the art is to adjust brightness to enhance the display of an image; this is commonly achieved via brightness enhancement techniques that often involve constraining brightness of pixel values to be adjusted, as is evident in Park S.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the adjustment function that adjusts brightness of an image of Sun as modified with the adjustment of brightness via constraining maximum brightness of adjusted pixel values of Park S; Doing so would yield the predicable result of adjusting brightness to enhance image display.
Regarding Claim 11, the image processing method of Claim 4 is rejected over Sun as modified.
Sun as modified does not teach that the statistics are represented by a histogram of pixel values of the image.
However, Park S teaches that statistics are represented by a histogram of pixel values of the image (Paragraph [0086]: “For example, in a case of low illumination of the target portion 320, the image sensor may adjust the brightness of the target portion 320 through the brightness correction operation on the target portion 320. The brightness correction operation may be performed by, for example, the contrast stretching technique, the histogram equalization technique” Paragraph [0106]: “The y-histogram module 530 may calculate a histogram obtained by normalizing pixel values (e.g., color values and brightness values of pixels) of an image frame to be between values 0 to 255”).
Sun as modified and Park S are considered analogous in the art with respect brightness adjustment of images. A common motivation in the art is to adjust brightness to enhance the display of an image; this is commonly achieved via brightness correction methods such as the histogram equalization technique, which utilizes statistics of pixel values of the image, as is evident in Park S.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the adjustment function that adjusts brightness of an image of Sun as modified with the adjustment of brightness via the use of statistics in the form of a histogram of pixel values of the image of Park S; Doing so would yield the predicable result of adjusting brightness to enhance image display.
Regarding Claim 18, the method of Claim 15 is rejected over Sun as modified.
Sun as modified does not teach that the grading operation includes applying a filter to the physical-domain pixel values.
However, Park S teaches a grading operation includes applying a filter to the physical-domain pixel values (Paragraph [0086]: “For example, in a case of low illumination of the target portion 320, the image sensor may adjust the brightness of the target portion 320 through the brightness correction operation on the target portion 320. The brightness correction operation may be performed by, for example, the contrast stretching technique, the histogram equalization technique” Paragraph [0106]: “The y-histogram module 530 may calculate a histogram obtained by normalizing pixel values (e.g., color values and brightness values of pixels) of an image frame to be between values 0 to 255”. Notes: normalizing pixel values to be between 0 to 255 constrains max pixel values to a maximum of 255; The broadest reasonable interpretation of a filter for pixel values is a function that results in an adjustment of the pixel values).
Sun as modified and Park S are considered analogous in the art with respect brightness adjustment of images. A common motivation in the art is to adjust brightness to enhance the display of an image; this is commonly achieved via brightness correction methods such as the histogram equalization technique, which utilizes statistics of pixel values of the image, as is evident in Park S.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the grading operation that adjusts brightness of an image of Sun as modified with the grading operation that adjusts brightness via the use of a filter of Park S; Doing so would yield the predicable result of adjusting brightness to enhance image display.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1), and in further view of Atkins (US 20200013151 A1).
Regarding Claim 6, the image processing method of Claim 4 is rejected over Sun as modified.
Sun as modified does not teach that the applying scales pixel values output by opto-optical transfer function process by the adjustment function.
However, Atkins teaches an applying scales pixel values output by opto-optical transfer function process by the adjustment function (Paragraph [0057]: “FIG. 2B illustrates an example chromaticity distribution of mapped pixel values (L′M′S′) as mapped from the scene pixel values (LMS) when the OOTF (102) is applied to the scene pixel values (LMS). As shown, applying the OOTF (102) in the LMS color space(s) reduces hue shifts in mapped colors as represented by the mapped pixel values in reference to scene colors as represented by the scene pixel values, but also increases saturations in the mapped colors well beyond an original color gamut, and even beyond what can be represented in a video signal format. Thus, further color remapping may be needed after applying the OOTF (102) in the LMS space(s)”. Notes: OOTF is short for opto-optical transfer function).
Sun as modified and Atkins are considered analogous in the art with respect to image adjustment for improved display. A common motivation in the art is to improve the viewing experience of an image, which is commonly achieved through adjusting pixel intensity or brightness through a remapping (function) after applying a transfer function (opto-optical transfer function), as is evident in Atkins.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the image processing method of Sun as modified with the application of an adjustment function to pixel values after applying an opto-optical transfer function; Doing so would yield the predictable result of improving the display of an image for viewing.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1), and in further view of Oh (US 20190158793 A1).
Regarding Claim 7, the image processing method of Claim 4 is rejected over Sun as modified.
Sun as modified does not teach that the applying scales sources pixel values of the image before inputting the scaled pixel values to an opto-optical transfer function process.
However, Oh teaches an applying scales sources pixel values of the image before inputting the scaled pixel values to an opto-optical transfer function process (Paragraph [0103]: “Referring again to FIG. 2, the first image processor 113 may convert the image data 11 to allow the processor 110 to apply a tone mapping curve to the image data 11. Particularly, the first image processor 113 may acquire data in a certain form, to which the tone mapping curve is applicable, by converting a transfer function (e.g., Electrical-Optical Transfer Function (EOTF)) to a certain standard (e.g., ST 2084 standard). For example, the transfer function may be gamma (i.e., brightness correction value) applied to the image data 11, and in this case, the first image processor 113 may release gamma from the image data 11 and convert the image data 11 into a form (e.g., a linear image data) to which an opto-optical Transfer function (OOTF) is applicable”. Notes: image data reads on pixel values).
Sun as modified and Oh are considered analogous in the art with respect to adjusting images for display purposes through adjusting pixel values. A common motivation in the art is to improve the viewing experience of an image, which is commonly achieved through adjusting pixel intensity or brightness through a function before applying an opto-optical transfer function (OOTF), as is evident in Oh.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the image processing method of Sun as modified with the adjustment function that adjusts pixel values before an opto-optical transfer function of Oh; Doing so would yield the predictable result of improving the display of an image for viewing.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1), and in further view of Liu (US 20040190789 A1).
Regarding Claim 8, the image processing method of Claim 4 is rejected over Sun as modified.
Sun as modified does not explicitly teach that the adjustment function is a non-linear transform curve.
However, Liu teaches that an adjustment function is a non-linear transform curve (Paragraph [0079]: “A "negative" gamma curve is a gamma curve where 1/Gamma is less than one; the luminance values for the pixels will gradually decrease in the values of the pixels, especially in the high-brightness region. In one embodiment, the exposure compensator applies a negative gamma curve with Gamma equal to 2.5 (1/Gamma=0.4). Alternatively, the exposure compensator performs some other adjustment on overexposed images, such as a linear transform or some other non-linear transform”).
Sun as modified and Liu are considered analogous in the art with respect to adjusting brightness of pixel values in images for display. Non-linear transform functions are well known in the art for brightness adjustment through pixel value adjustment, as is evident in Liu.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the image processing method of Sun as modified with the non-linear transform adjustment function of Liu; Doing so would yield the predictable result of adjusting pixel values through well-established means in the art.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1), and in further view of Deighton (US 20220014728 A1).
Regarding Claim 9, the image processing method of Claim 4 is rejected over Sun as modified.
Sun as modified does not teach that the adjustment function is a spline-based transform curve.
However, Deighton teaches that an adjustment function is a spline-based transform curve (Paragraph [0082]: “Because the pixel gain table does not include gain factors for every possible viewing angle, an LED viewing angle correction system according to the disclosure will interpolate between the gain factors stored in the table to calculate gain factors that transition smoothly as the viewing angle changes. In some embodiments, the interpolation might be bi-linear for ease of computation, while in others it might use non-linear function, such as bicubic or spline-based interpolation”).
Sun as modified and Deighton are considered analogous in the art with respect to the adjustment of image brightness through pixel intensity. A common motivation in the art is to utilize non-linear functions for pixel adjustment. Spline-based interpolation is an example of non-linear function, and is used in the art for pixel adjustment, as is evident in Deighton.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the image processing method of Sun as modified with the spline-based transform adjustment function of Deighton; Doing so would yield the predictable result of adjusting image brightness through pixel intensity with an established method.
Claims 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1), and in further view of Furumoto (US 20150035848 A1).
Regarding Claim 10, the image processing method of Claim 4 is rejected over Sun as modified.
Sun as modified does not explicitly teach that the adjustment function is determined in part by characteristics of the display.
However, Furumoto teaches that an adjustment function is determined in part by characteristics of the display (Paragraph [0057]: “Note that there is no particular limitation regarding the method of deciding the extension. For example, the extension rate may be calculated by using a function representing the correspondence relation of the target brightness and the extension rate, or decided by using a table representing the foregoing correspondence relation”; Paragraph [0058]: “Moreover, the extension rate may or may not be decided for each region corresponding to the light source. When the display characteristics between the pixels (change in the modulation level of the light from the light-emitting unit relative to the change in the pixel value) are different, the extension rate may also be decided for each pixel. When the brightness of the displayed image change nonlinearly relative to the change of the pixel value, the extension rate of each pixel value may also be decided. When the brightness of the displayed image change nonlinearly relative to the change of the pixel value, the extension rate”; Paragraph [0059]: “The image processing unit 111 corrects the pixel value of the composite image data on the basis of the extension rate decided with the extension rate decision unit 110. Specifically, the image processing unit 111 multiplies the pixel value of the composite image data by the extension rate. Subsequently, the image processing unit 111 outputs the image data subject to image processing to the liquid crystal panel 112”) corresponding to the pixel value of the composite image data may also be decided for each pixel”).
Sun as modified and Furumoto are considered analogous in the art with respect to the adjustment of brightness of the display of an image through pixel adjustment. A common motivation in the art is to account adverse effects that result from display characteristics; this is often resolved through the functions that scale pixel changes in accordance with display characteristics (extension rate), which is evident in Furumoto.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the image processing method of Sun as modified with the adjustment function for pixel values accounting for display characteristics of Furumoto; Doing so would yield the predictable result of minimizing adverse display effects resulting from display characteristics.
Regarding Claim 17, the method of Claim 15 is rejected over Sun as modified.
Sun as modified does not teach that the grading operation includes adjusting the physical-domain pixel values according to parameters of a display to render the image.
However, Furumoto teaches a grading operation includes adjusting the physical-domain pixel values according to parameters of a display to render the image (Paragraph [0057]: “Note that there is no particular limitation regarding the method of deciding the extension. For example, the extension rate may be calculated by using a function representing the correspondence relation of the target brightness and the extension rate, or decided by using a table representing the foregoing correspondence relation”; Paragraph [0058]: “Moreover, the extension rate may or may not be decided for each region corresponding to the light source. When the display characteristics between the pixels (change in the modulation level of the light from the light-emitting unit relative to the change in the pixel value) are different, the extension rate may also be decided for each pixel. When the brightness of the displayed image change nonlinearly relative to the change of the pixel value, the extension rate of each pixel value may also be decided. When the brightness of the displayed image change nonlinearly relative to the change of the pixel value, the extension rate”; Paragraph [0059]: “The image processing unit 111 corrects the pixel value of the composite image data on the basis of the extension rate decided with the extension rate decision unit 110. Specifically, the image processing unit 111 multiplies the pixel value of the composite image data by the extension rate. Subsequently, the image processing unit 111 outputs the image data subject to image processing to the liquid crystal panel 112”) corresponding to the pixel value of the composite image data may also be decided for each pixel”).
Sun as modified and Furumoto are considered analogous in the art with respect to the adjustment of brightness of the display of an image through pixel adjustment. A common motivation in the art is to account adverse effects that result from display characteristics; this is often resolved through the functions that scale pixel changes in accordance with display characteristics (extension rate), which is evident in Furumoto.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the grading operation of Sun as modified with the grading operation accounting for display characteristics of Furumoto; Doing so would yield the predictable result of minimizing adverse display effects resulting from display characteristics.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1), and in further view of Herman (US 20200232895 A1) and Uchidate (US 20130027751 A1).
Regarding Claim 12, the image processing method of Claim 4 is rejected over Sun as modified.
Sun as modified does not teach that the statistics are represented by an average pixel value of the image.
However, Herman teaches that statistics are represented by an average pixel value of the image. (Paragraph [0072]: “The computer 34 may be programmed to compare a transparency of the liquid droplet 24 with a threshold transparency. The computer 34 may determine the transparency by comparing an average brightness (e.g., in lux) of the pixels identified as the liquid droplet 24 with an average brightness of a remainder of the pixel of the image data. The threshold transparency may be a maximum difference between such pixel brightness averages at which the hydrophobic coating is considered undegraded. In other words, the computer 34 may determine that the hydrophobic coating is undegraded when the computer 34 identifies that the difference between the average brightness of the pixels identified as the liquid droplet 24 and the average brightness of the remainder of the pixel of the image data is at or below the threshold transparency. The computer 34 may make such comparison when an average brightness of the pixels is above a specified threshold, e.g., indicating that is it day time”).
Sun as modified and Herman are considered analogous in the art with respect to determining an average brightness of captured image data. A common motivation in the art is to utilize statistical values such as average to define thresholds or otherwise representatively define attributes of an image such as brightness, as is evident in Herman.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the image processing method of Sun as modified with the use of an average pixel value of an image of Herman for to define image brightness; Doing so would yield the predictable result of the brightness of the captured image data being represented in a known statistical measure across pixels of an image.
Sun as modified does not teach that the statistics are represented by a maximum pixel value of the image.
However, Uchidate teaches that statistics are represented by a maximum pixel value of an image (Paragraph [0186]: “In this comparative example, the first halftone image data is data of three gradations (minimum: 0, maximum 2), and the quantization error data used as correction data is data of 256 gradations (minimum: 0, maximum 255). Since the maximum value of each pixel value is not equal between the first halftone image data and the correction data, normalization processing needs to be executed to make the maximum values equal to each other. In this comparative example, before correction processing, the output calculation unit 5032 multiplies each pixel value contained in the first halftone image data by 255/2 to normalize the maximum value of each pixel value to 255, making the maximum value equal to the maximum value of a pixel value contained in the correction data”).
Sun as modified and Uchidate are considered analogous in the art with respect to correcting images based on pixel values. A common motivation in the art is to normalize pixel values from an image according to a maximum value associated with the image, as doing so enables ease in comparison and adjustment, as is evident in Uchidate.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine image processing method of Sun as modified with use of a maximum pixel value as a statistic for image adjustment of Uchidate; Doing so would yield the predictable result of enabling ease of comparison and adjustment in images.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1), and in further view of Allen (US 20190096057 A1).
Regarding Claim 13, the image processing method of Claim 4 is rejected over Sun as modified.
Sun as modified does not teach that the statistics are represented by pixel percentiles of pixel values of the image.
However, Allen teaches that statistics are represented by pixel percentiles of pixel values of an image (Paragraph [0294]: “The “ApplyPercentileThreshold” operation, in step 9005, inputs the dark_field_image, the percentile_table, and a minimum_percentile_threshold. The percentile of each pixel intensity in the dark_field_image is looked up using the percentile_table. If the pixel's percentile value in the dark_field_image is less than the minimum_percentile_threshold then the pixel is converted to a 0. Otherwise, the pixel is converted to a 255. The binary_dark_field_image 8042 is the result of the ApplyPercentileThreshold operation on the dark_field_image 8041, in step 9006. The binary_dark_field_image 8042 (shown in FIG. 74) is created from image 8041 which is shown in FIG. 73 using a minimum_percentile_threshold of 0.79”. Notes: pixel percentile values are utilized to map pixel values to new pixel values, impacting the brightness of the image).
Sun as modified and Allen are considered analogous in the art with respect to the adjustment of pixel values to adjust image brightness. A common motivation in the art is to adjust brightness to more clearly define an image, where doing so is commonly achieved through utilizing statistical measures such as percentile to threshold brightness values, as is evident in Allen.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the image processing method of Sun with the use of pixel percentiles of pixel values of an image of Allen; Doing so would yield the predictable result of thresholding brightness in an image to more clearly define an image.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1), and in further view of Kudo (US 10805553 B2).
Regarding Claim 14, the image processing method of Claim 4 is rejected over Sun as modified.
Sun as modified does not teach for a rendering application involving a composite of two images: the generating is performed independently on each image, generating respective adjustment functions based on statistics of the respective image's pixel values and measured brightness, the respective adjustment functions are merged into a composite adjustment function with each adjustment function scaled in association with its respective image's brightness, and the applying applies the composite adjustment function to the pixel values of the composite image.
However, Kudo teaches for a rendering application involving a composite of two images (Claim 13: “generating a composite image by superimposing (1) the second image in which the brightness is adjusted using the second sensitivity value onto (2) the first image in which the brightness is adjusted using the first sensitivity value”):
the generating is performed independently on each image (Claim 13: “generating a composite image by superimposing (1) the second image in which the brightness is adjusted using the second sensitivity value onto (2) the first image in which the brightness is adjusted using the first sensitivity value”),
generating respective adjustment functions based on statistics of the respective image's pixel values and measured brightness (Claim 13: “calculating a first sensitivity value of the first image to allow brightness of the first image to be equal to a predetermined value or to be a value within a first range from the predetermined value, and calculating a second sensitivity value of the second image relating to brightness of the second image based on the first sensitivity value and a predetermined ratio”; Claim 1: “calculation circuitry configured to calculate a first sensitivity value of the first image to allow brightness of the first image to be equal to a predetermined value or to be a value within a first range from the predetermined value, and to calculate a second sensitivity value of the second image relating to brightness of the second image based on the first sensitivity value and a predetermined ratio”; Column 5, line 39-45: “In step S204, the evaluation calculation circuitry 135 calculates the evaluation value of the background image. The range that the brightness is evaluated in the background image may be the entire background image, or the photometry area set in step S202. The brightness of the background image may be an average of the brightness values of each pixel, or an integration of the maximum RGB pixel values”),
the respective adjustment functions are merged into a composite adjustment function with each adjustment function scaled in association with its respective image's brightness, and the applying applies the composite adjustment function to the pixel values of the composite image (Claim 13: “generating a composite image by superimposing (1) the second image in which the brightness is adjusted using the second sensitivity value onto (2) the first image in which the brightness is adjusted using the first sensitivity value; receiving a plurality of types of the second light”. Notes: because the generating of the composite image includes the adjustment functions for each image, the entire process is considered to utilize a “merged adjustment function” on a composite image).
Sun as modified and Kudo are considered analogous in the art with respect to adjusting the brightness of images for display. A common motivation in the art is to accurately adjust brightness for specific images; this is commonly achieved by adjusting brightness in images based on the images themselves, as opposed to an all-purpose adjustment function to adjust brightness in a uniform manner across multiple pictures, as is evident in Kudo.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the image processing method of Sun as modified with the composite adjustment function of Kudo; Doing so would yield the predictable result of effectively adjusting the brightness in composite images.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1), and in further view of Magielse (US 20180160501 A1).
Regarding Claim 16, the method of Claim 15 is rejected over Sun as modified.
Sun as modified does not explicitly teach that the grading operation includes applying a color palette to the physical-domain pixel values.
However, Magielse teaches a grading operation includes applying a color palette to the physical-domain pixel values (Paragraph [0073]: “The color palette P(I) generated by the color quantize 58 for an image I may for example be a reduced-color version of that image itself, generated by mapping ranges of pixel values to color bins, whereby each bin constitutes a single color value representing a range of (now indistinguishable) colors in the original image. Alternatively, the color palette P(I) may be come other data structure describing a range of colors in the image. For instance, it may constitute an ordered list of a number of the most frequently occurring colors in that image I or of the most frequently occurring color bins in a color quantized version of that image”).
Sun as modified and Magielse are considered analogous in the art with respect to the adjustment of pixel intensity values for display. A common motivation in the art is to simplify or reduce color variation in an image for a more basic display of an image. This is commonly achieved via color binning and a color palette, as is evident in Magielse.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the grading operation of Sun as modified with the use of a color palette in a grading operation of Magielse; Doing so would yield the predictable result of enabling a simpler display experience of an image.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (WO 2019105305 A1), in view of Price (US 20230117006 A1), and in further view of Knorr (US 20150279113 A1).
Regarding Claim 19, the method of Claim 15 is rejected over Sun as modified.
Sun as modified does not teach that the grading operation includes blending a content item with the physical-domain representation of the source image.
However, Knorr teaches a grading operation includes blending a content item with the physical-domain representation of the source image (Paragraph [0005]: “In Augmented Reality (AR), a view of a real environment, such as a video image of the real environment, is combined with an overlay of one or more virtual objects in a spatial relationship to the real environment. For many AR applications, the virtual objects shall seamlessly integrate in the view so that real and virtual objects cannot be distinguished”).
Sun as modified and Knorr are considered analogous in the art with respect to the capture of a real environment and adjustment of pixel values resulting in an updated display of an environment. A common motivation in the art is to simulate scenarios within a real environment, of which blending content that did not previously exist in the environment within the image is a commonly utilized method of simulation, as is evident in Knorr.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the grading operation of Sun as modified with the grading operation of Knorr that blends content into an image; Doing so would yield the predictable result of an image depicting an environment with content blended in.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/ Examiner, Art Unit 2614
/KENT W CHANG/ Supervisory Patent Examiner, Art Unit 2614