Detailed Action
1. Claims 1-20 are pending in this Application.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to amendment
3. Applicant’s response to the last Office Action filed on 03/11/2026 has been entered and made of record.
4. Claims1-5 and 12 have been amended.
.
Response to Argument
5. The Applicant’s argument filed 06/06/2026 is fully consider. For Examiner response see discussion below.
6 The corrected sentence is: "The applicant has substantially amended claims 1 and 12 by adding the new limitation wherein forming the output image comprises, for each pixel position of the output image, selecting a pixel value from one of the plurality of raw images captured at different integration times, and has argued that the applied prior art does not specifically teach this newly added limitation. Based on the applicant's amendment and argument, the 35 U.S.C. § 102 rejection is expressly withdrawn. However, further search and consideration of a new prior art reference, 'Multiple Exposure Fusion for High Dynamic Range Image Acquisition' by Takao et al., which teaches the added limitation, has been found.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 1-2, 5-6 and 12-13 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Alves; James (hereafter Alves ), US 20170195605 A1, pub., 07/06/2017, in view of Takao et al. ( hereafter Takao) “Multiple Exposure Fusion for High Dynamic Range Image Acquisition”, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 1, JANUARY 2012.
As to claim 1, Alves teaches A method implemented by one or more processors included in a vehicle (Abstract, [0004], A digital camera control system, where the system therefore provides a method to rapidly determine an optimum camera settings for any time of day and ensures the camera is always ready to capture at least the minimum required contrast image of a fast moving transient object that include includes a vehicle on a motorway), the method comprising:
obtaining a plurality of raw images of a real-world scene about the vehicle ( [0068]-[0069], a method of obtaining different regains of vehicle that includes two regiones two regions one for the vehicle plates in the shade 1711 and a second region for the vehicle plates located in the sun 1710, and background regions of the license ), the raw images being associated with different integration times for individual pixels which form the raw images ( [0044], digital camera control system is to set sensor parameters the exposure control parameters of integration time, reset times, reset voltages and amplifier gains to ensure that images of the plate (i.e., two regiones of the plates ) have sufficient contrast so that a features can be recognized against a background);
applying an analog gain to the raw images ([0068], Fig. 1, the camera further has the ability to adjust the gain of the amplifier 104 (FIG. 1)), the analog gain being selected based on a lux estimate for the real-world scene ([0116], shown in FIGS. 25-26 the gain on the image sensor electronics is increased if at the time of the image acquisition event there is no retro-reflection of the sun off the object thereby reducing the range of values of luminance over which the system will operate without saturation but increasing the signal for the region of interest. It is known that lux is the unit of luminance ); and
forming an output image based on a combination of the plurality of images, wherein each pixel of the output image is based on a corresponding pixel of an individual raw image selected based on its integration time ([0043], a plurality of slopes are used to optimize the acquired image data and enable sufficient contrast to recognize characters on the plate in images acquired at integration times and gains required to recognize characters on a rapidly moving vehicle in a variety of lighting conditions. The resultant output 111 is an 8 bit still image of the object of interest, in this case a vehicle and its license plate driving on a roadway).
However, it is noted that Alves does not specifically teach “wherein forming the output image comprises, for each pixel position of the output image, selecting a pixel value from one of the plurality of raw images captured at different integration times”
On the other hand in the same field of endeavor method for High Dynamic
Range Image Acquisition by Takao teaches wherein forming the output image comprises, for each pixel position of the output image, selecting a pixel value from one of the plurality of raw images captured at different integration times (page 359, section B: Photometric Camera Calibration, the relationship between irradiance i and the amount of lights L that we measure through some sensor can be expressed by
L=i [Symbol font/0xB4] t
where t is the exposure time (shutter speed). The integration time corresponds to the exposure time (shutter speed). Takao specifically teaches a method where the multiple exposure images are taken by varying the exposure time of a camera with other settings fixed. Then, the images are merged to create the HDRI by Ihdr
PNG
media_image1.png
76
442
media_image1.png
Greyscale
where Im hdr is a value of the pixel m of the th exposure image. N is the number of images, tn is the exposure time of nth image, and w is a weighting function. The weighting function has small values for the underexposed and overexposed pixels, and these pixel values are ignored. If a scene is completely static and the images are aligned, and the images
combine using equation (4). If there is motion between images, ghosting artifact may appear. The images are aligned and combine the images by removing the ghosting artifact.)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the method of constructing high dynamic range images (HDRIs) by combining multiple exposures and estimating pixel irradiance, as taught by Takao, into the system of Alves.
The motivation for doing so would have been to allow users of Alves to produce a single image with a wider range of dark and bright details than any standard photo can capture. By combining several shots taken at different light levels, this process reveals hidden textures in deep shadows and bright highlights simultaneously."
As to claim 2, Takao teaches obtaining the plurality of raw images comprises using a high dynamic range ("HDR") camera to obtain the plurality of raw images (page 359, section B: Photometric Camera Calibration Photometric Camera multiple exposure images are taken by varying the exposure time of a camera with other settings fixed. Then, the images are merged to create the HDRI ).
Regarding the motivation statement, the amotivation applied to claim 1 above equally applies to claim 2.
As to claim 5, Alves teaches applying the analog gain to the plurality of raw images comprising applying analog gain to raw sensor information of the plurality of raw images ([0044], 0044] The purpose of the invented digital camera control system is to set sensor parameters the exposure control parameters of integration time, reset times, reset voltages and amplifier gains to ensure that images of the plate have sufficient contrast so that a features can be recognized against a background).
Regarding claim 6, while Alves teaches individual pixels of the output image are selected from corresponding pixels of the raw images by identifying a corresponding pixel which is not saturated ([0005], whereas for a portion of a license plate in direct sunlight there is a naturally high contrast so that the gain can be lowered to prevent saturation of the image sensor), but fails to teach “identifying a corresponding pixel which is not saturated and which is associated with the longest integration time”
On the other hand Takao teaches “identifying a corresponding pixel which is not saturated and which is associated with the longest integration time (page 359, section C: Weighting Function, Photometric Camera Calibration In (4), the weighting function is introduced because underexposed or overexposed regions are much less reliable than the regions of middle intensities. Thus, in the conventional methods, the weight is specified to be small for pixel values near saturation values 0 and 1 and high for the middle intensities. Two examples for the weighting functions used in the conventional methods [6], [7] are shown in Fig. 1. The role of the weight is to discard saturated pixels. The region where the pixel values are close to 0 or 255 is backed up by other exposure with larger weight ( longer shutter speed i.e., longer integration time).
Regarding the motivation statement, the amotivation applied to claim 1 above equally applies to claim 6.
Claim 12 is rejected the same as claim 1 except claim 12 is directed to a system claim. the rejection of claim 1 includes all the limitation of claim 12. Thus, argument analogous to that presented above for claim 1 is applicable to claim 12.
Claim 13 is rejected the same as claim 2 except claim 13 is directed to a system claim. The rejection of claim 2 includes all the limitation of claim 13. Thus, argument analogous to that presented above for claim 2 is applicable to claim 13.
Claim 15 is rejected the same as claim 6 except claim 15 is directed to a system claim. The rejection of claim 6 includes all the limitation of claim 15. Thus, argument analogous to that presented above for claim 6 is applicable to claim 15.
7. Claims 3- 4,8-11, 14 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Alves , US 20170195605 A1, in view of Takao “Multiple Exposure Fusion for High Dynamic Range Image Acquisition” , further in view of PINHASOV; et al., (hereafter PINHASOV), US20220060619A, filed on 01/26/2021
Regarding claim 3, while modified Alves teaches the limitation of claim 1, but fails to teach the limitation of claim 3.
On the other hand PINHASOV teaches obtaining the plurality of raw images comprising using a Bayer filter to obtain color information that includes a color value of an individual color channel for individual pixels of the plurality of raw images ([0047], A Bayer color filters include red color filters, blue color filters, and green color filters, with each pixel of the image generated based on red light data from at least one photodiode covered in a red color filter, blue light data from at least one photodiode covered in a blue color filter, and green light data from at least one photodiode covered in a green color filter), and wherein a demosaic process is applied to interpolate the color information for individual pixels([0050] The image processor 150 may perform a number of tasks, such as de-mosaicing, color space conversion, image frame downsampling, pixel interpolation. It is known that a demosaic process is explicitly applied to interpolate color information in digital imaging)) ,
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate image a demosaic process taught by PINHASOV into modified Alves.
The suggestion/motivation for doing so would have been to allow user of modified Alves to reconstruct full color data early, preventing false color artifacts and harsh clipping around bright light sources during the blending process.
As to claim 4, PINHASOV teaches selecting the analog gain to be applied to the plurality of raw images comprising selecting the analog gain according to a lookup table([0068], [0128], the ISP tuning parameters can also include additional parameters, such as gamma, gain, luminance, shading, edge enhancement, color correction (CC), color mapping (CM) (e.g., based on a 2D look-up table) and the lux estimate for the real-world scene ([0120] The different ISP tuning parameter can include, for example, gain, luminance, shading, edge enhancement, image combining for high dynamic range, where the luminance is associated to lux).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate image a demosaic process taught by PINHASOV into modified Alves.
The suggestion/motivation for doing so would have been to allow user of Alves to reconstruct full color data early, preventing false color artifacts and harsh clipping around bright light sources during the blending process
As to claim 8, PINHASOV teaches determining a lux estimate for the real-world scene is based at least in part on intensity information from a most sensitive color channel of a plurality of color channels. ( [0047], For instance, Bayer color filters include red color filters, blue color filters, and green color filters, with each pixel of the image generated based on red light data from at least one photodiode covered in a red color filter, blue light data from at least one photodiode covered in a blue color filter, and green light data from at least one photodiode covered in a green color filter. It is known that Green is the most sensitive color in Bayer filters, which is why they contain twice as many green filters (50%) as red (25%) or blue (25%). This design mimics the human eye, which is most sensitive to green light and perceives it as the primary contributor to brightens).
Regarding the motivation statement, the amotivation applied to claim 3 above equally applies to claim 8
As to claim 9 PINHASOV teaches before applying the time ([0050] The image processor 150 may perform a number of tasks that includes, automatic exposure
(AE), and merging of image frames to form an HDR image. It is also known that HDR is fundamentally related to integration time through the combination of multiple exposures (short and long) to capture both bright highlights and dark shadows).
Regarding the motivation statement, the amotivation applied to claim 3 above equally applies to claim 9.
As to claim 10, PINHASOV teaches applying a digital gain to the output image ([0143], an analog gain, a digital gain,).
Regarding the motivation statement, the amotivation applied to claim 3 above equally applies to claim 10.
As to claim 11, PINHASOV wherein selecting the digital gain is based at least in part on a saturation level associated with the raw images ([0013], an ISO, an analog gain, a digital gain, a denoising, a sharpening, a tone mapping, a color saturation. It is known that the digital gain of a camera is related to color saturation. Specifically increasing digital gain (often associated with raising the ISO) boosts both the signal and, crucially, the noise, which can lead to a reduction in overall color saturation and a decrease in color accuracy).
Regarding the motivation statement, the amotivation applied to claim 3 above equally applies to claim 11.
Claim 14 is rejected the same as claim 3 except claim 14 is directed to a system claim. The rejection of claim 3 includes all the limitation of claim 14. Thus, argument analogous to that presented above for claim 3 is applicable to claim 14.
Claim 17 is rejected the same as combination of claim 9 and 10 except claim 17 is directed to a system claim. The rejection of claims 9 and 10 includes all the limitation of claim 17. Thus, argument analogous to that presented above for claims 9 and 10 is applicable to claim 17.
8. Claims 7 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Alves , US 20170195605 A1, in view of Takao “Multiple Exposure Fusion for High Dynamic Range Image Acquisition”, further in view of KR 101401855 B1, pub. 05/29/2014.
Regarding claim 7, while Alves teaches obtaining the plurality of raw images comprises capturing a first raw image of the real-world(this limitation discussed in claim 1 above ), but fails to teach “capturing a first raw image of the real-world scene for around 14 to 16 milliseconds, capturing a second raw image of the real-world scene for around 0.5 to 1.5 ms, and capturing a third raw image of the real-world scene for around 1/20 to 1/20 ms”
On the other hand KR 101401855B1 teaches capturing a first raw image of the real-world scene for around 14 to 16 milliseconds, capturing a second raw image of the real-world scene for around 0.5 to 1.5 ms, and capturing a third raw image of the real-world scene for around 1/20 to 1/20 ms (page 6 last paragraph, when capturing a still image by the image pickup section 3, an operation of continuously receiving a captured image can be executed as a so-called continuous shutter function. For example, the image
capturing control unit 6 may sequentially capture captured image data at predetermined
time intervals such as several tens of milliseconds, several hundreds of milliseconds, one second, and several seconds in accordance with a single shutter operation by the user Processing is performed. That is, in the imaging section 3, the captured image data can be generated on a frame-by-frame basis and output to the imaging control section 6.)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method of capturing image data at predetermined, high-frequency intervals—specifically in the range of tens of milliseconds taught by KR 101401855B1into modified Alves.
The suggestion/motivation for doing so would have been to allow user of Alves to eliminating motion blur and visualize fast-moving, transient, or complex events.
Claim 16 is rejected the same as claim 7 except claim 16 is directed to a system claim. The rejection of claim 7 includes all the limitation of claim 16. Thus, argument analogous to that presented above for claim 7 is applicable to claim 16.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Mekonen Bekele whose telephone number is (469) 295-9077.The examiner can normally be reached on Monday -Friday from 9:00AM to 6:50 PM Eastern Time.
If attempt to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Eng, George can be reached on (571) 272-7495.The fax phone number for the organization where the application or proceeding is assigned is 571-237-8300. Information regarding the status of an application may be obtained from the patent Application Information Retrieval (PAIR) system. Status information for published application may be obtained from either Private PAIR or Public PAIR.
Status information for unpublished application is available through Privet PAIR only.
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have question on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.217-919 (tool-free)
/MEKONEN T BEKELE/Primary Examiner, Art Unit 2699
Prosecution Insights