DETAILED ACTION
1. Claims 1-20 are pending in the application.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Election /Restriction.
3a. During the interview dated 3/5/2026 Examiner and the Applicant’s representative Attorney Eric L. Sophir discussed a possible restriction between the following two groups I and II.
I. Claims 1-17 are drawn to techniques of enhancing images using an imaging sensor system, and classified in G06T5/50 and G06T3/4015.
II. Claims 18-20 drawn to a method of assembling a camera system on a vehicle, and classified in B60R1/00.
Applicant’s representative hereby elects Group I without traverse.
3b. Claims 18-30 are withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected invention, there being no allowable generic or linking claim
Restriction/Election
4. Restriction to one of the following inventions is required under 35 U.S.C. 121:
I. Claims 1-17 are drawn to techniques of enhancing images using an imaging sensor system, and classified in G06T5/50 and G06T3/4015
II. Claims 18-20 drawn to a method of assembling a camera system on a vehicle, and classified in B60R1/00.
The inventions are distinct, each from the other because of the following reasons:
Inventions I and II are related as subcombinations disclosed as usable together in a single combination. The subcombinations are distinct from each other if they are shown to be separately usable. In the instant case, invention I has separate utility such image enhancement
(see Fig.2b and 3), invention II has separate utility such as assembling different cameras in different section of a vehicle for real time viewing argument for a driver (see Figs. 1 and 4). See MPEP 806.05(d).
4. Because these inventions are distinct for the reasons given above and have acquired a separate status in the art as shown by their different classification, restriction for examination purposes as indicated is proper.
5. Applicant is reminded that upon the cancellation of claims to a non-elected invention, the inventor ship must be amended in compliance with 37 CFR 1.48(b) if one or more of the currently named inventors is no longer an inventor of at least one claim remaining in the application. Any amendment of inventor ship must be accompanied by a petition under 37 CFR 1.48(b) and by the fee required under 37 CFR 1.17(I).
6. Applicant is advised that the reply to this requirement to be complete must include an election of the invention to be examined even though the requirement be traversed (37 CFR 1.143).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless -
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
7. Claims 1,5 and 12 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Alves; James (hereafter Alves ), US 20170195605 A1, pub., 07/06/2017
As to claim 1, Alves teaches A method implemented by one or more processors included in a vehicle ( Abstract, [0004], A digital camera control system, where the system therefore provides a method to rapidly determine an optimum camera settings for any time of day and ensures the camera is always ready to capture at least the minimum required contrast image of a fast moving transient object that include includes a vehicle on a motorway), the method comprising:
obtaining a plurality of raw images of a real-world scene about the vehicle ( [0068]-[0069], a method of obtaining different regains of vehicle that includes two regiones two regions one for the vehicle plates in the shade 1711 and a second region for the vehicle plates located in the sun 1710, and background regions of the license ), the raw images being associated with different integration times for individual pixels which form the raw images ( [0044], digital camera control system is to set sensor parameters the exposure control parameters of integration time, reset times, reset voltages and amplifier gains to ensure that images of the plate (i.e., two regiones of the plates ) have sufficient contrast so that a features can be recognized against a background);
applying an analog gain to the raw images ([0068], Fig. 1, the camera further has the ability to adjust the gain of the amplifier 104 (FIG. 1)), the analog gain being selected based on a lux estimate for the real-world scene ([0116], shown in FIGS. 25-26 the gain on the image sensor electronics is increased if at the time of the image acquisition event there is no retro-reflection of the sun off the object thereby reducing the range of values of luminance over which the system will operate without saturation but increasing the signal for the region of interest. It is known that lux is the unit of luminance ); and
forming an output image based on a combination of the plurality of images, wherein each pixel of the output image is based on a corresponding pixel of an individual raw image selected based on its integration time ([0043], a plurality of slopes are used to optimize the acquired image data and enable sufficient contrast to recognize characters on the plate in images acquired at integration times and gains required to recognize characters on a rapidly moving vehicle in a variety of lighting conditions. The resultant output 111 is an 8 bit still image of the object of interest, in this case a vehicle and its license plate driving on a roadway).
As to claim 5, Alves teaches applying the analog gain to the plurality of raw images comprising applying analog gain to raw sensor information of the plurality of raw images ([0044], 0044] The purpose of the invented digital camera control system is to set sensor parameters the exposure control parameters of integration time, reset times, reset voltages and amplifier gains to ensure that images of the plate have sufficient contrast so that a features can be recognized against a background).
Claim 12 is rejected the same as claim 1 except claim 12 is directed to a system claim. the rejection of claim 1 includes all the limitation of claim 12. Thus, argument analogous to that presented above for claim 1 is applicable to claim 12.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 2-4,8-11, 13-14 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Alves ), US 20170195605 A1, in view of PINHASOV; et al., (hereafter PINHASOV), US20220060619A, filed on 01/26/2021
Regarding claim 2, while Alves obtaining the plurality of raw images comprising using a ([0071], the regions of the data containing real image data are retained and scaled to cover the full dynamic range of the output while regions containing non-valid data points arising from noise or errors are rejected. This ensures all the useful data of the image and the full 8-bit dynamic range is retained while rejecting errors and noise), but fails to teach “high dynamic range ("HDR")”.
On the other hand PINHASOV teaches obtaining the plurality of raw images comprising using a high dynamic range ("HDR") camera to obtain the plurality of raw images ([0013], image combining for high dynamic range (HDR))
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a well- known high dynamic rang (HDR) taught by PINHASOV into Alves.
The suggestion/motivation for doing so would have been to allow user of Alves to generate high quality images by delivering a wider range of luminosity, producing deeper blacks, brighter whites, and a broader color palette using the features of HDR.
As to claim 3, PINHASOV teaches obtaining the plurality of raw images comprising using a Bayer filter to obtain color information that includes a color value of an individual color channel for individual pixels of the plurality of raw images ([0047], A Bayer color filters include red color filters, blue color filters, and green color filters, with each pixel of the image generated based on red light data from at least one photodiode covered in a red color filter, blue light data from at least one photodiode covered in a blue color filter, and green light data from at least one photodiode covered in a green color filter), and wherein a demosaic process is applied to interpolate the color information for individual pixels([0050] The image processor 150 may perform a number of tasks, such as de-mosaicing, color space conversion, image frame downsampling, pixel interpolation. It is known that a demosaic process is explicitly applied to interpolate color information in digital imaging))
As to claim 4, PINHASOV teaches selecting the analog gain to be applied to the plurality of raw images comprising selecting the analog gain according to a lookup table([0068], [0128], the ISP tuning parameters can also include additional parameters, such as gamma, gain, luminance, shading, edge enhancement, color correction (CC), color mapping (CM) (e.g., based on a 2D look-up table) and the lux estimate for the real-world scene ([0120] The different ISP tuning parameter can include, for example, gain, luminance, shading, edge enhancement, image combining for high dynamic range, where the luminance is associated to lux).
As to claim 8, PINHASOV teaches determining a lux estimate for the real-world scene is based at least in part on intensity information from a most sensitive color channel of a plurality of color channels. ( [0047], For instance, Bayer color filters include red color filters, blue color filters, and green color filters, with each pixel of the image generated based on red light data from at least one photodiode covered in a red color filter, blue light data from at least one photodiode covered in a blue color filter, and green light data from at least one photodiode covered in a green color filter. It is known that Green is the most sensitive color in Bayer filters, which is why they contain twice as many green filters (50%) as red (25%) or blue (25%). This design mimics the human eye, which is most sensitive to green light and perceives it as the primary contributor to brightens).
As to claim 9 PINHASOV teaches before applying the time ([0050] The image processor 150 may perform a number of tasks that includes, automatic exposure
(AE), and merging of image frames to form an HDR image. It is also known that HDR is fundamentally related to integration time through the combination of multiple exposures (short and long) to capture both bright highlights and dark shadows).
As to claim 10, PINHASOV teaches applying a digital gain to the output image ([0143], an analog gain, a digital gain,).
As to claim 11, PINHASOV wherein selecting the digital gain is based at least in part on a saturation level associated with the raw images ([0013], an ISO, an analog gain, a digital gain, a denoising, a sharpening, a tone mapping, a color saturation. It is known that the digital gain of a camera is related to color saturation. Specifically increasing digital gain (often associated with raising the ISO) boosts both the signal and, crucially, the noise, which can lead to a reduction in overall color saturation and a decrease in color accuracy).
Claim 13 is rejected the same as claim 2 except claim 13 is directed to a system claim. The rejection of claim 2 includes all the limitation of claim 13. Thus, argument analogous to that presented above for claim 2 is applicable to claim 13.
Claim 14 is rejected the same as claim 3 except claim 14 is directed to a system claim. The rejection of claim 3 includes all the limitation of claim 14. Thus, argument analogous to that presented above for claim 3 is applicable to claim 14.
Claim 17 is rejected the same as combination of claim 9 and 10 except claim 17 is directed to a system claim. The rejection of claims 9 and 10 includes all the limitation of claim 17. Thus, argument analogous to that presented above for claims 9 and 10 is applicable to claim 17.
8. Claims 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Alves , US 20170195605 A1, in view of Scott; Basil Henry (hereafter Scott), US 20130076910A, pub. 03/28/2013.
Regarding claim 6, while Alves teaches individual pixels of the output image are selected from corresponding pixels of the raw images by identifying a corresponding pixel which is not saturated ([0005], whereas for a portion of a license plate in direct sunlight there is a naturally high contrast so that the gain can be lowered to prevent saturation of the image sensor), but fails to teach “identifying a corresponding pixel which is not saturated and which is associated with the longest integration time”
On the other hand Scott teaches identifying a corresponding pixel which is not saturated and which is associated with the longest integration time (claim 7, a second re-changed
integration time and a second re-changed reset time, wherein said second re-changed
integration time is shorter than said second changed integration time and said second re-changed reset time is longer than said second changed reset time, and wherein if said small pixel output voltages indicate that said small pixels are unsaturated while using said second changed integration time and a second changed reset time).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method of identifying pixels that are not saturated while utilizing the longest possible integration time taught by Scott into Alves.
The suggestion/motivation for doing so would have been to allow user of Alves to generate maximizing the dynamic range. Specifically maximizes the Signal-to-Noise Ratio (SNR) while preserving detail in both bright and faint areas of the scene. It is known that a method identifying pixels that are not saturated while utilizing the longest possible integration time known as maximizing the dynamic range.
Claim 15 is rejected the same as claim 6 except claim 15 is directed to a system claim. The rejection of claim 6 includes all the limitation of claim 15. Thus, argument analogous to that presented above for claim 6 is applicable to claim 15.
8. Claims 7 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Alves , US 20170195605 A1, in view of KR 101401855 B1, pub. 05/29/2014.
Regarding claim 7, while Alves teaches obtaining the plurality of raw images comprises capturing a first raw image of the real-world(this limitation discussed in claim 1 above ), but fails to teach “capturing a first raw image of the real-world scene for around 14 to 16 milliseconds, capturing a second raw image of the real-world scene for around 0.5 to 1.5 ms, and capturing a third raw image of the real-world scene for around 1/20 to 1/20 ms”
On the other hand KR 101401855B1 teaches capturing a first raw image of the real-world scene for around 14 to 16 milliseconds, capturing a second raw image of the real-world scene for around 0.5 to 1.5 ms, and capturing a third raw image of the real-world scene for around 1/20 to 1/20 ms (page 6 last paragraph, when capturing a still image by the image pickup section 3, an operation of continuously receiving a captured image can be executed as a so-called continuous shutter function. For example, the image
capturing control unit 6 may sequentially capture captured image data at predetermined
time intervals such as several tens of milliseconds, several hundreds of milliseconds, one second, and several seconds in accordance with a single shutter operation by the user Processing is performed. That is, in the imaging section 3, the captured image data can be generated on a frame-by-frame basis and output to the imaging control section 6.)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method of capturing image data at predetermined, high-frequency intervals—specifically in the range of tens of milliseconds taught by KR 101401855B1into Alves.
The suggestion/motivation for doing so would have been to allow user of Alves to eliminating motion blur and visualize fast-moving, transient, or complex events.
Claim 16 is rejected the same as claim 7 except claim 16 is directed to a system claim. The rejection of claim 7 includes all the limitation of claim 16. Thus, argument analogous to that presented above for claim 7 is applicable to claim 16.
Contact Information
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Mekonen Bekele whose telephone number is (469) 295-9077.The examiner can normally be reached on Monday -Friday from 9:00AM to 6:50 PM Eastern Time.
If attempt to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Eng, George can be reached on (571) 272-7495.The fax phone number for the organization where the application or proceeding is assigned is 571-237-8300. Information regarding the status of an application may be obtained from the patent Application Information Retrieval (PAIR) system. Status information for published application may be obtained from either Private PAIR or Public PAIR.
Status information for unpublished application is available through Privet PAIR only.
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have question on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.217-919 (tool-free)
/MEKONEN T BEKELE/Primary Examiner, Art Unit 2699