Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-5 are pending in this application [7/1/2024].
Claims 1-5, Abstract and Specification have been preliminary amended [7/1/2024].
Drawings
The replacement drawings were received on 7/1/2024. These drawings are accepted for examination.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
Acknowledgment is made of applicant's claim for foreign priority based on an application filed in Turkey on 12/29/2021. It is noted, however, that applicant has not filed a certified copy of the TR-2021/021665 application as required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 7/29/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claim 1 is objected to because of the following informalities: Based on preliminary amendment, Claim 1, line 8 recites ‘and third LDR cameras’ without any punctuation mark “,” or “;”. Examiner treats as ‘and third LDR cameras,’ Appropriate correction is required.
Claim 1 is objected to because of the following informalities: Claim 1, line 17 recites the limitation ‘the tone mapping step’. There is insufficient antecedent basis for this limitation in the claim. Examiner treats as ‘a tone mapping step’. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 and 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Wendel et al. (US-2019/0208111) in view of Khanna, M. (HDR Imaging: What is an HDR image anyway?) [hereinafter Khanna].
As to Claim 1, Wendel teaches ‘A high-performance automatic object detection system using High Dynamic Range (HDR) images obtained from Low Dynamic Range (LDR) cameras, comprising the following steps: - operating first, second, and third LDR cameras with different exposure values and located in different positions [Figs 8A-B, par 0138-0139, 0175-0180, 0200-0201 – a camera system mounted on an autonomous vehicle and used for object detection including first, second and third image sensors having different exposure settings including variable exposure, resulting in different dynamic ranges and located in different positions, where the image sensor captures may be combined into a single HDR, 24-bit image such that the combined image may be processed by one of the object-identification algorithms optimized for HDR, 24-bit images]; - recording all image points that are difficult to see due to differences in illumination in the scene to be captured, with variable duration exposure values so that details can be noticed, by the first and third LDR cameras [par 0176-0180 – together with the dynamic range of the first and third image sensors including having variable/modified exposure settings and enabling to capture details of lower luminance than the second image sensor having fixed exposure settings], - combining the images recorded according to the exposure times and normalized and made combinable according to the exposure times to generate an HDR image as an HDR image on the second camera and saving this image as an input to the object detection algorithm [Figs 8A-B, par 0093, 0138-0139, 0175, 0200-0201 – image sensor captures are stored and may be combined into a single HDR, 24-bit image such that the combined image may be processed by one of the object-identification algorithms optimized for HDR, 24-bit images], - saving images, which are the outputs of a tone mapping step, using HDR images as input and trained together with the object detection algorithm, as input to the object detection algorithm and automatically detecting objects [par 0002, 0039, 0139, 0175 – combining the captured images into a single HDR, 24-bit image may include performing tone mapping and allow for the use of an expanded library of object-identification algorithms, where in object identification may involve a machine-learning algorithm]’.
Wendel does not disclose expressly ‘making images recorded by the first and third LDR cameras combinable after gamma correction by equalizing them according to their exposure times, and by taking into account the pixel disparity values of the cameras at different locations’, although Wendel teaches three different cameras have different properties, including variably dynamic ranges and being in different locations, are combined into a single HDR, 24-bit image such that the combined image may be processed by one of the object-identification algorithms optimized for HDR, 24-bit images.
Khanna in the proposed combination teaches ‘making images recorded by the first and third LDR cameras combinable after gamma correction by equalizing them according to their exposure times, and by taking into account the pixel disparity values of the cameras at different locations [pg 8-9, 12-15, 17-18 – bracketing and merging three images captured at multiple exposure values, saving them as a single HDR image while performing a CRF function and tone mapping by de-linearisation (i.e., gamma correction) allowing to compensate for a non-linear display using neural networks and optical flow assigning displacement vectors]’.
Wendel and Khanna are analogous art because they are from the same field of endeavor, image processing systems. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to include bracketing and merging based on gamma correction and tone mapping, as taught by Khanna. The motivation for doing so would have been to combining images of different exposures having a wide range of light intensity levels of a scene. Therefore, it would have been obvious to combine Khanna with Wendel to obtain the invention as specified in claim 1.
As to Claim 4, Wendel in the proposed combination of Khanna teaches ‘wherein the tone mapping block that transmits images to the automatic detection unit is selected as a learnable algorithm and the automatic detection and tone mapping blocks are trained together [par 0039, 0139, 0141, 0169, 0175, 0200-0201 – performing tone mapping to approximate HDR images, and performing object recognition on HDR images using a machine-learned object recognition algorithm]’.
As to Claim 5, Wendel teaches ‘wherein the first and third cameras are positioned on the sides and the second camera is positioned in the middle [Fig 8A (810, 820, 830) – second image sensor in the middle of first and third image sensors]’.
Claim(s) 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Wendel et al. in view of Khanna, M. and further in view of Ninan et al. (US-9,420,200)].
As to Claim 2, Wendel in view of Khanna teaches all of the claimed elements/features as recited in independent claim 1. Wendel in view of Khanna does not disclose expressly ‘wherein the usable pixels from the second camera are detected while generating the HDR image and saved as direct input to generate an HDR image’.
Ninan in the proposed combination teaches ‘wherein the usable pixels from the second camera are detected while generating the HDR image and saved as direct input to generate an HDR image [col 8, lines 13-36, col 9, line 45-col 10, line 45 – flagging underexposure and overexposure pixels in the left and right input images, computing disparity for unflagged pixels, interpolating disparity for flagged pixels and merging shifted pixels derived from the left and right input scanlines to form high dynamic range output pixels in the shifted view by using a weight-based method]’.
Wendel in view of Khanna are analogous art with Ninan because they are from the same field of endeavor, image processing systems. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to include different processes on flagged/unflagged pixels of different images to generate high dynamic range images from narrow dynamic range image sensors, as taught by Ninan. The motivation for doing so would have been to improving captured image details in dim areas of a scene with elements having different exposure settings. Therefore, it would have been obvious to combine Ninan with Wendel in view of Khanna to obtain the invention as specified in claim 2.
As to Claim 3, Ninan teaches ‘wherein the unusable pixels from the second camera are combined with the pixels from the first and third cameras by weighting while generating the HDR image [col 8, lines 13-36, col 9, line 45-col 10, line 45 – flagging underexposure and overexposure pixels in the left and right input images, computing disparity for unflagged pixels, interpolating disparity for flagged pixels and merging shifted pixels derived from the left and right input scanlines to form high dynamic range output pixels in the shifted view by using a weight-based method]’.
Wendel in view of Khanna are analogous art with Ninan because they are from the same field of endeavor, image processing systems. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to include different processes on flagged/unflagged pixels of different images to generate high dynamic range images from narrow dynamic range image sensors, as taught by Ninan. The motivation for doing so would have been to improving captured image details in dim areas of a scene with elements having different exposure settings. Therefore, it would have been obvious to combine Ninan with Wendel in view of Khanna to obtain the invention as specified in claim 3.
Conclusion
The prior art made of record
a. US Publication No. 2019/0208111
b. Khanna, M. (HDR Imaging: What is an HDR image anyway?)
c. US Publication No. 9,420,200
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
d. US Publication No. 2023/0121217
e. US Publication No. 2025/0069191
f. US Publication No. 2016/0277743
g. WO Publication No. 2023/083466A1
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MIYA J CATO whose telephone number is (571)270-3954. The examiner can normally be reached M-F, 830-530.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi Sarpong can be reached at 571.270.3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MIYA J CATO/Primary Examiner, Art Unit 2681
Prosecution Insights