Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Response to Amendment
Claims 1-3 and 5-7 are pending in this application [8/12/2026].
Claims 1-3 and 5 have been amended [8/12/2026].
Claim 4 has been cancelled [8/12/2026].
Claims 6-7 have been added [8/12/2026].
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-3 and 5-7 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument based on newly applied reference He et al. (US-2024/0320810).
Applicant's arguments filed 8/12/2026 have been fully considered but they are not persuasive. On pages 7-9 of Applicant’s Arguments, Applicant argues ‘the Examiner’s stated rationale does not establish that amended claim 1 would have been obvious. Merely characterizing Wendel and Khanna as belonging to the broad field of “image processing systems” does not provide an articulated reason with rational underpinning, for modifying Wendel to arrive at the particular claimed architecture… Examiner’s rationale likewise fails to establish the obviousness of claim 2… Moreover, the Examiner’s generalized objective of improving details in dim image regions does not explain why a skilled artisan would modify Ninan’s two-view disparity-and-interpolation process to detect usable pixels specifically…’.
In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, Wendel, Khanna and Ninan are all image processing systems that processes input images that are captured, who perform different tasks that ultimately teaches the claimed invention. Whether it is a “broad field” or a “narrow field”, each of the prior art references perform image processing which is what the claimed invention is performing.
Applicant's arguments filed 8/12/2026 have been fully considered but they are not persuasive. On page 9 last paragraph to page 11 of Applicant’s Arguments, Applicant argues there is no suggestion or motivation to make the proposed modification over Wendel, Khanna, and Ninan per In re Gordon because the combination must not destroy functionality of original art; the proposed combination is therefore technically unsatisfactory and does not render claim 1 obvious.
In response to applicant's argument that there is no suggestion or motivation to make the proposed modification over Wendel, Khanna, and Ninan per In re Gordon because the combination must not destroy functionality of original art; the proposed combination is therefore technically unsatisfactory and does not render claim 1 obvious, a recitation of the intended use of the claimed invention must result in a structural difference between the claimed invention and the prior art in order to patentably distinguish the claimed invention from the prior art. If the prior art structure is capable of performing the intended use, then it meets the claim.
In addition, a motivation to combine may be found explicitly or implicitly in market force; design inventive; ‘the interrelated teachings of multiple patents’; ‘any need or problem known in the field of endeavor at the time of invention and addressed by the patent’; and the background knowledge, creativity, and common sense of the person of ordinary skill. Thus, the rationale supporting a conclusion of obviousness between Wendel, Khanna, and Ninan are known work in one field of endeavor, image processing systems, that prompts variation of their teachings in the same field.
Applicant's arguments filed 8/12/2026 have been fully considered but they are not persuasive. On page 11 last paragraph to page 13 of Applicant’s Arguments, Applicant argues there is no suggestion or motivation to make the proposed modification over Wendel, Khanna, and Ninan per In re Ratti because the claimed combination cannot change the principle of operation of the primary reference or render the reference inoperable for its intended purpose.
In response to applicant's argument that there is no suggestion or motivation to make the proposed modification over Wendel, Khanna, and Ninan per In re Ratti because the claimed combination cannot change the principle of operation or require substantial reconstruction of the primary reference or render the reference inoperable for its intended purpose, a recitation of the intended use of the claimed invention must result in a structural difference between the claimed invention and the prior art in order to patentably distinguish the claimed invention from the prior art. If the prior art structure is capable of performing the intended use, then it meets the claim.
In addition, a motivation to combine may be found explicitly or implicitly in market force; design inventive; ‘the interrelated teachings of multiple patents’; ‘any need or problem known in the field of endeavor at the time of invention and addressed by the patent’; and the background knowledge, creativity, and common sense of the person of ordinary skill. Lastly, the selection of a known material/process based on its suitability for its intended use supports a prima facie obviousness determination in Sinclair & Carroll Co. v. Interchemical Corp., 325 U.S. 327, 65 USPQ 297 (1945). Thus, the rationale supporting a conclusion of obviousness between Wendel, Khanna, and Ninan are known work in one field of endeavor, image processing systems, that prompts variation of their teachings in the same field.
Applicant's arguments filed 8/12/2026 have been fully considered but they are not persuasive. On page 13 last paragraph to page 15 of Applicant’s Arguments, Applicant argues that the proposed combinations of Wendel, Khanna, and Ninan are based on impermissible hindsight reconstruction.
In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971).
Applicant's arguments filed 8/12/2026 have been fully considered but they are not persuasive. On page 15 last paragraph to page 17 of Applicant’s Arguments, Applicant argues that the inventors recognized a problem in the art that Wendel, Khanna, and Ninan did not recognize, namely, that independently performing tone mapping to generate a visually suitable HDR image does not necessarily preserve or emphasize the information most useful for downstream automatic object detection, and that improved object-detection performance therefore requires the tone-mapping sub-block and the object-detection sub-block to be trained end-to-end together… implementing amended claim 1 would require substantial additional system architecture and associated expense.
In response to applicant's argument that Wendel, Khanna, and Ninan did not recognize, namely, that independently performing tone mapping to generate a visually suitable HDR image does not necessarily preserve or emphasize the information most useful for downstream automatic object detection, and that improved object-detection performance therefore requires the tone-mapping sub-block and the object-detection sub-block to be trained end-to-end together and would require substantial additional system architecture and associated expense, the fact that the inventor has recognized another advantage which would flow naturally from following the suggestion of the prior art cannot be the basis for patentability when the differences would otherwise be obvious. See Ex parte Obiaya, 227 USPQ 58, 60 (Bd. Pat. App. & Inter. 1985). In addition, newly applied reference He et al. teaches high dynamic range (HDR) images are online processed with object detection and tone mapping to low dynamic range (LDR) image data, where offline image processing is performed by storing an output of online image processing using online transmission provided to deep neural networks, thus consuming fewer computational resources for storage and processing tasks to obtain a more suitable human viewing [Fig 5A (502), par 0032-0033, 0037, 0040-0041, 0082-0085, 0109, 0114-0119].
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Wendel et al. (US-2019/0208111) in view of Khanna, M. (HDR Imaging: What is an HDR image anyway?) [hereinafter Khanna] and further in view of He et al. (US-2024/0320810).
As to Claim 1, Wendel teaches ‘A high-performance automatic object detection system using High Dynamic Range (HDR) images obtained from Low Dynamic Range (LDR) cameras, comprising the following steps: operating first, second, and third LDR cameras with different exposure values and located in different positions [Figs 8A-B, par 0138-0139, 0175-0180, 0200-0201 – a camera system mounted on an autonomous vehicle and used for object detection including first, second and third image sensors having different exposure settings including variable exposure, resulting in different dynamic ranges and located in different positions, where the image sensor captures may be combined into a single HDR, 24-bit image such that the combined image may be processed by one of the object-identification algorithms optimized for HDR, 24-bit images]; recording all image points that are difficult to see due to differences in illumination in the scene to be captured, with variable duration exposure values so that details are noticed by the first and third LDR cameras [par 0176-0180 – together with the dynamic range of the first and third image sensors including having variable/modified exposure settings and enabling to capture details of lower luminance than the second image sensor having fixed exposure settings]; combining the images recorded according to the exposure times and normalized and made combinable according to the exposure times to generate an HDR image on the second camera and saving this image as an input to the object detection algorithm [Figs 8A-B, par 0093, 0138-0139, 0175, 0200-0201 – image sensor captures are stored and may be combined into a single HDR, 24-bit image such that the combined image may be processed by one of the object-identification algorithms optimized for HDR, 24-bit images]; saving images, which are outputs of a tone mapping step, using the HDR images as input to the object detection algorithm and automatically detecting objects [par 0002, 0039, 0139, 0175 – combining the captured images into a single HDR, 24-bit image may include performing tone mapping and allow for the use of an expanded library of object-identification algorithms, where in object identification may involve a machine-learning algorithm]’.
Wendel does not disclose expressly ‘making images recorded by the first and third LDR cameras combinable after gamma correction by equalizing them according to their exposure times; by taking into account the pixel disparity values of the cameras at different locations; wherein the tone mapping step is performed by a tone mapping sub-block, and the tone mapping sub-block and an object detection sub-block including the object detection algorithm are trained end-to-end together’, although Wendel teaches three different cameras have different properties, including variably dynamic ranges and being in different locations, are combined into a single HDR, 24-bit image such that the combined image may be processed by one of the object-identification algorithms optimized for HDR, 24-bit images.
Khanna in the proposed combination teaches ‘making images recorded by the first and third LDR cameras combinable after gamma correction by equalizing them according to their exposure times; and by taking into account the pixel disparity values of the cameras at different locations [pg 8-9, 12-15, 17-18 – bracketing and merging three images captured at multiple exposure values, saving them as a single HDR image while performing a CRF function and tone mapping by de-linearisation (i.e., gamma correction) allowing to compensate for a non-linear display using neural networks and optical flow assigning displacement vectors]’.
Wendel and Khanna are analogous art because they are from the same field of endeavor, image processing systems. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to include bracketing and merging based on gamma correction and tone mapping, as taught by Khanna. The motivation for doing so would have been to combining images of different exposures having a wide range of light intensity levels of a scene. Therefore, it would have been obvious to combine Khanna with Wendel to obtain the invention.
Wendel in view of Khanna does not disclose expressly ‘wherein the tone mapping step is performed by a tone mapping sub-block, and the tone mapping sub-block and an object detection sub-block including the object detection algorithm are trained end-to-end together’.
He in the proposed combination teaches ‘wherein the tone mapping step is performed by a tone mapping sub-block, and the tone mapping sub-block and an object detection sub-block including the object detection algorithm are trained end-to-end together [Fig 5A (502), par 0032-0033, 0037, 0040-0041, 0082-0085, 0109, 0114-0119 – high dynamic range (HDR) images are online processed with object detection and tone mapping to low dynamic range (LDR) image data, where offline image processing is performed by storing an output of online image processing (i.e., end-to-end together) using online transmission provided to deep neural networks]’.
Wendel in view of Khanna are analogous art with He because they are from the same field of endeavor, image processing systems. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to include deep neural networks for processing input image data including tone mapping and object detection, as taught by He. The motivation for doing so would have been to consuming fewer computational resources for storage and processing tasks to obtain a more suitable human viewing. Therefore, it would have been obvious to combine He with Wendel in view of Khanna to obtain the invention as specified in claim 1.
As to Claim 5, Wendel teaches ‘wherein the first and third cameras are positioned on the sides and the second camera is positioned in the middle [Fig 8A (810, 820, 830) – second image sensor in the middle of first and third image sensors]’.
Claims 2-3 and 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Wendel et al. in view of Khanna, M., He et al. and further in view of Ninan et al. (US-9,420,200)].
As to Claim 2, Wendel in view of Khanna and He teaches all of the claimed elements/features as recited in independent claim 1. Wendel in view of Khanna and He does not disclose expressly ‘wherein the usable pixels from the second camera are detected while generating the HDR image and saved as direct input to generate an HDR image’.
Ninan in the proposed combination teaches ‘wherein the usable pixels from the second camera are detected while generating the HDR image and saved as direct input to generate an HDR image [col 8, lines 13-36, col 9, line 45-col 10, line 45 – flagging underexposure and overexposure pixels in the left and right input images, computing disparity for unflagged pixels, interpolating disparity for flagged pixels and merging shifted pixels derived from the left and right input scanlines to form high dynamic range output pixels in the shifted view by using a weight-based method]’.
Wendel in view of Khanna and He are analogous art with Ninan because they are from the same field of endeavor, image processing systems. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to include different processes on flagged/unflagged pixels of different images to generate high dynamic range images from narrow dynamic range image sensors, as taught by Ninan. The motivation for doing so would have been to improving captured image details in dim areas of a scene with elements having different exposure settings. Therefore, it would have been obvious to combine Ninan with Wendel in view of Khanna and He to obtain the invention as specified in claim 2.
As to Claim 3, Ninan teaches ‘wherein the unusable pixels from the second camera are combined with the pixels from the first and third cameras by weighting while generating the HDR image [col 8, lines 13-36, col 9, line 45-col 10, line 45 – flagging underexposure and overexposure pixels in the left and right input images, computing disparity for unflagged pixels, interpolating disparity for flagged pixels and merging shifted pixels derived from the left and right input scanlines to form high dynamic range output pixels in the shifted view by using a weight-based method]’.
Wendel in view of Khanna and He are analogous art with Ninan because they are from the same field of endeavor, image processing systems. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to include different processes on flagged/unflagged pixels of different images to generate high dynamic range images from narrow dynamic range image sensors, as taught by Ninan. The motivation for doing so would have been to improving captured image details in dim areas of a scene with elements having different exposure settings. Therefore, it would have been obvious to combine Ninan with Wendel in view of Khanna and He to obtain the invention as specified in claim 3.
As to Claim 6, Wendel in the proposed combination teaches ‘wherein the first and third cameras are positioned on the sides and the second camera is positioned in the middle [Fig 8A (810, 820, 830) – second image sensor in the middle of first and third image sensors]’.
As to Claim 7, Wendel in the proposed combination teaches ‘wherein the first and third cameras are positioned on the sides and the second camera is positioned in the middle [Fig 8A (810, 820, 830) – second image sensor in the middle of first and third image sensors]’.
Conclusion
The prior art made of record
a. US Publication No. 2024/0320810
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MIYA J CATO whose telephone number is (571)270-3954. The examiner can normally be reached M-F, 830-530.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi Sarpong can be reached at 571.270.3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MIYA J CATO/Primary Examiner, Art Unit 2681
