DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/26/2026 has been entered.
Response to Amendment
This Office Action is in response to the Request for Continued Examination filed on 05/26/2026.
Status of the Claims:
Claim(s) 1 and 15-17 has/have been amended.
Claim(s) 1-17 is/are pending in this Office Action.
Response to Arguments
Applicant’s arguments are deemed moot since they are directed to the newly added claim limitations, not previously presented and not against the previous rejected limitations. Newly added limitations are addressed below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 5-6 and 10-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over IDS provided reference US 2008/0219654 to Border et al. (hereinafter Border) in view of US 9,571,741 to Ramsay (hereinafter Ramsay).
Regarding independent claim 1, Border teaches a method comprising:
determining, based on captured sensor data representative of a scene (auto exposure detector 46, see Fig. 1), a likelihood of exposure-related defects in the scene to be captured by multiple image-capture devices, the likelihood of exposure-related defects including a first likelihood of blur defects and a second likelihood of high-noise defects (the primary capture stage is set for a relatively long exposure so that the digital noise in the image is low, but any motion present either from movement of the camera or from movement of objects in the scene results in motion blur. Simultaneously, the secondary capture stage is set for a relatively fast exposure so that digital noise in the image is higher, but the motion blur is less. See par. [0150]);
determining, based on the first likelihood, to apply a first exposure time to decrease the blur defect (the secondary capture stage is set for a relatively fast exposure so that digital noise in the image is higher, but the motion blur is less, see par. [0150]);
determining, based on the second likelihood, to apply a second exposure time, the second exposure time longer than the first exposure time, to decrease the high-noise defect (the primary capture stage is set for a relatively long exposure so that the digital noise in the image is low, see par. [0150]);
providing the first and second image captures to an image-merging module to create a single image from the first and second image captures (a modified image is then created by replacing portions of the primary image with portions of the secondary image, see par. [0150]).
But Border fails to clearly specify “determining a brightness value to capture multiple images of the scene;
determining, based on the first exposure time, a first adjustable gain value for a first image- capture device of the multiple image-capture devices to achieve the brightness value;
determining, based on the second exposure time, a second adjustable gain value for a second image-capture device of the multiple image-capture devices to achieve the brightness value;
causing the first image-capture device of the multiple image-capture devices to capture a first image of the scene using the first exposure time and the first adjustable gain value and the second image-capture device of the multiple image-capture devices to capture a second image of the scene using the second exposure time and the second adjustable gain value”.
However, Ramsay teaches “determining a brightness value to capture multiple images of the scene (For each photograph, shutter speed and digital gain act together to determine how much light is incident on the light-capturing digital sensor of a camera. This is referred to as exposure, which can be described quantitatively by an exposure value (EV), see Ramsay column 4 lines 17-21).
determining, based on the first exposure time, a first adjustable gain value for a first image- capture device of the multiple image-capture devices to achieve the brightness value (Different combinations of shutter speed and digital gain can produce a photograph with the same EV. A plot of all such combinations is known as a constant exposure curve, hence the gain can be adjusted to obtain the desired exposure value (first exposure time), see Ramsay column 4 lines 21-24);
determining, based on the second exposure time, a second adjustable gain value for a second image-capture device of the multiple image-capture devices to achieve the brightness value (Different combinations of shutter speed and digital gain can produce a photograph with the same EV. A plot of all such combinations is known as a constant exposure curve, hence the gain can be adjusted to obtain the desired exposure value (second exposure time), see Ramsay column 4 lines 21-24);
causing the first image-capture device of the multiple image-capture devices to capture a first image of the scene using the first exposure time and the first adjustable gain value (Border teaches that second image stage 2 with zoom lens 4 and image sensor 14, see Fig. 1 captures using a second exposure time, see Border par. [0150], and as combined with Ramsay teaches adjusting the gain to in order to achieve the desired second exposure time, see Ramsay column 4 lines 25-27) and the second image-capture device of the multiple image-capture devices to capture a second image of the scene using the second exposure time and the second adjustable gain value (Border teaches that image stage 1 with zoom lens 2 and image sensor 12, see Fig. 1 captures using a first exposure time, see Border par. [0150] and as combined with Ramsay teaches adjusting the gain to in order to achieve the desired second exposure time, see Ramsay column 4 lines 25-27))”.
References are analogous art because they are from the same field of endeavor and/or are reasonably pertinent to the particular problem with which the applicant was concerned because they relate to exposure control.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above system as taught by Border, by incorporating the teachings of Ramsay.
One of ordinary skill in the art would have been motivated to do this modification in order to produce a photograph with the same EV with different combinations of shutter speed and digital gain as suggested by Ramsay (see column 4 lines 21-23).
Regarding claim 2, Border in view of Ramsay teaches a method described in claim 1, wherein one or more additional image-capture devices are used to capture one or more additional image captures of the scene, and wherein providing the first and second image captures provides the additional image captures to the image-merging module (step 276 continuously capture video, see Border Fig. 5).
Regarding claim 5, Border in view of Ramsay teaches a method described in claim 1, wherein the first and second image captures are captured at a same brightness and wherein the brightness is defined by a sensor gain multiplied by an exposure time (the gain of the secondary image is increased so the average pixel values in the secondary image match those of the primary image, see Border par. [0150]).
Regarding claim 6, Border in view of Ramsay teaches a method described in claim 1, wherein the sensor data includes non-imaging data collected from a radar system usable to determine movement in the scene to be captured (camera GPS and electronic compass to provide camera pointing direction to determine scene movement, see Border par. [0079]).
Regarding claim 10, Border in view of Ramsay teaches a method described in claim 1, wherein the sensor data is imaging data collected by the image-capture device (can also use image data to form a range map to enable motion tracking, see Border par. [0077]).
Regarding claim 11, Border in view of Ramsay teaches a method described in claim 1, further comprising determining an object of focus based on the sensor data and further comprises using the image-merging module to create the single image of the scene by incorporating the first image capture for the object of focus and incorporating the second image capture for a remaining background portion of the scene (can also use image data to enable object extraction, see Border par. [0077], a modified image is then created by replacing portions of the primary image with portions of the secondary image, see Border pars. [0120, 0150]).
Regarding claim 12, Border in view of Ramsay teaches a method described in claim 11, wherein the second image capture is incorporated to create a motion scene in the background portion, the motion scene in the background portion being a blurred image-capture indicating motion within the scene (different augmentation of modifications are contemplated with the primary and secondary image, and includes video or a series of images, hence to convey motion, see Border pars. [0144-0146]).
Regarding claim 13, Border in view of Ramsay teaches a method described in claim 1, wherein the first and second image captures are multi-frame image captures, and the single image created by the image-merging module is a multi-frame image, the multi-frame image including multiple single-frame image captures captured in succession (different augmentation of modifications are contemplated with the primary and secondary image, and includes video or a series of images, hence multi-frame, see Border pars. [0144-0146]).
Regarding claim 14, Border in view of Ramsay teaches a method described in claim 1, further comprising displaying the single image created from the image-merging module (comprises a display 70 to display captured images, see Border Fig. 1 and par. [0127]).
Regarding independent claim(s) 15, claim(s) is/are drawn to the apparatus corresponding to the method of using same as claimed in claim(s) 1 and is/are rejected for the same reasons used above.
Regarding claim 16, Border in view of Ramsay teaches the method described in claim 1,
wherein the first sensor gain is greater than the second sensor gain (the gain of the secondary image is increased so the average pixel values in the secondary image match those of the primary image, see Border par. [0150]).
Regarding independent claim(s) 17, claim(s) is/are drawn to the non-transitory computer-readable storage medium used by the corresponding method in claim(s) 1 and is/are rejected for the same reasons used above.
Claim(s) 3-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Border in view of Ramsay as applied to claim 1 above, and further in view of US 2021/0374931 to Kumar et al. (hereinafter Kumar).
Regarding claim 3, Border in view of Ramsay discloses the claimed invention except for “wherein determining the likelihood of the exposure-related defects is determined, at least partially, through machine learning based on previous image captures”.
However, Kumar teaches “wherein determining the likelihood of the exposure-related defects is determined, at least partially, through machine learning based on previous image captures (systems, methods, and computer storage media for detecting and classifying an exposure defect in an image using neural networks, see abstract)”.
References are analogous art because they are from the same field of endeavor and/or are reasonably pertinent to the particular problem with which the applicant was concerned because they relate to exposure defects in digital cameras.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above method as taught by Border in view of Ramsay, by incorporating the teachings as taught by Kumar.
One of ordinary skill in the art would have been motivated to do this modification in order to efficiently and accurately detect and classify exposure detects in images using a neural network as suggested by Kumar (see par. [0003]).
Regarding claim 4, Border in view of Ramsay discloses the claimed invention except for “wherein determining the first or second exposure time is determined, at least partially, through machine learning based on previous image-captures captured using different exposure times”.
However, Kumar teaches “wherein determining the first or second exposure time is determined, at least partially, through machine learning based on previous image-captures captured using different exposure times (the neural network(s) may be trained and used to predict exposure levels (e.g., overexposure, underexposure, good exposure) for each digital image, see par. [0019])”.
References are analogous art because they are from the same field of endeavor and/or are reasonably pertinent to the particular problem with which the applicant was concerned because they relate to exposure determination in digital cameras.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above method as taught by Border in view of Ramsay, by incorporating the teachings as taught by Kumar.
One of ordinary skill in the art would have been motivated to do this modification in order to efficiently and accurately detect and classify exposure levels in images using a neural network as suggested by Kumar (see par. [0019]).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Border in view of Ramsay as applied to claim 1 above, and further in view of US 2022/0366584 to CHOI et al. (hereinafter CHOI).
Regarding claim 7, Border in view of Ramsay teaches a method described in claim 1 except for “wherein the sensor data includes non-imaging data collected from a flicker sensor usable to determine a banding defect in the scene to be captured”.
However, CHOI teaches “wherein the sensor data includes non-imaging data collected from a flicker sensor usable to determine a banding defect in the scene to be captured (electronic device comprising multiple cameras 221, 222, 223 and 224, and a flicker sensor 310 that includes non-imaging data, see pars. [0003, 0061, 0066] and Fig. 3)”.
References are analogous art because they are from the same field of endeavor and/or are reasonably pertinent to the particular problem with which the applicant was concerned because they relate to image capturing devices with multiple cameras.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above method as taught by Border in view of Ramsay, by incorporating the teachings as taught by CHOI.
One of ordinary skill in the art would have been motivated to do this modification in order to determine the best camera to capture based on the flicker detection as suggested by CHOI (see par. [0203]).
Claim(s) 8-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Border in view of Ramsay and CHOI as applied to claim 7 above, and further in view of IDS provided reference US 2021/0029290 to Okuike (hereinafter Okuike).
Regarding claim 8, Border in view of Ramsay and CHOI teaches a method described in claim 7 except for “wherein causing the second image-capture device to capture the second image at the second exposure time causes the second exposure time to be greater than a time associated with a frequency of flickering of light within the scene to be captured, the frequency collected by the flicker sensor”.
However, Okuike teaches an imaging device comprising two or more imaging devices 10 and 20 (see Fig. 2) “wherein causing the second image-capture device to capture the second image at the second exposure time causes the second exposure time to be greater than a time associated with a frequency of flickering of light within the scene to be captured, the frequency collected by the flicker sensor (causing the second image device 20 to capture at a second exposure at a time associated with the flicker frequency (see lighting from LED in Figs. 5-7) and imaging device 20 exposure in Figs. 5-7, see pars. [0132-0134])”.
References are analogous art because they are from the same field of endeavor and/or are reasonably pertinent to the particular problem with which the applicant was concerned because they relate to image capturing devices with multiple cameras.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above method as taught by Border in view of Ramsay and CHOI, by incorporating the teachings as taught by Okuike.
One of ordinary skill in the art would have been motivated to do this modification in order to obtain an appropriate captured image that allows recognition of an object even when a flicker phenomenon is occurring as suggested by Okuike (see par. [0010]).
Regarding claim 9, Border in view of Ramsay, CHOI and Okuike teaches a method described in claim 8, wherein the second exposure time is at least 8.33 milliseconds and the second image is a band-free image (the second image device 20 to capture at a second exposure at a time associated with the flicker frequency of 120 Hz, hence 1/120Hz = 8.33 milliseconds, see Okuike pars. [0132-0134])”.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANGEL L GARCES-RIVERA whose telephone number is (571)270-7268. The examiner can normally be reached Mon-Fri 9AM-5PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached at 571-727-7564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANGEL L GARCES-RIVERA/Examiner, Art Unit 2637
/Timothy J Henn/Primary Examiner, Art Unit 2639
Prosecution Insights