DETAILED ACTION
I. Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
II. Response to Amendment
A. The response, filed April 9, 2026, has been entered and made of record. Claims 1-8,12,14-17, and 19-21 are pending in the application.
B. Correction. In the previous Office action, the examiner accidentally noted that the allowability of claim 19 was withdrawn in the heading of section IV. However, its allowability remains.
III. Response to Arguments
Applicant’s arguments regarding independent claims 1 and 12 and Nayar et al. have been considered but are not persuasive. Applicant argues that the claims differ from Nayar et al. because the reference divides out exposure differences between the pixels under mask portions with differing transparency in its normalization process. Consequently, according to Applicant, Nayar et al. would not be interpolating between two first and second luminance values since the averaging performed during interpolation would be of pixel signals with identical values that differ only due to noise. The examiner disagrees with this position but thanks Applicant for the consideration given to Nayar et al. and the presentation of new arguments regarding the reference. Applicant’s contention that the averaging in Nayar’s interpolation is of identical pixel values differing only due to noise is only true if the scene presents the same brightness at each position. It is also might be true that Applicant’s first and second luminance values are identical, for example, if the scene is a checkerboard with alternating patterns of portions irradiating a first light intensity and portions irradiating a second light intensity half of the first that transmit through the filter of Applicant’s Fig. 3A. Realistically, the scenes captured by Nayar’s camera and Applicant’s system have more varying detail, which is illustrated in Nayar’s Fig. 10 at the boundary. Even if exposure differences are divided out, signals from pixels behind mask cells with differing transparency will still present different luminance such that an average of them would be an interpolation between them. However, the difference between Applicant’s system and Nayar et al. is not lost on the examiner. To better distinguish the claims, the examiner suggests incorporating a limitation of the sort, “wherein the first luminance and second luminance are different even if light of equal intensity transmits to the first plurality of pixels and the second plurality of pixels.”
Applicant further takes issue with the examiner’s position that interpolation between two luminance values may be performed even if one of the values is excluded from an averaging (interpolation) process. First, the Patent Trial and Appeal Board affirmed this reading of interpolation in the decision dated September 25, 2025. Second, Applicant expressly discloses excluding saturated pixels from interpolation in Fig. 8 and still interpolating between values associated with them. Deleting the phrase “according to a light condition” does not help Applicant’s argument. The issue is whether exclusion itself removes the possibility of an interpolation, regardless of the reason for exclusion. In Nayar et al., hypothetically, a pixel whose signal value is immediately below the saturation threshold and is excluded from interpolation would be interpolated between in the averaging process.
IV. Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3,5,8,10-13,15,17,20, and 21 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nayar et al. (US # 6,864,916).
As to claim 1, Nayar et al. teaches a method comprising:
exposing a first plurality of pixels of an image sensor (Fig. 4B, pixel corresponding to brightest cell “102”) a second plurality of pixels of the image sensor (Fig. 4B, pixel corresponding to darkest cell “105”), and a third plurality of pixels of the image sensor (Fig. 4B, pixel corresponding to either bright cell “103” or dark cell “104”) to light substantially synchronously (col. 7, lines 39-41), wherein the first plurality of pixels are interspersed with the second plurality of pixels and the third plurality of pixels in a repeating array (Fig. 4A; Fig. 5B);
controlling the exposing such that the first plurality of pixels has a first exposure and the second plurality of pixels has a second exposure and the third plurality of pixels has a third exposure, and wherein the first exposure differs from the second exposure and the third exposure (col. 8, lines 60-64);
determining both a first luminance from the first exposure and a second luminance from the second exposure (Fig. 9; {The Examiner interprets the first luminance and the second luminance as the pixel values corresponding to the brightest mask cell and the darkest mask cell, respectively.}); and
determining a composite luminance (Fig. 9, interpolated pixel “129”) from the first luminance and the second luminance (The Examiner submits that the interpolated pixel “129” is determined from the pixel signal of a brightest cell and the pixel signal of a darkest cell even if they are not used in Nayar’s interpolation process. That is, in the case of saturation, the pixel value of the darkest mask cell will be used in interpolation but the pixel value of the brightest mask cell will not. However, even if not used in interpolation, its value is used to indicate saturation and that it should not be used in the interpolation process. Conversely, in the case of blackening, the pixel value of the brightest mask cell will be used in interpolation, and the pixel value of the darkest mask cell will be used to indicate blackening and that it should not be used in interpolation.}), wherein the composite luminance is determined by interpolating between the first luminance and the second luminance (col. 11, lines 2-9).
As to claim 2, Nayar et al. teaches the method of claim 1, wherein the interpolating comprises one or more of a bicubic, a simple average (col. 10, lines 59-65), a bicubic spline, a cubic nearest-neighbor or a bilinear.
As to claim 3, the Examiner interprets the term “short” in claim 10 as defined in para. [0064] on pp. 16 and 17 of the specification.
Further as to claim 3, Nayar et al. teaches the method of claim 1. Although it is not stated expressly in Nayar et al., the Examiner takes official notice to the idea of capturing video frames at 1/60th of a second as well known and expected in the art. One of ordinary skill in the art would have been obvious to quicken the frame rate of Nayar’s video camera to this well-known rate because this modification would be particularly advantageous in a camera that performs complicated post-capture processing, like the high dynamic range image calculation of Nayar et al., where reproduction of the image can be performed without noticeable delay after initiation of image capture.
As Applicant has failed to adequately traverse the official notice statement above, the subject matter of that statement is now considered admitted prior art. See MPEP 2144.03(c).
As to claim 5, Nayar et al. teaches the method of claim 1, wherein controlling the exposing such that the first plurality of pixels has the first exposure and the second plurality of pixels has the second exposure and the first exposure differs from the second exposure is performed with an optical device that is configured to transmit a relatively different amount of light to the first plurality of pixels as compared to the second plurality of pixels (col. 8, lines 60-64).
As to claim 8, Nayar et al. teaches the method of claim 5, wherein the first plurality of pixels are disposed immediately adjacent the second plurality of pixels in a sensing area so that the first plurality of pixels and second plurality of pixels would be subject to substantially a same amount of light if the optical device were not present (Fig. 4B).
As to claim 12, Nayar et al. teaches a system (Fig. 2, video camera “200”) comprising:
a first plurality of pixels (Fig. 4B, pixels corresponding to brightest cells “102”)
a second plurality of pixels (Fig. 4B, pixels corresponding to darkest cells “105”), and wherein the first plurality of pixels are interspersed with one another in a non-random pattern in a sensing area (Fig. 5B);
an optical device (Fig. 2, mask “101”; col. 9, lines 8 and 9, “…optical filter…”) configured to control an amount of exposure of the first pixel and the second pixel such that the first pixel has a first exposure and the second pixel has a second exposure (col. 8, lines 60-64), wherein the first exposure and the second exposure are taken simultaneously synchronously (col. 7, lines 39-41; note, also, the discussion in col. 1, lines 20-54), and wherein the first exposure differs from the second exposure (col. 8, lines 60-64);
a third pixel (Fig. 4B, pixel corresponding to either bright cell “103” or dark cell “104”) and the optical device is configured to control an amount of exposure of the third pixel such that the third pixel has a third exposure taken simultaneously and a for a substantially same time duration as the first exposure and the second exposure (col. 7, lines 39-41; note, also, the discussion in col. 1, lines 20-54), and wherein the first exposure, the second exposure, and the third exposure all differ from one another (col. 8, lines 60-64); and
a non-transitory processor-readable storage medium (Fig. 2, ROM “409”) storing processor executable instructions that, when executed by one or more processors (Fig. 2, processing unit “411”), cause the one or more processors to perform operations comprising:
determine both a first luminance from the first exposure and a second luminance from the second exposure (Fig. 9; {The Examiner interprets the first luminance and the second luminance as the pixel values corresponding to the brightest mask cell and the darkest mask cell, respectively.}); and
determine a composite luminance (Fig. 9, interpolated pixel “129”) from the first luminance and the second luminance (The Examiner submits that the interpolated pixel “129” is determined from the first and second luminance even if they are not used in Nayar’s interpolation process. That is, in the case of saturation, the pixel value of the darkest mask cell will be used in interpolation but the pixel value of the brightest mask cell will not. However, even if not used in interpolation, its value is used to indicate saturation and that it should not be used in the interpolation process. Conversely, in the case of blackening, the pixel value of the brightest mask cell will be used in interpolation, and the pixel value of the darkest mask cell will be used to indicate blackening and that it should not be used in interpolation.}), wherein the composite luminance is determined by interpolating between the first luminance and the second luminance (col. 11, lines 2-9) and according to a light condition (The Examiner interprets the light condition as blackening/saturation.).
As to claim 15, Nayar et al. teaches the system of claim 12, wherein the optical device includes an optical filter that is configured to transmit a relatively different amount of light to the first plurality of pixels as compared to the second plurality of pixels (col. 8, lines 60-64).
As to claim 17, Nayar et al. teaches the system of claim 12, wherein the first plurality of pixels includes at least a first pixel disposed immediately adjacent at least a second pixel of the plurality of pixels in the sensing area so that the first pixel and second pixel would be subject to substantially a same amount of light if the optical device were not present (Fig. 4B).
As to claim 20, Nayar et al. teaches the system of claim 12, wherein the first plurality of pixels includes at least a first pixel disposed immediately adjacent at least a second pixel of the plurality of pixels in the sensing area (Fig. 4B).
As to claim 21, Nayar et al. teaches the method of claim 1, wherein all pixels of the repeating array comprise either the first plurality of pixels, the second plurality of pixels or the third plurality of pixels (col. 8, line 64 – col. 9, line 2).
V. Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
A. Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Nayar et al. (US # 6,864,916) in view of Georgiev et al. (US # 8,345,144) and further in view of Ciurea et al. (US 2018/0013945)
As to claim 4, Nayar et al. teaches the method of claim 1. The claim differs from Nayar et al. in that it requires that the first exposure comprises substantially a full exposure and the second exposure comprises substantially fifty percent exposure.
In the same field of endeavor, Georgiev et al. discloses an optical mask for an image sensor that filters incoming light to transmit a plurality of different amounts of light to respective pixels of the image sensor. The mask includes a plurality of elements that pass all incoming light to the image sensor (Fig. 37A, 100% mask elements; col. 36, lines 56-60). Further in the same field of endeavor, Ciurea et al. teaches that it is known to capture multiple images with different exposure settings to produce a high dynamic range image, where a first exposure setting can be set to half the value of another exposure setting ([0007]). In light of the teaching of Georgiev et al. and Ciurea et al., it would have been obvious to one of ordinary skill in the art to design Nayar’s brightest mask cell to transmit all incoming light and Nayar’s darkest mask cell to transmit half of the incoming light because an artisan of ordinary skill in the art would recognize that this mask design would be particularly advantageous when producing high dynamic range images of low-light scenes.
As to claim 14, Nayar et al. teaches the system of claim 12. The claim differs from Nayar et al. in that it requires that the first exposure comprises substantially a full exposure and the second exposure comprises substantially fifty percent exposure.
In the same field of endeavor, Georgiev et al. discloses an optical mask for an image sensor that filters incoming light to transmit a plurality of different amounts of light to respective pixels of the image sensor. The mask includes a plurality of elements that pass all incoming light to the image sensor (Fig. 37A, 100% mask elements; col. 36, lines 56-60). Further in the same field of endeavor, Ciurea et al. teaches that it is known to capture multiple images with different exposure settings to produce a high dynamic range image, where a first exposure setting can be set to half the value of another exposure setting ([0007]). In light of the teaching of Georgiev et al. and Ciurea et al., it would have been obvious to one of ordinary skill in the art to design Nayar’s brightest mask cell to transmit all incoming light, Nayar’s darkest mask cell to transmit half of the incoming light, and Nayar’s intermediate cells to transmit amounts of light between the amounts transmitted by the darkest and brightest cells because an artisan of ordinary skill in the art would recognize that this mask design would be particularly advantageous when producing high dynamic range images of low-light scenes.
B. Claims 6,7, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Nayar et al. (US # 6,864,916) in view of Fukuhara (US 2015/0256734)
As to claim 6, Nayar et al. teaches the method of claim 5. The claim differs from Nayar et al. in that it requires that the optical device comprises a lenslet array that is configured to transmit a relatively different amount of light to the first plurality of pixels as compared to the second plurality of pixels.
In the same field of endeavor, Fukuhara discloses an image sensor (Fig. 3, imaging element “301”) with pixels that receive either high-intensity light or low-intensity light (Fig. 3, H and L pixels). A microlens array (Fig. 3, lens array “302”) is positioned before the image sensor and includes lenses that are designed to transmit either low-intensity or high intensity light to the image sensor ([0078]). In light of the teaching of Fukuhara, it would have been obvious to one of ordinary skill in the art to use Fukuhara’s microlens array in lieu of Nayar’s mask to transmit the different amounts of light to the CCD because an artisan of ordinary skill in the art would recognize that Fukuhara’s microlens array would allow the resultant imaging system to capture light field images, thereby allowing for post-capture focusing.
As to claim 7, Nayar et al., as modified by Fukuhara, teaches the method of claim 6, wherein the lenslet array has a repeating pattern corresponding to the repeating array (See relevant cited portions of Nayar et al. and Fukuhara; {Nayar’s mask, which is replaced with Fukuhara’s microlens array, is repeating.}).
As to claim 16, Nayar et al. teaches the system of claim 12. The claim differs from Nayar et al. in that it requires that the optical device comprises a lenslet array that is configured to transmit a relatively different amount of light to the first plurality of pixels as compared to the second plurality of pixels.
In the same field of endeavor, Fukuhara discloses an image sensor (Fig. 3, imaging element “301”) with pixels that receive either high-intensity light or low-intensity light (Fig. 3, H and L pixels). A microlens array (Fig. 3, lens array “302”) is positioned before the image sensor and includes lenses that are designed to transmit either low-intensity or high intensity light to the image sensor ([0078]). In light of the teaching of Fukuhara, it would have been obvious to one of ordinary skill in the art to use Fukuhara’s microlens array in lieu of Nayar’s mask to transmit the different amounts of light to the CCD because an artisan of ordinary skill in the art would recognize that Fukuhara’s microlens array would allow the resultant imaging system to capture light field images, thereby allowing for post-capture focusing.
VI. Allowable Subject Matter
Claim 19 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. See the Office action dated 10/6/2022 of parent application, 16/802,205 for their reasons for allowability.
VII. Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J DANIELS whose telephone number is (571)272-7362. The examiner can normally be reached M-F 9:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached on 571-272-7564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
6/8/2026
/ANTHONY J DANIELS/Primary Examiner, Art Unit 2637
Prosecution Insights