Detailed Action
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Acknowledgments
2. Applicant’s arguments, filed on 04/14/2026, are acknowledged. Amended claims 1-2, 7-8, and 13-14 are acknowledged. Claims 1-20 remain pending and have been examined.
Response to Arguments
3. Applicant’s arguments have been fully considered but they are not persuasive with regards to independent claims 1, 7, and 13.
4. Regarding the amendment of dependent claim 2, previous rejection is withdrawn. However, upon further search and consideration, dependent claim 2 is rejected in view
of Chang (CN109040612A; see claim rejection below).
5. Applicant argues/remarks (pg. 1-2):
Gao and Chen, alone or in combination, do not teach or suggest the element of
detecting a preset target object in a first image shot by the camera before a current frame to determine a first target area in the first image, wherein the first target area is an area of the preset target object on which light compensation needs to be performed in the first image…”.
In addition, the Applicant states, Gao only calculates the overall environmental brightness of the entire frame and doesn’t detect any preset target object in the image, nor does Gao determine a target area based on such a detected object.
Further, Chen’s face detection serves as an on/off trigger for the fill light, not as a means to determine a target area used to calculate an exposure time period for light compensation.
6. The Examiner, respectfully, disagrees.
Gao (pg. 2, lines 1-7) teaches that the supplementary light operated by PWM signal output by the image signal processor includes obtaining exposure time of each image frame and the exposure time of each row of pixels of the image sensor and according to the exposure time of each image frame and each row of pixels, the high and low potentials of the PWM signal of the supplement lamp are obtained. Therefore, Gao teaches a particular timing with regards to particular row of pixels of an image frame wherein a supplement light is turned on.
Further, Chen does teach the determination of a target area because (pg. 1-2 lines 37-2) Chen teaches a preset condition in a particular mode to detect a face.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention that the teachings of Chen, to recognize a particular object in an image frame in a dark environment, can be implemented in the teaching of Gao’s thus to further enhance the timing of turning on/off of the supplemental light according to the obtaining of the exposure time of each image frame and the exposure time of each row of pixels, thereby optimizing on the usage of power supply.
Other arguments presented by the Applicant primarily depend on the addressed argument above.
Claim Rejections - 35 USC § 103
7. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. Claims 1, 3-7, and 9-20 are rejected under U.S.C. 103 being unpatentable over Gao (CN
111601046 A) in view of Chen (CN111447374A).
9. Regarding claim 1, claim 1 is rejected for reasons related to claim 7 (see claim 7 below).
10. Regarding claim 3, claim 3 is rejected for reasons related to claim 9 (see claim 9 below).
11. Regarding claim 4, claim 4 is rejected for reasons related to claim 10 (see claim 10
below).
12. Regarding claim 5, claim 5 is rejected for reasons related to claim 11 (see claim 11
below).
13. Regarding claim 6, claim 6 is rejected for reasons related to claim 12 (see claim 12
below).
14. Regarding claim 19, claim 19 is rejected for reasons related to claim 20 (see claim 20
below).
15. Regarding claim 7, an apparatus comprising
a camera comprising a photosensitive chip; at least one processor (…Gao on pg.
1, line 17-18 teaches a camera assembly including an image sensor and an image signal
processor…); and
at least one memory coupled to the at least one processor and storing programming instructions for execution by the at least one processor to cause the apparatus to perform operations (…pg. 5, line 7 teaches a memory which stores data and communicates with the signal processor…) comprising
determining a first exposure time period of a first target photosensitive chip row in the current frame based on an exposure start time of a first pixel row of an upper boundary of the first target area and an exposure end time of a second pixel row of a lower boundary of the first target area (…Gao, on page 2 line 3-7, teaches the obtaining the exposure time of each frame of image and the exposure time of each row of pixels of the image sensor. Accordingly, lamp supplementation (flash) is in accordance with the obtainment of turn-on and turn-off times, in accordance with exposure time of each frame and each row of pixels.
Gao, on Pg. 4-lines 14-24, teaches, based on a calculation of a histogram of a current frame, for current environment brightness BV, each line of pixels of each frame of image is exposed by rolling shutter and values of calculation are obtained; wherein first, the image signal processor calculates a starting point of the PWM wave high level; and secondly, the image signal processor controls the end point of the high level by calling the BV value; wherein the BV value is used to calculate the time that the LED needs to be turned on. Further, the BV value is a range [a, b] which is extracted from a boundary scene.
With reference to Fig. 2, a start exposure time and an end exposure time are depicted in correspondence wherein “LED” is on turned on…), wherein
the first target photosensitive chip row is in the photosensitive chip and generates image content in the first target area (…wherein Fig. 2 teaches a start exposure time and end exposure, with respect to an image sensor and its rows of pixels, an image sensor is taught by Gao…); and
indicating, based on the first exposure time period, an infrared light source to perform light compensation only in the first exposure time period when the photosensitive chip is exposed in the current frame (…pg. 1, lines 36-37 teach current environment brightness is compared with the preset brightness value preset in the image signal processor, and if the current environment brightness is low, a fill light is turned on; Gao specifies this to be the case for each frame obtained; thus each exposure period and fill light application is particular to each frame. Further, pg. 5, line 22 an infrared fill light is taught…).
Gao doesn’t further teach
detecting a preset target object in a first image shot by the came before a current frame to determine a first target area in the first image, wherein the first target area is an area of the preset target object on which light compensation needs to be performed in the first image (…however, Chen teaches a method and device for light supplement adjusting wherein pg. 3 lines 35-36 teach (S110) when a photographing device is in a night mode and a face that meets a fist preset condition is detected within a photographing range of the photographing device, a fill light is turned on. As such, the detected face maps to read on “a preset target object” which further is an area of the preset target object. The fill light is turned on subsequent to the detection of the face, therefore the fill light is turned on in a current frame wherein in a previous frame the face is detected.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention that the teachings of Chen, so to provide a fill light in night mode with respect to a preset condition specifying the determination of a face, further enhances the teachings of Gao wherein particular timings are determined for turning on supplemental lighting corresponding to row of exposure of an image sensor, thereby enhancing an object of interest in a dark environment of image capturing and also optimizing on the usage of power supply…).
16. Regarding claim 9, Gao in view of Chen teaches the apparatus according to claim 7 (see
claim 7 above), wherein the first target area is a face area in the first image (…though Gao
teaches an area, as such an environment of a driver’s surrounding being a target of observation, Gao does not teach a target area specifying a face being an object of a
target.
However, Chen teaches a method and device for light supplement adjusting
wherein pg. 3 lines 35-36 teach (S110) when a photographing device is in the night mode
and a face that meets the first preset condition is detected within the photographing
range of the photographing device, the fill light is turned on.
Therefore, it would have been obvious to one of ordinary skill in the art before the
effective filing date of the claimed invention that a face detection means to capture an
image of a driver can help to provide better observatory parameters thus to have a
plausible means to prevent situations that may cause harm to a driver and others of the
vicinity…).
17. Regarding claim 10, Gao in view of Chen teaches the apparatus according claim 7 (see
claim 7 above), wherein the photosensitive chip comprises
a plurality of photosensitive chip rows, and a plurality of pixel rows in the first image
correspond to the plurality of photosensitive chip rows (…pg. 1, line 18 teaches a camera
assembly including an image sensor and pg. 2 lines 3-4 teach rows of pixels of the image
sensor…), and
the determining a first exposure time period of a first target photosensitive chip row in
the current frame based on the first target area comprises:
determining the first exposure time period of the first target photosensitive chip row in
the current frame (…pg. 2, lines 3-4 teach the obtaining exposure time of each frame of
image and the exposure time of each row of pixels of the image sensor; further pg. 4 line
11 teaches that each line of pixels of each frame of image is exposed by rolling exposure
line by line, see Fig. 2 …);
determining the first target photosensitive chip row corresponding to one or more pixel
rows from the first pixel row of the upper boundary of the first target area to the second pixel row
of the lower boundary of the first target area in the first image (…Gao on pg. 4, lines 14-19
teaches a first point where the image signal processor calculates a starting point of the
PWM wave high level; and secondly, the image signal processor controls the end point
of the high level by calling the BV value; wherein the BV value is used to calculate the time that the LED needs to be turned on. Further, the BV value is a range [a, b] which is
extracted from a boundary scene. With reference to Fig. 2, a start exposure time and an
end exposure time are depicted in correspondence wherein “LED” is on turned on.
Chen further teaches, on pg. 6, lines 3-8, that the size of the face in the image can
be represented by pixel values and a first preset threshold is used to represent the size
of the face image.
Therefore, it would have been obvious to one of ordinary skill in the art before the
effective filing date of the claimed invention that a preset threshold can be used to
represent a face (target area) and thus process the pixels further for additional image
enhancement…).
18. Regarding claim 11, Gao in view of Chen teaches the apparatus according to claim 7
(see claim 7 above), wherein the operations further comprise:
determining a second exposure time period of a second target photosensitive chip row in
a subsequent frame based on the second target area, wherein the second target photosensitive
chip row is in the photosensitive chip and generates image content in the second target area;
and
indicating, based on the second exposure time period, the infrared light source to
perform light compensation in response to at least that the photosensitive chip is
exposed in the subsequent frame (…Gao, on pg.1, lines 30-34, teaches that the image signal processor calculates current environmental brightness according to each frame of image after exposure and if the current environment brightness is low, the fill light is turned on. The reference teaches frames of images. Therein Gao teaches a seemingly continuous process of
image capturing wherein current environmental brightness is calculated and accordingly
provide a fill light in low brightness environments. Hence, the process is repeated from
frame to frame, be it a second, third, or fourth…).
Gao doesn’t further teach wherein the operations further
comprise:
determining a second target area in a second image obtained in the current frame,
wherein the second target area is an area on which light compensation needs to be performed
in the second image (…however, Chen on pg. 6, lines 1-14 teaches the shooting device
can detect each frame of images in the video shot within a preset time period, and when
a human face is detected in one frame of image, the number of detections is increased. It
should be noted that, regardless of whether there are one or more faces in a frame of image, only the number of detections is increased by 1. If a face is detected in the 1#
frame image, the number of detections is +1, that is, the number of detections = 1, and 2
faces are detected in the 2# frame image, and then the number of detections is +1, that is,
the number of detections = 2. Thus, Chen teaches a second target area that may be
detected.
Therefore, it would have been obvious to one of ordinary skill in the art before the
effective filing date of the claimed invention that multiple target areas can be detected ,
as taught by Chen, and thus also be subjected to a fill light as taught by Gao…).
19. Regarding claim 12, Gao teaches the apparatus according to claim 7 (see claim 7
above), wherein the operations further comprise:
determining a third exposure time period of the first target photosensitive chip row in a
subsequent frame based on the first exposure time period; and
indicating, based on the third exposure time period, the infrared light source to perform
light compensation in response to at least that the photosensitive chip is exposed in the
subsequent frame (…Gao, on pg.1, lines 30-34, teaches that the image signal processor
calculates current environmental brightness according to each frame of image after
exposure and if the current environment brightness is low, the fill light is turned on. The
reference teaches frames of images. Therein Gao teaches a seemingly continuous
process of image capturing wherein current environmental brightness is calculated and
accordingly provide a fill light in low brightness environments…).
20. Regarding claim 13, an apparatus comprising:
a camera, the camera comprises: a photosensitive chip (…Gao on pg. 1, line 17-18
teaches a camera assembly including an image sensor and an image signal
processor…), and the operations comprise:
determining a first exposure time period of a first target photosensitive chip row in the
current frame based on an exposure start time of a first pixel row of an upper boundary of the
first target area and exposure end time of a second pixel row of a lower boundary of the first
target area (…Gao, on Pg. 4-lines 14-24, teaches, based on a calculation of a histogram of
a current frame, for current environment brightness BV, each line of pixels of each frame
of image is exposed by rolling shutter and values of calculation are obtained; wherein
first, the image signal processor calculates a starting point of the PWM wave high level;
and secondly, the image signal processor controls the end point of the high level by
calling the BV value; wherein the BV value is used to calculate the time that the LED
needs to be turned on. Further, the BV value is a range [a, b] which is extracted from a
boundary scene. In Fig. 2, a start exposure time and an end exposure time are depicted
in correspondence wherein “LED” is on turned on…); and
indicating, based on the first exposure time period, an infrared light source to perform
light compensation only in the first exposure time period when the photosensitive chip is
exposed in the current frame (…pg. 1, lines 36-37 teach current environment brightness is
compared with the preset brightness value preset in the image signal processor, and if
the current environment brightness is low, a fill light is turned on; Gao specifies this to
be the case for each frame obtained; thus each exposure period and fill light application
is particular to each frame. Further, pg. 5, line 22 teaches an infrared fill light…),
determining a first target area in a first image shot by the camera before a current frame,
wherein the first target area is an area on which light compensation needs to be performed in
the first image (…pg.1, lines 30-34 teach the image signal processor calculates current
environmental brightness according to each frame of image after exposure and if the
current environment brightness is low, the fill light is turned on.
Gao doesn’t further specify a particular object target.
However, Chen teaches a method and device for light supplement adjusting
wherein pg. 3 lines 35-36 teach S110 when a photographing device is in the night mode
and a face that meets a first preset condition is detected within the photographing range
of the photographing device, the fill light is turned on. Thus, a face that meets a first preset condition is viewed as a first target area.
Therefore, it would have been obvious to one of ordinary skill in the art before the
effective filing date of the claimed invention that a supplemental light relative to a
specific target area could have been determined according the teachings of Chen,
thereby identifying a particular target area that may be subjected for further image
processing…).
Further, Though Gao teaches a memory which stores data, Gao doesn’t further specify a non-transitory computer storage medium wherein the non-transitory computer storage medium
stores programming instructions for execution by at least one processor of an apparatus to
cause the apparatus to perform operations (…However, Chen teaches a storage device 104
may include one or more computer program products, and the computer program
products may include various forms of computer-readable storage media, such as non-
volatile memory.
Therefore, it would have been obvious to one of ordinary skill in the art before the
effective filing date of the claimed invention that a non-volatile storage memory as taught
by Chen could have been employed as a non-transitory storage medium thus to retain
executable computer programs for image signal processing, thus to produce in this
instance images that are at an acceptable brightness level…).
21. Regarding claim 15, Gao in view of Chen teaches the non-transitory computer storage
medium according to claim 13 (see claim 13 above), wherein the first target area is a face area
in the first image (…though Gao teaches an area, as such an environment of a driver’s
surrounding being a target of observation, Gao does not teach a target area specifying a
face being an object of target.
However, Chen teaches a method and device for light supplement adjusting
wherein pg. 3 lines 35-36 teaches (S110) when a photographing device is in the night
mode and a face that meets the first preset condition is detected within the
photographing range of the photographing device, the fill light is turned on.
Therefore, it would have been obvious to one skilled in the art before the effective
filing date of the claimed invention that a face detection means to capture an image of a
driver can help to provide better observatory parameters thus to have a plausible means
to prevent situations that may cause harm to a driver and others of the vicinity…).
22. Regarding claim 16, Gao in view of Chen teaches the non-transitory computer storage
medium according to claim 13 (see claim 13 above), wherein the photosensitive chip comprises:
a plurality of photosensitive chip rows, and a plurality of pixel rows in the first image
correspond to the plurality of photosensitive chip rows (…(Gao) pg. 1, line 18 teaches a
camera assembly including an image sensor and pg. 2 lines 3-4 teach rows of pixels of
the image sensor…), and
the determining a first exposure time period of a first target photosensitive chip row in
the current frame based on the first target area comprises:
determining the first target photosensitive chip row corresponding to one or more pixel rows from the first pixel row of the upper boundary of the first target area to the second pixel row
of the lower boundary of the first target area in the first image (…(Gao) pg. 2, lines 3-4 teach
the obtaining exposure time of each frame of image and the exposure time of each row of
pixels of the image sensor. Gao, on Pg. 4-lines 14-24, teaches, based on a calculation of
a histogram of a current frame, for current environment brightness BV, each line of
pixels of each frame of image is exposed by rolling shutter and values of calculation are
obtained; wherein first, the image signal processor calculates a starting point of the
PWM wave high level; and secondly, the image signal processor controls the end point
of the high level by calling the BV value; wherein the BV value is used to calculate the
time that the LED needs to be turned on. Further, the BV value is a range [a, b] which is
extracted from a boundary scene…); and
determining the first exposure time period of the first target photosensitive chip row in
the current frame (…(Gao) pg. 2, lines 3-4 teach the obtaining exposure time of each frame
of image and the exposure time of each row of pixels of the image sensor; further pg. 4
line 11 teaches that each line of pixels of each frame of image is exposed by rolling
exposure line by line, see Fig. 2…).
23. Regarding claim 17, Gao in view of Chen teaches the non-transitory computer storage
medium according to claim 13 (see claim 13 above), wherein the operations further comprise:
determining a second exposure time period of a second target photosensitive chip row in
a subsequent frame based on the second target area, wherein the second target photosensitive
chip row is in the photosensitive chip and generates image content in the second target area;
and indicating, based on the second exposure time period, the infrared light source to
perform light compensation in response to at least that the photosensitive chip is
exposed in the subsequent frame (…Gao on pg.1, lines 30-34, teaches that the image signal
processor calculates current environmental brightness according to each frame of image
after exposure and if the current environment brightness is low, the fill light is turned on.
The reference teaches frames of images. Therein Gao teaches a seemingly continuous
process of image capturing wherein current environmental brightness is calculated and
accordingly provide a fill light in low brightness environments. Hence, the process is
repeated from frame to frame, be it a second, third, or fourth frame…),
determining a second target area in a second image obtained in the current frame,
wherein the second target area is an area on which light compensation needs to be performed
in the second image (…Chen on pg. 6, lines 1-14 teaches the shooting device can detect
each frame of images in the video shot within a preset time period, and when a human
face is detected in one frame of image, the number of detections is increased. It should
be noted that, regardless of whether there are one or more faces in a frame of image,
only the number of detections is increased by 1. If a face is detected in the 1# frame image, the number of detections is +1, that is, the number of detections = 1, and 2 faces
are detected in the 2# frame image, and then the number of detections is +1, that is, the
number of detections = 2. Thus, Chen teaches a second target area that may be detected.
Therefore, it would have been obvious to one skilled in the art before the effective
filing date of the claimed invention that multiple target areas can be detected , as taught
by Chen, and thus also be subjected to a fill light as taught by Gao…).
24. Regarding claim 18, Gao in view of Chen teaches the non-transitory computer storage
medium according to claim 13, wherein the operations further comprise:
determining a third exposure time period of the first target photosensitive chip row in a
subsequent frame based on the first exposure time period; and indicating, based on the third
exposure time period, the infrared light source to perform light compensation in
response to at least that the photosensitive chip is exposed in the subsequent frame (…Gao, on pg.1, lines 30-34, teaches that the image signal processor calculates current
environmental brightness according to each frame of image after exposure and if the
current environment brightness is low, the fill light is turned on. The reference teaches
frames of images. Therein Gao teaches a seemingly continuous process of image
capturing wherein current environmental brightness is calculated and accordingly
provide a fill light in low brightness environments…).
25. Regarding claim 20, Gao in view of Chen teaches the apparatus according to claim 7,
wherein
the first exposure time period of the first target photosensitive chip row in the current
frame is determined as a time interval between the exposure start time of the first pixel row of
the upper boundary of the first target area and the exposure end time of the second pixel row of
the lower boundary of the first target area (…wherein the “on” period, parallel to a horizontal
time line in Fig. 2, can be viewed as a time interval, wherein that period specifically
corresponds to the first and last row of the sensor and its pixels. As addressed in claim
7, Gao, on page 2 line 3-7, teaches the obtaining the exposure time of each frame of
image and the exposure time of each row of pixels of the image sensor. Accordingly,
lamp supplementation (flash) is in accordance with the obtainment of turn-on and turn-
off times, in accordance with exposure time of each frame and each row of pixels…).
26. Claims 2, 8, and 14 are rejected under U.S.C. 103 being unpatentable over Gao (CN111601046A) in view of Chen (CN111447374A) and further view of Chang (CN109040612A).
27. Regarding claims 2 and 14, claims 2 and 14 are rejected for reasons related to claim 8(see claim 8 below).
28. Regarding claim 8, Gao in view of Chen teaches the apparatus according to claim 7. However, the combined reference doesn’t teach wherein detecting the preset target object comprises:
detecting the preset target object using a deep learning algorithm (…however, Chang teaches (pages 3-4, lines 31-7) an image processing method/device/apparatus including target object identification for target object enhancement wherein a target object in a photo/video for processing is identified to determine a location of the pixel of the target object; wherein the target object may be identified by a recognition algorithm, as such a machine learning algorithm or a deep learning algorithm.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention that a deep learning algorithm for object identification, as taught by Chang, could have been implemented in Chen’s teaching of light supplementing method/device, as part of detecting a target face, so to further determine a precise location of pixel location of the target object in an image frame.
Conclusion
29. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SURAFEL YILMAKASSAYE whose telephone number is (703)756-1910. The examiner can normally be reached Monday-Friday 8:30am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, TWYLER HASKINS can be reached at (571)272-7406. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SURAFEL YILMAKASSAYE/Examiner, Art Unit 2639
/TWYLER L HASKINS/Supervisory Patent Examiner, Art Unit 2639
Prosecution Insights