DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-18 are presented for examination.
Claim Objections
Claim 3 is objected to because of the following informalities:
Regarding Claim 3, the phrase “based on a determination that a foreign substance in on the front side of the camera lens” contains a grammatical error. Deletion of the bolded above word “in” would fix the grammatical error.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-10 and 12-18 are rejected under 35 U.S.C. 103 as being unpatentable over US PF PUB 2025/0085408 (hereinafter Han) in view of US20200103361 (hereinafter Wieczorek). Examiner notes that the pending claims do not enjoy the priority date of the application to which they claim priority as a continuation in part since the claims include limitations that were not completely supported by the application to which they claim priority.
Regarding Claim 1, Han teaches a camera system for a motor vehicle, comprising:
a camera lens (aperture window 1208, see FIGS. 12A – 12B);
an image sensor (image sensor 1204, [0118] clarifies that the term image sensor in the context of the specification also refers interchangeably to a video sensor ) configured to receive light from the camera lens (second return light signals 1205) and generate image data from the received light (image data 1216)
a light emitting device (LIDAR sensor 1202) configured to emit light onto a back side of the camera lens; and
a control device (1214) configured to
(a) cause the light emitting device (1202) to emit a pulse of light (transmission light signals 1201, [0146] teaches the use of a pulsed laser)
(b) receive image data from the image sensor for
(i) a first time period before the pulse of light is emitted,
(ii) a second time period while the pulse of light is being emitted and
(iii) a third period after the pulse of light has been emitted (image sensor 1204 can take the form of a video sensor as described in [0118] and a video sensor would generate image data both before and after the emission of emitted light)
(c) compare image data from the second time period with image data from the first and/or third time periods to determine an amount by which a brightness level increases while the pulse of light is being emitted ([0118] teaches the use of the LIDAR sensor emitter to illuminate the scene to provide sufficient light conditions for sensing the environment & [0174] describes how an at least partially blocked aperture window would result in a higher signal intensity compared to a light signal received during normal unblocked operation, see FIG. 14A illustrating an exemplary difference 1402 vs 1404)
(d) based on the amount by which the brightness level increases while the pulse of light is being emitted, determine whether a foreign substance is on a front side of the camera lens (the end of [0174] goes on to state that image data fused with LIDAR data can be used to identify blockages based on the intensity of the received light).
While Han does not specifically teach comparing previous and subsequent image data to image data collected during release of the pulse, FIG. 14A shows the collection of unblocked returns and blocked returns and the substantial differences between the two.
However, Wieczorek teaches identifying intensity levels likely to be caused by lens blockages by comparing the image data recorded during emission of the laser pulse with image data recorded outside emission windows ([0056] of Wieczorek describes measuring the intensity without the emitter being active to generate a reference value against which the signals are compared with during emission of the laser).
Han and Wieczorek both utilize intensity or brightness variations caused by light reflecting off a lens to identify obscuration on the front lens and describe removal of moisture using a heating element. A person having ordinary skill in the art at the time of filing would have found it obvious to modify the teachings of Han by utilizing updated reference values as taught by Wieczorek at [0056] to adapt to current conditions of the sensing device rather than relying solely upon factory or OEM established thresholds that could result in erroneous blockage identifications.
Regarding Claim 2, the combination of Han and Wieczorek teaches the camera system of claim 1, wherein the control device is configured to repeat processes (a)-(d) at a regular time interval ([0187] of Han describes repetition of the method 1500 shown in FIG. 15A over multiple cycles at a regular time interval corresponding to each frame of the data, see blocks 1509 and 1511).
Regarding Claim 3, the combination of Han and Wieczorek teaches the camera system of claim 1, further comprising:
a heating device configured to heat the camera lens ([0188] of Han describes the incorporation of a heater for removing condensation),
wherein the control device is further configured to cause the heating device to turn on, heating the camera lens, based on a determination that a foreign substance in on the front side of the camera lens ([0188] describes use of the heater for removing condensation in response to the controller determining a persistent partial window blockage of no less than one data collection cycle).
Regarding Claim 4, the combination of Han and Wieczorek teaches the camera system of claim 3, wherein the control device is further configured to
repeat processes (a)-(d) after the camera lens has been heated by the heating device (the process is repeated each frame cycle as describes in [0187] of Han), and
based on another determination that the foreign substance is on the front side of the camera lens, generate an alarm message ([0188] of Han describes informing the user if a partial window blockage cannot be removed after removal mechanisms such as the heater are applied).
Regarding Claim 5, the combination of Han and Wieczorek teaches the camera system of claim 4, wherein the alarm message is displayed to a driver of the vehicle indicating that the foreign substance is on the camera lens ([0188] describes informing the user if a partial window blockage cannot be removed after removal mechanisms such as the heater are applied).
Regarding Claim 6, the combination of Han and Wieczorek teaches the camera system of claim 1, wherein the light emitting device comprises a light emitting diode (LED) (FIG. 1 of Wieczorek shows display means (i.e. the light emitting device) as an LED).
Regarding Claim 7, the combination of Han and Wieczorek teaches the camera system of claim 1, wherein the increase in brightness level is caused by light from the light emitting device being reflected off of the camera lens (FIG. 14A of Han show blocked light causing large spikes in intensity 1404 and FIG. 14B of Han shows the spikes in intensity being caused by light reflecting off aperture window 1414 due to the presence of object 1418).
Regarding Claim 8, the combination of Han and Wieczorek teaches the camera system of claim 7, wherein the increase in brightness level is less when a foreign substance is on the front side of the camera lens than when no foreign substance is on the front side of the camera lens ([0118] of Han describes how illumination from the light source illuminates the scene leading to a brighter captured image however the failure of the light to illuminate the scene due to the foreign substance would lead to a reduction in light illuminating the scene).
Regarding Claim 9, the combination of Han and Wieczorek teaches the camera system of claim 2, wherein the image data from the image sensor is utilized to generate a camera image that is displayed to a driver of the vehicle ([0084] of Han describes displaying sensor data to the user), and wherein the image data generated during the second time period is not displayed as the camera image ([0174] describes how image data collected during emission of the light source is fused with LIDAR sensor data to help identify window blockages).
Regarding Claim 10, the combination of Han and Wieczorek teaches the camera system of claim 2, wherein the regular time interval is selected so as not to cause an interruption in the camera image that is perceptible by the driver of the vehicle ([0143] of Han describes how the operation of the imaging sensor and lidar sensor are synchronized to avoid errors).
Regarding Claim 12, the combination of Han and Wieczorek teaches the camera system of claim 1, wherein the control device is further configured to distinguish between two or more types of foreign substance based on the amount by which the brightness level increases while the pulse of light is being emitted ([0185] of Han describes identification of a type of object blocking the lens & [0056] of Wieczorek describes how the combination of Han and Wieczorek compare the brightness of the reflection relative to the brightness when the emitter is not active to make an initial identification of an object blocking the lens and therefore this increase in brightness is a basis for the ultimate identification of the foreign substance).
Regarding Claim 13, the combination of Han and Wieczorek teaches the camera system of claim 12, wherein the control device is further configured to cause a heating device to turn on, heating the camera lens, based on a determination that a water-based type of foreign substance is on the front side of the camera lens ([0188] of Han describes use of the heater for removing condensation in response to the controller determining a persistent partial window blockage of no less than one data collection cycle).
Regarding Claim 14, the combination of Han and Wieczorek teaches the camera system of claim 13, wherein the control device is further configured to generate an alarm message based on a determination that a non-water-based type of foreign substance is on the front side of the camera lens ([0188] describes issuing of an alert in response to detection of a blockage that is non-water based, e.g. plastic bags or leaves) .
Regarding Claim 15, the combination of Han and Wieczorek teaches the camera system of claim 13, wherein the control device is further configured to cause water to be sprayed on the camera lens based on a determination that a non-water-based type of foreign substance is on the front side of the camera lens ([0188] describes dispensing water to blow away non-water foreign substances).
Regarding Claims 16-18, they are rejected for the same reasons as claims 1, 3 and 4, respectively.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Han and Wieczorek in view of US Patent 10,585,194 (hereinafter Zhu).
Regarding Claim 11, the combination of Han and Wieczorek teaches the camera system of claim 10, but fails to teach the rest of the claim.
However, Zhu teaches wherein the image sensor is configured to capture image data at a frame rate that is greater than or equal to 60 frames per second (fps) and the regular time interval is greater than or equal to .5 seconds (Col 18, lines 32 – 35 teaches a specific use case in which the imaging sensor collects imagery for 0.5 second periods of time at 60 fps, Col 5 lines 18-25 mentions how the split between imaging sensor time to ToF time has various ratios).
Zhu and the combination of Han and Wieczorek both teach systems with multi-modal imaging and depth detection sensor configurations. Given that the combination of Han and Wieczorek is silent as to specific frame rates or duty cycles a person having ordinary skill in the art at the time of filing would have found it obvious to looked to the teachings of Zhu to help in selection of a camera frame rate and time sharing between depth sensing readings and imaging.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN WIGGER whose telephone number is (571)272-4208. The examiner can normally be reached 9:30am to 7:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Helal Algahaim can be reached at (571)270-5227. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BENJAMIN DAVID WIGGER/Examiner, Art Unit 3645
/HELAL A ALGAHAIM/SPE , Art Unit 3645
Prosecution Insights