DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on May 10, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: SYSTEM FOR DETECTING A WATER DROPLET ON A VEHICLE SENSOR
The abstract of the disclosure is objected to because of the following informality:
In line 2, “generate” should read “generated”
The disclosure is objected to because of the following informality:
In paragraph [0029], illumination device is labeled “110”, which is inconsistent with the prior references to illumination device and the corresponding object found in FIG. 1. Illumination device is referred to by the number “410” in paragraphs [0026] and [0027].
Appropriate correction is required.
Claim Objections
Claim 1 and 2 are objected to because of the following informalities:
Claim 1, line 5, “determine presence or absence” should read “determine the presence or absence”.
Claim 2, line 3, “determine adhesion of a water droplet” should read “determine the adhesion of a water droplet”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6 and 7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6 recites “wherein the arithmetic processing device determines that a water droplet has adhered when a frequency of a predetermined section on a low-brightness side of a histogram of a frame decreases from a frequency of the same section of a histogram of a previous frame”. The term “low-brightness” in claim 6 is a relative term which renders the claim indefinite. The term “low-brightness” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. “Low-brightness” is a subjective term without a defined standard of measurement and method of determination; thus, it is unclear whether “low-brightness” is asserted based on a threshold value related to pixel hue, whether “low-brightness” may potentially be impacted by exposure to sunlight and/or the darkness of night, or any other possible defining factors of “low-brightness”, when used in the context of claim 6. Accordingly, the scope of the term “low-brightness” is unclear and indefinite.
Claim 7 recites “the arithmetic processing device determines that a water droplet has adhered when a median value of a predetermined section of a histogram of a frame increases more than a median value of a predetermined section of a histogram of a previous frame”. A median value derived from a predetermined section of a histogram represents a computed statistical value (i.e. a single number) based on the underlying data. The claim does not explain how such a median value can “increase more” than a second value, nor does it specify the reference condition from which the increase is measured. It is unclear whether the limitation is claiming the device determines that a water droplet has adhered when a median value of a predetermined section of a histogram of a frame is a higher value than a previous frame’s median value of a histogram, whether the limitation is claiming the difference in median value from the previous to the new predetermined section must be higher than the previous median value, whether the limitation is claiming both the current and previous medians are somehow increasing and the current frame’s median must increase in value more than the previous frame’s value increases, whether a threshold of sorts is being used, or any other alternative systematic framework is being implemented. Accordingly, the scope of the comparison is unclear and indefinite.
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 7 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the enablement requirement and the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or use the invention. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
The disclosure language relating to claim 7 (see ¶ [0021]) equally mirrors the limitation recited in claim 7 and is rejected based on the same indefinite principles detailed in the 35 U.S.C. 112(b) rejection found above. Furthermore, the specification states “Alternatively, the arithmetic processing device 440 may detect the adhesion of a water droplet by comparing the median values in the monitoring sections” (¶ [0042]; ¶ [0043]), and states “…the water droplet can be detected based on the change in the median value of the histogram.” (¶ [0021]) without explicitly describing how the “increase more than” comparison method works. Accordingly, the specification does not reasonably convey possession of, nor enable, the full scope of the claimed system of determining that a water droplet has adhered based on median value of a predetermined section of histogram of a frame in comparison to a median value of a predetermined section of histogram of a previous frame.
For examination purposes, the comparison logic “increases more than” when comparing median values of a predetermined section of a histogram of a frame to a previous frame’s median value has been construed to mean a first median value is “a higher number value than” a second mean value, as in:
“the arithmetic processing device determines that a water droplet has adhered when a median value of a predetermined section of a histogram of a frame is a higher number value than a median value of a predetermined section of a histogram of a previous frame.”
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-4 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The limitations, under their broadest reasonable interpretation, cover mental process (concept performed in a human mind, including as observation, evaluation, judgment, opinion, organizing human activity and mathematical concepts and calculations). The claims recite determining the presence or absence of a water droplet based on a change in the same determination region of two consecutive frames of the image without significantly more. This judicial exception is not integrated into a practical application because the steps do not add meaningful limitations to be considered specifically applied to a particular technological problem to be solved. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the steps of the claimed invention can be done mentally and no additional features in the claims would preclude them from being performed as such except for the generic computer elements at high level of generality (e.g. processor, memory, operating system, etc.).
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that independent claim 1 is directed to a mental process (i.e. abstract idea) as shown below:
► STEP 1: Do the claims fall within one of the statutory categories?
YES, claim 1 is directed to a sensing system for determining the presence or absence of a water droplet.
► STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea?
Yes, the claim is directed toward a mental process and/or mathematical concepts (i.e. abstract idea).
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject
matter that are considered abstract ideas:
Mathematical concepts - mathematical relationships, mathematical formulas or
equations, mathematical calculations;
Certain methods of organizing human activity - fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes - concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
Independent claim 1 comprises a mental process that can be practicably performed in the human mind (or generic computers or components configured to perform the method) and, therefore, an abstract idea.
Regarding independent claim 1: the sensing system recites the steps of:
These limitations, as drafted, is a simple process that, under their broadest reasonable interpretation, covers performance of the limitations in the mind or by a human. The Examiner notes that under MPEP 2106.04(a)(2)(III), the courts consider a mental process (thinking) that "can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the 'basic tools of scientific and technological work' that are open to all."' 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ('"[M]ental processes[] and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work'" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584,589, 198 USPQ 193, 197 (1978) (same).
As such, a person could determine presence or absence of a water droplet based on a change in the same determination region of two consecutive frames of the image through observation, data collecting, and evaluation, either mentally or using a pen and paper. The mere nominal recitation that the various steps are being executed by a device (e.g. arithmetic processing device) does not take the limitations out of the mental process grouping. Thus, the claim recites a mental process.
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
NO, the claims do not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words "apply it" (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception;
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Independent claim 1 does not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application.
Regarding claim 1, the claim recites additional limitations:
“an arithmetic processing device configured to process an image generated by the image sensor” which amounts to data collecting and linking devices for evaluation, and fails to integrate the claim into a practical application. Arithmetic processing devices and image sensors are generic computers or components.
“wherein the arithmetic processing device is configured to determine presence or absence of a water droplet based on a change in the same determination region of two consecutive frames of the image” which amounts to observation, data collecting, and evaluation that can be done in the human mind. An arithmetic processing device is a generic computer or component.
The additional limitations found in claim 1 are generic computer components and/or insignificant pre/post-solution extra activity that do not add a meaningful limitation to the abstract idea because they amount to simply implementing the abstract idea in the apparatus claim. See MPEP 2106.05(g).
These limitations are recited at a high level of generality (i.e. as a general action or change being taken based on the results of the acquiring step) and amounts to mere post solution actions, which is a form of insignificant extra-solution activity. Further, the claims are claimed generically and are operating in their ordinary capacity such that they do not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
NO, the claims do not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
Independent claim 1 recites additional “image sensor” (line 2), but does not recite any additional elements that are not well understood, routine or conventional. The use of generic computer elements is a routine, well understood and conventional process that is performed by computers.
The additional limitations in claims 2-4 do not integrate the mental process into a practical application or add significantly more to the mental process. Claims 3 and 4 include the generic computer component of an “arithmetic processing device” described in claim 1, but fails to provide significant extra-solution activity.
Dependent claim 2 adds the additional mental processes of setting a determination region at a predetermined position. This is a mental process based on observation and evaluation, could be done either mentally or with a pen and paper, adds no structural components, and fails to remedy the abstract idea of claim 1.
Dependent claim 3 adds additional limitation wherein the arithmetic processing device divides each of the two consecutive frames into a plurality of determination regions and determines adhesion of a water droplet for each determination region. This amounts to generic computer processing for observation and evaluation purposes, could be done either mentally or with a pen and paper, and fails to remedy the abstract idea of claim 1.
Dependent claim 4 adds the limitation wherein the arithmetic processing device detects an object included in the image and to set the determination region so that the object which is detected is included. This amounts to generic computer processing for observation, data collecting, and evaluation purposes, could be done either mentally or with a pen and paper, and fails to remedy the abstract idea of claim 1.
Thus, since claim(s) 1-4 are: (a) directed toward an abstract idea, (b) do not recite additional elements that integrate the judicial exception into a practical application, and (c) do not recite additional elements that amount to significantly more than the judicial exception, it is clear that Claim(s) 1-4 are not eligible subject matter under 35 U.S.C 101.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Ikeda et al. (US 20180114089 A1).
Regarding Claim 1,
Ikeda et al. teaches: A sensing system comprising: an image sensor (Attachable matter removal system depicted in Fig. 3 includes a camera 2 that “has an image sensor … to photograph surroundings of the vehicle C using such an image sensor” (¶ [0060]). Ikeda et al. further teaches the system comprises an attachable matter detection apparatus 10 that “acquires a determination target area … from a photographic image. … (and) creates histograms of at least an edge intensity, luminance, and saturation for the determination target area acquired....” (Abstract). Accordingly, Ikeda et al.’s system is interpreted as equivalent to a sensing system because it includes a system with an image sensor that acquires images and uses corresponding images to detects areas.);
and an arithmetic processing device configured to process an image generated by the image sensor (Ikeda et al. teaches an attachable matter detection apparatus configured to process an image from the image sensor described above which requires performing arithmetic operations (e.g. comparison, determination, thresholding) (see Abstract “An attachable matter detection apparatus according to an embodiment includes an acquirement unit, a creation unit, and a determination unit….acquirement unit acquires a determination target area of an attachable matter from a photographic image…creation unit creates histograms of at least an edge intensity, luminance, and saturation for the determination target area…determination unit determines whether or not the attachable matter exists in the determination target area on the basis of a ratio of frequency of each grade in each of the histograms”; [0324] “detection threshold value is used by the detection unit 25 to compare with the similarity value input from the matching unit”; ¶ [0129] and FIG 7 illustrate processing sequences performed by the attachable matter detect apparatus.). Such operations performed by the attachable matter detection apparatus necessarily require a processing unit or equivalent circuitry. Therefore, Ikeda et al. teaches the attachable matter detection apparatus inherently includes an arithmetic processing device. For examination purposes, the attachable matter detection apparatus taught by Ikeda et al. will be referenced to as the arithmetic processing device throughout the remainder of the office action, found below.);
wherein the arithmetic processing device is configured to determine presence or absence of a water droplet based on a change in the same determination region of two consecutive frames of the image (see Fig 1D and ¶ [0075] “the attachable matter detection apparatus 10 sets a plurality of partitioned areas for a single frame of the camera image of the camera 2 and acquires each of the partitioned areas”; ¶ [0077] “attachable matter detection apparatus 10 determines whether or not the attachable matter exists in each partitioned area on the basis of a “change amount” between the current frame and the previous frame”; Ikeda et al. teaches “raindrops” as an example of “attachable matter…attached to a lens” (¶ [0004]). Examiner notes, “raindrops” are equivalent to the context of “water droplets” described in the “Summary of Invention” section of the instant application. Furthermore, “partitioned areas”, taught by Ikeda et al., are interpreted as equivalent to the “determination region” of the instant application because “region” is synonymous with “area” and the “partitioned area” is where Ikeda et al. teaches determinations are made (see [¶0075] “attachable matter detection apparatus 10 determines whether or not the attachable matter exists in each partitioned area”). Thus, Ikeda et al. teaches a sensing system equivalent to claim 1.).
Regarding Claim 2,
Ikeda et al. teaches the limitations of claim 1.
Ikeda et al. further teaches: wherein the determination region is set at a predetermined position (see ¶ [0075] “FIG. 1D, the attachable matter detection apparatus 10 sets a plurality of partitioned areas for a single frame of the camera image of the camera 2 and acquires each of the partitioned areas”; See further examples of diagrams illustrating settings of partitioned areas in FIGS. 8A to 8D (¶ [0018). Examiner notes, the “partitioned area” taught by Ikeda et al. is equivalent to a “determination region” of the instant application, as explained above in claim 1.).
Regarding Claim 3,
Ikeda et al. teaches the limitations of claim 1.
Ikeda et al. further teaches: wherein the arithmetic processing device is configured to divide each of the two consecutive frames into a plurality of determination regions and to determine adhesion of a water droplet for each determination region (Ikeda et al. teaches the attachable matter detection apparatus (i.e. arithmetic processing device, as detailed in claim 1) partitions areas into a plurality of partition areas (i.e. divides each frame “into a plurality of determination regions”), partitioned areas and other information is passed between the previous frame and the current frame (i.e. “two consecutive frames”), and the attachable matter detection apparatus determines whether attachable matter (i.e. “adhesion of a water droplet”) exists in each partitioned area (i.e. “determination region”) (see ¶¶ [0075]- [0078] “attachable matter detection apparatus 10 sets a plurality of partitioned areas for a single frame of the camera image of the camera 2 and acquires each of the partitioned areas (Step S1′).
The attachable matter detection apparatus 10 creates each histogram of the edge intensity, the luminance, and the saturation of each acquired partitioned area (Step S2′). Information including such histograms is stored for the previous frame one or more versions ago as well as the current frame.
The attachable matter detection apparatus 10 determines whether or not the attachable matter exists in each partitioned area on the basis of a “change amount” between the current frame and the previous frame…
By performing determination on the basis of the “change amount” between frames in this manner… it is possible to easily detect the raindrop as the attachable matter…”).).
Regarding Claim 4,
Ikeda et al. teaches the limitations of claim 1, including an arithmetic processing device (see claim 1).
Ikeda et al. further teaches: wherein the arithmetic processing device is configured to detect an object included in the image and to set the determination region so that the object which is detected is included.
Ikeda et al. teaches the attachable matter detection apparatus (i.e. arithmetic processing device) divides the captured image into partitioned areas and generating histograms for each partitioned area (¶¶ [0075]- [0077]). Ikeda et al. further teaches determining whether attachable matter (e.g., a water droplet) exists in each partitioned area based on histogram characteristics (¶ [0065]; ¶ [0151]). The attachable matter described by Ikeda et al. appears within the captured image and occupies pixels in a spatial region of the image; therefore, the attachable matter constitutes an object included in the image. Thus, when attachable matter is detected within a partitioned area, the partitioned area constitutes a determination region that includes the detected object. Accordingly, Ikeda et al. effectively set the determination region so that the detected object is included.
Regarding Claim 5,
Ikeda et al. teaches the limitations of claim 1.
Ikeda et al. further teaches: wherein the arithmetic processing device is configured to generate two histograms for the determination regions of the two frames (see ¶ [0075] “The attachable matter detection apparatus 10 creates each histogram of the edge intensity, the luminance, and the saturation of each acquired partitioned area (Step S2′). Information including such histograms is stored for the previous frame one or more versions ago as well as the current frame.”; See ¶¶ [0075]- [0078] (found above in claim 3) for further explanation of the frame-by-frame histogram creation process.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Ikeda et al. (US 20180114089 A1).
Regarding Claim 6,
Ikeda et al. teaches the limitations of claim 5.
Ikeda et al. further teaches: wherein the arithmetic processing device determines that a water droplet has adhered when a frequency of a predetermined section on a low-brightness side of a histogram of a frame decreases from a frequency of the same section of a histogram of a previous frame.
Ikeda et al. teaches using an attachable matter detection apparatus (i.e. arithmetic processing device, see claim 1 above for explanation of equivalence) to create an edge intensity histogram, a luminance histogram, and a saturation histogram for a determination target area (predetermined section) and determining whether attachable matter is present based on frequencies of each grade of the histograms (¶ [0065]). Ikeda et al. further teaches classifying luminance into “low”, “middle,” and “high” grades (¶ [0067). Ikeda et al. states that brightness (V or L) may be used as the luminance (¶ [00103]), thereby establishing that the “low” luminance grade corresponds to a low-brightness portion of the histogram.
Ikeda et al. also teaches creating each histogram for a current frame and storing histogram information for one or more previous frames (¶ [0076]) and determining whether attachable matter exists on the basis of a “change amount” between the current frame and the previous frame based on the frequencies of specific grades (¶ [0069]; ¶ [0077). Each grade therefore constitutes a predetermined section of the histogram whose frequency is compared between successive frames. In the second embodiment, Ikeda et al. teaches comparing the change amount of each grade of the histogram with predetermined detection conditions to determine whether attachable matter is present (¶ [0151). ¶ [0151] provides examples involving increases and decreases of histogram grades.
Ikeda et al. does no explicitly disclose determining that a water droplet has adhered when the frequency of the low-luminance (low-brightness) grade decreases from a previous frame. Ikeda’s example detection condition lists an increase of the middle luminance grade, but it does not explicitly describe using a decrease of the low-luminance grade as the operative detection condition.
Nevertheless, because Ikeda et al. computes a change amount for each grade of the luminance histogram between the same predetermined sections of frames, the change amount reflects whether the frequency of the low-luminance (low-brightness) grade increases or decreases relative to the previous frame. Ikeda’s detection framework is explicitly based on the change amount of each grade and includes positive and negative changes.
Therefore, it would have been obvious to a person of ordinary skill in the art at the time of the claimed invention to modify Ikeda et al.’s attachable matter detection apparatus (arithmetic processing device) to utilize a decrease in the frequency of a predetermined section of the low-brightness side (or “low luminance grade” taught by Ikeda et al.) between frames as a detection condition for adhered water droplets since Ikeda et al. already teaches determining attachable matter based on increases and decreases of histogram grades and computing change amounts for each luminance grade, from a previous frame to a current frame. Selecting a decrease of the low-brightness side of a histogram as the operative condition would have represented a predictable variation within the disclosed detection framework.
Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Ikeda et al. (US 20180114089 A1) in view of Hayashi et al. (US 20160371821 A1).
Based on the 35 U.S.C. 112 rejection for claim 7, found above, the comparison logic “increases more than” when comparing median values of a predetermined section of a histogram of a frame to a previous frame’s median value has been construed to mean a first median value is “a higher number value than” a second mean value.
Regarding Claim 7,
Ikeda et al. teaches the limitations of claim 5.
Ikeda et al. further teaches: wherein the arithmetic processing device determines that a water droplet has adhered when
As previously disclosed in claim 6 of the instant office action, Ikeda et al. teaches an attachable matter detection apparatus (i.e. arithmetic processing device, see claim 1 above for explanation of equivalence) creating histograms (e.g. edge intensity, luminance, and saturation histograms) based on determination target areas (¶ [0065]), classifying sections of the histogram by predefined grades (e.g. low, middle, and high) (see ¶ [0065]; ¶ [0067]), storing histogram data from previous frames and current frames (¶ [0076) for computing a “change amount” between frames based on the frequencies of specific grades (¶ [0069]; ¶ [0077), and determining the attachable matter (i.e. water droplets) based on those change amounts (¶ [0151]). Each grade therefore constitutes a predetermined section of the histogram whose frequency is compared between successive frames. In the second embodiment, Ikeda et al. teaches comparing the change amount of each grade of the histogram with predetermined detection conditions to determine whether attachable matter is present (¶ [0151). ¶ [0151] provides examples involving increases and decreases of histogram grades.
Ikeda et. al further teaches the histogram creation unit uses the edge, luminance, and saturation data from each pixel to calculate values that are used to create the histogram (¶ [0094]- [0104]); thus, subsequent determinations from histograms (e.g. computing change amounts, determining if a water droplet adheres, etc.) are based on these values.
Ikeda et al. fails to teach calculating a median value of a histogram section and using a median as the parameter compared across frames.
In a related art, Hayashi et al. teaches: image restoration processing device that includes a brightness information acquisition unit (Abstract). The image restoration processing device uses an optical system based on a source image (¶ [0012]). Hayashi et al.’s brightness information acquisition unit “acquires brightness values of respective pixels constituting the source image and the distribution of the brightness values as the brightness information” (¶ [0035]) and “acquires a median value of the histogram as the brightness information” (¶ [0050]). Hayashi et al. further teaches using the median value of the histogram from the brightness information acquisition unit to determine the restoration rate of the restoration processing and the sharpening rate of the sharpening (¶ [0050]- [0051]) and an imaging condition acquisition unit “which acquires imaging conditions of a frame corresponding to the source image data and a previous frame or a subsequent frame of the frame corresponding to the source image data in a time series” (¶ [0052]) and the brightness imaging conditions are subsequently used for adjusting the restoration rate or the sharpening rate based on brightness information of the previous frame (¶¶ [0052]- [0053]). Thus, the median value of the histogram of a previous frame and conditions are used for adjusting or determining data for a subsequent (i.e. current) frame.
Therefore, it would have been obvious to a person of ordinary skill in the art at the time of the claimed invention to modify Ikeda et al.’s attachable matter detection apparatus (arithmetic processing device) that determines attachable matter (i.e. adhered water droplets) based on whether predetermined sections of histogram-derived values change between frames to incorporate Hayashi et al.’s teachings of comparing median brightness values, derived from histograms, to determine image conditions. It would have been obvious to determine droplet adhesion in Ikeda et al. when a predetermined section of a current frame median value is a higher number value than the predetermined section of a previous frame’s median value as a predictable variation using a known statistical comparison metric, note the use of identifying increases and decreases in values across frames taught by Ikeda et al. The inventions lie in the same field of endeavor of identifying abnormalities like water or blurs found from camera data and using histograms based on image data across a plurality of frames for identifying and addressing the abnormalities using conditional methods. The motivation to combine the references is to increase the accuracy of identifying attachable matter detected from an imaging processing device and system by accounting for water droplets and blurs.
Regarding Claim 8,
Ikeda et al. teaches the limitations of claim 1.
Claim 8 follows the same principle of using an arithmetic processing device to determine that a water droplet has adhered when a value (claim 6: frequency of a predetermined section on a low-brightness side of a histogram; claim 7: median value of a predetermined section of a histogram of a frame) based on the determination region of a frame has either increased or decreased (claim 6: decreases; claim 7: increases) from the previous frame as taught in claims 6 and 7. Please refer back to claims 6 and 7 for corresponding teachings by Ikeda et al. and Hayashi et al. of wherein the arithmetic processing device determines that a water droplet has adhered when a value of a determination region of a frame decreases from a value of a determination region of a previous frame, and the motivations to combine.
Ikeda et al. fails to teach: using special frequency as the parameter for determining if a water droplet has adhered.
In a related art, Hayashi et al. teaches: image restoration processing is based on optical transfer characteristics including modulation transfer function (MTF) and phase transfer function (PTF), where PTF represents phase shift as a function of spatial frequency (¶ [0161]). Spatial frequency is a quantifiable variable used to describe image and optical behavior because these optical transfer characteristics vary with spatial frequency. Hayashi et al. also teaches the MTF and PTF are used in the restoration filter (¶ [0031]), and the restoration filter is applied to image frames (FIG 30; ¶ [0344]). Accordingly, spatial frequency, as taught by Hayashi et al., corresponds to a measurable image-related value that may be evaluated for a given image frame.
Therefore, it would have been obvious to a person of ordinary skill in the art at the time of the claimed invention to modify Ikeda et al.’s attachable matter detection apparatus (i.e. arithmetic processing device) to use a known image characteristic such as spatial frequency, taught by Hayashi et al., as the compared value in an existing frame to frame comparison and conditional framework taught by Ikeda et al. Further, since Ikeda et al. already teaches determining attachable matter (i.e. water droplets), found in partitioned areas (i.e. determination regions of a frame) based on increases and decreases of values from a previous frame to a current frame, it would have been obvious to compare the known spatial frequency characteristics, found in a determination region, frame by frame to determine whether or not a water droplet has adhered based on a decrease in spatial frequency from one frame to the next. Using the metrics taught by Hayashi et al. would be a predictable alternative means of detecting changes indicative of water droplet adhesion with no change in overall function. The inventions lie in the same field of endeavor of image analysis and detecting change between frames to identify abnormalities like water or blurs. The motivation to combine the references is to increase the accuracy of identifying attachable matter detected by accounting for additional characteristics found in images impacted by water droplets and blurs.
Claims 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Ikeda et al. (US 20180114089 A1) in view of Namba et al. (US 20180372869 A1).
Regarding Claim 9,
Ikeda et al. teach the limitations of claim 1.
Ikeda et al. fails to disclose: an illumination device configured to irradiate a field of view with illumination light, wherein the sensing system is a time of flight (ToF) camera configured to divide a field of view into a plurality of ranges with respect to a depth direction and to change a time difference between light emission and imaging for each range, making it possible to obtain a plurality of range images corresponding to the plurality of ranges.
In a related art, Namba et al. discloses: an illumination device configured to irradiate a field of view with illumination light (Namba et al. teaches an image acquiring apparatus, including a light emitting unit that emits pulse lights to a predetermined direction (¶ [0052]- [0053]; ¶ [0053] “the light emitting unit 5 emits pulse light to a predetermined direction (for example, forward from the vehicle V)”),
wherein the sensing system is a time of flight (ToF) camera (Namba et al. teaches the image acquiring apparatus also includes a high-speed camera (an image acquisition unit), and a timing controller (a timing control unit) (¶ [0052]), whereby the high-speed camera includes an image sensor (¶ [0057). The following excerpt is found in paragraph [0059] of Namba et al.’s disclosure:
PNG
media_image1.png
282
408
media_image1.png
Greyscale
Excerpt from Namba et al. (US 20180372869 A1)
The relationship described in the excerpt found above defines object distance based on the time required for emitted pulse light to travel to an object and return, which is the defining operational framework of time-of-flight (ToF) distance measurement. Thus, the timing controller and highspeed camera used to perform the task described in paragraph [0059] constitutes a time-of-flight camera system (ToF).)
configured to divide a field of view into a plurality of ranges with respect to a depth direction (Namba et al. teaches the timing controller controls timing by setting delay time and lengthening delay time (¶ [0059]) and dividing the field of view into a plurality of ranges with respect to a depth direction by teaching the timing controller varies the delay time for preset target distance ranges (¶¶ [0059] – [0061]). By repeatedly lengthening the delay time, as described in (¶ [0059]), the imaging range continuously moves forward away from the vehicle, thereby segmenting the scene by depth.)
and to change a time difference between light emission and imaging for each range (Namba et al. teaches delay time between light emission and gate opening changes for each target distance range (¶ [0061])), making it possible to obtain a plurality of range images corresponding to the plurality of ranges ( ¶ [0061] “ the high-speed camera 8 may acquire one image (one frame) with respect to each target distance range, or may acquire a plurality of images (several frames) with respect to each target distance range.”).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time of the claimed invention to modify Ikeda et al.’s attachable matter detection apparatus (i.e. arithmetic processing device) to incorporate a known illumination device, as taught by Namba et al.’s image acquiring apparatus, in order to irradiate a field of view with an illumination light and divide a field of view into a plurality of ranges with respect to a depth direction and to change a time difference between light emission and imaging for each range. Doing so would enable an arithmetic processing device, like the one taught by Ikeda et al., to obtain a plurality of range images corresponding to a plurality of ranges in varying lighting and environmental conditions; thereby improving the reliability of the detection device. The modification would involve coupling the known illumination and depth acquisition hardware (taught by Namba et al.) with the arithmetic processing device (taught by Ikeda et al.) which already processes image sensor data to detect water droplet adhesions. These references both lie in the same field of endeavor acquiring a plurality of image frames using a camera and image sensor for vehicle use. Such integration would yield the predictable result of “improving accuracy of detecting an attachable matter” outlined by Ikeda et al. (see Ikeda et al. ¶ [0005]).
Regarding Claim 10,
Ikeda et al. and Namba et al. teach the limitations of claim 9.
Ikeda et al. further teaches: wherein the arithmetic processing device is configured to compare a determination region of each of the plurality of (one) or more
Ikeda et al. teaches an attachable matter detection apparatus as an arithmetic processing device (Abstract; [0324]; [0129]; FIG 7; refer back to claim 1 for further explanation) configured to divide a captured image into partitioned determination areas (i.e. regions) and create edge intensity histograms for each partitioned area (¶ [0065]; ¶¶ [0076]- [0077]), storing histogram information for a previous frame, and determining the presence of attachable matter (e.g. raindrops, water droplets) based on a “change amount” between a current frame and a previous frame (¶¶ [0077] – [0078]). Ikeda et al. further discloses detection conditions involving decreases in edge intensity (¶ [0151] “…where the ‘high’ of the ‘edge intensity’ decreases…”). Edge intensity corresponds to image gradient, which represents local contrast. A decrease in the “high” edge intensity grade, taught by Ikeda et al., therefore corresponds to a decrease in contrast in the determination region, which is characteristic of a blur caused by adhered water droplets. Ikeda et al. determines the presence of attachable matter (e.g. water droplet) based on such changes (¶ [0151]).
Ikeda et al. fails to disclose using range images from a time-of-flight (ToF) system, and comparing two or more range images.
In a related art, Namba et al. teaches: an illumination device configured to emit pulse light (¶ [0052]- [0053]) and a sensing system configured to change a delay time between light emission and imaging (¶ [0059]) to obtain images corresponding to different preset target distance ranges, thereby acquiring “acquire a plurality of images (several frames) with respect to each target distance range.” (¶ [0061]). Thus, Namba et al. teaches a ToF imaging system to generate a plurality of range images corresponding to different depth ranges.
Therefore, it would have been obvious to a person of ordinary skill in the art at the time of the claimed invention to apply Ikeda et al.’s contrast detection technique to the plurality of images generated by Namba et al.’s ToF system in order to determine water droplet adhesion within range images because both references lie in the same field of endeavor of image-based detection systems for vehicles.
Namba et al. acquires a plurality of range images corresponding to different depth ranges (¶ [0061]). Applying Ikeda et al.’s contrast decrease determination method independently to each of those range images, and requiring the decrease in contrast condition to be satisfied in two or more range images before determining a water droplet adhered would have been an obvious robustness enhancement to reduce false positives, thus making the system more accurate. Applying a confirmation across multiple range images is a routine design choice within the skills of a person of ordinary skills in the art. Again, both inventions lie in the same field of endeavor of image-based detection systems for vehicles and the motivation to combine inventions is to provide a more robust and accurate method of identifying attachable matter, like adhered water droplets (see Ikeda et al. ¶ [0005]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAMUEL DAVID BAYNES whose telephone number is (571)272-0607. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.D.B./
Samuel D. Baynes
Examiner, Art Unit 2665
/Stephen R Koziol/Supervisory Patent Examiner, Art Unit 2665