Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed May 26, 2026 have been fully considered but they are not persuasive.
Regarding claim 1,
applicant states that “Tomoko's histogram processing is window-based image
processing. Tomoko obtains a histogram of horizontal edges in windows set at the position of a preceding vehicle. Tomoko does not disclose computing edge histogram data for each generated distance histogram.”. Examiner disagrees with this statement. Tomoko teaches a computation processor configured to compute edge histogram data (Fig. 8(C) is a histogram of each horizontal edge in the windows (1) to (5) in (B).
PNG
media_image1.png
536
626
media_image1.png
Greyscale
) for each distance histogram generated by the distance histogram generator (When such a distance image is used to vote in the table using the method shown in Figure 4, if the same parallax is lined up in the same direction (on the same x-coordinate), the number of votes for that direction and parallax value will be large, and the value of that position will be high (which reads on a “distance histogram”). Therefore, by searching for a position with a high value in the table of FIG. 5, it is possible to detect the presence or absence of an object ahead. In the example shown in Figure 5, votes are concentrated and have high values in the areas of disparity "19" in windows (3) and (4) (corresponding to the tree on the left), disparity "5" in window (5) (corresponding to the tree in the center), and disparity "15" in windows (8) to (16) (corresponding to the vehicle ahead). [0034]). The claim language does not preclude obtaining a histogram of horizontal edges using window-based image processing.
applicant states that “Masayuki does not disclose defining matching targets from
previously computed edge histogram data associated with previously generated distance histogram data. Masayuki also does not disclose applying the claimed first-threshold condition and second-threshold condition to the associated distance histogram data, determining a degree of closeness between the most recently computed edge histogram data and each matching target, and extracting matching edge histogram data based on that degree of closeness.”. Examiner disagrees with this statement. Please see rejection for claim 1.
applicant states that “The proposed combination would require changing Cheol's
object-candidate histogram analysis into a framework that associates distance histogram data with edge histogram data, changing Tomoko's window-based edge histograms into edge histogram data computed for each generated distance histogram, and changing Masayuki's object-candidate tracking into the claimed extractor logic. The references do not teach or suggest these changes.”. Examiner disagrees with this statement. It would have been prima facie obvious to one of ordinary skill in the art to have modified Cheol to incorporate the teachings of Tomoko to compute edge histogram data for each distance histogram generated by the distance histogram generator and hold the distance histogram generated by the distance histogram generator and the edge histogram data computed by the computation processor in association with each other in order to improve the reliability of measuring the distance and changes in distance between a vehicle ahead that is being followed. It would have been prima facie obvious to one of ordinary skill in the art to incorporate the teachings of Masayuki to define, as one or more matching targets, edge histogram data, associated with distance histogram data, wherein, for distance histogram data, a height difference is within a first threshold, and a change in speed is within a second threshold and to extract, based on the degree of closeness, from the matching targets, edge histogram data to be matched with the edge histogram data computed most recently and to calculate movement of the crossing object based on the extracted edge histogram data in order to recognize the environment around the vehicle to prevent accidents before they happen.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-2 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Cheol (Korea Patent Pub. No.: KR 10-1771146 B1), hereinafter Cheol, in view of Tomoko (Japan Patent Pub. No.: JP2000266539A), hereinafter Tomoko, further in view of Masayuki (PCT Patent Pub. No.: WO2016129403A1), hereinafter Masayuki.
Regarding claim 1, Cheol teaches an image processing apparatus comprising: a distance histogram generator configured to generate (The object candidate detection unit 110 may include a disparity image generation unit 111 and a histogram analysis unit 112 to detect object candidates. Page 3 6th paragraph), for each horizontal angle of view (In addition, the object candidate detection step may detect an object candidate by analyzing the uniformity of the vertical distribution of the histogram of the parallax image. Page 3 2nd paragraph) of an imaging unit (First, the stereo camera 200 may include a first camera 210 and a second camera 220 to generate a stereo image. Page 3 4th paragraph) mounted on a vehicle (More particularly, the present invention relates to a method and an apparatus for detecting a pedestrian, a vehicle, and the like using an image acquired through a stereo camera installed on a moving object such as a vehicle. Page 2 4th paragraph), a distance histogram as one-dimensional distance data (3 illustrates a depth image and a histogram distribution diagram used in the object candidate detection step according to an exemplary embodiment of the present invention. The horizontal axes of the histograms 310 and 320 in FIG. 3 denote the depth values of the pixels of the depth image, and the vertical axes denote the number of pixels of the depth value. Page 4 1st paragraph .
PNG
media_image2.png
494
784
media_image2.png
Greyscale
), based on distance data of a distance image captured by the imaging unit (Fig. 3 depth map), the distance image having a pixel value corresponding to a distance to a crossing object in captured images of a region in front of the vehicle (The parallax image generating unit 111 may convert the parallax image generated using camera parameters or the like into a depth image. For example, the depth image may be an image in which the distance from the camera to the object is expressed as a value from 0 to 255. Page 3 7th paragraph).
Cheol does not teach the following limitations as further recited, but Tomoko further teaches a computation processor configured to compute edge histogram data (Fig. 8(C) is a histogram of each horizontal edge in the windows (1) to (5) in (B).
PNG
media_image1.png
536
626
media_image1.png
Greyscale
) for each distance histogram generated by the distance histogram generator (When such a distance image is used to vote in the table using the method shown in Figure 4, if the same parallax is lined up in the same direction (on the same x-coordinate), the number of votes for that direction and parallax value will be large, and the value of that position will be high (which reads on a “distance histogram”). Therefore, by searching for a position with a high value in the table of FIG. 5, it is possible to detect the presence or absence of an object ahead. In the example shown in Figure 5, votes are concentrated and have high values in the areas of disparity "19" in windows (3) and (4) (corresponding to the tree on the left), disparity "5" in window (5) (corresponding to the tree in the center), and disparity "15" in windows (8) to (16) (corresponding to the vehicle ahead). [0034]); a memory (Reference numeral 9 denotes a calculation unit, which is composed of a microcomputer including, for example, a CPU, RAM, ROM, and the like. [0077]) configured to hold at least the distance histogram generated by the distance histogram generator and the edge histogram data computed by the computation processor in association with each other (Next, an embodiment will be described in which the method described above is used to detect edges (which reads on “the edge histogram data”) on a preceding vehicle being followed and to determine the rate of change of the inter-vehicle distance, thereby confirming the inter-vehicle distance (which reads on “the distance histogram” data)and improving its accuracy. [0059]. In other words, the distance histogram and the edge histogram data are held in association with each other.).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cheol to incorporate the teachings of Tomoko to compute edge histogram data for each distance histogram generated by the distance histogram generator and hold the distance histogram generated by the distance histogram generator and the edge histogram data computed by the computation processor in association with each other in order to improve the reliability of measuring the distance and changes in distance between a vehicle ahead that is being followed.
The combination of Cheol and Tomoko does not teach the following limitations as further recited, but Masayuki further teaches an extractor configured to define, as one or more matching targets, one or more pieces of the edge histogram data (The pedestrian detection unit 300 extracts three-dimensional objects using this parallax image, tracks the extracted three-dimensional object candidates in chronological order, and, when three-dimensional object candidates are stably extracted in chronological order, identifies whether the parallax shape and the outline shape based on the edges extracted (which reads on “the edge histogram data”) from the current image are likely to be those of a pedestrian (which reads on “one or more matching targets”). [0014]) previously computed by the computation processor (This will be explained with reference to the upper diagram in FIG. If the position of the pedestrian can be acquired in time series from two frames before, it is assumed that the position of the pedestrian from T-2 [frame] to the current T [frame] can be acquired. [0022]. Cheol teaches a distance histogram can be generated from distance image data (see rejection for claim 1).), included in a search range set by the image processing control processor (The position information generating unit 500 generates the three-dimensional position of the detected pedestrian. The positional accuracy information generating unit 400 generates positional accuracy information representing the accuracy of this positional information from the results of the parallax images. [0014]), and associated in the memory with one or more pieces of distance histogram data previously generated by the distance histogram generator (The pedestrian detection unit 300 extracts three-dimensional objects using this parallax image (which reads on “distance histogram data”), tracks the extracted three-dimensional object candidates in chronological order, and, when three-dimensional object candidates are stably extracted in chronological order, identifies whether the parallax shape and the outline shape based on the edges extracted from the current image are likely to be those of a pedestrian. [0014]), wherein, for each of the one or more pieces of distance histogram data, a height difference between the piece of distance histogram data and the distance histogram generated most recently by the distance histogram generator is within a first threshold (Taking into account the behavior of the vehicle, tracking of three-dimensional objects is performed by comparing the position and size of the three-dimensional object on the image predicted from the previous frame to the current frame, whether the predicted three-dimensional object is in a similar position, size, and disparity value (which reads on “height of distance histogram data”) within a certain threshold (which reads on “a first threshold”), and whether the position on the image of the previous frame is similar to the position on the image of the current frame. [0016]), and a change in speed (In step S04, the tracking unit 320 uses at least two pieces of information from the three-dimensional object position in the current frame, the three-dimensional object candidate position in the previous frame, the three-dimensional object speed information, the vehicle behavior, etc., to track the three-dimensional object candidate extracted independently for each frame in the processing of step S03. [0072]. It is common knowledge a change in speed can be calculated based on speed and time interval between frames.) calculated based on an amount of change between the piece of distance histogram data and the distance histogram generated most recently by the distance histogram generator (The effective parallax histogram (which reads on “the distance histogram”) thus projected in the vertical direction of the image is used to calculate the degree of separation from surrounding objects. [0047]) is within a second threshold (Taking into account the behavior of the vehicle, tracking of three-dimensional objects is performed by comparing the position and size of the three-dimensional object on the image predicted from the previous frame to the current frame, whether the predicted three-dimensional object is in a similar position, size, and disparity value within a certain threshold, and whether the position on the image of the previous frame is similar to the position on the image of the current frame. [0016]. In step S04, the tracking unit 320 uses at least two pieces of information from the three-dimensional object position in the current frame, the three-dimensional object candidate position in the previous frame, the three-dimensional object speed information, the vehicle behavior, etc., to track the three-dimensional object candidate extracted independently for each frame in the processing of step S03. [0072]), determine a degree of closeness (Taking into account the behavior of the vehicle, tracking of three-dimensional objects is performed by comparing the position and size of the three-dimensional object on the image predicted from the previous frame to the current frame, whether the predicted three-dimensional object is in a similar position, size, and disparity value within a certain threshold, and whether the position on the image of the previous frame is similar to the position on the image of the current frame. [0016]) between the edge histogram data computed most recently by the computation processor and each of the one or more matching targets (This will be explained with reference to the upper diagram in FIG. If the position of the pedestrian can be acquired in time series from two frames before, it is assumed that the position of the pedestrian from T-2 [frame] to the current T [frame] can be acquired. [0022]. Tomoko teaches an edge histogram can be generated from distance image data (see rejection for claim 1).), and extract, based on the degree of closeness (Taking into account the behavior of the vehicle, tracking of three-dimensional objects is performed by comparing the position and size of the three-dimensional object on the image predicted from the previous frame to the current frame, whether the predicted three-dimensional object is in a similar position, size, and disparity value within a certain threshold, and whether the position on the image of the previous frame is similar to the position on the image of the current frame. [0016]), from the one or more matching targets, edge histogram data (The pedestrian detection unit 300 extracts three-dimensional objects using this parallax image, tracks the extracted three-dimensional object candidates in chronological order, and, when three-dimensional object candidates are stably extracted in chronological order, identifies whether the parallax shape and the outline shape based on the edges extracted from the current image are likely to be those of a pedestrian. [0014]) to be matched with the edge histogram data computed most recently by the computation processor (Taking into account the behavior of the vehicle, tracking of three-dimensional objects is performed by comparing the position and size of the three-dimensional object on the image predicted from the previous frame to the current frame, whether the predicted three-dimensional object is in a similar position, size, and disparity value within a certain threshold, and whether the position on the image of the previous frame is similar to the position on the image of the current frame. [0016]) (In step S10, the movement information prediction unit 530 predicts the pedestrian's destination using the pedestrian's position information and position accuracy information. [0077]) in a vehicle width direction (
PNG
media_image3.png
678
620
media_image3.png
Greyscale
), based on a difference value between the extracted edge histogram data (The pedestrian detection unit 300 extracts three-dimensional objects using this parallax image, tracks the extracted three-dimensional object candidates in chronological order, and, when three-dimensional object candidates are stably extracted in chronological order, identifies whether the parallax shape and the outline shape based on the edges extracted from the current image are likely to be those of a pedestrian. [0014]) most recently (For this reason, movement prediction is performed using the instantaneous value of the position accuracy information acquired by the position accuracy information generating unit 400. As shown in the movement prediction (a) using the position accuracy information at the bottom right of FIG. 17, positions with poor accuracy using the position accuracy information are treated as excluded from the data for movement prediction. [0024].
PNG
media_image3.png
678
620
media_image3.png
Greyscale
); and an image processing control processor configured to execute image processing control (In step S02, the parallax image generating unit 200 performs stereo matching processing using the images captured by the imaging unit 100 of the stereo camera, and generates parallax images. [0070]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Masayuki to define, as one or more matching targets, edge histogram data, associated with distance histogram data, wherein, for distance histogram data, a height difference is within a first threshold, and a change in speed is within a second threshold and to extract, based on the degree of closeness, from the matching targets, edge histogram data to be matched with the edge histogram data computed most recently and to calculate movement of the crossing object based on the extracted edge histogram data in order to recognize the environment around the vehicle to prevent accidents before they happen.
Claim 2, unamended and is rejected based on the combination of Cheol, in view of Tomoko, further in view of Masayuki. The grounds of rejection established in the last Office Action is fully incorporated herein.
Apparatus claim 10 is drawn to the apparatus as claimed in claim 1. Therefore apparatus claim 10 corresponds to apparatus claim 1, and is rejected for the same reasons of obviousness as used above.
Claims 3-4, unamended and are rejected based on the combination of Cheol (Korea Patent Pub. No.: KR 10-1771146 B1), hereinafter Cheol, in view of Tomoko (Japan Patent Pub. No.: JP2000266539A), hereinafter Tomoko, further in view of Masayuki (PCT Patent Pub. No.: WO2016129403A1), hereinafter Masayuki, further in view of Kazutoshi (Japan Patent No.: JP3650205B2), hereinafter Kazutoshi. The grounds of rejection established in the last Office Action is fully incorporated herein.
Claim 5, unamended and is rejected based on the combination of Cheol (Korea Patent Pub. No.: KR 10-1771146 B1), hereinafter Cheol, in view of Tomoko (Japan Patent Pub. No.: JP2000266539A), hereinafter Tomoko, further in view of Masayuki (PCT Patent Pub. No.: WO2016129403A1), hereinafter Masayuki, further in view of Kazutoshi (Japan Patent No.: JP3650205B2), hereinafter Kazutoshi, further in view of Masumi (Japan Patent Pub. No.: JP2014-182629A), hereinafter Masumi. The grounds of rejection established in the last Office Action is fully incorporated herein.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEI ZHAO whose telephone number is (703)756-1922. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, VU LE can be reached at (571)272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEI ZHAO/Examiner, Art Unit 2668
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights