DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-5 are pending for examination in the application filed 12/29/2025. Claims 1-4 have been amended and claim 5 is new.
Priority
Acknowledgement is made of Applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent application JP2023-004468 filed on 01/16/2023.
Response to Arguments and Amendments
The 35 U.S.C. 112(f) interpretation of claims 1-4 has been withdrawn in light of the amendments. Applicant’s arguments with respect to claims 1-5 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument, as facilitated by the newly added amendments.
Claim Objections
New claim 5 is objected to because of the following informalities: claim 5 recites “the stop line searching area is set is response to…”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Amended claim 4 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 4 recites the limitation "wherein the ECU is further configured to set the stop line searching area based on the number of the far-side traffic lights in response to detecting only the far-side traffic light”. There is insufficient antecedent basis for “the far-side traffic lights” limitation in the claim. Claim 1, which claim 4 depends from, only refers to one far-side traffic light. Additionally, claim 4 refers to “in response to detecting only the far-side traffic light”, which makes it unclear if/how the other far-side traffic lights are detected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Nishimura (US20220375233A1) in view of Ben (US20160318490A1).
Regarding claim 1, Nishimura teaches a stop line searching device comprising: an electronic control unit (ECU) configured to acquire an image of an area in front of an own vehicle captured by a camera provided on the own vehicle ([0026] the information processing circuit (110, 120, 130, 140, 150, 160) may also be used as an electronic control unit (ECU) used for other control related to the vehicle. [0014] FIG. 1 is a block diagram illustrating a configuration of a traffic signal recognition device according to the present embodiment. As shown in FIG. 1, the traffic signal recognition device according to the present embodiment includes an imaging unit 71. [0015] Here, the imaging unit 71, the in-vehicle sensor 73, and the vehicle control device 400 are mounted on a vehicle. [0029] Then, the stop position acquisition unit 120 acquires the position of the stop line corresponding to the extracted traffic signal from the map information);
detect whether one or both of a near- side traffic light and a far-side traffic light are present in the acquired image, the near-side traffic light and the far-side traffic light being installed at intersections in front of the own vehicle ([0037] The selection of the target traffic signal will be described with reference to FIG. 3. FIG. 3 is a schematic view showing an example of the positional relationship between the vehicle and the plurality of traffic signals. In FIG. 3, it is shown that a vehicle traveling at a speed V is located at a position PS (coordinates x=0), a traffic signal TS1, a traffic signal TS2, and a traffic signal TS3 exist in the traveling direction of the vehicle, and the stop lines corresponding to each of the above traffic signals TS1, TS2, TS3 are at the position P1 (coordinates x=D1), the position P2 (coordinates x=D2), and the position P3 (coordinates x=D3). [0007] According to the present invention, it is possible to recognize traffic signals that need to be detected while suppressing an increase in computational load even when the distance between intersections is relatively short);
and set a stop line searching area to search for a stop line at intersections based on whether one or both of the near-side traffic light and the far-side traffic light are detected at intersections in the acquired image (See Fig. 3. [0029] Then, the stop position acquisition unit 120 acquires the position of the stop line corresponding to the extracted traffic signal from the map information. The stop position acquisition unit 120 may search for the map information acquired by the map information acquisition unit 75 based on the current position and posture of the vehicle acquired by the self-position acquisition unit 110, and may extract a traffic signal located in the traveling direction of the vehicle. Further, the stop position acquisition unit 120 may extract a traffic signal within a range that can be imaged by the imaging unit 71. Further, the stop position acquisition unit 120 may extract both a traffic signal and a stop line corresponding to the extracted traffic signal from the image captured by the imaging unit 71, and may acquire the position of the stop line corresponding to the extracted traffic signal by acquiring the position of the traffic signal and the stop line with respect to the vehicle).
Nishimura does not explicitly teach the near-side traffic light and the far-side traffic light being installed at an intersection in front of the own vehicle, and the far-side traffic light being installed on a far side of the intersection as viewed from the own vehicle; and set a stop line searching area to search for a stop line at the intersection based on whether one or both of the near-side traffic light and the far-side traffic light are detected at the intersection in the acquired image, wherein a length of the stop line searching area that is set when only the far-side traffic light is detected at the intersection in the acquired image is longer than the length of the stop line searching area that is set when both the near-side traffic light and the far-side traffic light are detected at the intersection in the acquired image, the length of the stop line searching area being the distance the stop line searching area extends in front of the own vehicle.
Ben, in the same field of endeavor of traffic light detection, teaches the near-side traffic light and the far-side traffic light being installed at an intersection in front of the own vehicle, and the far-side traffic light being installed on a far side of the intersection as viewed from the own vehicle ([0165] Distinguishing between relevant and irrelevant (or less relevant) traffic lights on a road may be complex. FIG. 9 illustrates examples of traffic light detection. In FIG. 9, vehicle 200a is traveling on a multilane road. Each lane of the road is associated with a different traffic light fixture. Vehicle 200a is approaching an intersection and is traveling in a lane designated for proceeding through the intersection and to the opposite side across the intersection. Also shown in FIG. 9, vehicle 200b is traveling in a lane that allows traffic to continue to travel straight and through the intersection or that allows traffic to make a right turn. The traffic light fixture associated with the lane in which vehicle 200b is traveling includes a right-turn traffic light. As another example, in FIG. [9], vehicle 200a reaches a junction of non-perpendicular roads. Multiple traffic light fixtures (e.g., traffic light fixtures 912 and 924), including some that do not regulate the lane in which vehicle 200a is traveling, may be visible to vehicle 200a due to the orientation of the junction);
and set a stop line searching area to search for a stop line at the intersection based on whether one or both of the near-side traffic light and the far-side traffic light are detected at the intersection in the acquired image, wherein a length of the stop line searching area that is set when only the far-side traffic light is detected at the intersection in the acquired image is longer than the length of the stop line searching area that is set when both the near-side traffic light and the far-side traffic light are detected at the intersection in the acquired image, the length of the stop line searching area being the distance the stop line searching area extends in front of the own vehicle ([0172] As described above, traffic light detection module 810 may detect traffic light from one or more images captured by a camera, and may determine the relevancy of the detected traffic light to the lane in which vehicle 200 is traveling. For example processing unit 110 may determine the relevancy of each of the plurality of traffic light fixtures (or traffic lights) captured within one or more images to vehicle 200. [0174] Other assessments may be applicable to the decision making process. For example, processing unit 110 may assess the distance of each of the plurality of traffic light fixtures with respect to the vehicle. For example, processing unit 110 may execute instructions installed within distance determination module 815 to assess the distance between the vehicle and each traffic light fixture. Processing unit 110 may determine that a closest traffic light fixture (e.g., having the smallest distance to the vehicle) is the most relevant to the vehicle. [0178] The location of the traffic light fixtures in a junction can be before, after, or in the middle of the junction. Identifying the position of each traffic light fixture in the junction may be used, for example, to determine the relevancy of the traffic light fixtures. In some embodiments, processing unit 110 may estimate the distance of one or more traffic light fixtures with respect to vehicle 200 to create a 3D model of a junction. [0181] Using the 3D model, processing unit 110 may identify a relevant traffic light when vehicle 200 is more than a predetermined distance (e.g., 50 meters, 75 meters, 100 meters, or 125 meters) from the junction. As another example, processing unit 110 may compare the relative distance between recognized traffic light fixtures in the junction to the 3D model to determine the distance to the junction's stop line).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of Nishimura with the teachings of Ben to search for stop line based on the detection of near-side and far-side traffic lights because "system 100 should base any determined system response (including warning to driver and other navigational response) on the status of the traffic light associated with the more relevant fixture 914" [Ben 0167] and "The distance to the junction (e.g., the end of the current lane in which the vehicle is travelling that meets one or more lanes of one or more crossing roads) may be determined based on the distance from the vehicle to the stop line…Based on the distance to the junction, system 100 (e.g., via processing unit 110) may determine whether one or more conditions for causing a system response are satisfied. The system response may include a warning or alert notification to a driver, or other navigational responses such as a turn, a lane shift, an acceleration, a deceleration (e.g., by a braking), maintaining current speed, and the like" [Ben 0154].
Regarding claim 5, Nishimura and Ben teach the device of claim 1. Nishimura further teaches wherein the ECU is further configured to determine a lighting state of the detected near-side traffic light or the far-side traffic light in the acquired image, wherein the stop line searching area is set is response to the ECU determining that the lighting state of the detected near-side traffic light or the far-side traffic light indicates a passage prohibition state or a transition state ([0042] The setting of the detection area will be described with reference to FIG. 4. FIG. 4 is a schematic diagram showing an example of setting a detection area on the captured image. FIG. 4 shows how the detection area R1, the detection area R2, and the detection area R3 are set corresponding to the traffic signal TS1, the traffic signal TS2, and the traffic signal TS3, respectively. [0043] When the traffic signal TS1 and the traffic signal TS2 are selected as the target traffic signals and the traffic signal TS3 is not selected as the target traffic signal as shown in FIG. 3, the detection area R3 corresponding to the traffic signal TS3 is not set. [0046] The determination unit 150 executes image processing on the detection area, detects the target traffic signal in the detection area, and determines the display state of the target traffic signal. [0029] Then, the stop position acquisition unit 120 acquires the position of the stop line corresponding to the extracted traffic signal from the map information. The stop position acquisition unit 120 may search for the map information acquired by the map information acquisition unit 75 based on the current position and posture of the vehicle acquired by the self-position acquisition unit 110, and may extract a traffic signal located in the traveling direction of the vehicle. Further, the stop position acquisition unit 120 may extract a traffic signal within a range that can be imaged by the imaging unit 71. Further, the stop position acquisition unit 120 may extract both a traffic signal and a stop line corresponding to the extracted traffic signal from the image captured by the imaging unit 71, and may acquire the position of the stop line corresponding to the extracted traffic signal by acquiring the position of the traffic signal and the stop line with respect to the vehicle. [0047] The “color signal” indicated by the traffic signal includes a “green signal”, a “yellow signal”, and a “red signal”. The meaning of “color signal” is determined by the traffic regulations that the vehicle should follow. For example, “green signal” means “may proceed” and “red signal” means “stop at the stop position”. The “yellow signal” means “stop at the stop position unless it is not possible to stop safely because it is close to the stop position”).
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Nishimura in view of Ben and Sato (JP2009015504A).
Regarding claim 2, Nishimura and Ben teach the device of claim 1. Nishimura further teaches wherein the ECU is further configured to set the stop line searching area in response to detecting only a far-side traffic light in the acquired image ([0062] In step S111, the detection area setting unit 140 sets the detection area corresponding to the target traffic signal on the image captured by the imaging unit 71. [0063] In step S121, the determination unit 150 sets the variable i to 1. [0064] In step S123, the determination unit 150 determines the display state of the target traffic signal (i-th target traffic signal) closest to the vehicle among the N target traffic signals. Specifically, image processing is executed on the detection area corresponding to the i-th target traffic signal closest to the vehicle, and the determination unit 150 determines the display state of the target traffic signal. [0072] Since the number of detection areas set on the image can be reduced, the computational load on the detection area setting unit 140 and the determination unit 150 can be reduced).
Nishimura does not teach set the stop line searching area based on the number of lanes extending in an intersecting direction intersecting a travel direction in which the own vehicle enters the intersection in response to detecting only a far-side traffic light in the acquired image.
Sato teaches set the stop line searching area based on the number of lanes extending in an intersecting direction intersecting a travel direction in which the own vehicle enters the intersection in response to detecting only a far-side traffic light in the acquired image (See Sato Fig. 6-7 below. [pg. 7 para. 5] The traffic regulation position detection device (1) according to claim 8 is the traffic regulation position detection device according to claim 7, wherein the road information acquisition means (13) is provided with the traffic regulation as road information. Information on the number of lanes and the lane width of the road intersecting with the vehicle is acquired, and the position detecting means (13) detects the position of traffic regulation based on the number of lanes and the lane width of the road. [pg. 13 para. 4] In S42, the CPU 41 calculates the distance L from the road end point to the temporary stop line in order to specifically identify the position of the temporary stop line on the road based on the road information acquired in S23 to S25. To do. Specifically, it is calculated as L = (number of lanes / 2) × lane width + sidewalk width + predetermined distance (constant 3 m). Thereby, the detailed position of the temporary stop line on the traveling road of the own vehicle is detected. See Sato Fig. 13-14 below).
PNG
media_image1.png
823
305
media_image1.png
Greyscale
PNG
media_image2.png
817
570
media_image2.png
Greyscale
PNG
media_image3.png
908
1273
media_image3.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of Nishimura with the teachings of Sato to set the stop line searching area based on the number of lanes in the intersecting direction because "it is possible to detect the exact position on the road for the latest and a wide range of the stop line. Based on the detected position of the temporary stop line, for example, guidance for the temporary stop line and vehicle control can be performed at appropriate timing" [Sato pg. 14 para. 1].
Regarding claim 3, Nishimura, Ben, and Sato teach the device of claim 2. Nishimura further teaches wherein the ECU is further configured to set an area extending toward the own vehicle side as the stop line searching area using an end portion, on the own vehicle side, of a road including a lane extending in the intersecting direction as a starting point ([0023] The map information acquisition unit 75 acquires map information indicating the structure of the road on which the vehicle travels. The map information acquired by the map information acquisition unit 75 includes road structure information such as absolute lane positions, lane connection relationships, and relative positional relationships. Further, the map information acquired by the map information acquisition unit 75 may include facility information such as a parking lot and a gas station. In addition, the map information may include the position information of the traffic signal, the type of the traffic signal, the position of the stop line corresponding to the traffic signal, and the like).
PNG
media_image4.png
408
637
media_image4.png
Greyscale
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Nishimura in view of Ben and Tsuji (JP2020109560A).
Regarding claim 4, Nishimura and Ben teach the device of claim 1. Nishimura further teaches wherein the ECU is further configured to set the stop line searching area in response to detecting only the far-side traffic light ([0062] In step S111, the detection area setting unit 140 sets the detection area corresponding to the target traffic signal on the image captured by the imaging unit 71. [0063] In step S121, the determination unit 150 sets the variable i to 1. [0064] In step S123, the determination unit 150 determines the display state of the target traffic signal (i-th target traffic signal) closest to the vehicle among the N target traffic signals. Specifically, image processing is executed on the detection area corresponding to the i-th target traffic signal closest to the vehicle, and the determination unit 150 determines the display state of the target traffic signal. [0072] Since the number of detection areas set on the image can be reduced, the computational load on the detection area setting unit 140 and the determination unit 150 can be reduced).
Nishimura does not teach the set the stop line searching area based on the number of the far-side traffic lights in response to detecting only the far-side traffic light.
Tsuji, in the same field of endeavor of traffic light and stop line detection, teaches set the stop line searching area based on the number of the far-side traffic lights in response to detecting only the far-side traffic light (See Tsuji Fig. 3A below. [pg. 6 para. 7] In addition, when the number of traffic signals corresponding to the stop line is one or more, the position shift amount calculation unit 65 sets the “switching distance” longer as the number of traffic signals corresponding to the stop line increases. [pg. 8-9] Further, the traffic light recognition method and the traffic light recognition device according to the present embodiment may set the first distance longer as the number of traffic lights corresponding to the stop line increases.
PNG
media_image5.png
898
454
media_image5.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of Nishimura with the teachings of Tsuji to "By setting the first distance to be long, the timing of switching from the second image to the first image becomes early. The traffic signals corresponding to the same stop line are more likely to be captured in the first image captured in a wider angle range than the second image, and by detecting a plurality of traffic signals with the same first image, the traffic signals are continuously detected" [Tsuji pg. 8-9].
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jacqueline R Zak whose telephone number is (571)272-4077. The examiner can normally be reached M-F 9-5. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACQUELINE R ZAK/Examiner, Art Unit 2666
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666