Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 05/27/2025 has been entered, claims 1-7, 10-20 remain pending in the application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4, 5, 6, 11, 12, 13, 15, 16 are rejected under 35 U.S.C. 103 as being unpatentable by Pronovost (US12012108) in view of in view of Moed (US20090074249) and Tamilarasan (US20220176957).
Regarding claim 1, Pronovost teaches the method for traffic sign prediction, the method comprising:
determining a plurality of roadway characteristics (col 9 lines 23-35 disclosing determining road conditions, traffic conditions, driving environment, city or rural);
determining a plurality of road user information (col. 6 line 37 – col 7 line 35 disclosing detecting road users);
determining a plurality of weather-related information (col 9 lines 23-35 disclosing plurality of weather conditions);
determining a predicted traffic sign based at least in part on the plurality of roadway characteristics, the plurality of road user information, and the plurality of weather-related information (col. 8 last paragraph - col. 9 lines 35 disclosing predicting a stop sign based on heuristics including plurality of road user information, plurality of weather information and roadway conditions such as lane markings, accidents, traffic conditions);
and performing an action based at least in part on the predicted traffic sign (at least col. 9 last paragraph- col.10 first paragraph discloses adding the stop sign feature to the map. Col. 30 disclosing controlling the vehicle based on the updated map).
Wherein performing the action includes at least Control a path of the vehicle by adjusting a path planning and control algorithm of an automated driving system based at least in part on the predicted traffic sign (col. 29 line 30-col 30 first paragraph disclosing the control the vehicle based on the changed map based on the sign predicted wherein the execution of driving maneuver and navigation).
Pronovost does not teach evaluating the predicted sign with a predetermined ruleset, wherein the predetermined ruleset includes rules from traffic laws, rules, or standards; Modifying the predicted traffic sign in response to determining that the predicted traffic sign is contrary to the predetermined ruleset;
Tamilarasan teaches evaluating the predicted sign with a predetermined ruleset, wherein the predetermined ruleset includes rules from traffic laws, rules, or standards ([0020], [0024]-[0025] disclosing the evaluation of the camera detected direct speed limit sign by comparing to indirect speed limit data from context information such as regional speed limit policies “standard, or traffic laws or rules”);
Modifying the predicted traffic sign in response to determining that the predicted traffic sign is contrary to the predetermined ruleset [0035]-[0037] disclosing the regional speed limit rule and determining based on the verification that the speed limit read from a sign by the camera is an error, thus ignoring the predicted speed limit sign and controlling the speed limit based on the determined speed. The claim does not require a real sign to be seen or detected, the modification of the speed detected by the sign to a speed of the road indicated by the GPS/map is interpreted as modifying the predicted traffic sign. A sign is also interpretable as a non physical sign such as indication of a traffic pattern such as speed or stopping. [0040]-[0045] disclosing determining if a camera based speed limit conforms to regional policies to verify if the camera detected and predicted speed limit is correct);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed to incorporate the teaching of Tamilarasan in order to improve safety by verifying a stop sign as taught by Tamilarasan [0002], [0020]. While Tamilarasan does not explicitly state the predicted sign, it is obvious to combine the evaluation method with the predicted sign in order to verify the predicted sign and improve prediction accuracy which also improves driving safety.
While it is believed that Tamilarasan teaches the modification of the sign, Moed is further cited to teach the limitation.
Moed teaches modifying a predicted sign ([0038]-[0045] disclosing the modifying the prediction by removing some of the predicted signs based on further verification of signs).
it would have been obvious to one of ordinary skill in the art to have modified the teaching of Pronovost as modified by Tamilarasan to incorporate the teaching of Moed of modifying a predicted sign in order to accurately determine a sign and removing wrong signs that fall below a threshold as taught by Moed thus improving driving safety and avoiding wrong vehicle decisions and improving safety.
Regarding claim 2, Pronovost as modified by Moed and Tamilarasan teaches the method of claim 1, wherein determining the plurality of roadway characteristics further comprises: determining the plurality of roadway characteristics using at least one of: a perception sensor and data retrieved from a remote server, wherein the plurality of roadway characteristics includes at least one roadway geometry characteristic, at least one roadway condition characteristic, and at least one roadside environment characteristic (Pronovost col. 2 last paragraph disclosing the vehicle captures sensor data and receives data from an infrastructure. Col. 6 line 23-35 disclosing the infrastructure is a remote server. col. 8 last paragraph - col. 10 lines 17 disclosing the detected lane markings and intersection “roadway geometry”, accidents “roadway conditions”, vehicles on shoulder “roadside environmental characteristic).
Regarding claim 4, Pronovost as modified by Moed and Tamilarasan teaches the method of claim 1, wherein determining the plurality of weather- related information further comprises: determining the plurality of weather-related information using at least one of: a perception sensor and data retrieved from a remote server (Pronovost col. 21 lines 39-62 disclosing perception sensors detect weather condition), wherein the plurality of weather-related information includes at least one precipitation weather condition and at least one visibility weather condition (Pronovost col. 9 lines 21-35 disclosing the weather condition includes fog “visibility condition” and rain “precipitation condition”).
Regarding claim 5, Pronovost as modified by Moed and Tamilarasan teaches the method of claim 1, wherein determining the predicted traffic sign further comprises: determining the predicted traffic sign using a rule-based algorithm, wherein the rule-based algorithm takes the plurality of roadway characteristics, the plurality of road user information, and the plurality of weather-related information as inputs and produces the predicted traffic sign as output (Pronovost at least col. 9 lines 21-35 and cols 8-10 disclosing the heuristics using all indicated data above to include road user information, weather information and roadway characteristic to determine the sign as an output to be added to the map using Machine learning and based on a set of given combination of rules).
Regarding claim 6, Pronovost as modified by Moed and Tamilarasan teaches the method of claim 1. Pronovost as modified by Moed and Tamilarasan does not explicitly teach wherein determining the predicted traffic sign further comprises: determining the predicted traffic sign using a fuzzy logic algorithm, wherein the However, Pronovost teaches predicting the algorithm to detect a sign based on the weather, traffic condition and road user information (col. 8- 10 disclosing the heuristics including a table to take an input and mapping rules to determine features “signs” to add to map).
Moed further teaches a fuzzy logic algorithm to predict a plurality of possible predicted traffic signs and a corresponding plurality of confidence values as output; to determine traffic sign with confidence values ([0026]-[0027] disclosing a fuzzy logic that predicts plural sign candidates with confidence values and chooses the signs based on the confidence level. [0038]-[0045] disclosing the process of determining the sign with confidence).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed and Tamilarasan to incorporate the teaching of Moed of a fuzzy logic algorithm to predict a plurality of possible predicted traffic signs and a corresponding plurality of confidence values as output; to determine traffic sign with confidence values in order to determine the sign accurately by discriminating the sign and remove a wrong sign when confidence fall below a threshold as taught by Moed which improves driving and reduces accidents. Pronovost teaches the determining of a sign using algorithm based on the weather, traffic and road user information, Moed teaches the fuzzy algorithm method to determine the sign, it would be obvious to try to use the fuzzy algorithm with the information of Pronovost which would yield predictable results. The combination is obvious to one of ordinary skill in the art yielding predictable results. Also the use of equations is a simple design choice, Since the invention failed to provide novel or unexpected results from the usage of said claimed formula, use of any mathematical means, including that of the claimed invention, would be an obvious matter of design choice within the skill of the art.
Claims 11, 12, 13, 15, 16 are rejected under 35 U.S.C. 103 as being unpatentable by Pronovost (US12012108) in view of in view of Moed (US20090074249).
Regarding claim 11, Pronovost teaches a system for traffic sign prediction for a vehicle, the system comprising:
A plurality of vehicle sensors including a vehicle communication system (Col. 2 last paragraph disclosing the vehicle sensors to detect the environment data. Col. 20 last paragraph disclosing a communication system);
An automated system (at least col. 20 lines 40-60 disclosing the autonomous vehicle with autonomous brakes, steering);
A controller in electrical communication with the plurality of vehicle sensors and the automated driving system wherein the controller is programmed to (col. 20 and at least col.26 first paragraph disclosing a controller connected to the sensors):
determining a plurality of roadway characteristics using at least one of the plurality of vehicle sensors (col 9 lines 23-35 disclosing determining road conditions, traffic conditions, driving environment, city or rural. Col. 2 last paragraph disclosing the vehicle sensors to detect the environment data);
determining a plurality of road user information using at least one of the plurality of vehicle sensors (col. 6 line 37 – col 7 line 35 disclosing detecting road users. Col. 2 last paragraph disclosing the vehicle sensors to detect the environment data);
determining a plurality of weather-related information using at least one of the plurality of vehicle sensors (col 9 lines 23-35 disclosing plurality of weather conditions. Col. 2 last paragraph disclosing the vehicle sensors to detect the environment data. col. 21 lines 39-62 disclosing perception sensors detect weather condition);
determining a predicted traffic sign based at least in part on the plurality of roadway characteristics, the plurality of road user information, and the plurality of weather-related information (col. 8 last paragraph - col. 9 lines 35 disclosing predicting a stop sign based on heuristics including plurality of road user information, plurality of weather information and roadway conditions such as lane markings, accidents, traffic conditions);
Control a path of the vehicle by adjusting a path planning and control algorithm of an automated driving system based at least in part on the predicted traffic sign (col. 29 line 30-col 30 first paragraph disclosing the control the vehicle based on the changed map based on the sign predicted wherein the execution of driving maneuver and navigation).
Pronovost does not explicitly teach wherein determining the predicted traffic sign further comprises: determining the predicted traffic sign using a fuzzy logic algorithm, wherein the However, Pronovost teaches predicting the algorithm to detect a sign based on the weather, traffic condition and road user information (col. 8- 10 disclosing the heuristics including a table to take an input and mapping rules to determine features “signs” to add to map).
Moed further teaches a fuzzy logic algorithm to predict a plurality of possible predicted traffic signs and a corresponding plurality of confidence values as output; to determine traffic sign with confidence values ([0026]-[0027] disclosing a fuzzy logic that predicts plural sign candidates with confidence values and chooses the signs based on the confidence level. [0038]-[0045] disclosing the process of determining the sign with confidence).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost to incorporate the teaching of Moed of a fuzzy logic algorithm to predict a plurality of possible predicted traffic signs and a corresponding plurality of confidence values as output; to determine traffic sign with confidence values in order to determine the sign accurately by discriminating the sign and remove a wrong sign when confidence fall below a threshold as taught by Moed which improves driving and reduces accidents. Pronovost teaches the determining of a sign using algorithm based on the weather, traffic and road user information, Moed teaches the fuzzy algorithm method to determine the sign, it would be obvious to try to use the fuzzy algorithm with the information of Pronovost which would yield predictable results. The combination is obvious to one of ordinary skill in the art yielding predictable results. Also the use of equations is a simple design choice, Since the invention failed to provide novel or unexpected results from the usage of said claimed formula, use of any mathematical means, including that of the claimed invention, would be an obvious matter of design choice within the skill of the art.
Regarding claim 12, Pronovost as modified by Moed teaches the system of claim 11, wherein the plurality of vehicle sensors further includes at least one perception sensor, and wherein to determine the plurality of roadway characteristics, the controller further programmed to (Pronovost Col. 2 last paragraph disclosing the vehicle sensors to detect the environment data. at least col.26 first paragraph disclosing a controller connected to the sensors):
Perform a plurality of measurements of an environment surrounding the vehicle sing the at least one perception sensor (Pronovost Col. 2 last paragraph disclosing the vehicle sensors to detect the environment data);
determining the plurality of roadway characteristics using at least one of: a perception sensor and data retrieved from a remote server, wherein the plurality of roadway characteristics includes at least one roadway geometry characteristic, at least one roadway condition characteristic, and at least one roadside environment characteristic (Pronovost col. 2 last paragraph disclosing the vehicle captures sensor data and receives data from an infrastructure. Col. 6 line 23-35 disclosing the infrastructure is a remote server. col. 8 last paragraph - col. 10 lines 17 disclosing the detected lane markings and intersection “roadway geometry”, accidents “roadway conditions”, vehicles on shoulder “roadside environmental characteristic).
Regarding claim 13, Pronovost as modified by Moed teaches the system of claim 11, wherein the plurality of sensors further includes a vehicle communication system, and wherein to determine the plurality of roadway characteristics the controller is further programmed to (Pronovost col. 2 last paragraph disclosing receiving the plurality of information from other vehicles “v2v” or infrastructure “v2x”. col. 25 second paragraph disclosing the communication connections. Col. 26 first paragraph disclosing the controller):
Receive at least one of a V2V message and a V2X message including the plurality of roadway characteristics using the vehicle communication system(Pronovost col. 2 last paragraph disclosing receiving the plurality of information from other vehicles “v2v” or infrastructure “v2x”)., wherein the plurality of roadway characteristics includes at least one roadway geometry characteristic, at least one roadway condition characteristic, and at least one roadside environment characteristic (Pronovost col. 2 last paragraph disclosing the vehicle captures sensor data and receives data from an infrastructure. Col. 6 line 23-35 disclosing the infrastructure is a remote server. col. 8 last paragraph - col. 10 lines 17 disclosing the detected lane markings and intersection “roadway geometry”, accidents “roadway conditions”, vehicles on shoulder “roadside environmental characteristic).
Regarding claim 15, Pronovost as modified by Moed teaches the system of claim 11, wherein the plurality of vehicle sensors further includes at least one perception sensor and a vehicle communication sensor, wherein determining the plurality of weather- related information (Pronovost col. 20 and 26 cited above disclose communication sensor and col. 21 lines 39-62 disclosing perception sensor). the controller further configured to: determining the plurality of weather-related information using at least one of: a perception sensor and data retrieved from a remote server (Pronovost col. 21 lines 39-62 disclosing perception sensors detect weather condition), wherein the plurality of weather-related information includes at least one precipitation weather condition and at least one visibility weather condition (Pronovost col. 9 lines 21-35 disclosing the weather condition includes fog “visibility condition” and rain “precipitation condition”).
Regarding claim 16, Pronovost as modified by Moed teaches the system of claim 11. Pronovost as modified by Moed does not teach wherein to determine the predicted traffic sign, the controller is further programmed to: validate the predicted traffic sign by evaluating the predicted traffic sign with a predetermined ruleset; and modify the predicted traffic sign in response to determining that the predicted traffic sign is contrary to the predetermined ruleset.
Specifically, Moed teaches validate the predicted traffic sign by evaluating the predicted traffic sign with a predetermined ruleset [0038]-[0045] disclosing after determining the predicted traffic signs, further discriminating the signs against another colored image and filtering out the signs based on a rule of matching a predetermined set of signs).
modify the predicted traffic sign in response to determining that the predicted traffic sign is contrary to the predetermined ruleset ([0038]-[0045] disclosing modifying the prediction by removing some of the predicted signs based on the further verification of the sign).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed to incorporate the teaching of Moed of validate the predicted traffic sign by evaluating the predicted traffic sign with a predetermined ruleset and modify the predicted traffic sign in response to determining that the predicted traffic sign is contrary to the predetermined ruleset in order to determine the sign accurately by discriminating the sign and remove a wrong sign when confidence fall below a threshold as taught by Moed which improves driving and reduces accidents.
Claims 3 are rejected under 35 U.S.C. 103 as being unpatentable by Pronovost (US12012108) in view of Moed (US20090074249) and Tamilarasan (US20220176957) and Asakura (US20170010612) and St. Romain (US20210396528).
Regarding claim 3, Pronovost as modified by Moed and Tamilarasan teaches the method of claim 1, wherein determining the plurality of road user information further comprises:
detecting a plurality of road users (Pronovost col. 8 last paragraph - col. 9 lines 35 disclosing the detected construction vehicle, a disabled vehicle);
detecting a plurality of roadside users (Pronovost col. 8 last paragraph - col. 9 lines 35 disclosing the detected person on the shoulder. Col. 20 lines 18-40 disclosing detecting any number of objects and features within environment);
determining a plurality of predicted paths of each of the plurality of road users and the plurality of roadside users (Pronovost col. 22 line 44-60 disclosing predicting the paths of multiple road users); and
determining the plurality of road user information, wherein the plurality of road user information includes at least a (Pronovost col. 8 last paragraph – col 10 discloses the information including users on road and shoulder. col. 22 line 44-60 disclosing plurality of object path detection).
Pronovost as modified by Moed and Tamilarasan does not teach plurality of predicted paths the plurality of roadside users. Quantity of road users, quantity of roadside user.
Asakura teaches Quantity of road users ([0059] disclosing estimating the traffic signal based on the quantity of road users).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed and Tamilarasan to incorporate the teaching of Asakura of quantity of road users in order to determine a state of a signal as taught by Asakura [0059], determining a sign of a signal is a predicted sign. It is also obvious to combine the teaching of Asakura with the prediction of Pronovost to predict a state of a light indicative of a traffic sign yielding predictable results and improving safety.
Pronovost as modified by Moed and Tamilarasan and Asakura does not teach plurality of predicted paths the plurality of roadside users. quantity of roadside user.
St. Romain teaches plurality of predicted paths the plurality of roadside users. quantity of roadside user ([0053] disclosing monitoring activity of other vehicles and determining that the other vehicles trajectory is driving on the shoulder and determine the number of vehicles driving on the shoulder. At least [0066] disclosing tracking the trajectory of other vehicles, the tracking of trajectories is interested as predicting trajectory since the tracking has to be done by sensors to predict a trajectory behavior).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed and Tamilarasan and Asakura to incorporate the teaching of St. Romain of plurality of predicted paths the plurality of roadside users. quantity of roadside user in order to determine a construction zone based on the behavior of a number of vehicles driving on the shoulder as taught by St. Romain [0053]. It would have been obvious to try to use the method of determining the work zone using the number of vehicles and predicted paths as a solution to predict the sign relating to a work zone with reasonable expectation of success. The combination is also obvious to predict a traffic sign relating to a work zone yielding predictable results.
Claims 7 are rejected under 35 U.S.C. 103 as being unpatentable by Pronovost (US12012108) in view of in view of in view of Moed (US20090074249) and Tamilarasan (US20220176957) and Beaurepaire (US20240044661).
Regarding claim 7, Pronovost as modified by Moed and Tamilarasan teaches the method of claim 1. Pronovost as modified by Moed and Tamilarasan does not explicitly teach wherein determining the predicted traffic sign further comprises: determining the predicted traffic sign using a machine learning algorithm, wherein the machine learning algorithm takes the plurality of roadway characteristics, the plurality of road user information, and the plurality of weather-related information as inputs and produces a plurality of possible predicted traffic signs and a corresponding plurality of confidence values as output; and determining the predicted traffic sign to be one of the plurality of possible predicted traffic signs based at least in part on the corresponding plurality of confidence values. However, Pronovost teaches predicting the algorithm to detect a sign based on the weather, traffic condition and road user information (col. 8- 10 disclosing he heuristics including a table to take an input and mapping rules to determine features “signs” to add to map).
Pronovost as modified by Moed and Tamilarasan does not teach determining the predicted traffic sign using a machine learning algorithm to output the sign; plurality of possible predicted traffic signs and corresponding plurality of confidence level and determining the predicted traffic sign to be one of the plurality of possible predicted traffic signs based at least in part on the corresponding plurality of confidence values.
Moed further teaches outputting plurality of possible predicted traffic signs and corresponding plurality of confidence level and determining the predicted traffic sign to be one of the plurality of possible predicted traffic signs based at least in part on the corresponding plurality of confidence values ([0026]-[0027] disclosing a fuzzy logic that predicts plural sign candidates with confidence values and chooses the signs based on the confidence level. [0038]-[0045] disclosing the process of determining the sign with confidence).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed and Tamilarasan to incrorporate the teaching of Moed of a fuzzy logic algorithm to predict a plurality of possible predicted traffic signs and a corresponding plurality of confidence values as output; to determine traffic sign with confidence values in order to determine the sign accurately by discriminating the sign and remove a wrong sign when confidence fall below a threshold as taught by Moed which improves driving and reduces accidents. Pronovost teaches the determining of a sign using algorithm based on the weather, traffic and road user information, Moed teaches the fuzzy algorithm method to determine the sign, it would be obvious to try to use the fuzzy algorithm with the information of Pronovost which would yield predictable results. The combination is obvious to one of ordinary skill in the art yielding predictable results. Also the use of equations is a simple design choice, Since the invention failed to provide novel or unexpected results from the usage of said claimed formula, use of any mathematical means, including that of the claimed invention, would be an obvious matter of design choice within the skill of the art.
Beaurepaire teaches determining the predicted traffic sign using a machine learning algorithm to output the sign ([0073] disclosing machine learning to predict a traffic sign).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost to include a machine learning algorithm of Beaurepaire using the information of Pronovost as input to output a traffic sign, the combination would yield predictable results. The use of machine learning is an obvious improvement to the prediction results which increases safety of the vehicle.
Claims 10, 18 are rejected under 35 U.S.C. 103 as being unpatentable by Pronovost (US12012108) in view of Moed (US20090074249) and Tamilarasan (US20220176957) and Wagner (US20110261197).
Regarding claim 10, Pronovost as modified by Moed and Tamilarasan teaches teaches the method of claim 1, wherein performing the action based at least in part on the predicted traffic sign further comprises: and adjusting a path planning and control algorithm of an automated driving system based at least in part on the predicted traffic sign (Pronovost col. 29 line 30-col 30 first paragraph disclosing the control the vehicle based on the changed map based on the sign predicted wherein the execution of driving maneuver and navigation).
Pronovost as modified by Moed and Tamilarasan does not teach transmitting the predicted traffic sign to at least one of: a remote vehicle and a remote server;
Wagner teaches transmitting the predicted traffic sign to at least one of: a remote vehicle and a remote server ([0029] disclosing transmitting the information of a detected traffic sign to another vehicle).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost to incorporate the teaching of Wagner of transmitting the predicted traffic sign to at least one of: a remote vehicle and a remote server in order to inform other vehicles of a information of a road sign as taught by Wagner [0041] which improves driving safety.
Regarding claim 18, Pronovost teaches a system for traffic sign prediction for a vehicle, the system comprising:
A plurality of vehicle sensors, wherein the plurality of vehicle sensors include at least one perception sensor, and wherein the plurality of vehicle sensors further include at least vehicle communication system (Col. 2 last paragraph disclosing the vehicle sensors to detect the environment data. Col 25 disclosing vehicle communication system);
An automated driving system (col. 29 line 30-col 30 first paragraph disclosing the control the vehicle based on the changed map based on the sign predicted wherein the execution of driving maneuver and navigation by an automated driving);
A controller in electrical communication with the plurality of vehicle sensors, wherein the controller is programmed to (at least col.26 first paragraph disclosing a controller connected to the sensors):
determining a plurality of roadway characteristics using at least one of the plurality of vehicle sensors, wherein the plurality of roadway characteristics includes at least one roadway geometry characteristic, at least one roadway condition characteristic and at least one roadside environment characteristic (col 9 lines 23-35 disclosing determining road conditions, traffic conditions, driving environment, city or rural. Col. 2 last paragraph disclosing the vehicle sensors to detect the environment data. Col. 6 line 23-35 disclosing the infrastructure is a remote server. col. 8 last paragraph - col. 10 lines 17 disclosing the detected lane markings and intersection “roadway geometry”, accidents “roadway conditions”, vehicles on shoulder “roadside environmental characteristic);
determining a plurality of road user information using at least one of the plurality of vehicle sensors (col. 6 line 37 – col 7 line 35 disclosing detecting road users. Col. 2 last paragraph disclosing the vehicle sensors to detect the environment data);
determining a plurality of weather-related information using at least one of the plurality of vehicle sensors, wherein the plurality of weather related information includes at least one precipitation weather condition and at least one visibility weather condition (col 9 lines 23-35 disclosing plurality of weather conditions such as fog “visibility condition” and rain “precipitation”. Col. 2 last paragraph disclosing the vehicle sensors to detect the environment data. col. 21 lines 39-62 disclosing perception sensors detect weather condition);
determining a predicted traffic sign based at least in part on the plurality of roadway characteristics, the plurality of road user information, and the plurality of weather-related information (col. 8 last paragraph - col. 9 lines 35 disclosing predicting a stop sign based on heuristics including plurality of road user information, plurality of weather information and roadway conditions such as lane markings, accidents, traffic conditions);
Control a path of the vehicle by adjusting a path planning and control algorithm of an automated driving system based at least in part on the predicted traffic sign (col. 29 line 30-col 30 first paragraph disclosing the control the vehicle based on the changed map based on the sign predicted wherein the execution of driving maneuver and navigation).
Pronovost does not teach transmitting the predicted traffic sign to at least one of: a remote vehicle and a remote server;
Wagner teaches transmitting the predicted traffic sign to at least one of: a remote vehicle and a remote server ([0029] disclosing transmitting the information of a detected traffic sign to another vehicle).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost to incorporate the teaching of Wagner of transmitting the predicted traffic sign to at least one of: a remote vehicle and a remote server in order to inform other vehicles of a information of a road sign as taught by Wagner [0041] which improves driving safety.
Pronovost does not teach evaluating the predicted sign with a predetermined ruleset, wherein the predetermined ruleset includes rules from traffic laws, rules, or standards; Modifying the predicted traffic sign in response to determining that the predicted traffic sign is contrary to the predetermined ruleset;
Tamilarasan teaches evaluating the predicted sign with a predetermined ruleset, wherein the predetermined ruleset includes rules from traffic laws, rules, or standards ([0020], [0024]-[0025] disclosing the evaluation of the camera detected direct speed limit sign by comparing to indirect speed limit data from context information such as regional speed limit policies “standard, or traffic laws or rules”);
Modifying the predicted traffic sign in response to determining that the predicted traffic sign is contrary to the predetermined ruleset [0035]-[0037] disclosing the regional speed limit rule and determining based on the verification that the speed limit read from a sign by the camera is an error, thus ignoring the predicted speed limit sign and controlling the speed limit based on the determined speed. The claim does not require a real sign to be seen or detected, the modification of the speed detected by the sign to a speed of the road indicated by the GPS/map is interpreted as modifying the predicted traffic sign. A sign is also interpretable as a non physical sign such as indication of a traffic pattern such as speed or stopping);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed to incorporate the teaching of Tamilarasan in order to improve safety by verifying a stop sign as taught by Tamilarasan [0002], [0020]. While Tamilarasan does not explicitly state the predicted sign, it is obvious to combine the evaluation method with the predicted sign in order to verify the predicted sign and improve prediction accuracy which also improves driving safety.
While it is believed that Tamilarasan teaches the modification of the sign, Moed is further cited to teach the limitation.
Moed teaches modifying a predicted sign ([0038]-[0045] disclosing the modifying the prediction by removing some of the predicted signs based on further verification of signs).
it would have been obvious to one of ordinary skill in the art to have modified the teaching of Pronovost as modified by Tamilarasan to incorporate the teaching of Moed of modifying a predicted sign in order to accurately determine a sign and removing wrong signs that fall below a threshold as taught by Moed thus improving driving safety and avoiding wrong vehicle decisions and improving safety.
Claims 14 are rejected under 35 U.S.C. 103 as being unpatentable by Pronovost (US12012108) in view of Moed (US20090074249) and Asakura (US20170010612) and St. Romain (US20210396528).
Regarding claim 14, Pronovost as modified by Moed teaches the system of claim 11, wherein to determine the plurality of road user information the controller is further programmed to:
detecting a plurality of road users (Pronovost col. 8 last paragraph - col. 9 lines 35 disclosing the detected construction vehicle, a disabled vehicle);
detecting a plurality of roadside users (Pronovost col. 8 last paragraph - col. 9 lines 35 disclosing the detected person on the shoulder. Col. 20 lines 18-40 disclosing detecting any number of objects and features within environment);
determining a plurality of predicted paths of each of the plurality of road users and the plurality of roadside users (Pronovost col. 22 line 44-60 disclosing predicting the paths of multiple road users); and
determining the plurality of road user information, wherein the plurality of road user information includes at least a (Pronovost col. 8 last paragraph – col 10 discloses the information including users on road and shoulder. col. 22 line 44-60 disclosing plurality of object path detection).
Pronovost as modified by Moed does not teach plurality of predicted paths the plurality of roadside users. Quantity of road users, quantity of roadside user.
Asakura teaches Quantity of road users ([0059] disclosing estimating the traffic signal based on the quantity of road users).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed to incorporate the teaching of Asakura of quantity of road users in order to determine a state of a signal as taught by Asakura [0059], determining a sign of a signal is a predicted sign. It is also obvious to combine the teaching of Asakura with the prediction of Pronovost to predict a state of a light indicative of a traffic sign yielding predictable results and improving safety.
Pronovost as modified by Moed and Asakura does not teach plurality of predicted paths the plurality of roadside users. quantity of roadside user.
St. Romain teaches plurality of predicted paths the plurality of roadside users. quantity of roadside user ([0053] disclosing monitoring activity of other vehicles and determining that the other vehicles trajectory is driving on the shoulder and determine the number of vehicles driving on the shoulder. At least [0066] disclosing tracking the trajectory of other vehicles, the tracking of trajectories is interested as predicting trajectory since the tracking has to be done by sensors to predict a trajectory behavior).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed and Asakura to incorporate the teaching of St. Romain of plurality of predicted paths the plurality of roadside users. quantity of roadside user in order to determine a construction zone based on the behavior of a number of vehicles driving on the shoulder as taught by St. Romain [0053]. It would have been obvious to try to use the method of determining the work zone using the number of vehicles and predicted paths as a solution to predict the sign relating to a work zone with reasonable expectation of success. The combination is also obvious to predict a traffic sign relating to a work zone yielding predictable results.
Claims 17 are rejected under 35 U.S.C. 103 as being unpatentable by Pronovost (US12012108) in view of Moed (US20090074249) and Wagner (US20110261197).
Regarding claim 17, Pronovost as modified by Moed teaches the system of claim 11, comprising an automated driving system in electrical communication with the controller, wherein the plurality of vehicle sensors include vehicle communication system and the controller is further programmed to: (Pronovost col. 20 and at least col.26 first paragraph disclosing a controller connected to the sensors).
Pronovost as modified by Moed does not teach transmitting the predicted traffic sign to at least one of: a remote vehicle and a remote server;
Wagner teaches transmitting the predicted traffic sign to at least one of: a remote vehicle and a remote server ([0029] disclosing transmitting the information of a detected traffic sign to another vehicle).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed to incorporate the teaching of Wagner of transmitting the predicted traffic sign to at least one of: a remote vehicle and a remote server in order to inform other vehicles of a information of a road sign as taught by Wagner [0041] which improves driving safety.
Claims 19 are rejected under 35 U.S.C. 103 as being unpatentable by Pronovost (US12012108) in view of Moed (US20090074249) and Wagner (US20110261197), Asakura (US20170010612) and St. Romain (US20210396528).
Regarding claim 19, Pronovost as modified by Moed and Wagner teaches the system of claim 18, the controller is further programmed to (Pronovost co. 26 disclosing the controller):
detecting a plurality of road users using the plurality of vehicle sensors (Pronovost col. 8 last paragraph - col. 9 lines 35 disclosing the detected construction vehicle, a disabled vehicle. col.2 last paragraph disclosing detecting road users using sensors);
detecting a plurality of roadside users using the plurality of vehicle sensors (Pronovost col. 8 last paragraph - col. 9 lines 35 disclosing the detected person on the shoulder. Col. 20 lines 18-40 disclosing detecting any number of objects and features within environment. col.2 last paragraph disclosing detecting road users using sensors);
determining a plurality of predicted paths of each of the plurality of road users and the plurality of roadside users (Pronovost col. 22 line 44-60 disclosing predicting the paths of multiple road users); and
determining the plurality of road user information, wherein the plurality of road user information includes at least a (Pronovost col. 8 last paragraph – col 10 discloses the information including users on road and shoulder. col. 22 line 44-60 disclosing plurality of object path detection).
Pronovost as modified by Moed and Wagner does not teach plurality of predicted paths the plurality of roadside users. Quantity of road users, quantity of roadside user.
Asakura teaches Quantity of road users ([0059] disclosing estimating the traffic signal based on the quantity of road users).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed and Wagner to incorporate the teaching of Asakura of quantity of road users in order to determine a state of a signal as taught by Asakura [0059]. determining a sign of a signal is a predicted sign. It is also obvious to combine the teaching of Asakura with the prediction of Pronovost to predict a state of a light indicative of a traffic sign yielding predictable results and improving safety.
Pronovost as modified by Moed and Wagner and Asakura does not teach plurality of predicted paths the plurality of roadside users. quantity of roadside user.
St. Romain teaches plurality of predicted paths the plurality of roadside users. quantity of roadside user ([0053] disclosing monitoring activity of other vehicles and determining that the other vehicles trajectory is driving on the shoulder and determine the number of vehicles driving on the shoulder. At least [0066] disclosing tracking the trajectory of other vehicles, the tracking of trajectories is interested as predicting trajectory since the tracking has to be done by sensors to predict a trajectory behavior).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Moed and Wagner and Asakura to incorporate the teaching of St. Romain of plurality of predicted paths the plurality of roadside users. quantity of roadside user in order to determine a construction zone based on the behavior of a number of vehicles driving on the shoulder as taught by St. Romain [0053]. It would have been obvious to try to use the method of determining the work zone using the number of vehicles and predicted paths as a solution to predict the sign relating to a work zone with reasonable expectation of success. The combination is also obvious to predict a traffic sign relating to a work zone yielding predictable results.
Claim 20 are rejected under 35 U.S.C. 103 as being unpatentable by Pronovost (US12012108) in view of Moed (US20090074249) and Wagner (US20110261197), Asakura (US20170010612) and St. Romain (US20210396528) and Moed (US20090074249).
Regarding claim 20, Pronovost as modified by Moed and Wagner and Asakura and St. Romain teaches the system of claim 19, wherein to determine the predicted traffic sign, the controller is further programmed to: determine the predicted traffic sign using a prediction algorithm, wherein the prediction algorithm takes the plurality of roadway characteristics, the plurality of road user information, and the plurality of weather-related information as inputs and produce possible sign prediction (Pronovost col. 8- 10 disclosing the heuristics including a table to take an input and mapping rules to determine features “signs” to add to map, the information including the plurality of road users, weather and roadway characteristics and predicts a feature of the road “traffic sign” such as a stop sign).
Pronovost as modified by Moed and Wagner and Asakura and St. Romain further teach produces a plurality of possible predicted traffic signs and a corresponding plurality of confidence values as output, and wherein the prediction algorithm is at least one of: a rule-based algorithm, a fuzzy logic algorithm, and a machine learning algorithm; determine the predicted traffic sign to be one of the plurality of possible predicted traffic signs based at least in part on the corresponding plurality of confidence values; validate the predicted traffic sign by evaluating the predicted traffic sign with a predetermined ruleset; and modify the predicted traffic sign in response to determining that the predicted traffic sign is contrary to the predetermined ruleset.
Moed further teaches produces a plurality of possible predicted traffic signs and a corresponding plurality of confidence values as output, and wherein the prediction algorithm is at least one of: a rule-based algorithm, a fuzzy logic algorithm, and a machine learning algorithm; determine the predicted traffic sign to be one of the plurality of possible predicted traffic signs based at least in part on the corresponding plurality of confidence values ([0026]-[0027] disclosing a fuzzy logic that predicts plural sign candidates with confidence values and chooses the signs based on the confidence level. [0038]-[0045] disclosing the process of determining the sign with confidence).
validate the predicted traffic sign by evaluating the predicted traffic sign with a predetermined ruleset [0038]-[0045] disclosing after determining the predicted traffic signs, further discriminating the signs against another colored image and filtering out the signs based on a rule of matching a predetermined set of signs).
modify the predicted traffic sign in response to determining that the predicted traffic sign is contrary to the predetermined ruleset ([0038]-[0045] disclosing modifying the prediction by removing some of the predicted signs based on the further verification of the sign).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teaching of Pronovost as modified by Wagner and Asakura and St. Romain to incrorporate the teaching of Moed of a fuzzy logic algorithm to predict a plurality of possible predicted traffic signs and a corresponding plurality of confidence values as output; to determine traffic sign with confidence values, validate the predicted traffic sign by evaluating the predicted traffic sign with a predetermined ruleset and modify the predicted traffic sign in response to determining that the predicted traffic sign is contrary to the predetermined ruleset in order to determine the sign accurately by discriminating the sign and remove a wrong sign when confidence fall below a threshold as taught by Moed which improves driving and reduces accidents. Pronovost teaches the determining of a sign using algorithm based on the weather, traffic and road user information, Moed teaches the fuzzy algorithm method to determine the sign, it would be obvious to use the fuzzy algorithm with the information of Pronovost which would yield predictable results. The invention fails to provide a novel outcome of the equation, the use of equations is an obvious design choice.
Response to Arguments
Applicant’s arguments filed on 05/27/2025 have been fully considered but they are not fully persuasive.
Applicant’s arguments regarding the 101 rejection have been considered and the 101 rejection withdrawn based on the amended claims reciting control of the vehicle.
Applicant’s arguments regarding the 102/103 rejections are mute since they are directed towards newly added matter of validation of the predicted signs based on the predetermined rules being traffic laws, rules or standards.
In response to arguments that are directed towards claims 6, 8, 9 which are not just towards the new matter: In short, Moed already teaches validating a set of predicted signs and modifying the prediction based on rules, the newly cited reference Tamilarasan teaches validating and modifying a predicted speed sign by using standards and rules and laws of the region. Moed also already teaches the modifying of the sign based on rules, so combining the technique used by Tamilarasan or substituting the validation method of Tamilarasan would solve the same problem of validating a sign and is obvious yielding predictable results. same applies to the argument of claim 6, Pronovost teaches the algorithm using all the information indicated by the claim, Moed teaches a different method “fuzzy algorithm” which is considered an equation that is an obvious design choice outputting the same outcome, the fuzzy algorithm can substitute the algorithm of Pronovost yielding predictable results. Regarding claims 8-9, the use of the fuzzy algorithm to predict sign is a principle of the invention of Moed, thus Moed in itself is not altered, Moed is cited to state that fuzzy algorithm is known in the detection of sign, thus it is obvious to try the method of fuzzy algorithm with reasonable expectation of success, fuzzy algorithm is also based on equations which is obvious design choice. The method of Moed is interpreted as fuzzy algorithm, a fuzzy algorithm compares the input to candidates and gives multiple candidates with corresponding confidences for each candidate instead of giving one simple output.
With respect to arguments regarding claim 7, Beaurepaire is not cited to teach the fuzzy algorithm, but to teach machine learning algorithm for the sign detection.
With respect to arguments regarding claim 10, Wagner is not cited to teach the fuzzy algorithm, but to teach transmitting information.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to
applicant's disclosure. The prior art cited in PTO-892 and not mentioned above disclose related devices and methods.
US20190325736 discloses validating a road sign.
US20090070031 discloses detecting a red light by the stopping and go of the own vehicle but validates it with other vehicles detection.
US20170278398 discloses detecting absence and presence of road sign.
US20210104024 discloses visibility of sign affected by smoke or fog.
US20220073106 disclosing detecting stop sign by other vehicles stopping.
US20200074851 discloses detecting light of orthogonal traffic based on other vehicle’s movement.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMAD O EL SAYAH whose telephone number is (571)270-7734. The examiner can normally be reached on M-Th 6:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramon Mercado can be reached on (571) 270-5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOHAMAD O EL SAYAH/Examiner, Art Unit 3658A
Prosecution Insights