DETAILED ACTION
Notice of Pre-AIA or AIA Status
Claims 1-20 are pending in this application. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 and 12 are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Schaefer et al. (US PGPub US 2024/0166204 A1, filed November 17, 2023, hereby referred to as “Schaefer”).
Consider Claims 1 and 12.
Schaefer teaches:
1. A method for detecting one or more approaching vehicles comprising: / 12. A system for detecting one or more approaching vehicles, the system comprising: (Schaefer: abstract, A computer-implemented method for collision threat assessment of a vehicle includes obtaining context information for the surrounding of the vehicle, including information about at least one road user. The method includes determining ego occupancy information for multiple possible future locations of the vehicle at multiple future points in time based on the context information. The method includes determining road user occupancy information for multiple possible future locations of the at least one road user at multiple future points in time based on the context information. The method includes fusing the ego occupancy information and the road user occupancy information to obtain fused occupancy information at each future point in time. The method includes determining a collision threat value based on the fused occupancy information., [0005] The complexity of a traffic scene depends on a number of aspects. For example, complex road and lane structures (different intersections, round-abouts, multi-lane roads, etc.), various numbers and types of the road users (vehicles, pedestrians, cyclist, etc.) may be in the same scene and they may interact with each other and thus affect their future behaviors. Different dynamic movements of the road users are possible (e.g. straight driving, turning or stopped vehicles, highly dynamic pedestrian maneuverability, etc.). The future maneuvers of road users are unknown and multiple future maneuvers are often feasible and possible. Thus, anticipating other road users' movements in the future (e.g. a few seconds) does not only depend on their current dynamic, but also on the whole driving context around them. This normally results in multiple possible and feasible maneuvers which requires a multi-modality prediction. [0008]-[0012], Figures 1)
1. transforming one or more images of surroundings of an ego vehicle captured by an infrastructure camera to a bird’s eye view of the surroundings, / 12. an infrastructure camera operable to capture one or more images of surroundings of an ego vehicle, (Schaefer: [0008] The present invention addresses the future collision threat assessment by a computer-implemented method according to claim 1, a computer system according to claim 13, a vehicle according to claim 14 and a computer program according to claim 15. [0009] The method according to the invention comprises the steps: obtaining context information for the surrounding of the vehicle comprising information about at least one road user; determining ego occupancy information for a plurality of possible future locations of the vehicle at a plurality of future points in time based on the context information; determining road user occupancy information for a plurality of possible future locations of the at least one road user at the plurality of future points in time based on the context information; fusing the ego occupancy information and the road user occupancy information to obtain fused occupancy information at each future point in time; and determining a collision threat value based on the fused occupancy information.)
1.wherein the surroundings include a ramp, one or more lanes of a freeway, and the one or more approaching vehicles, and the ramp and the lanes share at least one merging point; / 12. wherein the surroundings include a ramp, one or more lanes of a freeway, and the one or more approaching vehicles, and the ramp and the lanes share at least one merging point; (Schaefer: [0005] For example, complex road and lane structures (different intersections, round-abouts, multi-lane roads, etc.), various numbers and types of the road users (vehicles, pedestrians, cyclist, etc.) may be in the same scene and they may interact with each other and thus affect their future behaviors. Different dynamic movements of the road users are possible (e.g. straight driving, turning or stopped vehicles, highly dynamic pedestrian maneuverability, etc.), [0015], [0040] An example of how the occupancy information can be estimated in steps 120 and 130 is illustrated in FIG. 2 which depicts a traffic scene at an intersection in bird-eye perspective. FIG. 2 shows the past trajectories 201 a and 202 a of two vehicles and of a pedestrian 203 a. More precisely, each past trajectory is sampled at three discrete points in time, wherein the squared dots represent vehicle trajectories and crosses represent the pedestrian trajectory. In an actual application, the past trajectories or discrete samples thereof could be obtained by onboard sensors of the ego vehicle. FIG. 2 also depicts the future trajectories 201 b and 202 b of the vehicles and of the pedestrian 203 b. The future trajectories are those that the vehicles and pedestrians had actually followed in that particular scene and are also denoted as the ground truths. [0041]-[0042], Figures 1-2)
1.generating a static segmentation map and one or more dynamic objects in the static segmentation map based on the bird’s eye view; / 12. one or more processors operable to: transform the one or more images of surroundings to a bird’s eye view of the surroundings; generate a static segmentation map and one or more dynamic objects in the static segmentation map based on the bird’s eye view; (Schaefer: [0045] The method according to the invention as illustrated in FIG. 1 may also some comprise optional steps which will now be described in more detail. In optional step 115 the context information which had been obtained in step 110 is filtered, i.e. a part of the context information is intentionally not provided to steps 120 and 130 which estimate the occupancy information of the ego vehicle and of at least one other road user. The filtering may affect the dynamic context information and/or the static context information. Dynamic context information represents information about the at least one road user such as its position and velocity. Static context information represents information about the environment of the vehicle which may restricts road users' movements including movements of the ego vehicle. Examples include roads, lanes and intersections. In the following, different examples of filtering context information are described. [0047] In a second example, no dynamic context information is provided for the estimation of the ego vehicle's occupancy information. Instead, planned maneuver information of the ego vehicle is provided for the estimation. Planned maneuver information may be provided by the vehicle's direction indicator corresponding to the intent of the driver to turn left or right. [0048] This is illustrated in more detail in FIGS. 3A-C which show an ego vehicle 301 at an intersection. The ego vehicle has the possibility to drive straight ahead or to turn right. Both options are possible and feasible at the time frame shown in FIGS. 3A-C. Thus, without knowing the intention of the vehicle or its driver, a multi-modal prediction results as shown in FIG. 3A in which the occupancy information indicates that driving straight ahead 302 is about as possible as turning right 303. [0049] In FIG. 3B, trajectory information is provided to the prediction. In this case, the vehicle or its driver intend to drive straight ahead and the estimated occupancy information indicates that there will most probably not be a right turn. In contrast, in FIG. 3C the vehicle or its driver intend to make a right turn and the estimated occupancy information indicates that the vehicle will most probably not drive straight ahead.)
1.predicting a lane occupancy of the ramp and the lanes based on range estimation of the ego vehicle and the one or more dynamic objects; / 12. predict a lane occupancy of the ramp and the lanes based on range estimation of the ego vehicle and the one or more dynamic objects; (Schaefer: [0050] The effects of considering trajectory information for the ego vehicle for the collision estimation is demonstrated in more detail in FIG. 4 which relates to the same traffic scene as depicted in FIGS. 3A-C. In FIG. 4 the estimated occupancy information for the ego vehicle 301 is shown taking into account the ego vehicle's intention (or of its driver) to turn right. Accordingly, the occupancy information is highest for the right turn 303. In FIG. 4 the estimation of the occupancy information for a pedestrian 401 is also shown who is just about crossing the street near the intersection (e.g. at a crosswalk). Indeed, the occupancy information indicates that the pedestrian 401 will likely be located in the middle of the street within the next few seconds. Fusing the occupancy information for the ego vehicle shown in FIG. A with the occupancy information for the pedestrian shown in FIG. 4 indicates that there is some risk of collision at grid point 402. In this case a warning could be provided to the driver or automatic braking could be started. Notably, if the intention of the vehicle or its driver had been to drive straight ahead instead of making a right turn, the estimated collision risk would have been much lower (compare FIG. 4 with FIG. 3B). [0051] In the first and second example of how filtering context information may be used, the whole dynamic context information is not provided for the estimation of the ego vehicle's occupancy information but only the static context is provided. One could have more detailed control over the context information by integrating with onboard sensing features. As an example, driver's gaze detection could help with filtering out the road users which may not have been seen by the driver or restrict certain static context information for the ego prediction. This could help imitating what the ego driver may see (or may not see), thus also anticipating the ego vehicle's driver future behavior, which eventually could help with more accurate threat assessment. For example, if a driver properly saw a jaywalking pedestrian, then the AEB function may be more fine-tuned to avoid unnecessary warning or braking and, at the same time, would provide sufficient safety.)
1.and transmitting information about the predicted lane occupancy to the ego vehicle./ 12. and transmit information about the predicted lane occupancy to the ego vehicle. (Schaefer: [0039] If it is determined in step 150 that there is a risk of collision of the ego vehicle with some other road user based on the collision threat value, a collision warning may be provided. The collision warning may by output to a driver and/or used to control functions of the ego vehicle. For example, in response to a collision warning indication, an automatic braking may be engaged. Generally, the collision warning indication may be provided to an ADAS (Advanced Driver-Assistance System) function. In another example, the collision warning indication may be uploaded to a central server, for example for later assessment and analysis. The warning may be transmitted via cellular radio. [0050]-[0051] In the first and second example of how filtering context information may be used, the whole dynamic context information is not provided for the estimation of the ego vehicle's occupancy information but only the static context is provided. One could have more detailed control over the context information by integrating with onboard sensing features. As an example, driver's gaze detection could help with filtering out the road users which may not have been seen by the driver or restrict certain static context information for the ego prediction. This could help imitating what the ego driver may see (or may not see), thus also anticipating the ego vehicle's driver future behavior, which eventually could help with more accurate threat assessment. For example, if a driver properly saw a jaywalking pedestrian, then the AEB function may be more fine-tuned to avoid unnecessary warning or braking and, at the same time, would still be able to provide sufficient safety.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Schaefer et al. (US PGPub US 2024/0166204 A1, filed November 17, 2023, hereby referred to as “Schaefer”), in view of Epstein et al. (US PGPub US2023/0294742A1, filed March 17, 2022), hereby referred to as “Epstein”.
Consider Claims 1 and 12.
Schaefer teaches:
1. A method for detecting one or more approaching vehicles comprising: / 12. A system for detecting one or more approaching vehicles, the system comprising: (Schaefer: abstract, A computer-implemented method for collision threat assessment of a vehicle includes obtaining context information for the surrounding of the vehicle, including information about at least one road user. The method includes determining ego occupancy information for multiple possible future locations of the vehicle at multiple future points in time based on the context information. The method includes determining road user occupancy information for multiple possible future locations of the at least one road user at multiple future points in time based on the context information. The method includes fusing the ego occupancy information and the road user occupancy information to obtain fused occupancy information at each future point in time. The method includes determining a collision threat value based on the fused occupancy information., [0005] The complexity of a traffic scene depends on a number of aspects. For example, complex road and lane structures (different intersections, round-abouts, multi-lane roads, etc.), various numbers and types of the road users (vehicles, pedestrians, cyclist, etc.) may be in the same scene and they may interact with each other and thus affect their future behaviors. Different dynamic movements of the road users are possible (e.g. straight driving, turning or stopped vehicles, highly dynamic pedestrian maneuverability, etc.). The future maneuvers of road users are unknown and multiple future maneuvers are often feasible and possible. Thus, anticipating other road users' movements in the future (e.g. a few seconds) does not only depend on their current dynamic, but also on the whole driving context around them. This normally results in multiple possible and feasible maneuvers which requires a multi-modality prediction. [0008]-[0012], Figures 1)
1. transforming one or more images of surroundings of an ego vehicle captured by an infrastructure camera to a bird’s eye view of the surroundings, / 12. an infrastructure camera operable to capture one or more images of surroundings of an ego vehicle, (Schaefer: [0008] The present invention addresses the future collision threat assessment by a computer-implemented method according to claim 1, a computer system according to claim 13, a vehicle according to claim 14 and a computer program according to claim 15. [0009] The method according to the invention comprises the steps: obtaining context information for the surrounding of the vehicle comprising information about at least one road user; determining ego occupancy information for a plurality of possible future locations of the vehicle at a plurality of future points in time based on the context information; determining road user occupancy information for a plurality of possible future locations of the at least one road user at the plurality of future points in time based on the context information; fusing the ego occupancy information and the road user occupancy information to obtain fused occupancy information at each future point in time; and determining a collision threat value based on the fused occupancy information.)
1.wherein the surroundings include a ramp, one or more lanes of a freeway, and the one or more approaching vehicles, and the ramp and the lanes share at least one merging point; / 12. wherein the surroundings include a ramp, one or more lanes of a freeway, and the one or more approaching vehicles, and the ramp and the lanes share at least one merging point; (Schaefer: [0005] For example, complex road and lane structures (different intersections, round-abouts, multi-lane roads, etc.), various numbers and types of the road users (vehicles, pedestrians, cyclist, etc.) may be in the same scene and they may interact with each other and thus affect their future behaviors. Different dynamic movements of the road users are possible (e.g. straight driving, turning or stopped vehicles, highly dynamic pedestrian maneuverability, etc.), [0015], [0040] An example of how the occupancy information can be estimated in steps 120 and 130 is illustrated in FIG. 2 which depicts a traffic scene at an intersection in bird-eye perspective. FIG. 2 shows the past trajectories 201 a and 202 a of two vehicles and of a pedestrian 203 a. More precisely, each past trajectory is sampled at three discrete points in time, wherein the squared dots represent vehicle trajectories and crosses represent the pedestrian trajectory. In an actual application, the past trajectories or discrete samples thereof could be obtained by onboard sensors of the ego vehicle. FIG. 2 also depicts the future trajectories 201 b and 202 b of the vehicles and of the pedestrian 203 b. The future trajectories are those that the vehicles and pedestrians had actually followed in that particular scene and are also denoted as the ground truths. [0041]-[0042], Figures 1-2)
1.generating a static segmentation map and one or more dynamic objects in the static segmentation map based on the bird’s eye view; / 12. one or more processors operable to: transform the one or more images of surroundings to a bird’s eye view of the surroundings; generate a static segmentation map and one or more dynamic objects in the static segmentation map based on the bird’s eye view; (Schaefer: [0045] The method according to the invention as illustrated in FIG. 1 may also some comprise optional steps which will now be described in more detail. In optional step 115 the context information which had been obtained in step 110 is filtered, i.e. a part of the context information is intentionally not provided to steps 120 and 130 which estimate the occupancy information of the ego vehicle and of at least one other road user. The filtering may affect the dynamic context information and/or the static context information. Dynamic context information represents information about the at least one road user such as its position and velocity. Static context information represents information about the environment of the vehicle which may restricts road users' movements including movements of the ego vehicle. Examples include roads, lanes and intersections. In the following, different examples of filtering context information are described. [0047] In a second example, no dynamic context information is provided for the estimation of the ego vehicle's occupancy information. Instead, planned maneuver information of the ego vehicle is provided for the estimation. Planned maneuver information may be provided by the vehicle's direction indicator corresponding to the intent of the driver to turn left or right. [0048] This is illustrated in more detail in FIGS. 3A-C which show an ego vehicle 301 at an intersection. The ego vehicle has the possibility to drive straight ahead or to turn right. Both options are possible and feasible at the time frame shown in FIGS. 3A-C. Thus, without knowing the intention of the vehicle or its driver, a multi-modal prediction results as shown in FIG. 3A in which the occupancy information indicates that driving straight ahead 302 is about as possible as turning right 303. [0049] In FIG. 3B, trajectory information is provided to the prediction. In this case, the vehicle or its driver intend to drive straight ahead and the estimated occupancy information indicates that there will most probably not be a right turn. In contrast, in FIG. 3C the vehicle or its driver intend to make a right turn and the estimated occupancy information indicates that the vehicle will most probably not drive straight ahead.)
1.predicting a lane occupancy of the ramp and the lanes based on range estimation of the ego vehicle and the one or more dynamic objects; / 12. predict a lane occupancy of the ramp and the lanes based on range estimation of the ego vehicle and the one or more dynamic objects; (Schaefer: [0050] The effects of considering trajectory information for the ego vehicle for the collision estimation is demonstrated in more detail in FIG. 4 which relates to the same traffic scene as depicted in FIGS. 3A-C. In FIG. 4 the estimated occupancy information for the ego vehicle 301 is shown taking into account the ego vehicle's intention (or of its driver) to turn right. Accordingly, the occupancy information is highest for the right turn 303. In FIG. 4 the estimation of the occupancy information for a pedestrian 401 is also shown who is just about crossing the street near the intersection (e.g. at a crosswalk). Indeed, the occupancy information indicates that the pedestrian 401 will likely be located in the middle of the street within the next few seconds. Fusing the occupancy information for the ego vehicle shown in FIG. A with the occupancy information for the pedestrian shown in FIG. 4 indicates that there is some risk of collision at grid point 402. In this case a warning could be provided to the driver or automatic braking could be started. Notably, if the intention of the vehicle or its driver had been to drive straight ahead instead of making a right turn, the estimated collision risk would have been much lower (compare FIG. 4 with FIG. 3B). [0051] In the first and second example of how filtering context information may be used, the whole dynamic context information is not provided for the estimation of the ego vehicle's occupancy information but only the static context is provided. One could have more detailed control over the context information by integrating with onboard sensing features. As an example, driver's gaze detection could help with filtering out the road users which may not have been seen by the driver or restrict certain static context information for the ego prediction. This could help imitating what the ego driver may see (or may not see), thus also anticipating the ego vehicle's driver future behavior, which eventually could help with more accurate threat assessment. For example, if a driver properly saw a jaywalking pedestrian, then the AEB function may be more fine-tuned to avoid unnecessary warning or braking and, at the same time, would still be able to provide sufficient safety.)
1.and transmitting information about the predicted lane occupancy to the ego vehicle./ 12. and transmit information about the predicted lane occupancy to the ego vehicle. (Schaefer: [0039] If it is determined in step 150 that there is a risk of collision of the ego vehicle with some other road user based on the collision threat value, a collision warning may be provided. The collision warning may by output to a driver and/or used to control functions of the ego vehicle. For example, in response to a collision warning indication, an automatic braking may be engaged. Generally, the collision warning indication may be provided to an ADAS (Advanced Driver-Assistance System) function. In another example, the collision warning indication may be uploaded to a central server, for example for later assessment and analysis. The warning may be transmitted via cellular radio. [0050]-[0051] In the first and second example of how filtering context information may be used, the whole dynamic context information is not provided for the estimation of the ego vehicle's occupancy information but only the static context is provided. One could have more detailed control over the context information by integrating with onboard sensing features. As an example, driver's gaze detection could help with filtering out the road users which may not have been seen by the driver or restrict certain static context information for the ego prediction. This could help imitating what the ego driver may see (or may not see), thus also anticipating the ego vehicle's driver future behavior, which eventually could help with more accurate threat assessment. For example, if a driver properly saw a jaywalking pedestrian, then the AEB function may be more fine-tuned to avoid unnecessary warning or braking and, at the same time, would still be able to provide sufficient safety.)
Even if Schaefer does not specifically teach limitations from Claim 2 for: “a static segmentation map comprises labeled regions representing static objects and structures of roads”
Epstein teaches:
1. A method for detecting one or more approaching vehicles comprising: / 12. A system for detecting one or more approaching vehicles, the system comprising: (Epstein: abstract, A method for vehicle behavior prediction is described. The method includes detecting a vehicle entering an unmarked road segment. The method also includes determining a set of exit paths available for the detected vehicle to reach different exit lanes of the unmarked road segment. The method further includes predicting an exit path taken by the detected vehicle from the set of exit paths available for the detected vehicle to reach the different exit lanes of the unmarked road segment. The method also includes planning a trajectory of an ego vehicle according to the predicted exit path taken by the detected vehicle to reach an exit lane of the unmarked road segment. [0005]-[0008])
1. transforming one or more images of surroundings of an ego vehicle captured by an infrastructure camera to a bird’s eye view of the surroundings, / 12. an infrastructure camera operable to capture one or more images of surroundings of an ego vehicle, (Epstein:[0057]-[0063], [0057] FIG. 5 is a diagram illustrating a birds-eye-view of a roadway environment, including an intersection having several roads crisscrossing each other, according to aspects of the present disclosure. FIG. 5 illustrates an intersection 570 of a roadway environment 500 having six lanes (e.g., 511, 521, 531, 541, 551, and 561). The six lanes (e.g., 511, 521, 531, 541, 551, and 561) may correspond to a first road 510, a second road 520, a third road 530, a fourth road 540, a fifth road 550, and a sixth road 560. In addition, the first road 510 and the fourth road 540 may correspond to the same road. Similarly, the second road 520 and the fifth road 550 may correspond to the same road. Also, the third road 530 and the sixth road 560 may correspond to the same road. [0060] FIG. 6 is a diagram 600 further illustrating the birds-eye-view of the roadway environment 500 of FIG. 5 , in which an exit lane of a detected vehicle is determined, according to aspects of the present disclosure. As shown in FIG. 6 , several different paths are determined that the first detected vehicle 572 can take to each of the exit lanes (e.g., 511, 521, 531, 541, 551, and 561) as the first detected vehicle 572 enters the intersection 570. For example, a first path 610 may correspond to the exit lane 511 of the first road 510, and a second path (not shown) may correspond to the exit lane 521 of the second road 520, if the first detected vehicle 572 performs a U-turn. In addition, a third path 630 may correspond to the exit lane 531 of the third road 530, a fourth path 640 may correspond to the exit lane 541 of the fourth road 540, a fifth path 650 may correspond to the exit lane 551 of the fifth road 550, and a sixth path 660 may correspond to the exit lane 561 of the sixth road 560.)
1.wherein the surroundings include a ramp, one or more lanes of a freeway, and the one or more approaching vehicles, and the ramp and the lanes share at least one merging point; / 12. wherein the surroundings include a ramp, one or more lanes of a freeway, and the one or more approaching vehicles, and the ramp and the lanes share at least one merging point; (Epstein: [0050]-[0055], [0056] FIG. 4 is a diagram illustrating an overview of a roadway environment, including an ego vehicle in the first lane of the roadway and a detected vehicle approaching an intersection, according to aspects of the present disclosure. In this example, a roadway environment 400 includes a first roadway 410, having a first lane 412 in which an ego vehicle 420 is traveling and a second lane 414. In addition, the roadway environment 400 includes a second roadway 430, having a first lane 432 in which a detected vehicle 440 is traveling and a second lane 434. In this example, the detected vehicle 440 approaching the intersection 450 from the second roadway 430 has three choices. The detected vehicle 440 can go straight on the second roadway 430, or turn left or right onto the first roadway 410. The lane that the detected vehicle 440 travels to after crossing the intersection 450 is referred to as the “exit lane.” Detecting the exit lane used by the detected vehicle becomes more complex as the traffic environments become more complex, for example, as shown in FIG. 5 . [0057] FIG. 5 is a diagram illustrating a birds-eye-view of a roadway environment, including an intersection having several roads crisscrossing each other, according to aspects of the present disclosure. FIG. 5 illustrates an intersection 570 of a roadway environment 500 having six lanes (e.g., 511, 521, 531, 541, 551, and 561). The six lanes (e.g., 511, 521, 531, 541, 551, and 561) may correspond to a first road 510, a second road 520, a third road 530, a fourth road 540, a fifth road 550, and a sixth road 560. In addition, the first road 510 and the fourth road 540 may correspond to the same road. Similarly, the second road 520 and the fifth road 550 may correspond to the same road. Also, the third road 530 and the sixth road 560 may correspond to the same road.)
1.generating a static segmentation map and one or more dynamic objects in the static segmentation map based on the bird’s eye view; / 12. one or more processors operable to: transform the one or more images of surroundings to a bird’s eye view of the surroundings; generate a static segmentation map and one or more dynamic objects in the static segmentation map based on the bird’s eye view; (Epstein: [0055] The vehicle perception module 312 may be configured to detect a vehicle entering an unmarked road segment based on images from the sensor module 302. The vehicle paths determination module 314 may be configured to determine a set of exit paths available for the detected vehicle to reach different exit lanes of the unmarked road segment. In these aspects of the present disclosure, the vehicle path prediction module 316 is configured to predict an exit path taken by the detected vehicle from the set of exit paths available for the detected vehicle to reach the different exit lanes of the unmarked road segment. Once the exit path taken by the detected vehicle is predicted, the vehicle trajectory planner module 318 performs motion planning a trajectory of the car 350 for maneuvering to avoid the predicted path taken by a detected vehicle to reach an exit lane of the unmarked road segment (e.g., an intersection), as shown in FIG. 4 . [0056] FIG. 4 is a diagram illustrating an overview of a roadway environment, including an ego vehicle in the first lane of the roadway and a detected vehicle approaching an intersection, according to aspects of the present disclosure. In this example, a roadway environment 400 includes a first roadway 410, having a first lane 412 in which an ego vehicle 420 is traveling and a second lane 414. In addition, the roadway environment 400 includes a second roadway 430, having a first lane 432 in which a detected vehicle 440 is traveling and a second lane 434. In this example, the detected vehicle 440 approaching the intersection 450 from the second roadway 430 has three choices. The detected vehicle 440 can go straight on the second roadway 430, or turn left or right onto the first roadway 410. The lane that the detected vehicle 440 travels to after crossing the intersection 450 is referred to as the “exit lane.” Detecting the exit lane used by the detected vehicle becomes more complex as the traffic environments become more complex, for example, as shown in FIG. 5. [0060] FIG. 6 is a diagram 600 further illustrating the birds-eye-view of the roadway environment 500 of FIG. 5 , in which an exit lane of a detected vehicle is determined, according to aspects of the present disclosure. As shown in FIG. 6 , several different paths are determined that the first detected vehicle 572 can take to each of the exit lanes (e.g., 511, 521, 531, 541, 551, and 561) as the first detected vehicle 572 enters the intersection 570. For example, a first path 610 may correspond to the exit lane 511 of the first road 510, and a second path (not shown) may correspond to the exit lane 521 of the second road 520, if the first detected vehicle 572 performs a U-turn. In addition, a third path 630 may correspond to the exit lane 531 of the third road 530, a fourth path 640 may correspond to the exit lane 541 of the fourth road 540, a fifth path 650 may correspond to the exit lane 551 of the fifth road 550, and a sixth path 660 may correspond to the exit lane 561 of the sixth road 560.)
1.predicting a lane occupancy of the ramp and the lanes based on range estimation of the ego vehicle and the one or more dynamic objects; / 12. predict a lane occupancy of the ramp and the lanes based on range estimation of the ego vehicle and the one or more dynamic objects; (Epstein: [0060] FIG. 6 is a diagram 600 further illustrating the birds-eye-view of the roadway environment 500 of FIG. 5 , in which an exit lane of a detected vehicle is determined, according to aspects of the present disclosure. As shown in FIG. 6 , several different paths are determined that the first detected vehicle 572 can take to each of the exit lanes (e.g., 511, 521, 531, 541, 551, and 561) as the first detected vehicle 572 enters the intersection 570. For example, a first path 610 may correspond to the exit lane 511 of the first road 510, and a second path (not shown) may correspond to the exit lane 521 of the second road 520, if the first detected vehicle 572 performs a U-turn. In addition, a third path 630 may correspond to the exit lane 531 of the third road 530, a fourth path 640 may correspond to the exit lane 541 of the fourth road 540, a fifth path 650 may correspond to the exit lane 551 of the fifth road 550, and a sixth path 660 may correspond to the exit lane 561 of the sixth road 560. [0062] In the example shown in FIG. 6 , the spline representing the sixth path 660 is determined as the most likely path taken by the first detected vehicle 572. The other splines, associated with the first path 610, the third path 630, the fourth path 640 and the fifth path 650, are determined to be less likely. The various paths (e.g., 610, 630, 640, 650, and 660) may be determined by the vehicle paths determination module 314 of FIG. 3 . In addition, the prediction regarding the path likelihood may be performed by the vehicle path prediction module 316 based on a curvature of the splines. For example, the spline representing the sixth path 660 is shown as being relatively smooth (e.g., having a lowest curvature) and generally agrees with the vehicle’s current heading based on the past history 662 of previous locations and headings of the first detected vehicle 572. By contrast, the splines representing the other paths (e.g., 610, 630, 640, and 650) exhibit significant curvatures. By evaluating the curvatures of the splines, the vehicle path prediction module 316 can predict which spline represents the path the first detected vehicle 572 is most likely to take, according to some aspects of the present disclosure.)
1.and transmitting information about the predicted lane occupancy to the ego vehicle./ 12. and transmit information about the predicted lane occupancy to the ego vehicle. (Epstein: [0033] In this example, the transceiver 332 may receive/transmit vehicle-to-vehicle traffic state information for the vehicle behavior planner 310 to/from connected vehicles within the vicinity of the car 350. [0036] The measurements of the first sensor 306 and the second sensor 304 may be processed by the processor 320, the sensor module 302, the vehicle behavior planner 310, the onboard unit 324, the locomotion module 326, the location module 328 the communication module 330, and/or the controller module 340. In conjunction with the computer-readable medium 322, the measurements of the first sensor 306 and the second sensor 304 are processed to implement the functionality described herein. In one configuration, the data captured by the first sensor 306 and the second sensor 304 may be transmitted to a connected vehicle via the transceiver 332. The first sensor 306 and the second sensor 304 may be coupled to the car 350 or may be in communication with the car 350.)
2. The method of claim 1, wherein the static segmentation map comprises labeled regions representing static objects and structures of roads. (Epstein: [0064] FIG. 7 is a flowchart illustrating a method for vehicle behavior prediction, according to aspects of the present disclosure. The method 700 begins at block 702, in which a vehicle is detected entering an unmarked road segment. For example, in FIG. 5 , the ego vehicle 420 has entered the intersection 570 from the first road 510, and the first detected vehicle 572 is identified within the intersection 570 by the ego vehicle 420. In some aspects of the present disclosure, a vehicle perception module 312 of the vehicle behavior planning system 300 of FIG. 3 is mounted within the ego vehicle 420 having the sensor module 302 (e.g., camera/radar/sonar/LIDAR). The vehicle perception module 312 may provide object detection and tracking system functionality to determine a location and a heading of the first detected vehicle 572. [0065] At block 704, a set of exit paths available for the detected vehicle to reach different exit lanes of the unmarked road segment are determined. For example, as shown in FIG. 6 , several different paths are determined that the first detected vehicle 572 can take to each of the exit lanes (e.g., 511, 521, 531, 541, 551, and 561) as the first detected vehicle 572 enters the intersection 570. For example, a first path 610 may correspond to the exit lane 511 of the first road 510, and a second path (not shown) may correspond to the exit lane 521 of the second road 520, if the first detected vehicle 572 performed a U-turn. In addition, a third path 630 may correspond to the exit lane 531 of the third road 530, a fourth path 640 may correspond to the exit lane 541 of the fourth road 540, a fifth path 650 may correspond to the exit lane 551 of the fifth road 550, and a sixth path 660 may correspond to the exit lane 561 of the sixth road 560. [0066] At block 706, an exit path taken by the detected vehicle is predicted from the set of exit paths available for the detected vehicle to reach the different exit lanes of the unmarked road segment. For example, in FIG. 6 , the spline representing the sixth path 660 is predicted as the most likely path taken by the first detected vehicle 572. The other splines, associated with the first path 610, the third path 630, the fourth path 640 and the fifth path 650, are determined to be less likely. A prediction regarding the path likelihood may be performed by the vehicle path prediction module 316 based on a curvature of the splines. For example, the spline representing the sixth path 660 is shown as being relatively smooth (e.g., having a lowest curvature) and generally agrees with the vehicle’s current heading based on the past history 662 of previous locations and headings of the first detected vehicle 572. By contrast, the splines representing the other paths (e.g., 610, 630, 640, and 650) exhibit significant curvatures. By evaluating the curvatures of the splines, the vehicle path prediction module 316 can predict which spline represents the path the first detected vehicle 572 is most likely to take, according to some aspects of the present disclosure.)
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify Schaefer’s vehicular collision threat assessment machine learning model with the improved algorithm for lane association and transition as disclosed by Epstein as they are both directed towards the same field of endeavor for autonomous vehicular navigation using image processing and analytics. The determination of obviousness is predicated upon the following findings: One skilled in the art would have been motivated to modify Schaefer in order to improve the overall collision threat learning model with a more accurate ranked spline-based path selection algorithm as proposed by Epstein. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and programming techniques, without changing a “fundamental” operating principle of Schaefer, while the teaching of Epstein continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of .”improved path identification of detected vehicles at an intersection” (Epstein: [0030]). It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Consider Claim 2.
The combination of Schaefer and Epstein teaches:
2. The method of claim 1, wherein the static segmentation map comprises labeled regions representing static objects and structures of roads. (Schaefer: [0005], [0015] The context information may include static context information, wherein static context information represents information about the environment of the vehicle. Static context information[n may represent the environment of the vehicle which restricts road users' movements including movements of the ego vehicle. Examples include roads, lanes and intersections. By considering static context information, the possible future locations of the ego vehicle and the at least one road user can be restricted to possible locations. Impossible or improbable locations (e.g. off-road) can be excluded. In this way, the prediction becomes more accurate and false warnings are avoided or at least reduced. [0016] The static context information may be represented at least in part by map data and/or traffic rules. Map data is readily available as modern cars often comprise a route guidance system. Traffic rules like for example left- or right-hand driving, one-way roads, right of way, etc. are often also contained in the route guidance system and may be used as static context information improving the collision prediction. Epstein: [0064] FIG. 7 is a flowchart illustrating a method for vehicle behavior prediction, according to aspects of the present disclosure. The method 700 begins at block 702, in which a vehicle is detected entering an unmarked road segment. For example, in FIG. 5 , the ego vehicle 420 has entered the intersection 570 from the first road 510, and the first detected vehicle 572 is identified within the intersection 570 by the ego vehicle 420. In some aspects of the present disclosure, a vehicle perception module 312 of the vehicle behavior planning system 300 of FIG. 3 is mounted within the ego vehicle 420 having the sensor module 302 (e.g., camera/radar/sonar/LIDAR). The vehicle perception module 312 may provide object detection and tracking system functionality to determine a location and a heading of the first detected vehicle 572. [0065] At block 704, a set of exit paths available for the detected vehicle to reach different exit lanes of the unmarked road segment are determined. For example, as shown in FIG. 6 , several different paths are determined that the first detected vehicle 572 can take to each of the exit lanes (e.g., 511, 521, 531, 541, 551, and 561) as the first detected vehicle 572 enters the intersection 570. For example, a first path 610 may correspond to the exit lane 511 of the first road 510, and a second path (not shown) may correspond to the exit lane 521 of the second road 520, if the first detected vehicle 572 performed a U-turn. In addition, a third path 630 may correspond to the exit lane 531 of the third road 530, a fourth path 640 may correspond to the exit lane 541 of the fourth road 540, a fifth path 650 may correspond to the exit lane 551 of the fifth road 550, and a sixth path 660 may correspond to the exit lane 561 of the sixth road 560. [0066])
Consider Claim 3 and 13.
The combination of Schaefer and Epstein teaches:
3. The method of claim 2, wherein the static segmentation map comprises one or more background regions, one or more lane mark regions, and one or more road boundary regions./ 13. The system of claim 12, wherein the static segmentation map comprises labeled regions representing static objects and structures of roads, one or more background regions, one or more lane mark regions, and one or more road boundary regions. (Epstein: [0052] Some aspects of the present disclosure describe a system for determining the likelihood that a detected vehicle chooses a given exit lane at an intersection. Some intersections are created when two roads cross each other. In these cases, a detected vehicle approaching the intersection from one of the roads has three choices. The vehicle can go straight, turn left, or turn right. As described, the lane the vehicle travels to after crossing the intersection is referred to as an “exit lane.” In some aspects of the present disclosure, the likelihood that a detected vehicle utilizes a particular exit lane is determined by ranking on one or more calculated splines based on the particular exit lane and a current vehicle position. In one configuration, the system is mounted within the car 350 having the sensor module (e.g., camera/radar/sonar/LIDAR), which are provided to an object detection and tracking system (e.g., a vehicle perception module 312) to determine the location and heading of a detected vehicle. [0061] In some aspects of the present disclosure, each of the different paths (e.g., 610, 630, 640, 650, and 660) is ranked based on spline information corresponding to a path of the first detected vehicle 572. In some aspects of the present disclosure, a vehicle perception module 312 of the vehicle behavior planning system 300 of FIG. 3 is mounted within the ego vehicle 420 having the sensor module 302 (e.g., camera/radar/sonar/LIDAR). The vehicle perception module 312 may provide object detection and tracking system functionality to determine a location and a heading of the first detected vehicle 572. In this example, the sixth path 660 is shown to include a past history 662 based on previous locations and headings of the first detected vehicle 572. In some aspects of the present disclosure, the vehicle perception module 312 provides a bounding box, a centroid (and/or front/rear axle location) a heading and velocity points from which a spline representing the sixth path 660 is generated. In addition, the birds-eye-view of the roadway environment 500 may be provided by an aerial drone, a satellite image, a map, or other like visual technique. Schaefer: [0015] The context information may include static context information, wherein static context information represents information about the environment of the vehicle. Static context information may represent the environment of the vehicle which restricts road users' movements including movements of the ego vehicle. Examples include roads, lanes and intersections. By considering static context information, the possible future locations of the ego vehicle and the at least one road user can be restricted to possible locations. Impossible or improbable locations (e.g. off-road) can be excluded. In this way, the prediction becomes more accurate and false warnings are avoided or at least reduced. [0016] The static context information may be represented at least in part by map data and/or traffic rules. Map data is readily available as modern cars often comprise a route guidance system. Traffic rules like for example left- or right-hand driving, one-way roads, right of way, etc. are often also contained in the route guidance system and may be used as static context information improving the collision prediction.)
Consider Claim 4 and 14.
The combination of Schaefer and Epstein teaches:
4. The method of claim 1, wherein the dynamic objects represent moving objects in the surroundings, and the moving objects comprises moving vehicles and pedestrians on the freeway and the ramp. / 14. The system of claim 12, wherein the dynamic objects represent moving objects in the surroundings, and the moving objects comprises moving vehicles and pedestrians on the freeway and the ramp. (Schaefer: [0040] An example of how the occupancy information can be estimated in steps 120 and 130 is illustrated in FIG. 2 which depicts a traffic scene at an intersection in bird-eye perspective. FIG. 2 shows the past trajectories 201 a and 202 a of two vehicles and of a pedestrian 203 a. More precisely, each past trajectory is sampled at three discrete points in time, wherein the squared dots represent vehicle trajectories and crosses represent the pedestrian trajectory. In an actual application, the past trajectories or discrete samples thereof could be obtained by onboard sensors of the ego vehicle. FIG. 2 also depicts the future trajectories 201 b and 202 b of the vehicles and of the pedestrian 203 b. The future trajectories are those that the vehicles and pedestrians had actually followed in that particular scene and are also denoted as the ground truths. [0041]-[0042], Figures 1-2, [0050] Epstein: [0004] Machine learning techniques for vehicle control using a network to select a vehicle control action (e.g., a selected speed / acceleration / steering angle) of the controlled (ego) vehicle is dependent on predicted actions of other vehicles and pedestrians, especially at intersections. A capability for determining the likelihood that a vehicle chooses a given exit lane from an intersection is desired. [0066] At block 706, an exit path taken by the detected vehicle is predicted from the set of exit paths available for the detected vehicle to reach the different exit lanes of the unmarked road segment. For example, in FIG. 6 , the spline representing the sixth path 660 is predicted as the most likely path taken by the first detected vehicle 572. The other splines, associated with the first path 610, the third path 630, the fourth path 640 and the fifth path 650, are determined to be less likely. A prediction regarding the path likelihood may be performed by the vehicle path prediction module 316 based on a curvature of the splines. For example, the spline representing the sixth path 660 is shown as being relatively smooth (e.g., having a lowest curvature) and generally agrees with the vehicle’s current heading based on the past history 662 of previous locations and headings of the first detected vehicle 572. By contrast, the splines representing the other paths (e.g., 610, 630, 640, and 650) exhibit significant curvatures. By evaluating the curvatures of the splines, the vehicle path prediction module 316 can predict which spline represents the path the first detected vehicle 572 is most likely to take, according to some aspects of the present disclosure.)
Consider Claim 5 and 15.
The combination of Schaefer and Epstein teaches:
5. The method of claim 1, wherein the method further comprises displaying the lane occupancy on a lane indicator light. / 15. The system of claim 12, wherein the one or more processors are further operable to display the lane occupancy on a lane indicator light. (Scaefer: [0014] The computer-implemented method may further comprise the step of triggering an ADAS (Advanced Driver Assistance Systems) functionality if the collision threat value exceeds a predetermined threshold at least one future point in time. For example, in response to a collision warning indication, an automatic braking may be engaged. In another example, a collision warning indication may be output if the predetermined threshold is exceeded. The collision warning indication may be output to a driver and/or may be logged. Alternatively, or additionally, the collision warning indication may be uploaded to a central server, for example for later assessment and analysis. [0017] The context information may include dynamic context information, wherein the dynamic context information represents information about the ego vehicle and/or the at least one road user. Information about the ego vehicle and/or the at least one road user may comprise for example position and velocity. Considering dynamic context information can not only improve the prediction of the location of the at least one road user but also of the ego vehicle. [0018] The computer-implemented method may further comprise the step of filtering out road users by selecting a subset of road users in the vicinity of the vehicle and basing the step of determining ego occupancy information on the selected subset of road users. As explained, this may avoid the unnecessary triggering of an ADAS functionality and/or a collision warning in case of road users that have been recognized by the ego vehicle's driver and for which the probability of a collision is rather low. [0019] The step of obtaining context information for the vehicle may comprise obtaining maneuver information relating to the future trajectory of the vehicle, and the step of determining ego occupancy information may additionally be based on the maneuver information. In one example, maneuver information may be provided by the vehicle's direction indicator corresponding to the intent of the driver to turn left or right. In another example, the onboard route guidance system may provide information about the route the drive is likely to follow based on the planned route. In yet another example, a control unit of an autonomous or semi-autonomous vehicle may provide the planned maneuvers of the vehicle. Using planned maneuver information improves the accuracy of the prediction of possible future locations of the ego vehicle but also of other road users in view of interactions between road users.)
Consider Claim 6 and 16.
The combination of Schaefer and Epstein teaches:
6. The method of claim 1, wherein the infrastructure camera is located at or near the merging point. / 16. The system of claim 12, wherein the infrastructure camera is located at or near the merging point. (Epstein: [0052] Some aspects of the present disclosure describe a system for determining the likelihood that a detected vehicle chooses a given exit lane at an intersection. Some intersections are created when two roads cross each other. In these cases, a detected vehicle approaching the intersection from one of the roads has three choices. The vehicle can go straight, turn left, or turn right. As described, the lane the vehicle travels to after crossing the intersection is referred to as an “exit lane.” In some aspects of the present disclosure, the likelihood that a detected vehicle utilizes a particular exit lane is determined by ranking on one or more calculated splines based on the particular exit lane and a current vehicle position. In one configuration, the system is mounted within the car 350 having the sensor module (e.g., camera/radar/sonar/LIDAR), which are provided to an object detection and tracking system (e.g., a vehicle perception module 312) to determine the location and heading of a detected vehicle. [0061] In some aspects of the present disclosure, each of the different paths (e.g., 610, 630, 640, 650, and 660) is ranked based on spline information corresponding to a path of the first detected vehicle 572. In some aspects of the present disclosure, a vehicle perception module 312 of the vehicle behavior planning system 300 of FIG. 3 is mounted within the ego vehicle 420 having the sensor module 302 (e.g., camera/radar/sonar/LIDAR). The vehicle perception module 312 may provide object detection and tracking system functionality to determine a location and a heading of the first detected vehicle 572. In this example, the sixth path 660 is shown to include a past history 662 based on previous locations and headings of the first detected vehicle 572. In some aspects of the present disclosure, the vehicle perception module 312 provides a bounding box, a centroid (and/or front/rear axle location) a heading and velocity points from which a spline representing the sixth path 660 is generated. In addition, the birds-eye-view of the roadway environment 500 may be provided by an aerial drone, a satellite image, a map, or other like visual technique. Schaefer: [0015] The context information may include static context information, wherein static context information represents information about the environment of the vehicle. Static context information may represent the environment of the vehicle which restricts road users' movements including movements of the ego vehicle. Examples include roads, lanes and intersections. By considering static context information, the possible future locations of the ego vehicle and the at least one road user can be restricted to possible locations. Impossible or improbable locations (e.g. off-road) can be excluded. In this way, the prediction becomes more accurate and false warnings are avoided or at least reduced. [0016] The static context information may be represented at least in part by map data and/or traffic rules. Map data is readily available as modern cars often comprise a route guidance system. Traffic rules like for example left- or right-hand driving, one-way roads, right of way, etc. are often also contained in the route guidance system and may be used as static context information improving the collision prediction.)
Consider Claim 7 and 17.
The combination of Schaefer and Epstein teaches:
7. The method of claim 1, wherein the static segmentation map and the dynamic objects are generated further based on historical static segmentation maps of the surroundings. / 17. The system of claim 12, wherein the static segmentation map and the dynamic objects are generated further based on historical static segmentation maps of the surroundings. (Schaefer: [0040] An example of how the occupancy information can be estimated in steps 120 and 130 is illustrated in FIG. 2 which depicts a traffic scene at an intersection in bird-eye perspective. FIG. 2 shows the past trajectories 201 a and 202 a of two vehicles and of a pedestrian 203 a. More precisely, each past trajectory is sampled at three discrete points in time, wherein the squared dots represent vehicle trajectories and crosses represent the pedestrian trajectory. In an actual application, the past trajectories or discrete samples thereof could be obtained by onboard sensors of the ego vehicle. FIG. 2 also depicts the future trajectories 201 b and 202 b of the vehicles and of the pedestrian 203 b. The future trajectories are those that the vehicles and pedestrians had actually followed in that particular scene and are also denoted as the ground truths. Epstein: [0060] FIG. 6 is a diagram 600 further illustrating the birds-eye-view of the roadway environment 500 of FIG. 5 , in which an exit lane of a detected vehicle is determined, according to aspects of the present disclosure. As shown in FIG. 6 , several different paths are determined that the first detected vehicle 572 can take to each of the exit lanes (e.g., 511, 521, 531, 541, 551, and 561) as the first detected vehicle 572 enters the intersection 570. For example, a first path 610 may correspond to the exit lane 511 of the first road 510, and a second path (not shown) may correspond to the exit lane 521 of the second road 520, if the first detected vehicle 572 performs a U-turn. In addition, a third path 630 may correspond to the exit lane 531 of the third road 530, a fourth path 640 may correspond to the exit lane 541 of the fourth road 540, a fifth path 650 may correspond to the exit lane 551 of the fifth road 550, and a sixth path 660 may correspond to the exit lane 561 of the sixth road 560. [0062] In the example shown in FIG. 6 , the spline representing the sixth path 660 is determined as the most likely path taken by the first detected vehicle 572. The other splines, associated with the first path 610, the third path 630, the fourth path 640 and the fifth path 650, are determined to be less likely. The various paths (e.g., 610, 630, 640, 650, and 660) may be determined by the vehicle paths determination module 314 of FIG. 3 . In addition, the prediction regarding the path likelihood may be performed by the vehicle path prediction module 316 based on a curvature of the splines. For example, the spline representing the sixth path 660 is shown as being relatively smooth (e.g., having a lowest curvature) and generally agrees with the vehicle’s current heading based on the past history 662 of previous locations and headings of the first detected vehicle 572. By contrast, the splines representing the other paths (e.g., 610, 630, 640, and 650) exhibit significant curvatures. By evaluating the curvatures of the splines, the vehicle path prediction module 316 can predict which spline represents the path the first detected vehicle 572 is most likely to take, according to some aspects of the present disclosure.)
Consider Claim 8 and 18.
The combination of Schaefer and Epstein teaches:
8. The method of claim 1, wherein the method further comprises predicting moving speeds of the one or more dynamic objects and relative distances between the ego vehicle and the one or more dynamic objects. / 18. The system of claim 12, wherein the one or more processors are further operable to predict moving speeds of the one or more dynamic objects and relative distances between the ego vehicle and the one or more dynamic objects. (Schaefer: [0037] In step 140 the ego occupancy information and the road user occupancy information is fused to obtain fused occupancy information at each future point in time. Fusing may for example be done by assessing the ego vehicle and road user occupancy information at each possible future location at each future point in time individually. If the ego vehicle occupancy information is above a first predetermined threshold and the road user vehicle occupancy information is above a second predetermined threshold, it is inferred that there is a substantial risk of a collision. The first and second threshold may be different or identical. In another example, the ego vehicle occupancy information and the road user vehicle occupancy information is multiplied and the result is compared against a threshold. In general, any mathematical operation that leads to an estimate of the collision risk is suitable. [0038] In step 150 the processor determines a collision threat value based on the fused occupancy information. The value may for example correspond to the probability that the ego vehicle and the at least other road user will collide at least one future point in time. For example, if at step 140 the ego vehicle and road user occupancy information are fused by a mathematical operation (such as e.g. multiplication) that results in a single value, this value may represent the collision threat value. If at step 140 the ego vehicle occupancy information and the road user vehicle occupancy information are compared against first and second thresholds, respectively, the collision threat value may be the result of a logical operation, for example a logical OR operation taking as inputs the results of the threshold functions. Generally, the collision threat value may be continuous or may for example have discrete states (like for example “no risk”, “medium risk” and “high risk”). Generally, two states (“no risk”/“risk”) are sufficient. In case of a continuous collision threat value, this value may be compared against a threshold and exceeding this threshold may trigger further actions as will be described below. [0039] If it is determined in step 150 that there is a risk of collision of the ego vehicle with some other road user based on the collision threat value, a collision warning may be provided. The collision warning may by output to a driver and/or used to control functions of the ego vehicle. For example, in response to a collision warning indication, an automatic braking may be engaged. Generally, the collision warning indication may be provided to an ADAS (Advanced Driver-Assistance System) function. In another example, the collision warning indication may be uploaded to a central server, for example for later assessment and analysis. The warning may be transmitted via cellular radio. [0040] An example of how the occupancy information can be estimated in steps 120 and 130 is illustrated in FIG. 2 which depicts a traffic scene at an intersection in bird-eye perspective. FIG. 2 shows the past trajectories 201 a and 202 a of two vehicles and of a pedestrian 203 a. More precisely, each past trajectory is sampled at three discrete points in time, wherein the squared dots represent vehicle trajectories and crosses represent the pedestrian trajectory. In an actual application, the past trajectories or discrete samples thereof could be obtained by onboard sensors of the ego vehicle. FIG. 2 also depicts the future trajectories 201 b and 202 b of the vehicles and of the pedestrian 203 b. The future trajectories are those that the vehicles and pedestrians had actually followed in that particular scene and are also denoted as the ground truths. Epstein: [0049] The vehicle behavior planner 310 may be in communication with the sensor module 302, the processor 320, the computer-readable medium 322, the onboard unit 324, the locomotion module 326, the location module 328, the communication module 330, the transceiver 332, and the vehicle behavior planner 310. In one configuration, the vehicle behavior planner 310 receives sensor data from the sensor module 302. The sensor module 302 may receive the sensor data from the first sensor 306 and the second sensor 304. According to aspects of the disclosure, the sensor module 302 may filter the data to remove noise, encode the data, decode the data, merge the data, extract frames, or perform other functions. In an alternate configuration, the vehicle behavior planner 310 may receive sensor data directly from the first sensor 306 and the second sensor 304 to determine, for example, input traffic data images. [0050] Vehicle planning by machine learning is less effective in complicated traffic environments. For example, these traffic environments may involve complex interactions between vehicles, including situations where an ego vehicle (e.g., the car 350) maneuvers into and out of traffic lanes (e.g., intersections). Conventional machine learning techniques for planning vehicle control may use a network to plan an appropriate vehicle control action from input data relative to the car 350. For example, a selected speed / acceleration / steering angle of the car 350 may be planned as a vehicle control action to perform a vehicle control maneuver. Unfortunately, conventional machine learning techniques may be problematic when the car 350 encounters a complex intersection. [0051] For example, when an autonomous vehicle (AV), such as the car 350, is driving on a roadway, conventional approaches plan actions at discrete time intervals, as determined by a vehicle perception system.)
Consider Claim 9.
The combination of Schaefer and Epstein teaches:
9. The method of claim 8, wherein the method further comprises predicting collision probability between the ego vehicle and each of the one or more dynamic objects based on the lane occupancy, the moving speeds of the one or more dynamic objects, and the relative distances between the ego vehicle and the one or more dynamic objects. (Schaefer: [0037] In step 140 the ego occupancy information and the road user occupancy information is fused to obtain fused occupancy information at each future point in time. Fusing may for example be done by assessing the ego vehicle and road user occupancy information at each possible future location at each future point in time individually. If the ego vehicle occupancy information is above a first predetermined threshold and the road user vehicle occupancy information is above a second predetermined threshold, it is inferred that there is a substantial risk of a collision. The first and second threshold may be different or identical. In another example, the ego vehicle occupancy information and the road user vehicle occupancy information is multiplied and the result is compared against a threshold. In general, any mathematical operation that leads to an estimate of the collision risk is suitable. [0038] In step 150 the processor determines a collision threat value based on the fused occupancy information. The value may for example correspond to the probability that the ego vehicle and the at least other road user will collide at least one future point in time. For example, if at step 140 the ego vehicle and road user occupancy information are fused by a mathematical operation (such as e.g. multiplication) that results in a single value, this value may represent the collision threat value. If at step 140 the ego vehicle occupancy information and the road user vehicle occupancy information are compared against first and second thresholds, respectively, the collision threat value may be the result of a logical operation, for example a logical OR operation taking as inputs the results of the threshold functions. Generally, the collision threat value may be continuous or may for example have discrete states (like for example “no risk”, “medium risk” and “high risk”). Generally, two states (“no risk”/“risk”) are sufficient. In case of a continuous collision threat value, this value may be compared against a threshold and exceeding this threshold may trigger further actions as will be described below. [0039] If it is determined in step 150 that there is a risk of collision of the ego vehicle with some other road user based on the collision threat value, a collision warning may be provided. The collision warning may by output to a driver and/or used to control functions of the ego vehicle. For example, in response to a collision warning indication, an automatic braking may be engaged. Generally, the collision warning indication may be provided to an ADAS (Advanced Driver-Assistance System) function. In another example, the collision warning indication may be uploaded to a central server, for example for later assessment and analysis. The warning may be transmitted via cellular radio. [0040] An example of how the occupancy information can be estimated in steps 120 and 130 is illustrated in FIG. 2 which depicts a traffic scene at an intersection in bird-eye perspective. FIG. 2 shows the past trajectories 201 a and 202 a of two vehicles and of a pedestrian 203 a. More precisely, each past trajectory is sampled at three discrete points in time, wherein the squared dots represent vehicle trajectories and crosses represent the pedestrian trajectory. In an actual application, the past trajectories or discrete samples thereof could be obtained by onboard sensors of the ego vehicle. FIG. 2 also depicts the future trajectories 201 b and 202 b of the vehicles and of the pedestrian 203 b. The future trajectories are those that the vehicles and pedestrians had actually followed in that particular scene and are also denoted as the ground truths. Epstein: [0049] The vehicle behavior planner 310 may be in communication with the sensor module 302, the processor 320, the computer-readable medium 322, the onboard unit 324, the locomotion module 326, the location module 328, the communication module 330, the transceiver 332, and the vehicle behavior planner 310. In one configuration, the vehicle behavior planner 310 receives sensor data from the sensor module 302. The sensor module 302 may receive the sensor data from the first sensor 306 and the second sensor 304. According to aspects of the disclosure, the sensor module 302 may filter the data to remove noise, encode the data, decode the data, merge the data, extract frames, or perform other functions. In an alternate configuration, the vehicle behavior planner 310 may receive sensor data directly from the first sensor 306 and the second sensor 304 to determine, for example, input traffic data images. [0050] Vehicle planning by machine learning is less effective in complicated traffic environments. For example, these traffic environments may involve complex interactions between vehicles, including situations where an ego vehicle (e.g., the car 350) maneuvers into and out of traffic lanes (e.g., intersections). Conventional machine learning techniques for planning vehicle control may use a network to plan an appropriate vehicle control action from input data relative to the car 350. For example, a selected speed / acceleration / steering angle of the car 350 may be planned as a vehicle control action to perform a vehicle control maneuver. Unfortunately, conventional machine learning techniques may be problematic when the car 350 encounters a complex intersection. [0051] For example, when an autonomous vehicle (AV), such as the car 350, is driving on a roadway, conventional approaches plan actions at discrete time intervals, as determined by a vehicle perception system.)
Consider Claim 10.
The combination of Schaefer and Epstein teaches:
10. The method of claim 9, wherein the method further comprises: determining whether the collision probability is beyond a threshold probability; and in response to determining that the collision probability is beyond the threshold probability, warning the ego vehicle of a potential collision with at least one of the one or more dynamic objects. (Schaefer: [0037] In step 140 the ego occupancy information and the road user occupancy information is fused to obtain fused occupancy information at each future point in time. Fusing may for example be done by assessing the ego vehicle and road user occupancy information at each possible future location at each future point in time individually. If the ego vehicle occupancy information is above a first predetermined threshold and the road user vehicle occupancy information is above a second predetermined threshold, it is inferred that there is a substantial risk of a collision. The first and second threshold may be different or identical. In another example, the ego vehicle occupancy information and the road user vehicle occupancy information is multiplied and the result is compared against a threshold. In general, any mathematical operation that leads to an estimate of the collision risk is suitable. [0038] In step 150 the processor determines a collision threat value based on the fused occupancy information. The value may for example correspond to the probability that the ego vehicle and the at least other road user will collide at least one future point in time. For example, if at step 140 the ego vehicle and road user occupancy information are fused by a mathematical operation (such as e.g. multiplication) that results in a single value, this value may represent the collision threat value. If at step 140 the ego vehicle occupancy information and the road user vehicle occupancy information are compared against first and second thresholds, respectively, the collision threat value may be the result of a logical operation, for example a logical OR operation taking as inputs the results of the threshold functions. Generally, the collision threat value may be continuous or may for example have discrete states (like for example “no risk”, “medium risk” and “high risk”). Generally, two states (“no risk”/“risk”) are sufficient. In case of a continuous collision threat value, this value may be compared against a threshold and exceeding this threshold may trigger further actions as will be described below. [0039] If it is determined in step 150 that there is a risk of collision of the ego vehicle with some other road user based on the collision threat value, a collision warning may be provided. The collision warning may by output to a driver and/or used to control functions of the ego vehicle. For example, in response to a collision warning indication, an automatic braking may be engaged. Generally, the collision warning indication may be provided to an ADAS (Advanced Driver-Assistance System) function. In another example, the collision warning indication may be uploaded to a central server, for example for later assessment and analysis. The warning may be transmitted via cellular radio. [0040] An example of how the occupancy information can be estimated in steps 120 and 130 is illustrated in FIG. 2 which depicts a traffic scene at an intersection in bird-eye perspective. FIG. 2 shows the past trajectories 201 a and 202 a of two vehicles and of a pedestrian 203 a. More precisely, each past trajectory is sampled at three discrete points in time, wherein the squared dots represent vehicle trajectories and crosses represent the pedestrian trajectory. In an actual application, the past trajectories or discrete samples thereof could be obtained by onboard sensors of the ego vehicle. FIG. 2 also depicts the future trajectories 201 b and 202 b of the vehicles and of the pedestrian 203 b. The future trajectories are those that the vehicles and pedestrians had actually followed in that particular scene and are also denoted as the ground truths. Epstein: [0049] The vehicle behavior planner 310 may be in communication with the sensor module 302, the processor 320, the computer-readable medium 322, the onboard unit 324, the locomotion module 326, the location module 328, the communication module 330, the transceiver 332, and the vehicle behavior planner 310. In one configuration, the vehicle behavior planner 310 receives sensor data from the sensor module 302. The sensor module 302 may receive the sensor data from the first sensor 306 and the second sensor 304. According to aspects of the disclosure, the sensor module 302 may filter the data to remove noise, encode the data, decode the data, merge the data, extract frames, or perform other functions. In an alternate configuration, the vehicle behavior planner 310 may receive sensor data directly from the first sensor 306 and the second sensor 304 to determine, for example, input traffic data images. [0050] Vehicle planning by machine learning is less effective in complicated traffic environments. For example, these traffic environments may involve complex interactions between vehicles, including situations where an ego vehicle (e.g., the car 350) maneuvers into and out of traffic lanes (e.g., intersections). Conventional machine learning techniques for planning vehicle control may use a network to plan an appropriate vehicle control action from input data relative to the car 350. For example, a selected speed / acceleration / steering angle of the car 350 may be planned as a vehicle control action to perform a vehicle control maneuver. Unfortunately, conventional machine learning techniques may be problematic when the car 350 encounters a complex intersection. [0051] For example, when an autonomous vehicle (AV), such as the car 350, is driving on a roadway, conventional approaches plan actions at discrete time intervals, as determined by a vehicle perception system.)
Consider Claim 11 and 20.
The combination of Schaefer and Epstein teaches:
11. The method of claim 10, wherein the method further comprises in response to determining that the collision probability is beyond the threshold probability, operating the ego vehicle to avoid the potential collision. / 20. The system of claim 19, the one or more processors are further operable to, in response to determining that the collision probability is beyond the threshold probability, operate the ego vehicle to avoid the potential collision. (Schaefer: [0037] In step 140 the ego occupancy information and the road user occupancy information is fused to obtain fused occupancy information at each future point in time. Fusing may for example be done by assessing the ego vehicle and road user occupancy information at each possible future location at each future point in time individually. If the ego vehicle occupancy information is above a first predetermined threshold and the road user vehicle occupancy information is above a second predetermined threshold, it is inferred that there is a substantial risk of a collision. The first and second threshold may be different or identical. In another example, the ego vehicle occupancy information and the road user vehicle occupancy information is multiplied and the result is compared against a threshold. In general, any mathematical operation that leads to an estimate of the collision risk is suitable. [0038] In step 150 the processor determines a collision threat value based on the fused occupancy information. The value may for example correspond to the probability that the ego vehicle and the at least other road user will collide at least one future point in time. For example, if at step 140 the ego vehicle and road user occupancy information are fused by a mathematical operation (such as e.g. multiplication) that results in a single value, this value may represent the collision threat value. If at step 140 the ego vehicle occupancy information and the road user vehicle occupancy information are compared against first and second thresholds, respectively, the collision threat value may be the result of a logical operation, for example a logical OR operation taking as inputs the results of the threshold functions. Generally, the collision threat value may be continuous or may for example have discrete states (like for example “no risk”, “medium risk” and “high risk”). Generally, two states (“no risk”/“risk”) are sufficient. In case of a continuous collision threat value, this value may be compared against a threshold and exceeding this threshold may trigger further actions as will be described below. [0039] If it is determined in step 150 that there is a risk of collision of the ego vehicle with some other road user based on the collision threat value, a collision warning may be provided. The collision warning may by output to a driver and/or used to control functions of the ego vehicle. For example, in response to a collision warning indication, an automatic braking may be engaged. Generally, the collision warning indication may be provided to an ADAS (Advanced Driver-Assistance System) function. In another example, the collision warning indication may be uploaded to a central server, for example for later assessment and analysis. The warning may be transmitted via cellular radio. [0040] An example of how the occupancy information can be estimated in steps 120 and 130 is illustrated in FIG. 2 which depicts a traffic scene at an intersection in bird-eye perspective. FIG. 2 shows the past trajectories 201 a and 202 a of two vehicles and of a pedestrian 203 a. More precisely, each past trajectory is sampled at three discrete points in time, wherein the squared dots represent vehicle trajectories and crosses represent the pedestrian trajectory. In an actual application, the past trajectories or discrete samples thereof could be obtained by onboard sensors of the ego vehicle. FIG. 2 also depicts the future trajectories 201 b and 202 b of the vehicles and of the pedestrian 203 b. The future trajectories are those that the vehicles and pedestrians had actually followed in that particular scene and are also denoted as the ground truths. Epstein: [0049] The vehicle behavior planner 310 may be in communication with the sensor module 302, the processor 320, the computer-readable medium 322, the onboard unit 324, the locomotion module 326, the location module 328, the communication module 330, the transceiver 332, and the vehicle behavior planner 310. In one configuration, the vehicle behavior planner 310 receives sensor data from the sensor module 302. The sensor module 302 may receive the sensor data from the first sensor 306 and the second sensor 304. According to aspects of the disclosure, the sensor module 302 may filter the data to remove noise, encode the data, decode the data, merge the data, extract frames, or perform other functions. In an alternate configuration, the vehicle behavior planner 310 may receive sensor data directly from the first sensor 306 and the second sensor 304 to determine, for example, input traffic data images. [0050] Vehicle planning by machine learning is less effective in complicated traffic environments. For example, these traffic environments may involve complex interactions between vehicles, including situations where an ego vehicle (e.g., the car 350) maneuvers into and out of traffic lanes (e.g., intersections). Conventional machine learning techniques for planning vehicle control may use a network to plan an appropriate vehicle control action from input data relative to the car 350. For example, a selected speed / acceleration / steering angle of the car 350 may be planned as a vehicle control action to perform a vehicle control maneuver. Unfortunately, conventional machine learning techniques may be problematic when the car 350 encounters a complex intersection. [0051] For example, when an autonomous vehicle (AV), such as the car 350, is driving on a roadway, conventional approaches plan actions at discrete time intervals, as determined by a vehicle perception system.)
Consider Claim 19.
The combination of Schaefer and Epstein teaches:
19. The system of claim 18, wherein the one or more processors are further operable to: predict collision probability between the ego vehicle and each of the one or more dynamic objects based on the lane occupancy, the moving speeds of the one or more dynamic objects, and the relative distances between the ego vehicle and the one or more dynamic objects; determine whether the collision probability is beyond a threshold probability; and in response to determining that the collision probability is beyond the threshold probability, warn the ego vehicle of a potential collision with at least one of the one or more dynamic objects. (Schaefer: [0037] In step 140 the ego occupancy information and the road user occupancy information is fused to obtain fused occupancy information at each future point in time. Fusing may for example be done by assessing the ego vehicle and road user occupancy information at each possible future location at each future point in time individually. If the ego vehicle occupancy information is above a first predetermined threshold and the road user vehicle occupancy information is above a second predetermined threshold, it is inferred that there is a substantial risk of a collision. The first and second threshold may be different or identical. In another example, the ego vehicle occupancy information and the road user vehicle occupancy information is multiplied and the result is compared against a threshold. In general, any mathematical operation that leads to an estimate of the collision risk is suitable. [0038] In step 150 the processor determines a collision threat value based on the fused occupancy information. The value may for example correspond to the probability that the ego vehicle and the at least other road user will collide at least one future point in time. For example, if at step 140 the ego vehicle and road user occupancy information are fused by a mathematical operation (such as e.g. multiplication) that results in a single value, this value may represent the collision threat value. If at step 140 the ego vehicle occupancy information and the road user vehicle occupancy information are compared against first and second thresholds, respectively, the collision threat value may be the result of a logical operation, for example a logical OR operation taking as inputs the results of the threshold functions. Generally, the collision threat value may be continuous or may for example have discrete states (like for example “no risk”, “medium risk” and “high risk”). Generally, two states (“no risk”/“risk”) are sufficient. In case of a continuous collision threat value, this value may be compared against a threshold and exceeding this threshold may trigger further actions as will be described below. [0039] If it is determined in step 150 that there is a risk of collision of the ego vehicle with some other road user based on the collision threat value, a collision warning may be provided. The collision warning may by output to a driver and/or used to control functions of the ego vehicle. For example, in response to a collision warning indication, an automatic braking may be engaged. Generally, the collision warning indication may be provided to an ADAS (Advanced Driver-Assistance System) function. In another example, the collision warning indication may be uploaded to a central server, for example for later assessment and analysis. The warning may be transmitted via cellular radio. [0040] An example of how the occupancy information can be estimated in steps 120 and 130 is illustrated in FIG. 2 which depicts a traffic scene at an intersection in bird-eye perspective. FIG. 2 shows the past trajectories 201 a and 202 a of two vehicles and of a pedestrian 203 a. More precisely, each past trajectory is sampled at three discrete points in time, wherein the squared dots represent vehicle trajectories and crosses represent the pedestrian trajectory. In an actual application, the past trajectories or discrete samples thereof could be obtained by onboard sensors of the ego vehicle. FIG. 2 also depicts the future trajectories 201 b and 202 b of the vehicles and of the pedestrian 203 b. The future trajectories are those that the vehicles and pedestrians had actually followed in that particular scene and are also denoted as the ground truths. Epstein: [0049] The vehicle behavior planner 310 may be in communication with the sensor module 302, the processor 320, the computer-readable medium 322, the onboard unit 324, the locomotion module 326, the location module 328, the communication module 330, the transceiver 332, and the vehicle behavior planner 310. In one configuration, the vehicle behavior planner 310 receives sensor data from the sensor module 302. The sensor module 302 may receive the sensor data from the first sensor 306 and the second sensor 304. According to aspects of the disclosure, the sensor module 302 may filter the data to remove noise, encode the data, decode the data, merge the data, extract frames, or perform other functions. In an alternate configuration, the vehicle behavior planner 310 may receive sensor data directly from the first sensor 306 and the second sensor 304 to determine, for example, input traffic data images. [0050] Vehicle planning by machine learning is less effective in complicated traffic environments. For example, these traffic environments may involve complex interactions between vehicles, including situations where an ego vehicle (e.g., the car 350) maneuvers into and out of traffic lanes (e.g., intersections). Conventional machine learning techniques for planning vehicle control may use a network to plan an appropriate vehicle control action from input data relative to the car 350. For example, a selected speed / acceleration / steering angle of the car 350 may be planned as a vehicle control action to perform a vehicle control maneuver. Unfortunately, conventional machine learning techniques may be problematic when the car 350 encounters a complex intersection. [0051]
Conclusion
The prior art made of record in form PTO-892 and not relied upon is considered pertinent to applicant's disclosure.
PNG
media_image1.png
204
899
media_image1.png
Greyscale
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TAHMINA ANSARI whose telephone number is 571-270-3379. The examiner can normally be reached on IFP Flex - Monday through Friday 9 to 5.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’NEAL MISTRY can be reached on 313-446-4912. The fax phone numbers for the organization where this application or proceeding is assigned are 571-273-8300 for regular communications and 571-273-8300 for After Final communications. TC 2600’s customer service number is 571-272-2600.
Any inquiry of a general nature or relating to the status of this application or proceeding should be directed to the receptionist whose telephone number is 571-272-2600.
2674
August 21, 2026
/TAHMINA N ANSARI/Primary Examiner, Art Unit 2674
Prosecution Insights