Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1, 2, 4-10, and 12-21 are presented for examination.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b) which forms the basis for all indefiniteness rejections set forth in this Office action:
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 claims, in part, “applying, by at least one processor, a collision detection model to identify a collision between the first vehicle and the second vehicle based on the rate of change of the at least one spatial parameter of the bounding box…” The article “the” is missing prior to the claimed “at least one processor.” There is insufficient antecedent basis for this limitation in the claim. Claims 2, 4-8, 18, and 19 are rejected for the same reasons due to their dependency on claim 1.
Claim 9 claims, in part, “apply, by at least one processor, a collision detection model to identify a collision between the first vehicle and the second vehicle based on the rate of change of the at least one spatial parameter of the bounding box…” The article “the” is missing prior to the claimed “at least one processor.” There is insufficient antecedent basis for this limitation in the claim. Claims 10, 12-17, 20, and 21 are rejected for the same reasons due to their dependency on claim 9
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 4-7, 9, 10,12, and 14 are rejected under 35 USC 103 as being unpatentable over Gan et al., U.S. 2021/0403050 in view of Nakajima et al., U.S. 6,285, 778 and Liu et al., U.S.2019/0103026 and Rishi et al. , U.S.10,814,815.
On claim 1, Gan cites except as underlined:
A method for identifying a collision between a first vehicle and second vehicle comprising:
accessing image data, the image data including a set of images, each image of the set of images captured by an image capture device positioned at the second vehicle;
figure 3, [0032, 33], objects in bounding boxes 305, 315, and 325. The bounding boxes are envisioned from a perspective to the rear of the above vehicles. (The claimed “first and second vehicles” are presumed to be any one of the vehicles picture in figure 3).
And
[0030] The disclosed techniques and devices include a computer vision solution using cameras aimed away from a side of the autonomous vehicle to detect potentially interfering vehicles and objects. The images are processed to determine when and if it is “safe to go” (e.g., safe to proceed along a planned path) in a “T”-type and many other types of intersections. Cameras aimed toward space on the side of the autonomous vehicle (e.g., the areas surrounding the vehicle to the right and left) capture images, and image processing of the images captured by the cameras are used to generate bounding boxes for objects identified by the image processing, according to the disclosed technology.
applying, by at least one processor, an object detection model to each image in the set of images to determine a respective bounding box representation of the first vehicle from the perspective of the second vehicle for each image determine, by the at least one processor, for each image of the set of images, whether the first vehicle and the second vehicles are positioned in a respective common lane of travel;
when the first vehicle and the second vehicle are positioned in the common lane of travel;
see previous.
determining, by the at least one processor, a rate of change of at least one spatial parameter of the bounding box for the first vehicle across at least a subset of the set of images;
[0033] The distance to an object (e.g., a vehicle) and the speed of the object (e.g., relative to another object) can be identified by image processing which can also determine the bounding boxes for each object. The speed of an object can be determined, for example, from the time history of the bounding boxes associated with the object. Based on the distance to the object and the speed of the object, an amount of time before a possible crash between the object and the autonomous vehicle may be determined.
applying, by at least one processor, a collision detection model to identify a collision between the first vehicle and the second vehicle based on the rate of change of the at least one spatial parameter of the bounding box;
see above.
and output an indication of collision when the collision detection model identifies a collision between the first vehicle and the second vehicle; and
when the first vehicle and second vehicle are positioned in the respective common lane of travel;
determine, by the at least one processor that there is no collision between the first vehicle and the second vehicle.
[0033] The distance to an object (e.g., a vehicle) and the speed of the object (e.g., relative to another object) can be identified by image processing which can also determine the bounding boxes for each object. The speed of an object can be determined, for example, from the time history of the bounding boxes associated with the object. Based on the distance to the object and the speed of the object, an amount of time before a possible crash between the object and the autonomous vehicle may be determined. This estimated time before a possible collision or crash may be referred to as a time to crash (TTC).
(the above citation illustrates there is no collision between the vehicles because there is an amount of time remaining before there is a collision)
Regarding the above excepted non-italicized limitations, Gan disclosed a system employing the detection of objects collocated to a user’s vehicle wherein the distance and speeds between the objects were determined to be collision hazards. Gan did not disclose this system as detecting an actual collision.
However, in the same art of vehicle collision detection, Nakajima cites:
Col, 9, lines 60-65, Upon detection of a possible contact or collision, an alarm is sounded from a buzzer or an alarm message which is generated by a voice synthesizing means in the data processor 6 is issued. The alarm may also be displayed on the display device 10 to alert the driver.
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to include into Gan the collision detection features of Nakajima wherein an actual collision has occurred.
In Gan’s system, only potential collision hazards are detected, but this does not actual contact between the vehicles. This means no damage or injury has occurred. One of ordinary skill would have included Nakajima’s embodiment into Gan’s system to inform authorities and other concerned organizations to dispatch personnel to the accident site.
Regarding the excepted: each image of the set of images captured by an image capture device positioned at the second vehicle; and
determine, by the at least one processor, for each image of the set of images, whether the first vehicle and the second vehicles are positioned in a respective common lane of travel; when the first vehicle and the second vehicle are positioned in the common lane of travel;
Gan, as disclosed in figure 3, discloses
[0032] FIG. 3 depicts examples of bounding boxes generated from camera data. Bounding boxes 305, 315, and 325 are generated from images taken by one or more cameras. In FIG. 3, the bounding boxes 305, 315, and 325 are shown in the context of a representative image 300. The images are analyzed by a computer vision system or an image processor to identify objects and generate bounding boxes that outline the objects. In some implementations, deep learning or neural networks may be used to generate bounding boxes. Bounding box 305 corresponds to a semi-truck, bounding box 315 corresponds to a first pick-up truck, and bounding box 325 corresponds to a second pick-up truck.
The figure disclosed in Gan suggests the representative image is taken from the perspective of a vehicle behind the vehicles enclosed in the bounding boxes 305, 315, and 325. However, Gan doesn’t specifically disclose this as being a perspective from a vehicle to the rear of the above mentioned vehicles nor does Gan disclose where the first and second vehicles are positioned in a respective common lane of travel.
In the same art of vehicle collision detection systems, Liu cites:
[0028] FIG. 2B is a diagram of a cropped image frame 210 captured from a forward-facing field of view of a vehicle 140. In contrast to the full image frame shown in FIG. 2A, the image frame 210 shown in FIG. 2B includes cropped portions of the corresponding full image frame. The embodiment of FIG. 2B shows cropped portions using a first scale 220 and a second scale 230. The scales may vary in size, dimensions, or other attributes. The collision warning system 100 may crop a portion of the image frame 210 to focus on a detected object in front of the vehicle 140, such as another vehicle in the same lane on the road.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to include into Gan the features disclosed in Liu such that the claimed invention is realized. Liu discloses a known embodiment for determining if a first and second vehicle occupy the same lane and one of ordinary skill would have incorporated Liu’s feature into Gan to provide additional collision detection features from another perspective.
Regarding the excepted:
applying, by (the) at least one processor, a collision detection model to identify a collision between the first vehicle and the second vehicle based on the rate of change of the at least one spatial parameter of the bounding box;
and output an indication of collision when the collision detection model identifies a collision between the first vehicle and the second vehicle
Gan, as disclosed above, cites:
[0033]Based on the distance to the object and the speed of the object, an amount of time before a possible crash between the object and the autonomous vehicle may be determined. This estimated time before a possible collision or crash may be referred to as a time to crash (TTC).
Gan doesn’t disclose an actual collision.
In the same art of collision detection systems, Rishi cites:
Col. 4, lines 7-16 In an embodiment, the report generator may generate a crash report 1800 using output obtained from the IMU sensor data analytics system 102 and the video analytics system 104 to generate the crash report 1800. Referring to FIG. 2, various modules of the IMU sensor data analytics system 102 of the system 100, in accordance with an embodiment are disclosed. The IMU sensor data analytics system 102 may include a sensing system 202, a machine learning model 204, a first output module 206 and a sensor data processor 208.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to modify Gan’s embodiment using the crash reporting system of Rishi such that the claimed invention is realized. Rishi disclosed a known embodiment wherein if a collision is determined, a “crash report” is generated. One of ordinary skill would have provided such a feature to automatically contact law enforcement and other first responder organizations.
On claim 2, Gan cites except as underlined:
The method of claim 1, wherein if applying the object detection model to each image in the set of images to determine a respective bounding box representation of the first vehicle from the perspective of the second vehicle for each image does not result in a determination of any bounding boxes: identify no collision between the first vehicle and the second vehicle.
[0033] The distance to an object (e.g., a vehicle) and the speed of the object (e.g., relative to another object) can be identified by image processing which can also determine the bounding boxes for each object. The speed of an object can be determined, for example, from the time history of the bounding boxes associated with the object. Based on the distance to the object and the speed of the object, an amount of time before a possible crash between the object and the autonomous vehicle may be determined.
(The above passage indicates a warning, or a possibility, and not an actual occurrence of a crash between identified objects encircled with bounding boxes).
Furthermore, Gan cites:
[0035] A second challenge the technology disclosed herein overcomes is identification of the direction of travel of moving vehicles identified in successive images. Vehicles moving toward the autonomous vehicle camera will generate bounding boxes that are larger in size in images taken at later times compared to images taken at earlier times. Vehicles moving away from the autonomous vehicle will have bounding boxes that are smaller in size in images taken at later times compared to images taken at earlier times. For vehicles at a close range from the autonomous vehicle, successive bounding boxes can be combined to enable a quick determination of the direction of travel. For vehicles that are far away, a longer history may be used to make a reliable determination of the direction of travel. Distant objects must move a greater distance for the autonomous vehicle to be able to determine whether the bounding boxes are getting smaller or larger, and an estimate of their position will have more noise associated with it.
The claimed invention is interpreted to mean bounding boxes are not created when there are no collision hazards determined between collocated vehicles.
Gan discloses bounding boxes being created whenever vehicles are located within a detection distance of the autonomous vehicle. Gan does not disclose the feature of creating bounding boxes when a vehicle is detected from the autonomous vehicle. However, it would have been obvious to one of ordinary skill in the art at the time of the claimed invention to include into Gan the feature of determining a detected vehicle is not deemed to be a collision hazard. Gan, as above, discloses creating bounding boxes that become smaller as detected vehicles move away from the autonomous vehicle. Thus, unlike the claimed invention where no bounding boxes are created responsive to a likely threat of a colliding vehicle, Gan discloses an embodiment where the bounding box becomes smaller as it is moves away from the automated vehicle. In either case, each respective embodiment is disclosing a way to show to a user how a “non-threatening” vehicle is given less of a consideration as possible collision threat by either not constructing a bounding box or reducing the size of a bounding box to where the smaller the bounding box is shown. In either case, one of ordinary skill would have implement a similar strategy in making detected vehicles not considered a collision hazard by reducing their visibility to the autonomous vehicle.
On claim 4, Gan cites:
The method of claim 1, wherein accessing the image data comprises capturing the image data by the at least one image capture device positioned at the second vehicle.
[0023] and figure 2, Autonomous vehicle 230 has one or more cameras as described with respect to FIG. 1.
And
Figure 3 and [0030] The disclosed techniques and devices include a computer vision solution using cameras aimed away from a side of the autonomous vehicle to detect potentially interfering vehicles and objects. The images are processed to determine when and if it is “safe to go” (e.g., safe to proceed along a planned path) in a “T”-type and many other types of intersections. Cameras aimed toward space on the side of the autonomous vehicle (e.g., the areas surrounding the vehicle to the right and left) capture images, and image processing of the images captured by the cameras are used to generate bounding boxes for objects identified by the image processing, according to the disclosed technology
On claim 5, Gan cites:
The method of claim 1, wherein accessing the image data comprises retrieving the image data stored at a non-transitory processor-readable storage medium.
[0006] Another aspect of the disclosed embodiments relates to an apparatus for an autonomous driving vehicle that comprises at least one processor and a memory. The memory of the apparatus includes executable instructions that, when executed by the at least one processor, cause the apparatus to perform at least the following operations: receive a series of road images from a side-view camera sensor of the autonomous driving vehicle, wherein each image in the series of road images is taken at a different time; generate, for each object from objects captured in the series of road images, a series of bounding boxes in the series of road images, wherein each bounding box in the series of bounding boxes corresponds to an image in the series of road images; determine, for each object from the objects, a direction of travel or that the object is stationary; determine a speed of each object for which the direction of travel has been determined.
On claim 6, Gan and Nakajima cites:
The method of claim 1, wherein the at least one processor is positioned at the second vehicle, and outputting an indication of collision
(see the rejection of claim 1 citing Nakajima)
comprises transmitting the indication of collision by a communication interface to be received by a device remote from the second vehicle
[0064] The apparatus may use a machine learning device (which, for example, can be a part of the apparatus or, alternatively, can be a separate device that is, for example, externally or remotely located relative to the apparatus; for example, the machine learning device can be a remote server located “in the cloud”) to perform the process of determining whether the autonomous driving vehicle can safely move in a predetermined direction. (In this citation, coupled with the indication of a collision as previously discussed, the cited remote server would likely determine a collision has occurred since the server if a vehicle can safely move, a collision would also indicate the vehicle could not be safely moved).
On claim 7, Gan cites except as underlined:
The method of claim 6, further comprising outputting an alert to an operator of the device remote from the second vehicle.
Gan discloses:
[0064] The apparatus may use a machine learning device (which, for example, can be a part of the apparatus or, alternatively, can be a separate device that is, for example, externally or remotely located relative to the apparatus; for example, the machine learning device can be a remote server located “in the cloud”) to perform the process of determining whether the autonomous driving vehicle can safely move in a predetermined direction.
Gan also discloses:
[0033] The distance to an object (e.g., a vehicle) and the speed of the object (e.g., relative to another object) can be identified by image processing which can also determine the bounding boxes for each object. The speed of an object can be determined, for example, from the time history of the bounding boxes associated with the object. Based on the distance to the object and the speed of the object, an amount of time before a possible crash between the object and the autonomous vehicle may be determined.
And
Figure 9, communications interface 906 and [0059]
Gan doesn’t specifically disclose the excepted claim limitations. However, it would have been obvious to one of ordinary skill in the art at the time of the claimed invention, based on the given references, to provide an embodiment meeting the claimed limitations.
Gan, as disclosed above, includes a communications interface 900 for a vehicle collision detection system. Gan further discloses bounding boxes used to determine potential collision hazards detected by the respective collision detecting systems onboard on more vehicles shown in figure 3. Furthermore, Gan discloses a remote server located in the cloud “whether the autonomous driving vehicle can safely move in a predetermined direction.”
Taking these elements together, Gan provides a remote server from which the server directs information to the autonomous vehicle to safely move. In order to carry out the safe move, vehicles disclosed in figure 3 include collision determination features where the data disclosed in the bounding boxes are likely sent to the remote server using the communications interface disclosed in figure 9.
Accordingly, it would have been obvious to one of ordinary skill in the art at the time of the claimed invention to try and arrive at an embodiment realizing the claimed invention after being apprised with the known elements disclosed in the reference.
On claim 9, Gan, Nakajima, Liu, and Rishi cites:
A system for identifying a collision between a first vehicle and second vehicle comprising:
at least one processor; at least one non-transitory processor-readable storage medium storing processor-executable instructions which when executed cause the system to:
access image data, the image data including a set of images, each image of the et al. of images captured by an image capture device positioned at the second vehicle;
apply, by the at least one processor, an object detection model to each image in the set of images to determine a respective bounding box representation of the first vehicle from the perspective of the second vehicle for each image;
determine, by the at least one processor, for each image of the set of images whether the first vehicle and the second vehicle are positioned in a respective common lane of travel;
when the first vehicle and the second vehicle are positioned in the respective common lane of travel;
determine, by the at least one processor, a rate of change of at least one spatial parameter of the bounding box for the first vehicle across at least a subset of the set of images;
apply, by at least one processor, a collision detection model to identify a collision between the first vehicle and the second vehicle based on the rate of change of the at least one spatial parameter of the bounding box;
and output an indication of collision when the collision detection model identifies a collision between the first vehicle and the second vehicle; and
when the first vehicle and the second vehicle are not positioned in the respective common lane of travel;
figure 3, [0032, 33], objects in bounding boxes 305, 315, and 325. The bounding boxes are envisioned from a perspective to the rear of the above vehicles. (this passage discloses vehicles not in a common lane of travel)
determine, by the at least one processor that there is no collision between the first vehicle and the second vehicle.
See the rejection of claim 1, which discloses the same subject matter as claim 9 and is rejected for the same reasons.
On claim 10, Gan cites except as underlined:
The system of claim 9, wherein if the processor-executable instructions which cause the system to apply the object detection model to each image in the set of images to determine a respective bounding box representation of the first vehicle from the perspective of the second vehicle for each image does not result in a determination of any bounding boxes: the processor-executable instructions cause the system to identify no collision between the first vehicle and the second vehicle.
[0033] The distance to an object (e.g., a vehicle) and the speed of the object (e.g., relative to another object) can be identified by image processing which can also determine the bounding boxes for each object. The speed of an object can be determined, for example, from the time history of the bounding boxes associated with the object. Based on the distance to the object and the speed of the object, an amount of time before a possible crash between the object and the autonomous vehicle may be determined.
(The above passage indicates a warning, or a possibility, and not an actual occurrence of a crash between identified objects encircled with bounding boxes).
Furthermore, Gan cites:
[0035] A second challenge the technology disclosed herein overcomes is identification of the direction of travel of moving vehicles identified in successive images. Vehicles moving toward the autonomous vehicle camera will generate bounding boxes that are larger in size in images taken at later times compared to images taken at earlier times. Vehicles moving away from the autonomous vehicle will have bounding boxes that are smaller in size in images taken at later times compared to images taken at earlier times. For vehicles at a close range from the autonomous vehicle, successive bounding boxes can be combined to enable a quick determination of the direction of travel. For vehicles that are far away, a longer history may be used to make a reliable determination of the direction of travel. Distant objects must move a greater distance for the autonomous vehicle to be able to determine whether the bounding boxes are getting smaller or larger, and an estimate of their position will have more noise associated with it.
The claimed invention is interpreted to mean bounding boxes are not created when there are no collision hazards determined between collocated vehicles.
Gan discloses bounding boxes being created whenever vehicles are located within a detection distance of the autonomous vehicle. Gan does not disclose the feature of creating bounding boxes when a vehicle is detected from the autonomous vehicle. However, it would have been obvious to one of ordinary skill in the art at the time of the claimed invention to include into Gan the feature of determining a detected vehicle is not deemed to be a collision hazard. Gan, as above, discloses creating bounding boxes that become smaller as detected vehicles move away from the autonomous vehicle. Thus, unlike the claimed invention where no bounding boxes are created responsive to a likely threat of a colliding vehicle, Gan discloses an embodiment where the bounding box becomes smaller as it is moves away from the automated vehicle. In either case, each respective embodiment is disclosing a way to show to a user how a “non-threatening” vehicle is given less of a consideration as possible collision threat by either not constructing a bounding box or reducing the size of a bounding box to where the smaller the bounding box is shown. In either case, one of ordinary skill would have implement a similar strategy in making detected vehicles not considered a collision hazard by reducing their visibility to the autonomous vehicle.
On claim 12, Gan cites:
The system of claim 9, further comprising the at least one image capture device positioned at the second vehicle, wherein the processor-executable instructions which cause the system to access the image data cause the system to capture the image data by the at least one image capture device positioned at the second vehicle.
[0023] and figure 2, Autonomous vehicle 230 has one or more cameras as described with respect to FIG. 1.
And
figure 3, [0032, 33], objects in bounding boxes 305, 315, and 325. The bounding boxes are envisioned from a perspective to the rear of the above vehicles. (The claimed “first and second vehicles” are presumed to be any one of the vehicles picture in figure 3).
And
[0030] The disclosed techniques and devices include a computer vision solution using cameras aimed away from a side of the autonomous vehicle to detect potentially interfering vehicles and objects. The images are processed to determine when and if it is “safe to go” (e.g., safe to proceed along a planned path) in a “T”-type and many other types of intersections. Cameras aimed toward space on the side of the autonomous vehicle (e.g., the areas surrounding the vehicle to the right and left) capture images, and image processing of the images captured by the cameras are used to generate bounding boxes for objects identified by the image processing, according to the disclosed technology.
On claim 14, Gan cites:
The system of claim 9, wherein the processor executable instructions which cause the system to access the image data cause the system to retrieve the image data stored at the non-transitory processor readable storage medium.
[0006] Another aspect of the disclosed embodiments relates to an apparatus for an autonomous driving vehicle that comprises at least one processor and a memory. The memory of the apparatus includes executable instructions that, when executed by the at least one processor, cause the apparatus to perform at least the following operations: receive a series of road images from a side-view camera sensor of the autonomous driving vehicle, wherein each image in the series of road images is taken at a different time; generate, for each object from objects captured in the series of road images, a series of bounding boxes in the series of road images, wherein each bounding box in the series of bounding boxes corresponds to an image in the series of road images; determine, for each object from the objects, a direction of travel or that the object is stationary; determine a speed of each object for which the direction of travel has been determined.
Claims 8, 13, and 15-17 are rejected under 35 USC 103 as being unpatentable over Gan et al., U.S. 2021/0403050 in view of Nakajima et al., U.S. 6,285, 778, and Liu et al., U.S.2019/0103026 and Rishi et al. , U.S.10,814,815 and Sicconi et al., U.S. 2019/0213429.
On claim 8, Gan cites except as underlined:
The method of claim 6, further comprising transmitting, by a communication interface of the device remote from the second vehicle, the indication of collision to at least one other vehicle in a geographic region proximate a position of the second vehicle.
As in the rejection of claim 6, Gan disclosed a remote server working in conjunction with an autonomous vehicle, wherein the remote server “determin[es] whether the autonomous driving vehicle can safety move in a predetermined direction.” Implicit in this feature is the autonomous vehicle is sending the bounding box images to the remote server in order for the remote server to ascertain if the autonomous vehicle can be safely moved. Furthermore, and as previously discussed, Nakajima disclosed the feature of providing a notification upon an actual collision encountered between the automated vehicle and another vehicle. Neither reference speaks to the feature of transmitting the collision event data to a geographically proximate other vehicle.
In the same art of vehicle collision warning, Sicconi cites:
[0063] A driving risk model 1016 and accident detection/prevention unit 1017 analyzes features 1011 (e.g. vehicles ahead, pedestrian crossing the road, cyclists, animals, trees, road sign posts) from a road-facing camera 1004, features 1012 from a rear-facing camera 1005 such as tailgating vehicles coming too close, features 1013 from telematics data 1006 such as speed, acceleration, braking, cornering, engine load, fuel consumption, features 1014 from ambient data (2007) such as weather/traffic information.
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to include into Gan and Nakajima the telematics feature disclosed in Siccione such that the claimed embodiment is realized.
Siccione discloses a known embodiment for communicating information from one vehicle to another vehicle wherein each vehicle recognizes each other in a common communications network. One of ordinary skill would have communicated the collision event to the other vehicle in the telematics network to allow the non-collision vehicle’s driver to contact authorities on the accident.
On claim 13, Gan cites except as underlined:
The system of claim 12, wherein the system includes a telematic device positioned at the second vehicle, and the at least one processor and the at least one non-transitory processor-readable storage medium are included in the telematic device.
Gan, as previously stated, disclosed:
figure 3, [0032, 33], objects in bounding boxes 305, 315, and 325. The bounding boxes are envisioned from a perspective to the rear of the above vehicles. (The claimed “first and second vehicles” are presumed to be any one of the vehicles picture in figure 3).
And
[0030] The disclosed techniques and devices include a computer vision solution using cameras aimed away from a side of the autonomous vehicle to detect potentially interfering vehicles and objects. The images are processed to determine when and if it is “safe to go” (e.g., safe to proceed along a planned path) in a “T”-type and many other types of intersections. Cameras aimed toward space on the side of the autonomous vehicle (e.g., the areas surrounding the vehicle to the right and left) capture images, and image processing of the images captured by the cameras are used to generate bounding boxes for objects identified by the image processing, according to the disclosed technology.
Gan does not disclose using telematics.
In the same art of vehicle collision warning, Siccione cites:
[0063] A driving risk model 1016 and accident detection/prevention unit 1017 analyzes features 1011 (e.g. vehicles ahead, pedestrian crossing the road, cyclists, animals, trees, road sign posts) from a road-facing camera 1004, features 1012 from a rear-facing camera 1005 such as tailgating vehicles coming too close, features 1013 from telematics data 1006 such as speed, acceleration, braking, cornering, engine load, fuel consumption, features 1014 from ambient data (2007) such as weather/traffic information.
[0063] A driving risk model 1016 and accident detection/prevention unit 1017 analyzes features 1011 (e.g. vehicles ahead, pedestrian crossing the road, cyclists, animals, trees, road sign posts) from a road-facing camera 1004, features 1012 from a rear-facing camera 1005 such as tailgating vehicles coming too close, features 1013 from telematics data 1006 such as speed, acceleration, braking, cornering, engine load, fuel consumption, features 1014 from ambient data (2007) such as weather/traffic information.
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to include into Gan the telematics feature disclosed in Saccioni such that the claimed invention is realized. Saccioni discloses a known feature in which telematics is used in conjunction with collision determination devices as found in Nakajima and Gan and one of ordinary skill would have used telematics to communicate collision events to a receiving party.
On claim 15, Gan and Nakajima cites except as underlined:
The system of claim 9, further comprising a communication interface at the second vehicle, wherein the at least one processor is positioned at the second vehicle, and the processor-executable instructions
[0006] Another aspect of the disclosed embodiments relates to an apparatus for an autonomous driving vehicle that comprises at least one processor and a memory. The memory of the apparatus includes executable instructions that, when executed by the at least one processor, cause the apparatus to perform at least the following operations: receive a series of road images from a side-view camera sensor of the autonomous driving vehicle, wherein each image in the series of road images is taken at a different time; generate, for each object from objects captured in the series of road images, a series of bounding boxes in the series of road images, wherein each bounding box in the series of bounding boxes corresponds to an image in the series of road images; determine, for each object from the objects, a direction of travel or that the object is stationary; determine a speed of each object for which the direction of travel has been determined.
which cause the system to output an indication of collision cause the communication interface at the second vehicle to transmit the indication of collision to be received by a device remote from the second vehicle.
Gan previous disclosed:
[0064] The apparatus may use a machine learning device (which, for example, can be a part of the apparatus or, alternatively, can be a separate device that is, for example, externally or remotely located relative to the apparatus; for example, the machine learning device can be a remote server located “in the cloud”) to perform the process of determining whether the autonomous driving vehicle can safely move in a predetermined direction.
In other words, Gan’s embodiment is disclosing a communications feature wherein image data or bounding boxes are provided to the remote server for analysis. The responsive action from the server determines, for the autonomous vehicle, actions to safely move the vehicle in a predetermined direction.
Furthermore, as indicated in the rejection of claim 9, Nakajima disclosed an embodiment wherein a collision is detected and a collision alarm relayed to the driver.
However, neither reference discloses sending the information to another vehicle.
In the same art of vehicle collision warning, Saccioni cites:
[0063] A driving risk model 1016 and accident detection/prevention unit 1017 analyzes features 1011 (e.g. vehicles ahead, pedestrian crossing the road, cyclists, animals, trees, road sign posts) from a road-facing camera 1004, features 1012 from a rear-facing camera 1005 such as tailgating vehicles coming too close, features 1013 from telematics data 1006 such as speed, acceleration, braking, cornering, engine load, fuel consumption, features 1014 from ambient data (2007) such as weather/traffic information.
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to include into Gan and Nakajima the telematics feature disclosed in Saccioni such that the claimed embodiment is realized.
Saccioni discloses a known embodiment for communicating information from one vehicle to another vehicle wherein accident information is relayed via one vehicle to another via a telematics system. One of ordinary skill would have communicated the collision event to the other vehicle in the telematics network to allow the non-collision vehicle’s driver to contact authorities on the accident.
On claim 16, Gan cites except as underlined:
The system of claim 15, further comprising the device remote from the second vehicle,
[0064] for example, the machine learning device can be a remote server located “in the cloud”) to perform the process of determining whether the autonomous driving vehicle can safely move in a predetermined direction.
wherein the processor-executable instructions further
[0006] Another aspect of the disclosed embodiments relates to an apparatus for an autonomous driving vehicle that comprises at least one processor and a memory. The memory of the apparatus includes executable instructions that, when executed by the at least one processor, cause the apparatus to perform at least the following operations: receive a series of road images from a side-view camera sensor of the autonomous driving vehicle, wherein each image in the series of road images is taken at a different time; generate, for each object from objects captured in the series of road images, a series of bounding boxes in the series of road images, wherein each bounding box in the series of bounding boxes corresponds to an image in the series of road images; determine, for each object from the objects, a direction of travel or that the object is stationary; determine a speed of each object for which the direction of travel has been determined.
cause the system to output, by an output interface of the device remote from the second vehicle, an alert to an operator of the device remote from the second vehicle.
Gan discloses:
[0030] The disclosed techniques and devices include a computer vision solution using cameras aimed away from a side of the autonomous vehicle to detect potentially interfering vehicles and objects. The images are processed to determine when and if it is “safe to go” (e.g., safe to proceed along a planned path) in a “T”-type and many other types of intersections. Cameras aimed toward space on the side of the autonomous vehicle (e.g., the areas surrounding the vehicle to the right and left) capture images, and image processing of the images captured by the cameras are used to generate bounding boxes for objects identified by the image processing, according to the disclosed technology.
And
figure 3, [0032, 33], objects in bounding boxes 305, 315, and 325. The bounding boxes are envisioned from a perspective to the rear of the above vehicles.
Gan doesn’t specifically disclose the excepted claim limitations. However, it would have been obvious to one of ordinary skill in the art at the time of the claimed invention to try and include into Gan an arrangement of known elements disclosed in Gan to realize an embodiment meeting the claimed invention.
The claimed “device remote from the second vehicle” is the cited remote server. The remote server provides analysis and direction for “determining whether the autonomous driving vehicle can safely move in a predetermined direction.” As was disclosed in Gan, figure 3, bounding boxes are used to identify vehicles which are potential collision hazards. One of ordinary skill, apprised of the known but limited functions and structure disclosed in Gan, would have used the remote server, having inputs from the vehicles disclosed in figure 3, to inform the user of possible collision hazards.
On claim 17, Gan cites except as underlined:
The system of claim 15 further comprising the device remote from the second vehicle,
[0064] for example, the machine learning device can be a remote server located “in the cloud”) to perform the process of determining whether the autonomous driving vehicle can safely move in a predetermined direction.
wherein the processor-executable instructions
[0006] Another aspect of the disclosed embodiments relates to an apparatus for an autonomous driving vehicle that comprises at least one processor and a memory. The memory of the apparatus includes executable instructions that, when executed by the at least one processor, cause the apparatus to perform at least the following operations: receive a series of road images from a side-view camera sensor of the autonomous driving vehicle, wherein each image in the series of road images is taken at a different time; generate, for each object from objects captured in the series of road images, a series of bounding boxes in the series of road images, wherein each bounding box in the series of bounding boxes corresponds to an image in the series of road images; determine, for each object from the objects, a direction of travel or that the object is stationary; determine a speed of each object for which the direction of travel has been determined.
further cause the system to transmit, by a communication interface of the device remote from the second vehicle,
the indication of collision to at least one other vehicle in a geographic region proximate a position of the second vehicle.
Gan previous disclosed:
[0064] The apparatus may use a machine learning device (which, for example, can be a part of the apparatus or, alternatively, can be a separate device that is, for example, externally or remotely located relative to the apparatus; for example, the machine learning device can be a remote server located “in the cloud”) to perform the process of determining whether the autonomous driving vehicle can safely move in a predetermined direction.
In other words, Gan’s embodiment is disclosing a communications feature wherein image data or bounding boxes are provided to the remote server for analysis. The responsive action from the server determines, for the autonomous vehicle, actions to safely move the vehicle in a predetermined direction.
Furthermore, as indicated in the rejection of claim 9, Nakajima disclosed an embodiment wherein a collision is detected and a collision alarm relayed to the driver.
However, neither reference discloses sending the information to another vehicle.
In the same art of vehicle collision warning, Saccioni cites:
[0063] A driving risk model 1016 and accident detection/prevention unit 1017 analyzes features 1011 (e.g. vehicles ahead, pedestrian crossing the road, cyclists, animals, trees, road sign posts) from a road-facing camera 1004, features 1012 from a rear-facing camera 1005 such as tailgating vehicles coming too close, features 1013 from telematics data 1006 such as speed, acceleration, braking, cornering,
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to include into Gan and Nakajima the telematics feature disclosed in Saccioni such that the claimed embodiment is realized.
Saccioni discloses a known embodiment for communicating information from one vehicle to another vehicle wherein each vehicle recognizes each other in a common communications network. One of ordinary skill would have communicated the collision event to the other vehicle in the telematics network to allow the non-collision vehicle’s driver to contact authorities on the accident.
Claims 18-21 are rejected under 35 USC 103 as being unpatentable over Gan et al., U.S. 2021/0403050 in view of Nakajima et al., U.S. 6,285, 778 and Liu et al., U.S.2019/0103026 and Rishi et al. , U.S.10,814,815 and Kim, U.S. 2021/0197824.
On claim 18, Gan cites except:
The method of claim 1, wherein determining that the first vehicle and the second vehicle are positioned in a respective common lane of travel comprises:
applying a feature detection model to identify road lanes;
identifying a lane of travel of the of the second vehicle; and
determining that the first vehicle is in the lane of travel of the second vehicle.
In the rejection of claim 1, Liu cited:
The collision warning system 100 may crop a portion of the image frame 210 to focus on a detected object in front of the vehicle 140, such as another vehicle in the same lane on the road.
However, neither Gan nor Kim cited the excepted claim limitations.
In the similar art of lane keeping inventions, Kim cites:
[0135] As shown in FIG. 5B, the second controller 250 may recognize a driving pattern of another vehicle based on a distance change (d1->d2) between the recognized lane and any one part p of the other vehicle.
[0136] As illustrated in FIG. 5B, the second controller 250 may determine whether another vehicle has invaded or deviated from the lane based on the recognized position of the other vehicle and the position of the lane.
It would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to include into Gan the vehicle detecting features of Kim such that the claimed invention is realized.
Kim discloses an embodiment for determining the position of different vehicles in a lane. Since the cited controller 250 is measuring the respective distances d1 and d2 of the cited vehicles, it is clear and inherent the controller has identified the lane of travel for both vehicles as being in the same lane since the right side of the lane is used as a reference point to determine the respective distances d1 and d2 of the vehicles.
Furthermore, because these distances are different, it is also clear the controller is determining there are two distinct vehicles traveling the lane.
One of ordinary skill would have included Kim’s known features into Gan and the results of the substitution would have realized an embodiment including known elements to realize the claimed invention.
On claim 19, Gan cites except as underlined:
The method of claim 1, wherein determining that the first vehicle and the second vehicle are positioned in a respective common lane of travel comprises: determining a left-distance from a left image edge and a right-distance from a right image edge for a representation of the first vehicle in the image of the set of images; and determining that the first vehicle and the second vehicle are in a common lane of travel in response to a difference between the left-distance and the right-distance being within a horizontal distance threshold.
In the rejection of claim 18, which includes citations to Gan and Kim, which are applicable to the above claim limitations except as indicated, Kim disclosed:
[0135] As shown in FIG. 5B, the second controller 250 may recognize a driving pattern of another vehicle based on a distance change (d1->d2) between the recognized lane and any one part p of the other vehicle.
[0136] As illustrated in FIG. 5B, the second controller 250 may determine whether another vehicle has invaded or deviated from the lane based on the recognized position of the other vehicle and the position of the lane.
Furthermore, Kim discloses:
[0110] Here, the image information of the road may include an image of a lane and an image of another vehicle.
[0131] When the warning mode while driving is manually selected or when driving autonomously, the second controller 250 performs image processing on the image information acquired by the image acquirer 210 to recognize lanes and other vehicles on the road, recognizes the own lane in which the own vehicle is traveling based on the location information of the recognized lane, and recognizes the own lane and other vehicles running in the other lane based on the positions of both lanes in the own lane. The own lane and other vehicles driving in the other lane may be vehicles driving in front of the own vehicle.
In other words, Kim discloses the image information being used to determine distances d1 and d2 of the detected vehicles shown in figure 5B. However, the distances d1 and d2 refer to the right side of the land. There are no references involving the left-distance within the claimed horizontal threshold.
However, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to try and include an embodiment in which each disclosed distance also yields a left-distance measurement.
Kim already discloses, in figure 5B, distances d1, d2, a right lane boundary from which distances d1 and d2 are measured. While not specifically stated, Kim defined at least a lane to which the vehicles discloses are travelling. It is likely the lane on which the vehicles are traveling is uniform width. Furthermore, the width of the shown vehicles are known. Furthermore, if controller 250 can determine distance changes on the right for the vehicles, it likely can detect distance changes, and therefore, distances from the left side of the lane to each respective vehicle. Accordingly, one of ordinary skill, apprised of these known but limited quantities, would have arrived at an embodiment in which left-side distance measurements, as required by the claim, are disclosed.
On claim 20, Gan and Kim cites:
The system of claim 9, wherein the processor executable instructions which cause the at least one processor to determine whether the first vehicle and the second vehicle are positioned in a respective common lane of travel cause the at least one processor to: apply a feature detection model to identify road lanes; identify a lane of travel of the of the second vehicle; and determine whether the first vehicle is in the lane of travel of the second vehicle.
See the rejection of claim 18 which discloses the same subject matter as claim 20 and is rejected for the same reasons.
On claim 21, Gan and Kim cites:
The system of claim 9, wherein the processor-executable instructions which cause the at least one processor to determine whether the first vehicle and the second vehicle are positioned in a respective common lane of travel cause the at least one processor to:
determine a left-distance from a left image edge and a right-distance from a right image edge for a representation of the first vehicle in the image of the set of images; and determine that the first vehicle and the second vehicle are in a common lane of travel if a difference between the left-distance and the right-distance are within a horizontal distance threshold.
See the rejection of claim 18 which discloses the same subject matter as claim 21 and is rejected for the same reasons.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAL EUSTAQUIO whose telephone number is (571)270-7229. The examiner can normally be reached on 8am-5pm. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Brian Zimmerman, can be reached at (571) 272-3059. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application lnformation Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAlR only. For more information about the PAlR system, see http:/lpair-direct.uspto.gov. Should you have questions on access to the Private PAlR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAL J EUSTAQUIO/Examiner, Art Unit 2686
/BRIAN A ZIMMERMAN/Supervisory Patent Examiner, Art Unit 2686