Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 102(a)(1) as being unpatentable by US 20230391365 A1 to Ivanovic et al. (“Ivanovic”).
As to claim 1, Ivanovic teaches a method for vehicle blind spot object tracking, comprising: performing object detection using a machine learning model based on video data captured by one or more cameras associated with a vehicle in order to detect an object (Fig. 14B, ¶0112, may be used for depth-based object detection, especially for objects for which a neural network has not yet been trained); predicting an intent associated with the object based on one or more features associated with the object in the video data (¶0031, The system(s) may also input at least a portion of the input data and/or at least a portion of the goals data into a second model(s) that is trained to generate data (also referred to, in some examples, as “actions data”) representing one or more actions associated with the one or more navigational goals of the one or more objects. For instance, and for an object, the second model(s) may determine one or more predicted trajectories associated with the object over the given period of time. In some examples, to determine the one or more predicted trajectories, the second model(s) may predict, based on the goal(s), the controls of the object at future time steps within the given period of time, where the controls may include the respective location, velocity, acceleration, direction of travel, and/or the like associated with the object at each time step. The second model(s) may then forward integrate the controls through the object's dynamic model); applying a collision prediction algorithm based on the object, the intent associated with the object, and one or more measured attributes of the vehicle, in order to predict a proximity between the vehicle and the object in a given direction (Fig. 6, ¶0175); and generating, after determining an intent to move the vehicle in the given direction, an alert for presentation within the vehicle based on the predicted proximity between the vehicle and the object (¶0186-0187, FCW systems are designed to alert the driver to a hazard, so that the driver may take corrective action. FCW systems use a front-facing camera and/or RADAR sensor(s) 1460, coupled to a dedicated processor, DSP, FPGA, and/or ASIC, that is electrically coupled to driver feedback, such as a display, speaker, and/or vibrating component. FCW systems may provide a warning, such as in the form of a sound, visual warning, vibration and/or a quick brake pulse, AEB systems detect an impending forward collision with another vehicle or other object, and may automatically apply the brakes if the driver does not take corrective action within a specified time or distance parameter. AEB systems may use front-facing camera(s) and/or RADAR sensor(s) 1460, coupled to a dedicated processor, DSP, FPGA, and/or ASIC. When the AEB system detects a hazard, it typically first alerts the driver to take corrective action to avoid the collision and, if the driver does not take corrective action, the AEB system may automatically apply the brakes in an effort to prevent, or at least mitigate, the impact of the predicted collision. AEB systems, may include techniques such as dynamic brake support and/or crash imminent braking).
As to claim 2, Ivanovic teaches the method of claim 1, wherein the applying of the collision prediction algorithm is further based on route data for the vehicle, and wherein route data for the vehicle is based on a configured route associated with a satellite-based navigation system (¶0101, A steering system 1454, which may include a steering wheel, may be used to steer the vehicle 1400 (e.g., along a desired path or route) when the propulsion system 1450 is operating (e.g., when the vehicle is in motion). The steering system 1454 may receive signals from a steering actuator 1456. The steering wheel may be optional for full automation (Level 5) functionality,¶0104, satellite global navigation).
As to claim 3, Ivanovic teaches the method of claim 1, further comprising: estimating a distance and a trajectory associated with the object relative to the vehicle based on performing multiple object tracking (MOT) using a computer vision technique after the detecting of the object, wherein the applying of the collision prediction algorithm is based on the distance and the trajectory (¶0031, may determine one or more predicted trajectories associated with the object over the given period of time. In some examples, to determine the one or more predicted trajectories, the second model(s) may predict, based on the goal(s), the controls of the object at future time steps within the given period of time, where the controls may include the respective location, velocity, acceleration, direction of travel, and/or the like associated with the object at each time step. The second model(s) may then forward integrate the controls through the object's dynamic model).
As to claim 4, Ivanovic teaches the method of claim 3, wherein the one or more measured attributes comprises one or more of a speed of the vehicle (¶0104, speed of the vehicle), a steering wheel angle of the vehicle, or a heading of the vehicle.
As to claim 5, Ivanovic teaches the method of claim 4, wherein the applying of the collision prediction algorithm is further based on performing an optical flow, block matching, or Kalman filtering technique with respect to the video data to monitor movement of the object (¶0179, IMU sensor(s) 1466 may be implemented as a miniature, high performance GPS-Aided Inertial Navigation System (GPS/INS) that combines micro-electro-mechanical systems (MEMS) inertial sensors, a high-sensitivity GPS receiver, and advanced Kalman filtering algorithms to provide estimates of position, velocity, and attitude. As such, in some examples, the IMU sensor(s) 1466 may enable the vehicle 1400 to estimate heading without requiring input from a magnetic sensor by directly observing and correlating the changes in velocity from GPS to the IMU sensor(s) 1466. In some examples, the IMU sensor(s) 1466 and the GNSS sensor(s) 1458 may be combined in a single integrated unit).
As to claim 6, Ivanovic teaches the method of claim 1, further comprising: providing a sliding window of frames from the video data as inputs to a convolutional neural network (CNN) and a recurrent neural network (RNN) to detect one or more traffic lanes (¶0130, CNN and RCNN); extracting a segmentation mask of the object from the frames; and comparing boundaries of the one or more traffic lanes to the segmentation mask of the object to predict whether the object is in an adjacent lane to the vehicle, wherein the generating of the alert is further based on the predicting of whether the object is in the adjacent lane to the vehicle (¶0177, to illuminate vehicle surroundings up to approximately 200 m. A flash LIDAR unit includes a receptor, which records the laser pulse transit time and the reflected light on each pixel, which in turn corresponds to the range from the vehicle to the objects. Flash LIDAR may allow for highly accurate and distortion-free images of the surroundings to be generated with every laser flash. In some examples, four flash LIDAR sensors may be deployed, one at each side of the vehicle 1400. Available 3D flash LIDAR systems include a solid-state 3D staring array LIDAR camera with no moving parts other than a fan (e.g., a non-scanning LIDAR device). The flash LIDAR device may use a 5 nanosecond class I (eye-safe) laser pulse per frame and may capture the reflected laser light in the form of 3D range point clouds and co-registered intensity data. By using flash LIDAR, and because flash LIDAR is a solid-state device with no moving parts, the LIDAR sensor(s) 1464 may be less susceptible to motion blur, vibration, and/or shock, ¶0109).
As to claim 7, Ivanovic teaches the method of claim 6, wherein the comparing of the boundaries of the one or more traffic lanes to the segmentation mask of the object comprises computing an intersection over union (IoU) of the boundaries of the one or more traffic lanes to the segmentation mask of the object and comparing the IoU to a threshold (¶0145, The DLA may be used to run any type of network to enhance control and driving safety, including for example, a neural network that outputs a measure of confidence for each object detection. Such a confidence value may be interpreted as a probability, or as providing a relative “weight” of each detection compared to other detections. This confidence value enables the system to make further decisions regarding which detections should be considered as true positive detections rather than false positive detections. For example, the system may set a threshold value for the confidence and consider only the detections exceeding the threshold value as true positive detections. In an automatic emergency braking (AEB) system, false positive detections would cause the vehicle to automatically perform emergency braking, which is obviously undesirable. Therefore, only the most confident detections should be considered as triggers for AEB).
As to claim 8, Ivanovic teaches the method of claim 1, further comprising analyzing data captured using a rear- facing camera associated with the vehicle in order to predict a trajectory of the object, wherein the generating of the alert is further based on the predicting of the trajectory of the object (¶0190, BSW systems detects and warn the driver of vehicles in an automobile's blind spot. BSW systems may provide a visual, audible, and/or tactile alert to indicate that merging or changing lanes is unsafe. The system may provide an additional warning when the driver uses a turn signal. BSW systems may use rear-side facing camera(s) and/or RADAR sensor(s) 1460, coupled to a dedicated processor, DSP, FPGA, and/or ASIC, that is electrically coupled to driver feedback, such as a display, speaker, and/or vibrating component).
As to claim 9, Ivanovic teaches the method of claim 1, wherein the determining of the intent to move the vehicle in the given direction is based on one or more of: activation of a turn signal of the vehicle; or route data for the vehicle (¶0188, LDW systems provide visual, audible, and/or tactile warnings, such as steering wheel or seat vibrations, to alert the driver when the vehicle 1400 crosses lane markings. A LDW system does not activate when the driver indicates an intentional lane departure, by activating a turn signal. LDW systems may use front-side facing cameras, coupled to a dedicated processor, DSP, FPGA, and/or ASIC, that is electrically coupled to driver feedback, such as a display, speaker, and/or vibrating component).method of claim 1, wherein the generating of the alert comprises generating a graphical indicator of the object for display via a screen within the vehicle (¶0105, One or
As to claim 10, Ivanovic teaches the more of the controller(s) 1436 may receive inputs (e.g., represented by input data) from an instrument cluster 1432 of the vehicle 1400 and provide outputs (e.g., represented by output data, display data, etc.) via a human-machine interface (HMI) display 1434, an audible annunciator, a loudspeaker, and/or via other components of the vehicle 1400. The outputs may include information such as vehicle velocity, speed, time, map data (e.g., the High Definition (“HD”) map 1422 of FIG. 14C), location data (e.g., the vehicle's 1400 location, such as on a map), direction, location of other vehicles (e.g., an occupancy grid), information about objects and status of objects as perceived by the controller(s) 1436, etc. For example, the HMI display 1434 may display information about the presence of one or more objects (e.g., a street sign, caution sign, traffic light changing, etc.), and/or information about driving maneuvers the vehicle has made, is making, or will make (e.g., changing lanes now, taking exit 34B in two miles, etc.).
As to claim 11, see the rejection of claim 1.
As to claim 12, see the rejection of claim 2.
As to claim 13, see the rejection of claim 3.
As to claim 14, see the rejection of claim 4.
As to claim 15, see the rejection of claim 5.
As to claim 16, see the rejection of claim 6.
As to claim 17, see the rejection of claim 7.
As to claim 18, see the rejection of claim 8.
As to claim 19, see the rejection of claim 9.
As to claim 20, see the rejection of claim 1.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTINE A KURIEN whose telephone number is (571)270-5694. The examiner can normally be reached M-F; 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nathan Flynn can be reached at 571-272-1915. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTINE A KURIEN/Examiner, Art Unit 2421 /NATHAN J FLYNN/Supervisory Patent Examiner, Art Unit 2421
Prosecution Insights