DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/30/2026 has been entered.
Response to Arguments
In regards to Argument(s), Applicant(s) state(s) that, the combination of Ollis and Goel fails to teach or suggest a transformer network and a controller configured to predict a driving path of a target vehicle based on the transformer network. The Office Action alleges that Wang discloses these claimed features and also teaches that "the transformer network is configured to encode pieces of space information of the respective vehicles with respect to driving lanes of the respective vehicles," as recited in claim 1. Applicants respectfully traverse the rejection. The cited portions of Wang fail to disclose or suggest the claimed transformer network "configured to encode pieces of space information of the respective vehicles with respect to driving lanes of the respective vehicles", therefore, the rejection of 35 U.S.C. 103 should be removed, (Emphasis added, Remarks, page(s) 7-10).
Applicant’s arguments have been considered but are moot in view of the new ground(s) of rejection in view of Ollis et al (US 2023/0278581 A1) and Liu et al (US 2023/0356753 A1).
Office Action Summary
Claim(s) 3-4, 6-7, 9, 12-13, 15-16, and 18 is/are canceled.
Claim(s) 1-2 and 10-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ollis et al (US 2023/0278581 A1) in view of Liu et al (US 2023/0356753 A1).
Claim(s) 5 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ollis et al (US 2023/0278581 A1) in view of Goel (US 2023/0033177 A1), further in view of Takamatsu et al (US 2016/0090084 A1).
Claim(s) 8 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ollis et al (US 2023/0278581 A1) in view of Liu et al (US 2023/0356753 A1), further in view of Wang (US 2023/0211660 A1).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2 and 10-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ollis et al (US 2023/0278581 A1) in view of Liu et al (US 2023/0356753 A1).
Regarding claim(s) 1 and 10, Ollis teaches an apparatus for predicting a driving path of a vehicle, the apparatus comprising:
a camera configured to capture an image around an ego vehicle (Figure 2; and Paragraph [0070]: “such sensors may include, without limitation, a laser detection and ranging (LiDAR) system […] one or more cameras (e.g., visible spectrum cameras, infrared cameras, etc.) […] The sensor data can include information that describes the location of objects within the surrounding environment of AV 102, information about the environment itself, information about the motion of AV 102, information about a route of AV 102, and/or the like”);
a light detection and ranging (LiDAR) sensor configured to generate a point cloud around the ego vehicle (Figure 2; and Paragraph [0064]: “sensor data may include light detection and ranging (LiDAR) point cloud maps (e.g., map point data, etc.) associated with a geographic location (e.g., a location in three-dimensional space relative to the LiDAR system of a mapping vehicle in one or more roadways) of a number of points (e.g., a point cloud) that correspond to objects that have reflected a ranging laser of one or more mapping vehicles at the geographic location (e.g. an object such as a vehicle, bicycle, pedestrian, etc. in the roadway). As an example, sensor data may include LiDAR point cloud data that represents objects in the roadway, such as, other vehicles, pedestrians, cones, debris, and/or the like”); and
a controller configured to detect pieces of feature information of respective vehicles located around the ego vehicle based on the image and the point cloud and configured to predict a driving path of a target vehicle based on the pieces of feature information of the respective vehicles (Paragraph [0102] – Paragraph [0104]: “For example, prediction system 316 may process sensor data (e.g., from LiDAR, RADAR, camera images, etc.) in order to identify objects and/or features in and around the geospatial area of the autonomous vehicle. Detected objects may include traffic signals, roadway boundaries, vehicles, pedestrians, obstacles in the roadway, and/or the like. Perception detection 302 may use known object recognition and detection algorithms, video tracking algorithms, or computer vision algorithms (e.g., tracking objects frame-to-frame iteratively over a number of time periods, etc.) to perceive an environment of AV 102 [...] perception detection 302 may also determine, for one or more identified objects in the environment, a current state of the object. The state information may include, without limitation, for each object: current location; current speed and/or acceleration; current heading; current orientation; size/footprint; type (e.g., vehicle vs. pedestrian vs. bicycle vs. static object or obstacle); and/or other state information [...] Prediction system 316 may predict the future locations, trajectories, and/or actions of such objects perceived in the environment, based at least in part on perception information (e.g., the state data for each object) received from perception detection 302, the location information received from location system 312, sensor data, and/or any other data related to a past and/or current state of an object, the autonomous vehicle, the surrounding environment, and/or relationship(s)”).
Ollie fails to teach a storage configured to store a transformer network, training of which is completed; wherein the controller is further configured to predict the driving path of the target vehicle based on the transformer network, wherein the transformer network is configured to encode pieces of space information of the respective vehicles with respect to driving lanes of the respective vehicles.
However, Liu teaches a storage configured to store a transformer network, training of which is completed (Paragraph [0035]: “Further optionally, feeding respective states-related data 61 and segment-related data 62 to one or more neural networks may comprise—and/or the spatial/temporal NN feeding unit 106 may be adapted and/or configured for—feeding to a transformer-based network, e.g. supporting multi-head and/or self-attention operations”); and
wherein the controller is further configured to predict the driving path of the target vehicle based on the transformer network (Paragraph [0039]: “feeding output spatial- and temporal-processed respective states-related data 811 and segment-related data 821 […] to at least a first behavior-predicting neural network 9, may comprise—and/or the predicting NN feeding unit 107 may be adapted and/or configured […] by processing said data 811, 821, predict and output data 90 indicating predicted trajectories of one or more of the target road user(s) 31 in view of the one or more lane segments 4”),
wherein the transformer network is configured to encode pieces of space information of the respective vehicles with respect to driving lanes of the respective vehicles (Figure 1; Figure 2; Figure 3; Paragraph [0027]: “[…] view of an exemplifying lane-merge traffic scenario, the assessment system 1 is further—e.g. by means of a road users states obtaining unit 103—adapted and/or configured for obtaining states of road users 3 in the vehicle's 2 surroundings”; Paragraph [0031]: “by means of a data encoding unit 105—adapted and/or configured for encoding network input 5 comprising data associated with the lane segments 4, data associated with the road users states and historic data associated with the road users states and/or lane segments 4, into respective states-related data 61 associated with the road users states and segment-related data 62 associated with dynamic start and end boundaries of the lane segments 4”; Paragraph [0034]: “by means of a spatial/temporal NN feeding unit 106—adapted and/or configured for feeding the respective states-related data 61 and segment-related data 62 to one or more neural networks configured to encode the road users states in view of the lane segments 4 spatially and temporally, and output spatial- and temporal-processed respective states-related data 81 and segment-related data 82”; and Paragraph [0039]: “[…] by processing said data 811, 821, predict and output data 90 indicating predicted trajectories of one or more of the target road user(s) 31 in view of the one or more lane segments 4”).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to modify the trajectory prediction system of Ollis to utilize the transformer-based trajectory prediction architecture and lane-segment-aware spatial encoding techniques taught by Liu because Liu teaches that encoding road-user state information in view of lane segments and processing such information using a transformer-based network improves modeling of spatial and temporal relationships between vehicles and roadway lane structures, thereby improving the accuracy, robustness, and reliability of vehicle trajectory prediction, and the combination would have merely involved the predictable use of known trajectory prediction techniques to improve the performance of the system of Ollis with a reasonable expectation of success. This motivation for the combination of Ollis and Liu is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim(s) 2 and 11, Ollis as modified by Liu teaches the apparatus of claim 1, where Ollis teaches wherein the pieces of feature information of the respective vehicles include at least one of positions, speeds, heading angles, heading angle change rates, or driving lanes of the respective vehicles, or any combination thereof (Paragraph [0103]: “perception detection 302 may also determine, for one or more identified objects in the environment, a current state of the object. The state information may include, without limitation, for each object: current location; current speed and/or acceleration; current heading; current orientation; size/footprint; type (e.g., vehicle vs. pedestrian vs. bicycle vs. static object or obstacle); and/or other state information”).
Claim(s) 5 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ollis et al (US 2023/0278581 A1) in view of Liu et al (US 2023/0356753 A1), further in view of Takamatsu et al (US 2016/0090084 A1).
Regarding claim(s) 5 and 14, Ollis as modified by Liu teaches the apparatus of claim 1, but do not specifically teach wherein the controller is configured to track the detected vehicle and the detected traffic line based on a motion and measurement model (MAMM).
However, Takamatsu teaches wherein the controller is configured to track the detected vehicle and the detected traffic line based on a motion and measurement model (MAMM) (Paragraph [0025]: “Additionally, the sensor system 16 is capable of determining the distance from the left, right, front and rear of the vehicle 10 to a road boundary 28 or other stationary or moving objects. For example, the sensor system 16 is capable of detecting the road boundary 28, such as a curb, lane marker, etc., or other stationary or moving objects to the left and right of the vehicle 10. Additionally, the sensor system 16 can include internal sensors capable of determining the steering wheel angle, the steering wheel angular speed and the vehicle speed along the road 26. Based on this information, the controller 14 is capable of calculating the relative position, relative speed, angle of the vehicle 10 relative to the road boundary 28, and estimated future position of the host vehicle 10”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify to track the detected vehicle and the detected traffic line based on a motion and measurement model section of Ollis and Liu to incorporate the use to track the detected vehicle and the detected traffic line based on a motion and measurement model section of Takamatsu and one of ordinary skill in the art would have recognized that the results of the combination were predictable. One could look to Takamatsu to include to track the detected vehicle and the detected traffic line based on a motion and measurement model section. The driver assistance system may accurately determine the location of the host vehicle on an electronic map, also any additional information including the current or predicted vehicle position and any past vehicle position or any other suitable information. This motivation for the combination of Ollis, Liu and Takamatsu is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Claim(s) 8 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ollis et al (US 2023/0278581 A1) in view of Liu et al (US 2023/0356753 A1), further in view of Wang (US 2023/0211660 A1).
Regarding claim(s) 8 and 17, Ollis as modified by Liu teaches the apparatus of claim 1, but do not specifically teach wherein the transformer network is configured to predict positions of the respective vehicles at a future time point based on input vectors of the respective vehicles at a past time point and input vectors of the respective vehicles at a current time point.
However, Wang teaches wherein the transformer network is configured to predict positions of the respective vehicles at a future time point based on input vectors of the respective vehicles at a past time point and input vectors of the respective vehicles at a current time point (Paragraph [0024]: “Each time that a driver goes on a driving trip, this type of vectorized data about road geometries, other vehicles on the road, and other data may be collected [...] the data may be input to a transformer network as training data and the transformer network may be trained based on the training data to predict the vehicle acceleration at a future time step based on the data associated with a current time step. That is, the transformer network may be trained to predict the driver’s driving behavior in a variety of driving situations”; and Paragraph [0062]: “the transformer training module 322 may train the transformer network 600 to predict future trajectories of road agents based on training data comprising past driving data collected by the ego vehicle 104”).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to incorporate the vector-based trajectory prediction techniques of Wang into the combined system of Ollis and Liu because Wang teaches that utilizing vectorized representations of historical and current road-agent information in a transformer-based trajectory prediction framework improves prediction accuracy and enables more effective forecasting of future vehicle positions. The modification would have merely involved the application of a known vector-based transformer prediction technique to the trajectory prediction system of Ollis as modified by Liu and would have yielded predictable results. This motivation for the combination of Ollis, Liu and Wang is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Relevant Prior Art Directed to State of Art
Terazawa (US 2023/0147535 A1) are relevant prior art not applied in the rejection(s) above. Terazawa discloses a vehicle position estimation device mounted on a vehicle and estimating a current vehicle position, the vehicle position estimation device comprising a control unit configured by at least one processor, wherein the control unit includes: a localization unit performing a process for specifying a position of the vehicle on a map based on (i) position information of a landmark detected based on an image frame captured by a front camera and (ii) position information of the landmark registered in the map; and an adverse environment determination unit determining whether a surrounding environment of the vehicle is an adverse environment based on at least one of (i) information output from a sensor equipped to the vehicle or (ii) information output from a communication device equipped to the vehicle, the adverse environment causing a deterioration in an accuracy of object recognition that is performed using the image frame, when the vehicle does not travel in an overtaking lane or an acceleration lane, the control unit outputs, to a vehicle control module that automatically controls a traveling speed of the vehicle, a deceleration request signal to restrict the traveling speed of the vehicle in response to the adverse environment determination unit determining that the surrounding environment of the vehicle is the adverse environment, and when the vehicle travels in an overtaking lane or an acceleration lane, the control unit cancels output of the deceleration request signal in response to the adverse environment determination unit determining that the surrounding environment of the vehicle is the adverse environment.
Lee et al (US 2023/0077393 A1) are relevant prior art not applied in the rejection(s) above. Lee discloses a vehicle comprising: a sensor unit configured to sense a three-dimensional (3D) space; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to: acquire point cloud data for the sensed 3D space, distinguish individual object areas from the acquired point cloud data, acquire object information of an object identified by using an object classification model, track the sensed 3D space using the object information and information related to an object acquired from the individual object areas, and control driving of the vehicle based on information related to the tracked 3D space.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONGBONG NAH whose telephone number is (571) 272-1361. The examiner can normally be reached M - F: 9:00 AM - 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ONEAL MISTRY can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONGBONG NAH/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights