Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . The amendments provided 02/19/2026 have been entered and considered. Claims 1, 2, 3, 7, 11, 13, and 20 have been amended. No new matter has been added.
Response to Amendment
Claim Rejections - 35 USC § 112
In view of the amendments provided 02/19/2026, the 112 (b) rejections of the non-final rejection (11/20/2025) are hereby withdrawn.
Response to Arguments
Prior art rejections
On pages 9-11 of the remarks (02/19/2026), applicant contends that the relied upon prior art does not disclose the fusion of intermediate features, as would be required by the amended independent claims.
The examiner agrees, specifically due to the fused data being processed feature data rather than raw sensor data, as suggested by applicant on page 10 of the remarks (02/19/2026). As such, the 103 rejection of the non-final rejection 11/20/2025 is hereby withdrawn.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1, 2, 4-6, 11, 12, 14-16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (applicant provided non-patent literature " BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird’s-Eye View Representation"; hereinafter "Liu") in view of Gross et al. (applicant provided US publication 20230138112 A1; hereinafter "Gross") and Qiao et al. (non-patent literature titled “Adaptive Feature Fusion for Cooperative Perception using LiDAR Point Clouds”; hereinafter “Qiao”).
In re to claim 1, Liu teaches wherein: A method of fusing multi-modal sensor data, the method comprising: obtaining features for 3D data captured by a first sensor (Fig. 2 description lines 1-3 and Fig. 2 denote the extraction of features from LiDAR point clouds, understood as 3D data captured by the first sensor (understood as the LiDAR portion of the system));
obtaining features for images captured by second sensors (Fig. 2 description lines 1-3 and Fig. 2 denotes the capture of RGB image information from a camera. Thus, it is understood that the second sensors are the multi-view cameras (which are further suggested in section 3.2 para. 1 lines 5-6, which describes the use of the number of cameras in the generation of a camera feature point cloud));
flattening the features for 3D data to first features in bird eye view (Lidar projection flattening; section 3.1 para. 4 denotes the LiDAR to BEV projection flattening, understood to be the flattening of the 3D data to first features in bird eye view (being the resultant flattened LiDAR features to bird’s-eye view));
transforming the features for images into second features in bird eye view (section 3.2 para. 5 discloses that the methodology of the system performs camera image data transformation to bird’s-eye view (including the features for said image data). It is understood that the second features are the resultant transformed camera image data features);
concatenating the first features and the second features to obtain first concatenated multi- sensor features (section 3.3 discloses the concatenation of the BEV (bird’s-eye view) features, including that of the LiDAR feature and camera feature data. Thus, disclosing the concatenation of the first features and second features (each correspondent to the claims respectively)).
Liu does not explicitly teach wherein: sensors are on the ego vehicle nor does it explicitly teach the fusion of sensor data of the ego vehicle with sensor data of another vehicle to obtain fused multi-sensor data.
However, in a similar field of endeavor, Gross teaches wherein: sensors are on the ego vehicle (Fig. 1 and [0119] shows that individuals vehicles have sensors, with the ego vehicle being understood to be represented by Fig. 1 (103a), shown in communication with the other respective vehicles) while further teaching the fusion of sensor data ([0119] lines 3-10 disclose that the sensors that comprise Fig. 1 (105a) may include a plurality of sensors (including cameras and LiDAR)) of the ego vehicle with sensor data of another vehicle to obtain fused multi-sensor data ([0178] lines 9-14 discloses the combination of sensor data from sensors disposed on multiple vehicles, the resultant fused sensor data being understood as the fused multi-sensor data).
Gross, like Liu, teaches a multi-modality system that combines captured sensor data that may be leveraged by an autonomous system.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liu to combine sensor data from another vehicle with that of the ego vehicle, as taught by Gross, to arrive at the claimed invention discussed above. The motivation for the proposed modification would be to enable data sharing, thereby allowing the system to improve routing (as noted by Gross [0113] lines 1-4). Additionally, see Liu abstract lines 1-2 and section 6 subsection titled “societal impacts” which denote the Liu’s system’s use for autonomous driving.
Liu, in view of Gross, further teaches obtaining second concatenated multi-sensor features from another vehicle, wherein the second concatenated multi-sensor features are generated by the another vehicle from first features derived based on data from third sensors (Liu Fig. 2 description lines 1-3 and Liu Fig. 2 denote the extraction of features from LiDAR point clouds, understood as 3D data captured by a sensor. Gross Fig. 1 shows the communication of a plurality of vehicles with their own sensors. Thus, it is understood that the combination of Liu and Gross teaches another vehicle having a third sensor, by virtue of the second vehicle (Gross Fig. 1 (103b)) having a multi-sensor system that may include LiDAR (see Gross [0119] lines 3-10, which disclose that the sensors that comprise Gross Fig. 1 (105b) may include a plurality of sensors (including cameras and LiDAR). It is understood that the LiDAR of the second vehicle constitutes the third sensor) of the another vehicle and second features derived based on data from fourth sensors (Gross [0119] lines 3-10, which disclose that the sensors that comprise Gross Fig. 1 (105b) may include a plurality of sensors (including cameras and LiDAR). It is understood that the cameras of the second vehicle constitute fourth sensors) of the another vehicle (Liu section 3.3 discloses the concatenation of the BEV (bird’s-eye view) features, including that of the LiDAR feature and camera feature data. Thus, disclosing the concatenation of the third features and fourth features when done using the data from the third and fourth sensors (each correspondent to the claims respectively));
Liu in view of Gross does not explicitly teach wherein: there is fusion of processed feature data between vehicles.
However, in a similar field of endeavor, Qiao teaches wherein: there is fusion of processed feature data between vehicles (Fig. 2 shows the combination of processed feature data from sensors of a plurality of autonomous vehicles (being the CAV)).
Qiao, like Liu, teaches a system for autonomous vehicles that combines sensor data with consideration to BEV data. Further, Qiao, like Gross, is a system for multiple vehicles in communication with one another.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liu, in view of Gross, to combine processed feature data via fusion of said data from different vehicles, as taught by Qiao, to arrive at the claimed invention discussed above. The motivation for the proposed modification would be to enable the system to improved perception accuracy by having the ability to adapt feature selection for fusion of data, as is performed in Qiao section 4 para. 2, with its noted benefit acknowledged in Qiao section 5.4 para. 1 by indicating a higher perception accuracy (this is also a suggested benefit of such a system in Qiao’s abstract).
In re to claim 2 [dependent on claim 1], Liu in view of Gross and Qiao teaches wherein: obtaining features for 3D data captured by the third sensor of the another vehicle (Liu Fig. 2 description lines 1-3 and Liu Fig. 2 denote the extraction of features from LiDAR point clouds, understood as 3D data captured by a sensor. Gross Fig. 1 shows the communication of a plurality of vehicles with their own sensors. Thus, it is understood that the combination of Liu and Gross teaches another vehicle having a third sensor, by virtue of the second vehicle (Gross Fig. 1 (103b)) having a multi-sensor system that may include LiDAR (see Gross [0119] lines 3-10, which disclose that the sensors that comprise Gross Fig. 1 (105b) may include a plurality of sensors (including cameras and LiDAR). It is understood that the LiDAR of the second vehicle constitutes the third sensor);
obtaining features for images captured by the fourth sensors of the another vehicle (Gross [0119] lines 3-10, which disclose that the sensors that comprise Gross Fig. 1 (105b) may include a plurality of sensors (including cameras and LiDAR). It is understood that the cameras of the second vehicle constitute fourth sensors);
flattening the features for 3D data to third features in bird eye view (Lidar projection flattening; Liu section 3.1 para. 4 denotes the LiDAR to BEV projection flattening, understood to be the flattening of the 3D data to features in bird eye view (being the resultant flattened LiDAR features to bird’s-eye view). It is understood that the third features are the resultant flattened features when performed using data from the third sensor);
transforming the features for images into fourth features in bird eye view (Liu section 3.2 para. 5 discloses that the methodology of the system performs camera image data transformation to bird’s-eye view (including the features for said image data). It is understood that the fourth features are the resultant transformed camera image data features when performed using data from the fourth sensors); and
concatenating the third features and the fourth features to obtain the second concatenated multi-sensor features (Liu section 3.3 discloses the concatenation of the BEV (bird’s-eye view) features, including that of the LiDAR feature and camera feature data. Thus, disclosing the concatenation of the third features and fourth features when done using the data from the third and fourth sensors (each correspondent to the claims respectively)).
The reasons for combination are the same as provided above.
In re to claim 4 [dependent on claim 2], Liu, in view of Gross and Qiao, teaches wherein: the first sensor (Fig. 2 description lines 1-3 and Fig. 2 denote the extraction of features from LiDAR point clouds, understood as 3D data captured by the first sensor (understood as the LiDAR portion of the system)) and the third sensor are LiDAR sensors (Gross [0119] lines 3-10, which disclose that the sensors that comprise Gross Fig. 1 (105b) may include a plurality of sensors (including cameras and LiDAR). It is understood that the LiDAR of the second vehicle constitute third sensor) and the second sensors (Fig. 2 description lines 1-3 and Fig. 2 denotes the capture of RGB image information from a camera. Thus, it is understood that the second sensors are the multi-view cameras (which are further suggested in section 3.2 para. 1 lines 5-6, which describes the use of the number of cameras in the generation of a camera feature point cloud)) and the fourth sensors are camera sensors (Gross [0119] lines 3-10, which disclose that the sensors that comprise Gross Fig. 1 (105b) may include a plurality of sensors (including cameras and LiDAR). It is understood that the cameras of the second vehicle constitute fourth sensors).
The reasons for combination are the same as provided above.
In re to claim 5 [dependent on claim 1], Liu teaches wherein: the 3D data is a 3D LiDAR point cloud (Fig. 2 shows the LiDAR data as point cloud data); and
the features for the 3D data are obtained by inputting the 3D LiDAR point cloud into a LiDAR encoder (Fig. 2 shows that the LiDAR features are obtained using a LiDAR encoder).
In re to claim 6 [dependent on claim 1], Liu teaches wherein: the images are RGB images captured by cameras oriented in different directions (Fig. 2 shows that the system captures multi-view RGB images, thus capturing images in different directions (as they capture multiple different views, which is further corroborated by section 3.1 para. 1 lines 2-3)); and
the features for the images are obtained by inputting the RGB images into a camera encoder (Fig. 2 shows the use of a camera encoder to obtain camera features).
In re to claim 9 [dependent on claim 1], Liu, in view of Gross and Qiao, teaches wherein: the ego vehicle and the another vehicle are connected autonomous vehicles (Gross [0110] discloses the vehicles of Fig. 1 are autonomous, thus both the ego and the another vehicle are autonomous (each correspondent to the claims, respectively). Further, Gross Fig. 1 shows the two aforementioned vehicles are connected to one another via a wireless network Fig. 1 (112) (as corroborated by [0109] lines 15-27)).
The reasons for combination are the same as provided above.
As to claims 11, 12, and 14-16, they are the system that performs the method of claims 1,2, and 4-6 (respectively). As such, they recite similar limitations and are rejected for the same reasons as provided above.
As to claims 20, they are the system that performs the method of claim 1. As such, they recite similar limitations and are rejected for the same reasons as provided above.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Gross and Qiao, in further view of Borse and Agia et al. (US publication 20230267615 A1; hereinafter “Agia”).
In re to claim 3 [dependent on claim 1], Liu, in view of Gross and Qiao, does not explicitly teach wherein: decoding the fused multi-sensor features to identify objects external to the ego vehicle.
However, in a similar field of endeavor, Borse teaches wherein: decoding the fused multi-sensor features to identify objects external to the ego vehicle (Fig. 5 shows the combination of multi-sensor features in Fig. 5 (526), which shows the combination of Lidar and camera data. It further decodes the fusion in Fig. 5 (528) in order to produce a BEV map segmentation that identifies separate objects that make up the map segmented regions (see Fig. 5 (530)). Further, these segmented objects are external to the ego vehicle (being the vehicle that obtained the sensor data) due to the sensors collecting data from said vehicle’s surroundings (as suggested by [0027] lines 1-7, which disclose data capture for the vehicle’s environment)).
Borse further teaches operating the ego vehicle based on sensor data (Fig. 2A shows that the system uses the fused sensor data for motion planning and control. Thus, it is understood that the vehicle that collected the sensor data is operated based on the identified object (correspondent to the claims))
Borse, like Liu, captures multiple modalities of sensor data that is used for BEV perspective generation which may be used for an autonomous vehicle.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liu, in view of Gross, to decode fused multi-sensor data and operate a vehicle based on the sensor data, as taught by Borse, to arrive at the claimed invention discussed above. The motivation for the proposed modification would be to leverage machine learning to classify regions based on learned model, as is performed in Borse per Borse [0087] lines 10-12. Additionally, operating the vehicle based on the gathered sensor data enables the platform of the system to utilize the gathered information to improve its ability to understand its environment.
Liu, in view of Gross, Qiao, and Borse, does not explicitly teach wherein: operating the ego vehicle based on the identified objects.
However, in a similar field of endeavor, Agia teaches wherein: operating the ego vehicle based on the identified objects ([0052] discloses the use of the semantically segmented BEV image data generated by the system is used by the autonomous vehicle (understood as the ego vehicle) to plan pathing for said vehicle. Thus, as it uses semantic segmentation for pathing, it is understood to base operation on identified objects (being the segmented objects from the BEV image)).
Agia, like Liu, generates BEV perspectives in order to visualize an area.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liu, in view of Gross and Borse, to operate the ego vehicle based on identified objects, as taught by Borse, to arrive at the claimed invention discussed above. The motivation for the proposed modification would be to provide the autonomous vehicle a means of recognizing drivable locations (as is the benefit of semantic segmentation recognized in Liu section 3.4, which is performed by Borse per Fig. 5 (530) to recognize objects that compose a road) better enabling informed pathing (Borse Fig. 2A shows that the system decides pathing based on sensor data).
As to claim 13, it is the system that performs the method of claim 3. As such, it recites similar limitations and are rejected for the same reasons as provided above.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Gross and Qiao, in further view of Borse et al. (US publication 20240070541 A1; hereinafter “Borse”).
In re to claim 10 [dependent on claim 2], Liu, in view of Gross and Qiao, teaches wherein: the second sensors (Fig. 2 description lines 1-3 and Fig. 2 denotes the capture of RGB image information from a camera. Thus, it is understood that the second sensors are the multi-view cameras (which are further suggested in section 3.2 para. 1 lines 5-6, which describes the use of the number of cameras in the generation of a camera feature point cloud)) and the fourth sensors are camera sensors(Gross [0119] lines 3-10, which disclose that the sensors that comprise Gross Fig. 1 (105b) may include a plurality of sensors (including cameras and LiDAR). It is understood that the cameras of the second vehicle constitute fourth sensors).
Liu, in view of Gross and Qiao, further teaches that a plurality of vehicles with multiple sensors (Fig. 1 and [0119] lines 3-10 disclose that the sensors that comprise Fig. 1 (105a) may include a plurality of sensors) are in communication and that these sensors may include a radar ([0152] lines 1-4 discloses that the sensor system of a given vehicle may include radar)
Liu, in view of Gross and Qiao, does not explicitly teach wherein: the sensor obtaining features for 3D data for a given vehicle is a radar sensor.
However, in a similar field of endeavor, Borse teaches wherein: the sensor obtaining features for 3D data for a given vehicle is a radar sensor ([0027] lines 1-7 discloses the three-dimensional representation of a vehicle’s environment may be obtained from radar. Further, the obtained data is feature data (see Fig. 5 (522)) that is further flattened to be incorporated into a birds eye view representation (see Fig. 5 (524, 526, and 530))).
Borse, like Liu, captures multiple modalities of sensor data that is used for BEV perspective generation which may be used for an autonomous vehicle.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liu, in view of Gross, to use radar, as taught by Borse, to arrive at the claimed invention discussed above. The motivation for the proposed modification would be to increase the flexibility of system implementation by enabling it to use either LiDAR or radar, as is the case for Borse (per Borse[0027]).
As to claim 19, it is the system that performs the method of claim 10. As such, it recites similar limitations and are rejected for the same reasons as provided above.
Allowable Subject Matter
Claims 7 and 8 (as well as the similarly written 17 and 18) are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The reasons for objection are the same as provided on the non-final rejection (11/20/2025).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN M COOMBER whose telephone number is (571)270-0950. The examiner can normally be reached Monday - Friday 8:00am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEVIN M COOMBER/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights