DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
The amendment filed 05/18/2026 is entered.
Claims 1, 3, 5, 7, 9, and 11 are amended.
Claim 1-12 are pending.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims
particularly pointing out and distinctly claiming the subject matter which the
inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out
and distinctly claiming the subject matter which the applicant regards as his
invention.
Claim(s) 1 and 7 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claim 1, the claim recites the limitation “wherein the first fusion information is 2D and 3D space information.” It is unclear whether this limitation requires the first fusion information to be separate 2D and 3D spatial information, 5D information made up of 2D and 3D information, spatial information derived from 2D and 3D information, or something else. For examination purposes, the limitation is interpreted as requiring the first fusion information to be spatial information derived from 2D and 3D information. This rejections also applies to the corresponding limitation in Claim 7.
Regarding Claim 1, the claim recites the limitation “wherein the first fusion information is … within a coverage of the 2D driving image and the 3D point cloud information.” It is unclear whether “coverage” requires an overlapping coverage of the 2D driving image and the 3D point cloud information, a combined coverage of the 2D driving image and the 3D point cloud information, or something else. For examination purposes, the limitation is interpreted as limiting the first fusion information to an area where both 2D driving image information and 3D point cloud information are available. This rejections also applies to the corresponding limitation in Claim 7.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C.
102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the
statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a
new ground of rejection if the prior art relied upon, and the rationale supporting the rejection,
would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness
rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the
claimed invention is not identically disclosed as set forth in section 102, if the
differences between the claimed invention and the prior art are such that the
claimed invention as a whole would have been obvious before the effective filing
date of the claimed invention to a person having ordinary skill in the art to which
the claimed invention pertains. Patentability shall not be negated by the manner in
which the invention was made.
Claims 1-3, 5-9, and 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (Wang et al., “High Dimensional Frustum PointNet for 3D Object Detection from Camera, LiDAR, and Radar,” 2020) in view of Lin (Lin et al., “Deep Learning Derived Object Detection and Tracking Technology Based on Sensor Fusion of Millimeter-Wave Radar/Video and Its Application on Embedded Systems,” March 2, 2023).
Regarding Claim 1, Wang teaches:
A sensor fusion and object tracking system, comprising
a first fusion module configured to perform a first fusion process on a 2D driving image and 3D point cloud information to obtain first fusion information containing a plurality of recognized objects ([p. 1621]: “our method instead maps RGB values into a sequence of point clouds to aggregate 7D frustum (XY ZRGBT) from camera and LiDAR”; [p. 1624]: “basing detection on a sequence of camera images and 3D point clouds”; “estimate 3D bounding boxes in 7D point clouds”), wherein the first fusion information is 2D and 3D space information within a coverage of the 2D driving image and the 3D point cloud information ([p. 1621]: “due to restricting to 2D ROIs, our network avoids having a large 3D search space, while on the other hand, occluded objects can be detected more effectively by basing detection on a sequence of camera images and 3D point clouds”; [p. 1625]: “only the objects in the range of 50 meters are detected”); and
a second fusion module, being in signal communication with the first fusion module, the second fusion module being configured to perform a second fusion process on the first fusion information and 2D radar information to obtain second fusion information containing the recognized objects ([p. 1621]: “a radar feature map within the 2D ROI is extracted and concatenated to the colored point cloud features”; [p. 1624]: “a radar cluster is generated … which is able to deliver target points with 2D coordinates”; [p. 1625]: “8D radar PointNet”; “four box parameters”).
Wang further teaches:
generating a region of interest (ROI) from the first fusion information ([p. 1621]: “2D ROI”; [p. 1625]); and wherein the recognized objects inside the region of interest are used as a plurality of target objects ([p. 1624]: “3D bounding boxes”; [p. 1625]: “bounding box estimation”).
Wang does not explicitly teach:
wherein the second fusion information is used to compensate for a blind zone of the first fusion information;
wherein the second fusion information is used to generate the ROI; or
using objects detected in the ROI for subsequent detection and tracking.
However, Lin is in the field of sensor fusion and teaches:
a radar and camera fusion system that uses radar to generate a dynamic ROI when a default ROI does not contain an object, i.e., the default ROI has a blind zone (Lin [p. 13]: “we propose to use the mmWave radar sensor to find the area with the most objects and set it as the new ROI”; “When there is no object in the default ROI, the dynamic ROI we proposed can find the area where objects may appear followed by the successful detection of objects.”); and
using objects detected in the ROI for subsequent detection and tracking (Lin [p. 14]: “We select the bounding boxes as the input of the trackers.”; “we adopt the Kalman filter to implement the tracking”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Wang and use the second fusion information to compensate for a blind zone of the first fusion information and generate an ROI, and use objects detected in the ROI for subsequent detection and tracking, as taught by Lin, with a reasonable expectation of success. Wang teaches a camera, lidar, and radar fusion system that generates ROIs from its first (camera and lidar) fusion information ([p. 1625]). Lin teaches that radar information can be used to generate a new ROI when the default ROI contains no detected objects, followed by object detection and tracking (Lin [p. 13-14]). Applying Lin’s known radar-based dynamic ROI technique to Wang’s sensor fusion system would predictably enable detection and tracking of objects not detected by Wang’s default ROI.
Regarding Claim 7, Wang teaches: A sensor fusion and object tracking method, comprising steps:
receiving a 2D driving image and 3D point cloud information and performing a first fusion process on the 2D driving image and the 3D point cloud information by a first fusion module to obtain first fusion information containing a plurality of recognized objects ([p. 1621]: “our method instead maps RGB values into a sequence of point clouds to aggregate 7D frustum (XY ZRGBT) from camera and LiDAR”; [p. 1624]: “basing detection on a sequence of camera images and 3D point clouds”; “estimate 3D bounding boxes in 7D point clouds”), wherein the first fusion information is 2D and 3D space information within a coverage of the 2D driving image and the 3D point cloud information ([p. 1621]: “due to restricting to 2D ROIs, our network avoids having a large 3D search space, while on the other hand, occluded objects can be detected more effectively by basing detection on a sequence of camera images and 3D point clouds”; [p. 1625]: “only the objects in the range of 50 meters are detected”); and
receiving 2D radar information and performing a second fusion process on the first fusion information and the 2D radar information by a second fusion module to obtain second fusion information containing the recognized objects ([p. 1621]: “a radar feature map within the 2D ROI is extracted and concatenated to the colored point cloud features”; [p. 1624]: “a radar cluster is generated … which is able to deliver target points with 2D coordinates”; [p. 1625]: “8D radar PointNet”; “four box parameters”),
Wang further teaches:
generating a region of interest (ROI) from the first fusion information ([p. 1621]: “2D ROI”; [p. 1625]); and wherein the recognized objects inside the region of interest are used as a plurality of target objects ([p. 1624]: “3D bounding boxes”; [p. 1625]: “bounding box estimation”).
Wang does not explicitly teach:
wherein the second fusion information is used to compensate for a blind zone of the first fusion information;
wherein the second fusion information is used to generate the ROI; or
using objects detected in the ROI for subsequent detection and tracking.
However, Lin is in the field of sensor fusion and teaches:
a radar and camera fusion system that uses radar to generate a dynamic ROI when a default ROI does not contain an object, i.e., the default ROI has a blind zone (Lin [p. 13]: “we propose to use the mmWave radar sensor to find the area with the most objects and set it as the new ROI”; “When there is no object in the default ROI, the dynamic ROI we proposed can find the area where objects may appear followed by the successful detection of objects.”); and
using objects detected in the ROI for subsequent detection and tracking (Lin [p. 14]: “We select the bounding boxes as the input of the trackers.”; “we adopt the Kalman filter to implement the tracking”).
The rationale to modify Wang with the teachings of Lin persists from Claim 1.
Regarding Claims 2 and 8, Wang teaches: the system further comprising
… generating the region of interest … ([p. 1621]: “2D ROI”), and identifying the recognized objects inside the region of interest as the target objects ([. 1621]: “a radar feature map within the 2D ROI is extracted and concatenated to the colored point cloud features”).
Wang does not explicitly teach: an object tracking module being in signal communication with the second fusion module, receiving the second fusion information, and generating the region of interest within a field of view (FOV) of the 2D radar information according to the second fusion information.
However, Lin teaches: an object tracking module being in signal communication with the second fusion module ([p. 13]: “Dynamic ROI”; [p. 14]: “Tracking”), receiving the second fusion information ([p. 14]: “We select the bounding boxes as the input of the trackers”), generating the region of interest within a field of view (FOV) of the 2D radar information according to the second fusion information ([p. 13]: “Dynamic ROI”; [p. 16]: “the mmWave radar used in this work only detects around 50 m in a given field”; Figure 20), and identifying the recognized objects inside the region of interest as the target objects ([p. 14]: “We select the bounding boxes as the input of the trackers”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Wang and include an object tracking module that receives the second fusion information, generates a ROI within an FOV of the radar information, and identifies objects in the ROI, as taught by Lin, with a reasonable expectation of success. Using fusion information to generate a ROI within an FOV of a sensor and identifying objects in the ROI is considered ordinary and well-known in the art. Modifying Wang to generate a ROI and identify objects in the ROI as taught by Lin is beneficial for improving detection and tracking of objects (Lin [p. 20]).
Regarding Claims 3 and 9, Wang does not explicitly teach – but Lin teaches: wherein the object tracking module is further configured to:
perform a centroid tracking algorithm on the target objects inside the region of interest to generate target centroid coordinates of each of the target objects (Lin [p. 5]: “DBSCAN”; “center point”; [p. 14]: “bounding box”);
perform a Kalman filter on the target centroid coordinates of each of the target objects to obtain observed information of each of the target objects and then calculates predicted information based on the observed information (Lin [p. 14]: “we adopt the Kalman filter to implement the tracking”); and
track the target objects according to the observed information and the predicted information (Lin [p. 14]: “Tracking”; “Kalman filter”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Wang and generate centroid coordinates of each target object, and perform Kalman filter tracking of the target objects, as taught by Lin, with a reasonable expectation of success. Calculating centroid coordinates and using a Kalman filter for tracking are considered ordinary and well-known in the art, and both are beneficial for improving target tracking (Lin [p. 20]).
Regarding Claims 5 and 11, Wang teaches: wherein the first fusion information is space information of an external environment ([p 1621]: “environment perception”).
Regarding Claims 6 and 12, Wang teaches: wherein the first fusion module is further configured to perform feature extraction using a neural network on the 2D driving image and the 3D point cloud information to obtain a plurality of characteristic points ([p. 1621]: “a high dimensional convolution operator captures local features from a point cloud enhanced with color and temporal information”; [p. 1625]: “four sets of abstraction (SA) layers and four feature propagation layers”).
Claims 4 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (Wang et al., “High Dimensional Frustum PointNet for 3D Object Detection from Camera, LiDAR, and Radar,” 2020) in view of Lin (Lin et al., “Deep Learning Derived Object Detection and Tracking Technology Based on Sensor Fusion of Millimeter-Wave Radar/Video and Its Application on Embedded Systems,” March 2, 2023), as applied to Claims 1 and 7 above, and further in view of Uvarov (US 2019/0391587).
Regarding Claims 4 and 10, Wang does not explicitly teach: wherein the second fusion module performs a high-pass filtering process and a low-pass filtering process to filter out noise of the second fusion information.
However, Uvarov is in the field of deep learning for autonomous driving and teaches that data from multiple sensors may be combined into a single data component and pre-processed using high-pass and low-pass filtering to remove noise (Uvarov [0029]: “data from multiple sensors may be combined into a single data component”; [0030]: “high-pass”; “low-pass”; [0032]: “pre-processing includes removing noise from the component data”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Wang and perform high-pass and low-pass filtering to remove noise, as taught by Uvarov, with a reasonable expectation of success. High-pass and low-pass filtering are considered ordinary and well-known in the art, and removing noise is a well-known benefit of both high-pass and low-pass filtering.
Response to Arguments
Applicant’s amendments and arguments, filed 05/18/2026, regarding Claim Objections have been fully considered and are persuasive. The previous objections have been overcome.
Applicant’s arguments, filed 05/18/2026, regarding Claim Rejections under 35 USC 103 have been fully considered but they are not persuasive.
Applicant appears to argue that Wang does not teach a “hierarchical fusion process that deliberately separates the near-range and longer-range processing through a distance threshold.” In response to Applicant’s argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., a distance threshold, separate near-range and long-range processing) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Additionally, Examiner asserts that Wang teaches a two-stage sensor fusion processes ([p. 1621, 1624]) where the first fusion information is limited to a specific coverage ([p. 1625]: “only the objects in the range of 50 meters are detected”).
Applicant appears to argue that Lin does not teach “a world-coordinate multi-stage post-detection fusion architecture designed to resolve distance blind zones and optimize computational power.” In response to Applicant’s argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., world coordinates, post detection fusion architecture, optimizing computational power) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Additionally, Applicant’s argument that “the present invention is fundamentally different in terms of system architecture, operational sequence, and technical motivation from Lin” appears to be a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references.
In response to Applicant’s argument that “forcing the teachings of Lin into Wang’s system would fundamentally destroy Wang’s original core principle of operation,” the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NOAH Y. ZHU whose telephone number is (571) 270-0170. The examiner can normally be reached Monday-Friday, 8AM-4PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR).
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vladimir Magloire, can be reached on (571) 270-5144. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NOAH YI MIN ZHU/Examiner, Art Unit 3648
/BRADY W FRAZIER/Primary Examiner, Art Unit 3648
Prosecution Insights