Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to because 1412 in Fig. 14 should read “Perform cross-attention between the third set of features and the fourth set of features to obtain a second set of cross-attended features” to be consistent with the specification. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
a. In [0042] line 2, “first vehicle 204” should read "second vehicle 206".
b. In [0042] line 8, “first vehicle 204” should read "second vehicle 206".
Appropriate correction is required.
Claim Objections
Claim 8 is objected to because of the following informalities:
Claim 8 as recited is ambiguous as the word “bird’s” is misspelled. For the record, the examiner recommends claim 8 to be rewritten as follow, and interpretation will be as such until clarification is made of record or applicant accepts this proposal and makes changes accordingly.
8. The apparatus of claim 1, wherein the first 2D view is a bird’s eye view and the second 2D view is a front view.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-4, 8, 17, 19, 21-22 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Sun (PCT Patent Pub. No.: WO 2025043445 A1), hereinafter Sun, in view of Huang (US Patent Pub. No.: US 2025/0054286 A1), hereinafter Huang.
Regarding claim 1, Sun teaches an apparatus configured for object detection, comprising: one or more memories (An embodiment of the present invention further provides a computer device, which at least includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above method embodiments is implemented. Page 27 4th paragraph) configured to store a first two-dimensional (2D) view of a scene corresponding to a first viewpoint (Acquire images of the target three-dimensional image at two different viewpoints, and determine them as a first viewpoint image (i.e., a 2D view of a scene) and a second viewpoint image corresponding to the first viewpoint image respectively. Page 3 11th paragraph), a second 2D view of the scene corresponding to a second viewpoint (Acquire images of the target three-dimensional image at two different viewpoints, and determine them as a first viewpoint image and a second viewpoint image corresponding to the first viewpoint image respectively. Page 3 11th paragraph), a third 2D view of the scene corresponding to a third viewpoint (Acquire images of the target three-dimensional image at two different viewpoints, and determine them as a first viewpoint image (which reads on “a third 2D view of a scene”, as a multi-view camera can take multi-view images.) and a second viewpoint image corresponding to the first viewpoint image respectively. Page 3 11th paragraph. of course, they may also be two different images simultaneously captured by a multi-view camera. Page 9 last paragraph), and a fourth 2D view of the scene corresponding to a fourth viewpoint (Acquire images of the target three-dimensional image at two different viewpoints, and determine them as a first viewpoint image and a second viewpoint image (which reads on “a fourth 2D view of a scene”) corresponding to the first viewpoint image respectively. Page 3 11th paragraph); and one or more processors, coupled to the one or more memories (An embodiment of the present invention further provides a computer device, which at least includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above method embodiments is implemented. Page 27 4th paragraph), configured to: obtain a first set of features based on the first 2D view (Performing foreground object detection on the first view image to determine a foreground area. Page 30 2nd paragraph); obtain a second set of features based on the second 2D view (the position of the target area in the second view map (which reads on “a second set of features based on the second 2D view”.) and the predicted view map is the same as the position of the determined foreground area in the first view map. Page 18 7th paragraph), wherein the first 2D view and the second 2D view are based on input from a first input sensor (Optionally, the first view image and the corresponding second view image may be two different images simultaneously captured by a binocular camera. Page 9 last paragraph); obtain a third set of features based on the third 2D view (Performing foreground object detection on the first view image to determine a foreground area. Page 30 2nd paragraph); obtain a fourth set of features based on the fourth 2D view (the position of the target area in the second view map (which reads on “a fourth set of features based on the fourth 2D view”.) and the predicted view map is the same as the position of the determined foreground area in the first view map. Page 18 7th paragraph), wherein the third 2D view and the fourth 2D view are based on input from a second input sensor (of course, they may also be two different images simultaneously captured by a multi-view camera. Page 9 last paragraph. It is common knowledge that a multi-view camera integrates multiple lenses or sensors (In other words, the camera can include a second input sensor, besides the first input sensor) to capture different perspectives simultaneously.).
Sun does not teach the following limitations as further recited, but Huang further teaches perform cross-attention between the first set of features and the second set of features to obtain a first set of cross-attended features (The method of any of solutions 4-6, wherein the refining comprises performing a multi-level refinement wherein, at each layer of the multi-level refinement, a self-attention layer that acts on both the 2D features and the 3D features a first cross-attention layer that acts only on the 2D features and a second cross-attention layer that acts only on the 3D features are used. [0096]); perform cross-attention between the third set of features and the fourth set of features to obtain a second set of cross-attended features (The method of any of solutions 4-6, wherein the refining comprises performing a multi-level refinement wherein, at each layer of the multi-level refinement, a self-attention layer that acts on both the 2D features and the 3D features a first cross-attention layer that acts only on the 2D features and a second cross-attention layer that acts only on the 3D features are used. [0096]); and perform 3D object detection in the scene based on at least the first set of cross-attended features and the second set of cross-attended features (performing, a 3-dimensional (3D) feature extraction on the images; detecting objects in the images by fusing detection results from the 2D feature extraction and the 3D feature extraction. Abstract).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun to incorporate the teachings of Huang to perform 3D object detection in the scene based on the first set of cross-attended features and the second set of cross-attended features in order to allow the model to align and integrate information between two modalities or sequences and to have improved context understanding.
Regarding claim 3, Huang in the combination teaches the apparatus of claim 1, wherein the first input sensor is a light detection and ranging (LiDAR) sensor and the second input sensor is an image sensor (To enable object detection by a vehicle, sensors (e.g., lidar sensors) and cameras may be installed on the vehicle. [0030]).
Regarding claim 4, Huang in the combination teaches the apparatus of claim 1, wherein the one or more processors are configured to fuse the first set of cross-attended features with the second set of cross-attended features to obtain a set of fused features (performing, a 3-dimensional (3D) feature extraction on the images; detecting objects in the images by fusing detection results from the 2D feature extraction and the 3D feature extraction. Abstract), wherein the 3D object detection in the scene (JOINT 3D DETECTION AND SEGMENTATION USING BIRD'S EYE VIEW AND PERSPECTIVE VIEW. Title) based on at least the first set of cross-attended features (The method of any of solutions 4-6, wherein the refining comprises performing a multi-level refinement wherein, at each layer of the multi-level refinement, a self-attention layer that acts on both the 2D features and the 3D features a first cross-attention layer that acts only on the 2D features and a second cross-attention layer that acts only on the 3D features are used. [0096]) and the second set of cross-attended features (The method of any of solutions 4-6, wherein the refining comprises performing a multi-level refinement wherein, at each layer of the multi-level refinement, a self-attention layer that acts on both the 2D features and the 3D features a first cross-attention layer that acts only on the 2D features and a second cross-attention layer that acts only on the 3D features are used. [0096]) comprises the 3D object detection in the scene based on at least the set of fused features (performing, a 3-dimensional (3D) feature extraction on the images; detecting objects in the images by fusing detection results from the 2D feature extraction and the 3D feature extraction. Abstract).
Regarding claim 8, Huang in the combination teaches the apparatus of claim 1, wherein the first 2D view is a bird’s eye view and the second 2D view is a front view (JOINT 3D DETECTION AND SEGMENTATION USING BIRD'S EYE VIEW AND PERSPECTIVE VIEW. Title. FIG. 5 shows an example of a bird's eye view (BEV) of a scene. [0015].
PNG
media_image1.png
588
690
media_image1.png
Greyscale
. FIG. 4 shows an example of a perspective view (PV) of a scene. [0014].
PNG
media_image2.png
678
928
media_image2.png
Greyscale
).
Regarding claim 17, Sun in the combination teaches the apparatus of claim 1, wherein the third 2D view and the first 2D view are from a same viewpoint (of course, they may also be two different images simultaneously captured by a multi-view camera. Page 9 last paragraph. It is common knowledge that a multi-view camera integrates multiple lenses or sensors to capture images from different or the same viewpoint simultaneously.)).
Regarding claim 19, Huang in the combination teaches the apparatus of claim 1, further comprising the second input sensor comprising a camera (To enable object detection by a vehicle, sensors (e.g., lidar sensors) and cameras may be installed on the vehicle. [0030]), coupled to the one or more processors (The non-transitory computer readable storage medium includes code that when executed by a processor, causes the processor to perform the methods described in this patent document. [0008]), configured to obtain at least one image of the scene (FIG. 4 shows an example of a perspective view (PV) of a scene. [0014]), the input from the second input sensor comprising the at least one image of the scene (FIG. 4 shows an example of a perspective view (PV) of a scene. [0014].
PNG
media_image2.png
678
928
media_image2.png
Greyscale
).
Regarding claim 21, Huang in the combination teaches the apparatus of claim 1, further comprising the first input sensor comprising a LiDAR sensor (To enable object detection by a vehicle, sensors (e.g., lidar sensors) and cameras may be installed on the vehicle. [0030]), coupled to the one or more processors (The non-transitory computer readable storage medium includes code that when executed by a processor, causes the processor to perform the methods described in this patent document. [0008]), configured to generate a 3D point cloud representation of the scene, wherein the input of the first input sensor comprises the 3D point cloud representation of the scene (23. The method of any of solutions 17-22, wherein the one or more images comprise an red green blue (RGB) format, an RGB-D format, point cloud images, or radar images. [0114]).
Regarding claim 22, Huang in the combination teaches the apparatus of claim 21, wherein the LiDAR sensor is integrated into one of a vehicle, an extra-reality device, or a mobile device (To enable object detection by a vehicle, sensors (e.g., lidar sensors) and cameras may be installed on the vehicle. [0030]).
Method claim 24 is drawn to the method of using the corresponding apparatus claimed in claim 1. Therefore method claim 24 corresponds to apparatus claim 1 and is rejected for the same reasons of obviousness as used above.
Claims 2 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sun (PCT Patent Pub. No.: WO 2025043445 A1), hereinafter Sun, in view of Huang (US Patent Pub. No.: US 2025/0054286 A1), hereinafter Huang, further in view of Chen (FUTR3D: A Unified Sensor Fusion Framework for 3D Detection, arXiv:2203.10642v2 [cs.CV] 15 Apr 2023), hereinafter Chen.
Regarding claim 2, Sun and Huang teach all of the elements of the claimed invention as stated in claim 1 except for the following limitations as further recited. However, Chen teaches wherein: to obtain the first set of features, the one or more processors are configured to process the first 2D view (FUTR3D learns features from each modality independently. Since our framework does not make assumptions about the modalities used or their model architectures, our model works with any choices of feature encoders. This work focuses on three types of data: LiDAR point clouds, radar point clouds, and multi-view camera images. Page 3 right column 2nd paragraph. It is common knowledge that a multi-view camera integrates multiple lenses or sensors to capture different perspectives simultaneously.) through a first encoder to obtain the first set of features (First, the data from different sensor modalities can be encoded by their modality-specific feature encoders. Page 3 right column 1st paragraph); and to obtain the second set of features, the one or more processors are configured to process the second 2D view (This work focuses on three types of data: LiDAR point clouds, radar point clouds, and multi-view camera images. Page 3 right column 2nd paragraph. It is common knowledge that a multi-view camera integrates multiple lenses or sensors to capture different perspectives simultaneously.) through a second encoder to obtain the second set of features (First, the data from different sensor modalities can be encoded by their modality-specific feature encoders. Page 3 right column 1st paragraph).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun and Huang to incorporate the teachings of Chen to process the first 2D view through a first encoder to obtain the first set of features and to process the second 2D view through a second encoder to obtain the second set of features in order to improve feature detection.
Regarding claim 20, Chen in the combination teaches the apparatus of claim 1, further comprising a display, coupled to the one or more processors, configured to display bounding boxes around detected 3D objects in the scene (
PNG
media_image3.png
384
658
media_image3.png
Greyscale
).
Claims 5-7 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Sun (PCT Patent Pub. No.: WO 2025043445 A1), hereinafter Sun, in view of Huang (US Patent Pub. No.: US 2025/0054286 A1), hereinafter Huang, further in view of Goel (US Patent No.: US 12,475,725 B2), hereinafter Goel.
Regarding claim 5, Sun and Huang teach all of the elements of the claimed invention as stated in claim 1 except for the following limitations as further recited. However, Goel teaches wherein the one or more processors are configured to: transform a three-dimensional (3D) representation of the scene based on the input from the first input sensor (FIG. 10 depicts a block diagram of an example system for implementing various techniques described herein. Column 2 line 1.
PNG
media_image4.png
692
1060
media_image4.png
Greyscale
) to the first 2D view (Additionally or alternatively, image-based object detector 102 may voxelize a portion of a point cloud 114 to generate a 2D or 3D spatial representation of a particular object within the environment, or the environment itself. Column 11 line 41. In some cases, the image-based object detector 102 may perform 2D convolutions on multiple different views, such as a first set of 2D convolutions on the front view of a 3D bounding box, and second set of 2D convolutions on the side view of the 3D bounding box, and a third set of 2D convolutions on the top view of a 3D bounding box, and the combination of extracted features from each set of 2D convolutions may be used to perform the object detection functionalities. Column 6 line 55); and transform the 3D representation of the scene based on the input from the first input sensor to the second 2D view (Additionally or alternatively, image-based object detector 102 may voxelize a portion of a point cloud 114 to generate a 2D or 3D spatial representation of a particular object within the environment, or the environment itself. Column 11 line 41. In some cases, the image-based object detector 102 may perform 2D convolutions on multiple different views, such as a first set of 2D convolutions on the front view of a 3D bounding box, and second set of 2D convolutions on the side view of the 3D bounding box, and a third set of 2D convolutions on the top view of a 3D bounding box, and the combination of extracted features from each set of 2D convolutions may be used to perform the object detection functionalities. Column 6 line 55).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun and Huang to incorporate the teachings of Goel to transform a three-dimensional (3D) representation of the scene based on the input from the first input sensor to the first 2D view and the second 2D view in order to improve the efficiency and quality of the image-based object detection.
Regarding claim 6, Huang in the combination teaches the apparatus of claim 5, wherein the one or more processors are configured to: receive a 3D point cloud representation of the scene as the input from the first input sensor (23. The method of any of solutions 17-22, wherein the one or more images comprise an red green blue (RGB) format, an RGB-D format, point cloud images, or radar images. [0114]).
Goel in the combination further teaches generate a 3D voxel representation of the 3D point cloud, wherein the 3D representation of the scene comprises the 3D voxel representation of the 3D point cloud (Additionally or alternatively, image-based object detector 102 may voxelize a portion of a point cloud 114 to generate a 2D or 3D spatial representation of a particular object within the environment, or the environment itself. Column 11 line 41).
Regarding claim 7, Goel in the combination teaches the apparatus of claim 6, wherein: to transform the 3D representation of the scene to the first 2D view, the one or more processors are configured to geometrically project the 3D voxel representation of the 3D point cloud to the first 2D view (Additionally or alternatively, image-based object detector 102 may voxelize a portion of a point cloud 114 to generate a 2D or 3D spatial representation of a particular object within the environment, or the environment itself. Column 11 line 41. In some cases, the image-based object detector 102 may perform 2D convolutions on multiple different views, such as a first set of 2D convolutions on the front view of a 3D bounding box, and second set of 2D convolutions on the side view of the 3D bounding box, and a third set of 2D convolutions on the top view of a 3D bounding box, and the combination of extracted features from each set of 2D convolutions may be used to perform the object detection functionalities. Column 6 line 55); and to transform the 3D representation of the scene to the second 2D view, the one or more processors are configured to geometrically project the 3D voxel representation of the 3D point cloud to the second 2D view (Additionally or alternatively, image-based object detector 102 may voxelize a portion of a point cloud 114 to generate a 2D or 3D spatial representation of a particular object within the environment, or the environment itself. Column 11 line 41. In some cases, the image-based object detector 102 may perform 2D convolutions on multiple different views, such as a first set of 2D convolutions on the front view of a 3D bounding box, and second set of 2D convolutions on the side view of the 3D bounding box, and a third set of 2D convolutions on the top view of a 3D bounding box, and the combination of extracted features from each set of 2D convolutions may be used to perform the object detection functionalities. Column 6 line 55).
Regarding claim 15, Goel in the combination teaches the apparatus of claim 1, wherein the one or more processors are configured to: generate a three-dimensional (3D) representation of the scene from one or more 2D images of the scene (Techniques are discussed herein for generating three-dimensional (3D) representations of an environment based on two-dimensional (2D) image data. Abstract), the input from the second input sensor comprising the one or more 2D images of the scene (In some examples, an image-based object detector within an autonomous vehicle may receive 2D image data captured by a camera of the autonomous vehicle. Column 2 line 10); transform the 3D representation of the scene to the third 2D view (Operation 120 may include generating a 2D or 3D spatial grid, and performing convolutions based on a view (e.g., top-down view, side view, etc.) of the spatial grid. To generate a 2D or 3D view in operation 120, the image-based object detector 102 may generate a grid representation based on the 2D image data and depth data as described in more detail below. Column 8 line 3); and transform the 3D representation of the scene to the fourth 2D view (Operation 120 may include generating a 2D or 3D spatial grid, and performing convolutions based on a view (e.g., top-down view, side view, etc.) of the spatial grid. To generate a 2D or 3D view in operation 120, the image-based object detector 102 may generate a grid representation based on the 2D image data and depth data as described in more detail below. Column 8 line 3).
Regarding claim 16, Goel in the combination teaches the apparatus of claim 15, wherein to generate the 3D representation of the scene from the one or more 2D images of the scene, the one or more processors are configured to: generate a 3D point cloud representation of the scene based on the one or more 2D images of the scene (Using the 2D image data and associated depth data, an image-based object detector may generate 3D representations, including point clouds and/or 3D pixel grids, for the 2D image or particular regions of interest. Abstract).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (PCT Patent Pub. No.: WO 2025043445 A1), hereinafter Sun, in view of Huang (US Patent Pub. No.: US 2025/0054286 A1), hereinafter Huang, further in view of Zhou (Cross-View Transformers for Real-Time Map-View Semantic Segmentation, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 13760-13769), hereinafter Zhou.
Regarding claim 12, Sun and Huang teach all of the elements of the claimed invention as stated in claim 1 except for the following limitations as further recited. However, Zhou teaches wherein the one or more processors are configured to: obtain one or more additional sets of features (An image encoder produces a multi-scale feature representation of each input image. Page 13762 left column 2nd paragraph) based on one or more additional 2D views of the scene corresponding to additional viewpoints (In this section, we introduce our proposed architecture for semantic segmentation in the map-view from multiple camera views. Page 13762 left column 1st paragraph); perform cross-attention between the first set of features and each of the one or more additional sets of features to obtain one or more additional sets of cross-attended features (A cross-view cross-attention mechanism then aggregates multi-scale features into a shared map-view. Page 13762 left column 2nd paragraph.
representation.
PNG
media_image5.png
478
1052
media_image5.png
Greyscale
); and fuse the one or more additional sets of cross-attended features with the first set of cross-attended features and the second set of cross-attended features to obtain a set of fused features (A cross-view cross-attention mechanism then aggregates multi-scale features into a shared map-view. Page 13762 left column 2nd paragraph), wherein the 3D object detection in the scene based on at least the first set of cross-attended features and the second set of cross-attended features comprises the 3D object detection in the scene (Our architecture implicitly learns a mapping from individual camera views into a canonical map-view representation using a camera-aware cross-view attention mechanism. Abstract.
PNG
media_image6.png
904
1260
media_image6.png
Greyscale
) based on at least the set of fused features (A cross-view cross-attention mechanism then aggregates multi-scale features into a shared map-view. Page 13762 left column 2nd paragraph).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun and Huang to incorporate the teachings of Zhou to fuse the additional sets of cross-attended features, based on additional 2D views of the scene corresponding to additional viewpoints, with the first set of cross-attended features and the second set of cross-attended features in order to have improved context understanding.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (PCT Patent Pub. No.: WO 2025043445 A1), hereinafter Sun, in view of Huang (US Patent Pub. No.: US 2025/0054286 A1), hereinafter Huang, further in view of Vora (US Patent Pub. No.: US 2022/0371606 A1), hereinafter Vora.
Regarding claim 18, Sun and Huang teach all of the elements of the claimed invention as stated in claim 1 except for the following limitations as further recited. However, Vora teaches wherein the one or more processors are configured to: pillarize a three-dimensional (3D) representation of the scene to obtain the first 2D view (The pillar processing module 701 includes a pillar creating component 706, an encoder 710, and a 2D image creating component 714. [0108]. The set of measurements includes a plurality of data points that represent a plurality of objects in a 3D space surrounding the vehicle. Abstract); and pillarize the 3D representation of the scene to obtain the second 2D view (The 2D image creating component 714 is configured to receive the learned features output 712 from the encoder 710 and to process the learned features output 712 to generate the pseudo-image 716. The pseudo-image 716 is a 2D image that has more channels (e.g., 32, 64, or 128 channels) than a standard RGB image with 3 channels. [0136]. In some embodiments, the LiDAR sectors (i.e., 3D representation of the scene) are rotated to a canonical coordinate frame prior to being processed into pillars (in other words, a point cloud can be pillarized in different ways to obtain different 2D views. [0153]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun and Huang to incorporate the teachings of Vora to pillarize a three-dimensional (3D) representation of the scene to obtain the first and second 2D views in order to efficiently and quickly detecting 3D objects based on sensor data.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Sun (PCT Patent Pub. No.: WO 2025043445 A1), hereinafter Sun, in view of Huang (US Patent Pub. No.: US 2025/0054286 A1), hereinafter Huang, further in view of Wu (US Patent Pub. No.: US 2023/0351769 A1), hereinafter Wu.
Regarding claim 23, Sun and Huang teach all of the elements of the claimed invention as stated in claim 1 except for the following limitations as further recited. However, Wu teaches further comprising a modem, coupled to one or more antennas, and coupled to the one or more processors, wherein the modem and one or more antennas are configured to at least one of: receive at least one of one or more 2D representations of the scene or one or more indications of one or more 3D objects detected in the scene, wherein the first 2D view of the scene comprises at least one of the received one or more 2D representations of the scene; or send the one or more indications of one or more 3D objects detected in the scene to one or more devices (The vehicle 900 further includes a network interface 924 which may use one or more wireless antenna(s) 926 and/or modem(s) to communicate over one or more networks. For example, the network interface 924 may be capable of communication over LTE, WCDMA, UMTS, GSM, CDMA2000, etc. The wireless antenna(s) 926 may also enable communication between objects in the environment (e.g., vehicles, mobile devices, etc.), using local area network(s), such as Bluetooth, Bluetooth LE, Z-Wave, ZigBee, etc., and/or low power wide-area network(s) (LPWANs), such as LoRaWAN, SigFox, etc. In some embodiments, the stereo disparity machine learning hazard detector 102 may communicate detected hazardous objects to cloud based services or other ego-machines via the network interface 924. [0100]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sun and Huang to incorporate the teachings of Wu to configure the modem and antennas to send indications of 3D objects detected in the scene in order to warn other drivers of hazardous objects.
Allowable Subject Matter
Claims 9-11 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: the closest prior arts of record teach the apparatus of claim 1. However, none of them alone or in any combination teaches wherein: to perform cross-attention between the first set of features and the second set of features to obtain the first set of cross-attended features, the one or more processors are configured to perform cross-attention between the first set of features as a first query and the second set of features as a first key and value to obtain the first set of cross-attended features; the one or more processors are configured to perform cross-attention between the second set of features as a second query and at least the first set of features as a second key and value to obtain a third set of cross-attended features; and the one or more processors are configured to fuse the first set of cross-attended features and the second set of cross-attended features with the third set of cross-attended features to obtain a set of fused features, wherein the 3D object detection in the scene based on at least the first set of cross-attended features and the second set of cross-attended features comprises the 3D object detection in the scene based on at least the set of fused features as specified in claim 9.
Claims 13-14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: the closest prior arts of record teach the apparatus of claim 1. However, none of them alone or in any combination teaches wherein to perform 3D object detection in the scene, the one or more processors are configured to: generate a first centerness heatmap for an object class based on at least the first set of cross-attended features; generate a second centerness heatmap for the object class based on at least the second set of features; and perform cross-attention between the first centerness heatmap and the second centerness heatmap to obtain a third set of cross-attended features, wherein the 3D object detection in the scene based on at least the first set of cross-attended features and the second set of cross-attended features comprises the 3D object detection in the scene based on at least the third set of cross-attended features as specified in claim 13.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEI ZHAO whose telephone number is (703)756-1922. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, VU LE can be reached at (571)272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LEI ZHAO/Examiner, Art Unit 2668
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights