DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/28/2026 has been entered. Claims 1, 6, 14, 19, and 26 were amended. Claims 29 and 30 were canceled. Claims 31 and 32 were added. Claims 1-28 and 31-32 are pending in the application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4, 10-15, 17, 23-27, and 31 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cohen et al. (US 11062454) in view of Zhu et al. ("Cylindrical and asymmetrical 3d convolution networks for lidar segmentation," Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021) and Zeng et al. (US 2014/0035775).
Regarding claim 1, Cohen teaches/suggests: An apparatus comprising:
a processing system that includes one or more processors and one or more memories coupled to the one or more processors (Cohen Fig. 5: processor 518 and memory 520), the processing system configured to:
receive sensor data associated with a scene (Cohen col. 5 ll. 13-31 “the sensor data may comprise at least image data 106 and data that may be represented as a point cloud, which may be received from one or more types of sensors”);
generate a cylindrical representation associated with the scene (Cohen col. 5 ll. 13-31 “sensor data that may be represented as a point cloud may include radar data 108, lidar data 110 ... Points of the point cloud may be represented using any of a variety of coordinate systems (e.g., Euclidean, polar, spherical, cylindrical)”);
based on the to a discrete location in the environment surrounding the autonomous vehicle”).
Cohen further teaches/suggests a feature of the cylindrical representation (Cohen col. 9 ll. 20-39 “the perception engine 120 may additionally or alternatively determine a position of the autonomous vehicle 102 determined by a localization engine (not pictured, which may use any sensor data to localize the autonomous vehicle 102), data related to objects in the vicinity of the autonomous vehicle 102, route data that specifies a destination of the vehicle, global map data that identifies characteristics of roadways (e.g., features detectable in different sensor modalities useful for localizing the autonomous vehicle), local map data that identifies characteristics detected in proximity to the vehicle (e.g., locations and/or dimensions of buildings, trees, fences, fire hydrants, stop signs, and any other feature detectable in various sensor modalities), etc.”). Cohen is silent regarding a feature of the cylindrical representation is associated with an angular coordinate. Zhu, however, teaches/suggests a feature of the cylindrical representation is associated with an angular coordinate (Zhu Fig. 2 “these features are reassigned based on the [cylindrical] partition” §3.2 ¶2 “where radius ρ (distance to origin in x-y axis) and azimuth θ (angle from x-axis to y-axis) are calculated”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the coordinate system of Cohen to include the angular coordinate of Zhu for the cylindrical coordinates.
Nor does Cohen teach/suggest:
based on detecting that a feature of the cylindrical representation is associated with an angular coordinate that is included in a first region of the cylindrical representation, modify the cylindrical representation, wherein modifying the cylindrical representation includes relocating the feature from the first region to a second region that is different than the first region;
Zeng, however, teaches/suggests relocating the feature from the first region to a second region that is different than the first region (Zeng [0026] “Temporal constraints involve features that are relocated to a different cell between at least two time-displaced images”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the feature of Cohen as modified by Zhu to be relocated as taught/suggested by Zeng for temporal constraints. As such, Cohen as modified by Zhu and Zeng teaches/suggests:
based on detecting that a feature of the cylindrical representation is associated with an angular coordinate that is included in a first region of the cylindrical representation, modify the cylindrical representation, wherein modifying the cylindrical representation includes relocating the feature from the first region to a second region that is different than the first region (Cohen col. 5 ll. 13-31 “Points of the point cloud may be represented using any of a variety of coordinate systems (e.g., Euclidean, polar, spherical, cylindrical)” Zhu §3.2 ¶2 “where radius ρ (distance to origin in x-y axis) and azimuth θ (angle from x-axis to y-axis) are calculated” Zeng [0026] “Estimates are determined as to whether a feature can be relocated to another cell within the grid based on an elapsed time between the captured images”);
Regarding claim 2, Cohen as modified by Zhu and Zeng teaches/suggests: The apparatus of claim 1, wherein the feature is associated with a radial distance from an origin of the cylindrical representation, and wherein the processing system is further configured to modify the radial distance based on a radial adjustment value to generate the modified cylindrical representation (Zhu §3.2 ¶2 “where radius ρ (distance to origin in x-y axis) and azimuth θ (angle from x-axis to y-axis) are calculated” Zeng [0026] “Estimates are determined as to whether a feature can be relocated to another cell within the grid based on an elapsed time between the captured images”). In view of Zhu and Zeng, relocating the feature meets the radial adjustment value. The same rationale to combine as set forth in the rejection of claim 1 is incorporated herein.
Regarding claim 4, Cohen as modified by Zhu and Zeng teaches/suggests: The apparatus of claim 1, wherein the feature is associated with an angular distance from a polar axis of the cylindrical representation, and wherein the processing system is further configured to modify the angular distance based on an angular shift value to generate the modified cylindrical representation (Zhu §3.2 ¶2 “where radius ρ (distance to origin in x-y axis) and azimuth θ (angle from x-axis to y-axis) are calculated” Zeng [0026] “Estimates are determined as to whether a feature can be relocated to another cell within the grid based on an elapsed time between the captured images”). In view of Zhu and Zeng, relocating the feature meets the angular shift value. The same rationale to combine as set forth in the rejection of claim 1 is incorporated herein.
Regarding claim 10, Cohen as modified by Zhu and Zeng teaches/suggests: The apparatus of claim 1, wherein the one or more 3D perception operations include one or more of object detection, instance segmentation, lane detection, or road detection (Cohen col. 5 ll. 44-59 “The perception engine 120 may include one or more ML models and/or other computer-executable instructions for detecting, identifying, segmenting, classifying, and/or tracking objects from sensor data collected from the environment of the autonomous vehicle 102”).
Regarding claim 11, Cohen as modified by Zhu and Zeng teaches/suggests: The apparatus of claim 1, further comprising:
a first sensor configured to generate first sensor data; and a second sensor configured to generate second sensor data, wherein the sensor data includes the first sensor data and the second sensor data (Cohen col. 5 ll. 13-31 “the sensor data may comprise at least image data 106 and data that may be represented as a point cloud, which may be received from one or more types of sensors”).
Regarding claim 12, Cohen as modified by Zhu and Zeng teaches/suggests: The apparatus of claim 11, wherein the scene includes an object represented by both the first sensor data and the second sensor data (Cohen col. 9 ll. 20-39 “the perception engine 120 may additionally or alternatively determine a position of the autonomous vehicle 102 determined by a localization engine (not pictured, which may use any sensor data to localize the autonomous vehicle 102), data related to objects in the vicinity of the autonomous vehicle 102, route data that specifies a destination of the vehicle, global map data that identifies characteristics of roadways (e.g., features detectable in different sensor modalities useful for localizing the autonomous vehicle), local map data that identifies characteristics detected in proximity to the vehicle (e.g., locations and/or dimensions of buildings, trees, fences, fire hydrants, stop signs, and any other feature detectable in various sensor modalities), etc.”), and wherein one or more of continuity or linearity associated with the object is increased in the modified cylindrical representation as compared to the cylindrical representation (Cohen col. 5 ll. 13-31 “Points of the point cloud may be represented using any of a variety of coordinate systems (e.g., Euclidean, polar, spherical, cylindrical)” Zeng [0026] “Estimates are determined as to whether a feature can be relocated to another cell within the grid based on an elapsed time between the captured images”). In view of Cohen and Zeng, relocating the feature meets the continuity or the linearity. The same rationale to combine as set forth in the rejection of claim 1 is incorporated herein.
Regarding claim 13, Cohen as modified by Zhu and Zeng teaches/suggests: The apparatus of claim 11, wherein the apparatus corresponds to a vehicle, wherein the first sensor corresponds to a front-facing sensor of the vehicle, and wherein the second sensor corresponds to a rear-facing sensor of the vehicle or a side-facing sensor of the vehicle (Cohen col. 22 ll. 5-21 “the lidar sensors may include individual lidar sensors located at the corners, front, back, sides, and/or top of the vehicle 502”).
Claims 14-15, 17, and 23-25 recite limitation(s) similar in scope to those of claims 1-2, 4, 10, and 12-13, respectively, and are rejected for the same reason(s).
Claim 26 recites limitation(s) similar in scope to those of claim 1, and is rejected for the same reason(s). Cohen as modified by Zeng further teaches/suggests a non-transitory computer-readable medium storing instructions executable by one or more processors (Cohen Fig. 5: processor 518 and memory 520).
Claim 27 recites limitation(s) similar in scope to those of claims 2 and 4, and is rejected for the same reason(s).
Regarding claim 31, Cohen as modified by Zhu and Zeng teaches/suggests: The non-transitory computer-readable medium of claim 26, wherein modifying the cylindrical representation further includes:
adjusting the angular coordinate from a first angular coordinate value within the cylindrical representation to a second angular coordinate value within the modified cylindrical representation (Zhu §3.2 ¶2 “where radius ρ (distance to origin in x-y axis) and azimuth θ (angle from x-axis to y-axis) are calculated” Zeng [0026] “Estimates are determined as to whether a feature can be relocated to another cell within the grid based on an elapsed time between the captured images”); and
adjusting a cylindrical coordinate associated with the feature from a first cylindrical coordinate value within the cylindrical representation to a second cylindrical coordinate value within the modified cylindrical representation (Zhu §3.2 ¶2 “where radius ρ (distance to origin in x-y axis) and azimuth θ (angle from x-axis to y-axis) are calculated” Zeng [0026] “Estimates are determined as to whether a feature can be relocated to another cell within the grid based on an elapsed time between the captured images”).
The same rationale to combine as set forth in the rejection of claim 1 is incorporated herein.
Claim(s) 3, 5-9, 16, 18-22, and 28 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cohen et al. (US 11062454) in view of Zhu et al. ("Cylindrical and asymmetrical 3d convolution networks for lidar segmentation," Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021) and Zeng et al. (US 2014/0035775) as applied to claims 1, 14, and 26 above, and further in view of Harris et al. (US 6304265).
Regarding claim 3, Cohen, Zhu, and Zeng are silent regarding: The apparatus of claim 2, wherein the radial adjustment value is negative one. Harris, in view of Zhu and Zeng, teaches/suggests the radial adjustment value is negative one (Zhu §3.2 ¶2 “where radius ρ (distance to origin in x-y axis) and azimuth θ (angle from x-axis to y-axis) are calculated” Zeng [0026] “Estimates are determined as to whether a feature can be relocated to another cell within the grid based on an elapsed time between the captured images” Harris col. 10 ll. 18-31 “reflecting one point (V.sub.2.fwdarw.V.sub.2 ') across the W=0 boundary”). Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to modify the coordinate system of Cohen as modified by Zhu and Zeng to include the boundary of Harris to relocate the feature across such a boundary.
Regarding claim 5, Cohen, Zhu, and Zeng are silent regarding: The apparatus of claim 4, wherein the angular shift value is pi radians. Harris, in view of Zhu and Zeng, teaches/suggests the angular shift value is pi radians (Zhu §3.2 ¶2 “where radius ρ (distance to origin in x-y axis) and azimuth θ (angle from x-axis to y-axis) are calculated” Zeng [0026] “Estimates are determined as to whether a feature can be relocated to another cell within the grid based on an elapsed time between the captured images” Harris col. 10 ll. 18-31 “reflecting one point (V.sub.2.fwdarw.V.sub.2 ') across the W=0 boundary”). The same rationales to combine as set forth in the rejection of claims 1 and 3 is incorporated herein.
Regarding claim 6, Cohen, Zhu, and Zeng are silent regarding: The apparatus of claim 1, wherein a boundary between the first region and the second region corresponds to a particular value of the angular coordinate. Harris, in view of Zhu and Zeng, teaches/suggests a boundary between the first region and the second region corresponds to a particular value of the angular coordinate (Zhu §3.2 ¶2 “where radius ρ (distance to origin in x-y axis) and azimuth θ (angle from x-axis to y-axis) are calculated” Zeng [0026] “Estimates are determined as to whether a feature can be relocated to another cell within the grid based on an elapsed time between the captured images” Harris col. 10 ll. 18-31 “reflecting one point (V.sub.2.fwdarw.V.sub.2 ') across the W=0 boundary”). The same rationales to combine as set forth in the rejection of claims 1 and 3 is incorporated herein.
Regarding claim 7, Cohen as modified by Zhu, Zeng, and Harris teaches/suggests: The apparatus of claim 6, wherein the particular value is zero (Harris col. 10 ll. 18-31 “reflecting one point (V.sub.2.fwdarw.V.sub.2 ') across the W=0 boundary”). The same rationales to combine as set forth in the rejection of claims 1 and 3 is incorporated herein.
Regarding claim 8, Cohen as modified by Zhu, Zeng, and Harris teaches/suggests: The apparatus of claim 6, wherein the first region is associated with values of the angular coordinate of greater than or equal to zero, and wherein the second region is associated with values of the angular coordinate of less than zero (Zhu §3.2 ¶2 “where radius ρ (distance to origin in x-y axis) and azimuth θ (angle from x-axis to y-axis) are calculated” Zeng [0026] “Estimates are determined as to whether a feature can be relocated to another cell within the grid based on an elapsed time between the captured images” Harris col. 10 ll. 18-31 “reflecting one point (V.sub.2.fwdarw.V.sub.2 ') across the W=0 boundary”). The same rationales to combine as set forth in the rejection of claims 1 and 3 is incorporated herein.
Regarding claim 9, Cohen as modified by Zhu, Zeng, and Harris teaches/suggests: The apparatus of claim 6, wherein the processing system is further configured to reflect the feature across the boundary to generate the modified cylindrical representation (Zhu §3.2 ¶2 “where radius ρ (distance to origin in x-y axis) and azimuth θ (angle from x-axis to y-axis) are calculated” Zeng [0026] “Estimates are determined as to whether a feature can be relocated to another cell within the grid based on an elapsed time between the captured images” Harris col. 10 ll. 18-31 “reflecting one point (V.sub.2.fwdarw.V.sub.2 ') across the W=0 boundary”). The same rationales to combine as set forth in the rejection of claims 1 and 3 is incorporated herein.
Claims 16 and 18-22 recite limitation(s) similar in scope to those of claims 3 and 5-9, respectively, and are rejected for the same reason(s).
Claim 28 recites limitation(s) similar in scope to those of claims 3 and 5, and is rejected for the same reason(s).
Claim(s) 32 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cohen et al. (US 11062454) in view of Zhu et al. ("Cylindrical and asymmetrical 3d convolution networks for lidar segmentation," Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021) and Zeng et al. (US 2014/0035775) as applied to claim 26 above, and further in view of Schonberger et al. (US 2024/0161337).
Regarding claim 32, Cohen as modified by Zhu and Zeng teaches/suggests: The non-transitory computer-readable medium of claim 26, wherein the operations further comprise:
receiving light detection and ranging (LiDAR) sensor data from a LiDAR sensor system, wherein the sensor data includes the LiDAR sensor data (Cohen col. 5 ll. 13-31 “sensor data that may be represented as a point cloud may include radar data 108, lidar data 110”);
receiving image sensor data from one or more image sensors, wherein the sensor data further includes the image sensor data (Cohen col. 5 ll. 13-31 “the sensor data may comprise at least image data 106”);
generating a first point cloud based on the LiDAR sensor data (Cohen col. 5 ll. 13-31 “sensor data that may be represented as a point cloud may include radar data 108, lidar data 110”); and
wherein the cylindrical representation is generated based on the first point cloud (Cohen col. 5 ll. 13-31 “Points of the point cloud may be represented using any of a variety of coordinate systems (e.g., Euclidean, polar, spherical, cylindrical)”).
Cohen as modified by Zhu and Zeng does not teach/suggest:
generating a second point cloud based on the image sensor data, wherein generating the second point cloud includes lifting the image sensor data from a two-dimensional (2D) space to a three-dimensional (3D) space, and
wherein the cylindrical representation is generated based on the first point cloud and further based on the second point cloud.
Schonberger, however, teaches/suggests:
generating a second point cloud based on the image sensor data, wherein generating the second point cloud includes lifting the image sensor data from a two-dimensional (2D) space to a three-dimensional (3D) space (Schonberger [0020] “In the case where a 3D point-cloud is created from a set of reference images, this is done by establishing matches between the reference images to estimate their relative poses and triangulate a 3D model which is the 3D map ... These 2D-2D correspondences are then lifted to 3D2D matches and fed to a PnP solver to estimate the pose of the query image with respect to the 3D map”),
Before the effective filing date of the claimed invention, the substitution of one known element (the point cloud of Schonberger) for another (the image data of Cohen) would have been obvious to one of ordinary skill in the art because such substitutions would have yielded predictable results, namely, for the perception engine. As such, Cohen as modified by Zhu, Zeng, and Schonberger teaches/suggests:
wherein the cylindrical representation is generated based on the first point cloud and further based on the second point cloud (Cohen col. 10 line 60 – col. 11 line 17 “the different types of point cloud data may be aggregated into a single point cloud” col. 5 ll. 13-31 “sensor data that may be represented as a point cloud may include radar data 108, lidar data 110” Schonberger [0020] “In the case where a 3D point-cloud is created from a set of reference images, this is done by establishing matches between the reference images to estimate their relative poses and triangulate a 3D model which is the 3D map”).
Response to Arguments
Applicant's arguments filed on 05/28/2026 have been fully considered but they are moot in view of the new ground(s) of rejection set forth in this Office action.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 2018/0022472 – adjust angular coordinates
US 2023/0274466 – point cloud polar coordinates
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANH-TUAN V NGUYEN whose telephone number is 571-270-7513. The examiner can normally be reached on M-F 9AM-5PM ET. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JASON CHAN can be reached on 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANH-TUAN V NGUYEN/
Primary Examiner, Art Unit 2619
Prosecution Insights