DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application is being examined under the pre-AIA first to invent provisions.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
The pending application 18/272,773, filed on 17 July 2023, is a national stage application filed under 35 U.S.C. 371 of PCT/EP2022/051036, filed on 18 January 2023, and claims priority from foreign application GB2100683.8, filed on 19 January 2021 in the United Kingdom of Great Britain and Northern Ireland.
Response to Amendment
Applicant's amendment filed on 20 MAY 2026 has been entered. Claims 1, 8, 10, 17 and 23 have been amended. Claims 4-5,14, 16, 22 and 24 have been cancelled. Claims 1-3, 6-13, 15, 17-21 and 23 are still pending in this application, with claim 1, 10 and 23 being independent.
Response to Arguments
Applicant’s arguments regarding independent claims 1, 10 and 23, filed 20 MAY 2026 have been fully considered, but they are either not persuasive or moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant argues that independent claim 1 patentably distinguishes from Liu
Regarding the Examiner’s rejection of independent claim 1 under 35 U.S.C. 103 as unpatentable over Liu et al., the applicant argues that the cited reference fails to disclose all the features of the claimed invention, specifically “Liu does not disclose at least a radar point cloud having "for each pixel in the discretised image representation that has a non-zero occupancy channel value: (i) a Doppler channel containing a Doppler velocity of the corresponding point in the radar point cloud, or (ii) a radar cross section (RCS) channel containing an RCS value of the corresponding point in the radar point cloud for use by a machine learning (ML) perception component," as recited by claim 1.” (applicant’s remarks p. 8).
Applicant argues that “the channels in the RGB image in Liu are not affected by the values of the other channels.” (applicant’s remarks p. 9)
Applicant’s argument is moot in light of newly cited reference, Zhang et al. (US 2019/0147250 A1), where “The occupancy channel can be set to “1” if at least one point lies within the voxel cell or “0” if there is not a point which lies within the voxel cell. One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” (Zhang et al. ¶ [0025])
Applicant argues that independent claim 10 patentably distinguishes from Liu in view of Cohen and Fontijne
Regarding the Examiner’s rejection of independent claim 10 under 35 U.S.C. 103 as unpatentable over Liu et al. in view of Cohen et al. and Fontijne et al., the applicant argues that the cited reference fails to disclose all the features of the claimed invention, specifically “as discussed with respect to claim 1 above, Liu does not disclose at least a radar point cloud having "for each pixel in the discretised image representation that has a non-zero occupancy channel value: (i) a Doppler channel containing a Doppler velocity of the corresponding point in the radar point cloud, or (ii) a radar cross section (RCS) channel containing an RCS value of the corresponding point in the radar point cloud for use by a machine learning (ML) perception component" as recited by claim 10.” (applicant’s remarks p. 9).
Applicant’s argument is moot in light of newly cited reference, Zhang et al. (US 2019/0147250 A1), where “The occupancy channel can be set to “1” if at least one point lies within the voxel cell or “0” if there is not a point which lies within the voxel cell. One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” (Zhang et al. ¶ [0025])
Regarding the Examiner’s rejection of independent claim 10 under 35 U.S.C. 103 as unpatentable over Liu et al. in view of Cohen et al. and Fontijne et al., the applicant argues that the cited reference fails to disclose all the features of the claimed invention, specifically “Liu in any combination with Cohen or Fontijne does not disclose "the points of the radar point cloud being time-stamped, having been captured over a non-zero accumulation window" as recited by claim 10.” (applicant’s remarks p. 10)
Applicant argues that “At most, Cohen describes radar scans time-stamped as a whole, but does not disclose the points of a point cloud being time-stamped, having been captured over a non-zero accumulation window.” (applicant’s remarks p. 10)
Examiner respectfully disagrees. Cohen et al. discloses that “Clustering may be performed based on any physical parameters associated with the points (including, but not limited to, velocities, signal strength, location, nearest neighbors, a time stamp the measurement was performed, etc.) as well as corresponding threshold differences in any of the aforementioned parameters.” (Cohen et al. ¶ [0027]) In order for clustering to be performed based on a “time stamp the measurement was performed,” each of the points in the point cloud must be time-stamped. Therefore, applicant’s argument on this issue is not persuasive.
Examiner notes that this feature is also taught by newly cited Zhang et al. where Zhang et al. discloses that “In some implementations, time can be treated as a separate dimension.” (Zhang et al. ¶ [0026], [0046])
Regarding the Examiner’s rejection of independent claim 10 under 35 U.S.C. 103 as unpatentable over Liu et al. in view of Cohen et al. and Fontijne et al., the applicant argues that the cited reference fails to disclose all the features of the claimed invention, specifically “Liu in any combination with Cohen or Fontijne does not disclose "determining a motion model for the moving object cluster, by fitting one or more parameters of the motion model to the time-stamped points of that cluster," as recited by claim 10.” (applicant’s remarks p. 10)
Applicant argues that “Paragraph [0030] merely discloses: "the updated point cluster 136 may be useful to the autonomous vehicle 106 to identify objects, predict actions the objects may take, and/or maneuver in the environment relative to the objects, among other things" and "the updated point cluster 136 may be used to further characterize the object represented by the cluster." (applicant’s remarks p. 10)
Examiner respectfully disagrees. It would be obvious to one of ordinary skill in the art that a motion model must be determined in order to predict actions the object might take. Therefore, applicant’s argument on this issue is not persuasive.
Examiner notes that this feature is also taught by newly cited Zhang et al. where Zhang et al. discloses that “the prediction system 780 can determine a predicted motion trajectory along which a respective object is predicted to travel over time. A predicted motion trajectory can be indicative of a path that the object is predicted to traverse and an associated timing with which the object is predicted to travel along the path.” (Zhang et al. ¶ [0073])
Applicant argues that independent claim 23 patentably distinguishes from Liu in view of Moosman
Regarding the Examiner’s rejection of independent claim 23 under 35 U.S.C. 103 as unpatentable over Liu et al. in view of Moosmann et al., the applicant argues that the cited reference fails to disclose all the features of the claimed invention, specifically “as discussed with respect to claim 1 above, Liu does not disclose at least a radar point cloud having "for each pixel in the discretised image representation that has a non-zero occupancy channel value: (i) a Doppler channel containing a Doppler velocity of the corresponding point in the radar point cloud, or (ii) a radar cross section (RCS) channel containing an RCS value of the corresponding point in the radar point cloud for use by a machine learning (ML) perception component" as recited by claim 23.” (applicant’s remarks p. 11)
Applicant’s argument is moot in light of newly cited reference, Zhang et al. (US 2019/0147250 A1), where “The occupancy channel can be set to “1” if at least one point lies within the voxel cell or “0” if there is not a point which lies within the voxel cell. One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” (Zhang et al. ¶ [0025])
Regarding the Examiner’s rejection of independent claim 23 under 35 U.S.C. 103 as unpatentable over Liu et al. in view of Moosmann et al., the applicant argues that the cited reference fails to disclose all the features of the claimed invention, specifically “Liu in any combination with Mooseman does not disclose "wherein the radar point cloud is an accumulated radar point cloud comprising points accumulated over multiple radar sweeps" as recited by claim 23.” (applicant’s remarks p. 11)
Applicant argues that “The Office Action cites to paragraphs [0125] and [0138] of Liu as disclosing this limitation, however Liu does not disclose "an accumulated radar point cloud comprising points accumulated over multiple radar sweeps," as recited by claim 23, but instead discloses accumulating points over a single radar sweep.” (applicant’s remarks p. 11)
Examiner respectfully disagrees. For applicant’s benefit, portions of the cited reference(s) have been cited to aid in the review of the rejection(s). While every attempt has been made to be thorough and consistent within the rejection it is noted that the PRIOR ART MUST BE CONSIDERED IN ITS ENTIRETY, including disclosures that teach away from the claims. See MPEP 2141.02 VI. In this case, Liu et al. discloses “The radar device 2 outputs radar data for the entire observation area at each time at intervals (e.g., 50 ms) corresponding to the beam scanning period, and when the radar data for the object area at each time extracted from this radar data for the entire observation area at each time is visualized, a radar detection image is generated at a high frame rate (e.g., 20 fps).” (Liu et al. ¶ [0122]) The radar data for the entire observation area is captured over the beam scanning period, where the beam scan is considered to be a radar sweep. Liu et al. further discloses that the “radar data of the object area at multiple times, each extracted from radar data of the entire observation area at multiple times, is synthesized (integrated), and the synthesized radar data is converted into an image to generate a radar detection image of the object area.” (Liu et al. ¶ [0123]) The radar data of the entire observation area is captured via a beam scan, or radar sweep, at multiple times. Therefore, Liu et al. discloses that the points are accumulated over multiple radar sweeps, and applicant’s argument is not persuasive.
Regarding the Examiner’s rejection of independent claim 23 under 35 U.S.C. 103 as unpatentable over Liu et al. in view of Moosmann et al., the applicant argues that the cited reference fails to disclose all the features of the claimed invention, specifically “Liu in any combination with Mooseman does not disclose "wherein the accumulated radar point cloud includes points captured from an object that exhibits smearing effects caused by motion of the object during the multiple radar sweeps, and the discretised image representation retains the smearing effects" as recited by claim 23.” (applicant’s remarks p. 12)
Applicant argues that in the visualisations shown in the bottom rows of Figure 3 and 8 of Mooseman, the ego-motion is compensated as stated in the captions for Figures 3 and 8 of Moosmann which show that "odometry was used to compensate ego motion".”(applicant’s remarks p. 12)
Applicant’s argument is persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Insana et al. (US 2019/0154823 A1), where Insana et al. discloses the use of Type 2 Integration that involves image processing and “Pixels containing detections from static objects will accumulate the fastest, while pixels having detections from dynamic targets appear smeared as energy will be spread out across more adjacent pixels.” (Insana et al. ¶ [0037])
Claim Objections
Claims 1, 10 and 23 are objected to because of the following informalities:
In claim 1, line 8 recites “(i) a Doppler channel” and line 15 recites “(ii) the Doppler channel.’ Line 15 should recite “(i) the Doppler channel” to be consistent with line 8.
In claim 1, line 10 recites “(ii) a radar cross section (RCS) channel” and lines 15-16 recite “(iii) the RCS channels.” Lines 15-16 should recite “(ii) the RCS channel” to be consistent with line 10.
In claim 10, line 13 recites “(i) a Doppler channel” and line 20 recites “(ii) the Doppler channel.” Line 20 should recite “(i) the Doppler channel” to be consistent with line 13.
In claim 10, line 15 recites “(ii) a radar cross section (RCS) channel” and lines 20-21 recite “(iii) the RCS channels.” Lines 20-21 should recite “(ii) the RCS channel” to be consistent with line 15.
In claim 23, line 10 recites “(i) a Doppler channel” and line 17 recites “(ii) the Doppler channel.” Line 17 should recite “(i) the Doppler channel” to be consistent with line 10.
In claim 23, line 12 recites “(ii) a radar cross section (RCS) channel” and lines 17-18 recite “(iii) the RCS channels.” Lines 17-18 should recite “(ii) the RCS channel” to be consistent with line 23.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 6 and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2021/0311169 A1, cited by applicant in IDS dated 17 JUL 2023, previously relied upon by the examiner) in view of Zhang et al. (US 2019/0147250 A1, newly cited by the examiner).
Regarding claim 1 (Currently Amended), Liu et al. discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
A computer-implemented method (Liu et al. “The storage unit 13 stores radar data input from the radar device 2, programs executed by the processor constituting the control unit 12, and the like.” - ¶ [0040]) of perceiving structure in a radar point cloud, the method comprising:
generating a discretised image representation of the radar point cloud (Liu et al. The image generating unit 43 generates a radar detection image of the entire observation area based on radar data of the entire observation area.” - ¶ [0081]) having:
(ii) for each pixel in the discretised image representation
(i) a Doppler channel containing a Doppler velocity of the corresponding point in the radar point cloud (Liu et al. “Next, the image generating unit 32 sets the pixel value (values of each RGB channel) of the pixel at the position corresponding to the cell Cj based on the reflection intensity, Doppler velocity, and range of the cell Cj (ST206).” - ¶ [0089]), or
(ii) a radar cross section (RCS) channel containing an RCS value of the corresponding point in the radar point cloud for use by a machine learning (ML) perception component (Examiner notes that items (i) and (ii) are alternatives such that only one of (i) or (ii) is required);
inputting the discretised image representation to the machine learning (ML) perception component (Liu et al. “The object detection and discrimination unit 42 inputs the radar detection image of the entire observation area generated by the image generation unit 43 into a rained deep learning model…” - ¶ [0082]), which has been trained to extract information about structure exhibited in the radar point cloud from (i) the occupancy channel and: (ii) the Doppler channel, or (iii) the RCS channels (Liu et al. “Next, in the object detection and discrimination unit 42, the radar detection image of the entire observation area generated by the image generation unit 43 is input into the trained deep learning model, object detection and object discrimination are performed in the deep learning model, and the object discrimination result output from the deep learning model is obtained (ST112).” - ¶ [0086]; “Next, the object discrimination result and position information for each detected object are output (ST107).” - ¶ [0087]); and
wherein the ML perception component comprises a bounding box detector or other object detector (Liu et al. “in this embodiment, in addition to object discrimination, object detection to detect object regions is also performed using a deep learning model.” - ¶ [0079]), the extracted information comprising object position, orientation and/or size information for at least one detected object (Liu et al. “Then, the reflection intensity, Doppler velocity, and range of the selected cell Cj are obtained from the radar data of the entire observation area (ST205).” - ¶ [0088]).
Zhang et al. discloses:
generating a discretised image representation of the radar point cloud (Zhang et al. “The computing system can generate a two-dimensional voxel representation of the three-dimensional data (e.g., of the three-dimensional point clouds) that can be ingested by ta machine-learned model.” - ¶ [0021]) having:
(i) an occupancy channel indicating whether or not each pixel of the discretised image representation corresponds to a point in the radar point cloud (Zhang et al. “The occupancy channel can be set to “1” if at least one point lies within the voxel cell or “0” if there is not a point which lies within the voxel cell. One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” - ¶ [0025]) and:
(ii) for each pixel in the discretised image representation that has a non-zero occupancy channel value (Zhang et al. “One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” - ¶ [0025]):
(i) a Doppler channel containing a Doppler velocity of the corresponding point in the radar point cloud (Zhang et al. “Other sensor modalities that can be encoded into channel(s) include, for example, intensity, speed (e.g., of LIDAR returns), or other image features, etc.” - ¶ [0025]), or
(ii) a radar cross section (RCS) channel containing an RCS value of the corresponding point in the radar point cloud for use by a machine learning (ML) perception component (Examiner notes that items (i) and (ii) are alternatives such that only one of (i) or (ii) is required);
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Zhang et al. into the invention of Liu et al. to yield the invention of claim 1 above. Both Liu et al. and Zhang et al. are considered analogous arts to the claimed invention as they both disclose radar systems for vehicles that utilize machine learning for object detection. Liu et al. discloses the limitations of claim 1 outlined above. However, Liu et al. fails to explicitly disclose an occupancy channel indicating whether or not each pixel of the discretised image representation corresponds to a point in the radar point cloud. This feature is disclosed by Zhang et al. where “The occupancy channel can be set to “1” if at least one point lies within the voxel cell or “0” if there is not a point which lies within the voxel cell. One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” (Zhang et al. ¶ [0025]). The combination of Liu et al. and Zhang et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045]).
Regarding claim 2 (Original), Liu et al. as modified above discloses:
The method of claim 1, wherein the ML perception component has a neural network architecture (Liu et al. “Faster R-CNN (regions with convolutional neural network) is suitable deep with this search function.” - ¶ [0082]).
Regarding claim 3 (Original), Liu et al. as modified above discloses:
The method of claim 2, wherein the ML perception component has a convolutional neural network (CNN) architecture (Liu et al. “Faster R-CNN (regions with convolutional neural network) is suitable deep with this search function.” - ¶ [0082]).
Regarding claim 6 (Previously Presented), Liu et al. as modified above discloses:
The method of claim 1, wherein the radar point cloud is an accumulated radar point cloud comprising points accumulated over multiple radar sweeps (Liu et al. “The radar device 2 outputs radar data for the entire observation area at each time at intervals (e.g., 50 ms) corresponding to the beam scanning period, and when the radar data for the object area at each time extracted from this radar data for the entire observation area at each time is visualized, a radar detection image is generated at a high frame rate (e.g., 20 fps).” - ¶ [0122]; “The data synthesis unit 72 synthesizes (integrates) the radar data of the object area at a plurality of times acquired by the area data extraction unit 31, and generates synthesized radar data of the object area.” - ¶ [0125]; “Specifically, for example, when radar data at four different times is synthesized, it is determined that the timing for output is when the frame order is a multiple of four.” - ¶ [0138]; where radar data for the entire observation area is obtained during a beam scan, or radar sweep, and the radar data for the entire observation area is obtained multiple times, over multiple radar sweeps).
Regarding claim 19 (Previously Presented), Liu et al. as modified above discloses:
The method of claim 1, wherein the radar point cloud has only two spatial dimensions (Liu et al. “In this case, the area data extraction unit 31 performs coordinate conversion to convert the polar coordinate system of the radar into an XY orthogonal coordinate system.” - ¶ [0049]).
Regarding claim 20 (Previously Presented), Liu et al. as modified above discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
The method of claim 1
Zhang et al. discloses:
The method of claim 1, wherein the radar point cloud has three spatial dimensions (Zhang et al. “The sensor data can include, for example, a three-dimensional point cloud associated with an outdoor environment (e.g., surrounding an autonomous vehicle) and/or an indoor environment (e.g., a room, etc.).” - ¶ [0021]), and the discretised image representation additionally includes a height channel (Zhang et al. “For example, the two-dimensional voxel representation can be a two-dimensional bird's eye view of the voxel grid, with the vertical axis (e.g., a gravitational, z-axis) as a feature channel.” - ¶ [0026]).
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Zhang et al. into the invention of Liu et al. as modified above to yield the invention of claim 20. Both Liu et al. and Zhang et al. are considered analogous arts to the claimed invention as they both disclose radar systems for vehicles that utilize machine learning for object detection. Liu et al. discloses the limitations of claim 1. However, Liu et al. fails to explicitly disclose wherein the radar point cloud has three spatial dimensions, and the discretised image representation additionally includes a height channel. This feature is disclosed by Zhang et al. where “the two-dimensional voxel representation can be a two-dimensional bird's eye view of the voxel grid, with the vertical axis (e.g., a gravitational, z-axis) as a feature channel.” (Zhang et al. ¶ [0026]). The combination of Liu et al. and Zhang et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045]).
Claim(s) 7-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2021/0311169 A1, cited by applicant in IDS dated 17 JUL 2023, previously relied upon by the examiner) in view of Zhang et al. (US 2019/0147250 A1, newly cited by the examiner) as applied to claim 1 above, and further in view of Fontijne et al. (WO 2020/113160 A1, cited by applicant in IDS dated 17 JUL 2023, previously relied upon by the examiner).
Regarding claim 7 (Previously Presented), Liu et al. as modified above discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
The method of claim 1, wherein the points of the radar point cloud have been captured by a moving radar system (Liu et al. “it may be mounted on a vehicle and the discrimination results of surrounding objects may be used to control collision avoidance.” - ¶ [0035]), (Liu et al. “The radar device 2 outputs radar data for the entire observation area at each time at intervals (e.g., 50 ms) corresponding to the beam scanning period, and when the radar data for the object area at each time extracted from this radar data for the entire observation area at each time is visualized, a radar detection image is generated at a high frame rate (e.g., 20 fps).” - ¶ [0122]; “The data synthesis unit 72 synthesizes (integrates) the radar data of the object area at a plurality of times acquired by the area data extraction unit 31, and generates synthesized radar data of the object area.” - ¶ [0125]; “Specifically, for example, when radar data at four different times is synthesized, it is determined that the timing for output is when the frame order is a multiple of four.” - ¶ [0138]; where radar data for the entire observation area is obtained during a beam scan, or radar sweep, and the radar data for the entire observation area is obtained multiple times, over multiple radar sweeps).
Fontijne et al. discloses:
wherein ego motion of the radar system during the multiple radar sweeps is determined (Fontijne et al. “Next, the ego motion between frames 902 and 904 is obtained.” - ¶ [0075])
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Fontijne et al. into the invention of Liu et al. as modified above to yield the invention of claim 7. Liu et al., Zhang et al. and Fontijne et al. are considered analogous arts to the claimed invention as they disclose radar systems for vehicles that utilize machine learning for object detection. Liu et al. discloses the method of claim 1, wherein the points of the radar point cloud have been captured by a moving radar system (Liu et al. ¶ [0035]). However, Liu et al. fails to explicitly disclose wherein ego motion of the radar system during the multiple radar sweeps is determined and used to accumulate the points in a common static frame for generating the discretised image representation. This feature is disclosed by Fontijne et al. where the latent-space ego-motion compensation technique transforms the radar images received at different times to world coordinates using the second input frame as the world frame (Fontijne et al. latent-space ego-motion compensation technique, Fig. 9; ¶ [0074]-[0078]). The combination of Liu et al., Zhang et al. and Fontijne et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045]) and improve performance for object classification and detection (Fontijne et al. ¶ [0072]).
Regarding claim 8 (Currently Amended), Liu et al. as modified above discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
The method of claim 6, wherein the discretised image representation has (ii) the Doppler channel, and the points of the radar point cloud have been captured by a moving radar system (Liu et al. “it may be mounted on a vehicle and the discrimination results of surrounding objects may be used to control collision avoidance.” - ¶ [0035]),
Fontijne et al. discloses:
wherein the discretised image representation has (ii) the Doppler channel, and the points of the radar point cloud have been captured by a moving radar system (Fontijne et al. radar-camera sensor module 120 is mounted on vehicle 100, Fig. 1), wherein ego motion of the radar system, wherein ego motion of the radar system during the multiple radar sweeps is determined (Fontijne et al. “Next, the ego motion between frames 902 and 904 is obtained.” - ¶ [0075]), and wherein the Doppler velocities (Fontijne et al. “More specifically, an RNN has the ability to look at the position of the objects over multiple time steps (versus only at a given point in time). Based on the known position over a time period, computing the velocity is just a matter of calculating how fast that position has moved.” - ¶ [0060]) are ego motion-compensated Doppler velocities determined by compensating for the determined ego motion (Fontijne et al. latent-space ego-motion compensation technique, Fig. 9; ¶ [0074]-[0078]).
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Fontijne et al. into the invention of Liu et al. as modified above to yield the invention of claim 8 above. Liu et al., Zhang et al. and Fontijne et al. are considered analogous arts to the claimed invention as they disclose radar systems for vehicles that utilize machine learning for object detection. Liu et al. as modified above discloses the method of claim 6, wherein the points of the radar point cloud have been captured by a moving radar system (Liu et al. ¶ [0035]). However, Liu et al. fails to explicitly disclose wherein ego motion of the radar system during the multiple radar sweeps is determined, and wherein the Doppler velocities are ego motion-compensated Doppler velocities determined by compensating for the determined ego motion. This feature is disclosed by Fontijne et al. where the latent-space ego-motion compensation technique transforms the radar images received at different times to world coordinates using the second input frame as the world frame (Fontijne et al. latent-space ego-motion compensation technique, Fig. 9; ¶ [0074]-[0078]). The combination of Liu et al., Zhang et al. and Fontijne et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045]) and improve performance for object classification and detection (Fontijne et al. ¶ [0072]).
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2021/0311169 A1, cited by applicant in IDS dated 17 JUL 2023, previously relied upon by the examiner) in view of Zhang et al. (US 2019/0147250 A1, newly cited by the examiner) and Fontijne et al. (WO 2020/113160 A1, cited by applicant in IDS dated 17 JUL 2023, previously relied upon by the examiner) as applied to claim 7 above, and further in view of Mercep et al. (US 2018/0314921 A1, previously relied upon by the examiner).
Regarding claim 9 (Previously Presented), Liu et al. as modified above discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
The method of claim 7
Mercep et al. discloses:
wherein the ego motion is determined via odometry (Mercep et al. “The measurement integration system 310 can include an ego motion unit 313 to compensate for movement of at least one sensor capturing the raw measurement data 301, for example, due to the vehicle driving or moving in the environment. The ego motion unit 313 can estimate motion of the sensor capturing the raw measurement data 301, for example, by utilizing tracking functionality to analyze vehicle motion information, such as global positioning system (GPS) data, inertial measurements, vehicle odometer data, video images, or the like.” - ¶ [0038]).
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Mercep et al. into the invention of Liu et al. as modified above to yield the invention of claim 9. Liu et al., Zhang et al., Fontijne et al. and Mercep et al. and are considered analogous arts to the claimed invention as they disclose radar systems for vehicles that utilize machine learning for object detection. Liu et al. discloses the method of claim 7. However, Liu et al. fails to explicitly disclose wherein the ego motion is determined via odometry. Examiner notes that although Fontijne et al. does not explicitly disclose that the ego motion is determined via odometry, Fontijne et al. does disclose that the change in position of the radar sensor that is mounted on a vehicle is obtained via a sensor (Fontijne et al. “The ego motion between frames 902 and 904 is the change in the position of the radar sensor. This can be obtained in various ways, such as GPS or other sensors, or the neural network can estimate the motion, including rotation (i.e., a change in orientation of the vehicle 100).” - ¶ [0075]). Further, odometers are well known for measuring the change in position, or distance traveled, by a vehicle. The use of odometry is explicitly disclosed by Mercep et al. where “The ego motion unit 313 can estimate motion of the sensor capturing the raw measurement data 301, for example, by utilizing tracking functionality to analyze vehicle motion information, such as global positioning system (GPS) data, inertial measurements, vehicle odometer data, video images, or the like.” (Mercep et al. ¶ [0038]). The combination of Liu et al., Zhang et al., Fontijne et al. and Mercep et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045]), improve performance for object classification and detection (Fontijne et al. ¶ [0072]) and spatially align raw measurement data to world coordinates (Mercep et al. ¶ [0049]).
Claim(s) 10-13, 15 and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2021/0311169 A1, cited by applicant in IDS dated 17 JUL 2023, previously relied upon by the examiner) in view of Zhang et al. (US 2019/0147250 A1, newly cited by the examiner), Cohen et al. (US 2019/0391250 A1, previously relied upon by the examiner) and Fontijne et al. (WO 2020/113160 A1, cited by applicant in IDS dated 17 JUL 2023, previously relied upon by the examiner).
Regarding claim 10 (Currently Amended), Liu et al. discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
A computer system (Liu et al. object discrimination device 1 includes storage unit 13 and control unit 12, Fig. 8) for perceiving structure in a radar point cloud, the computer system comprising:
at least one memory configured to store computer-readable instructions (Liu et al. “The storage unit 13 stores radar data input from the radar device 2, programs executed by the processor constituting the control unit 12, and the like.” - ¶ [0040]); and
at least one processor coupled to the at least one memory and configured to execute the computer-readable instructions (Liu et al. “The control unit 12 is configured by a processor, and each unit of the control unit 12 is realized by the processor executing a program stored in the storage unit 13.” - ¶ [0041]), which upon execution cause the at least one processor to implement operations comprising:
generating a discretised image representation of the radar point cloud (Liu et al. The image generating unit 43 generates a radar detection image of the entire observation area based on radar data of the entire observation area.” - ¶ [0081]) having;
(ii) for each pixel in the discretised image representation
(i) a Doppler channel containing a Doppler velocity of the corresponding point in the radar point cloud (Liu et al. “Next, the image generating unit 32 sets the pixel value (values of each RGB channel) of the pixel at the position corresponding to the cell Cj based on the reflection intensity, Doppler velocity, and range of the cell Cj (ST206).” - ¶ [0089]), or
(ii) a radar cross section (RCS) channel containing an RCS value of the corresponding point in the radar point cloud for use by a machine learning (ML) perception component (Examiner notes that items (ii) and (iii) are alternatives such that only one of (ii) or (iii) is required);
inputting the discretised image representation to the machine learning (ML) perception component (Liu et al. “The object detection and discrimination unit 42 inputs the radar detection image of the entire observation area generated by the image generation unit 43 into a rained deep learning model…” - ¶ [0082]), which has been trained to extract information about structure exhibited in the radar point cloud from (i) the occupancy channel and: (ii) the Doppler channel, or (iii) the RCS channels (Liu et al. “Next, in the object detection and discrimination unit 42, the radar detection image of the entire observation area generated by the image generation unit 43 is input into the trained deep learning model, object detection and object discrimination are performed in the deep learning model, and the object discrimination result output from the deep learning model is obtained (ST112).” - ¶ [0086]; “Next, the object discrimination result and position information for each detected object are output (ST107).” - ¶ [0087]); and
wherein the ML perception component comprises a bounding box detector or other object detector (Liu et al. “in this embodiment, in addition to object discrimination, object detection to detect object regions is also performed using a deep learning model.” - ¶ [0079]), the extracted information comprising object position, orientation, and/or size information for at least one detected object (Liu et al. “Then, the reflection intensity, Doppler velocity, and range of the selected cell Cj are obtained from the radar data of the entire observation area (ST205).” - ¶ [0088]); and
Zhang et al. discloses:
generating a discretised image representation of the radar point cloud (Zhang et al. “The computing system can generate a two-dimensional voxel representation of the three-dimensional data (e.g., of the three-dimensional point clouds) that can be ingested by ta machine-learned model.” - ¶ [0021]) having;
(i) an occupancy channel indicating whether or not each pixel of the discretised image representation corresponds to a point in the radar point cloud (Zhang et al. “The occupancy channel can be set to “1” if at least one point lies within the voxel cell or “0” if there is not a point which lies within the voxel cell. One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” - ¶ [0025]) and
(ii) for each pixel in the discretised image representation that has a non-zero occupancy channel value (Zhang et al. “One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” - ¶ [0025]):
(i) a Doppler channel containing a Doppler velocity of the corresponding point in the radar point cloud (Zhang et al. “Other sensor modalities that can be encoded into channel(s) include, for example, intensity, speed (elg., of LIDAR regurnes), or other image features, etc.” - ¶ [0025]), or
(ii) a radar cross section (RCS) channel containing an RCS value of the corresponding point in the radar point cloud for use by a machine learning (ML) perception component (Examiner notes that items (i) and (ii) are alternatives such that only one of (i) or (ii) is required);
Cohen et al. discloses:
(ii) a radar cross section (RCS) channel containing an RCS value (Cohen et al. “In some example implementations, the first radar data 114 and/or the second radar data 116 may include position information indicative of a location of objects in the environment, e.g., a range and azimuth relative to the vehicle 106 or a position in a local or global coordinate system. The first sensor data 114 and/or the second sensor data 116 may also include signal strength information… In some instances, the signal strength may be a radar cross-section (RCS) measurement.” - ¶ [0021]) of the corresponding point in the radar point cloud for use by a machine learning (ML) perception component (Cohen et al. “In some implementations, the clustering component may utilize algorithmic processing, e.g., DBSCAN, computer learning processing, e.g., K-means unsupervised learning, and/or additional clustering techniques” - ¶ [0049]);
wherein the radar point cloud is transformed for generating the discretised image representation of the transformed radar point by:
applying clustering to the radar point cloud (Cohen et al. “At operation 118, the process 100 can determine one or more clusters from the first radar data 114.” - ¶ [0022]), and thereby identifying at least one moving object cluster within the radar point cloud (Cohen et al. “In the visualization 122, the positions of the detected points 124 may generally correspond to a position of one or more detected objects.” - ¶ [0022]), the points of the radar point cloud being time-stamped (Cohen et al. “For example, the timestamp may be a single parameter of a sensor measurement used to cluster the one or more points…” - ¶ [0034]), having been captured over a non-zero accumulation window (Cohen et al. “However, because the first radar sensor 108 and the second radar sensor 110 may have different pulse intervals and/or different scanning intervals, the scans may not be exactly at the same time. In some examples, scans occurring within about 100 milliseconds may be considered to be proximate in time or substantially simultaneous.” - ¶ [0026]),
determining a motion model for the moving object cluster, by fitting one or more parameters of the motion model to the time-stamped points of that cluster (Cohen et al. “As described herein, the updated point cluster 136 may be useful to the autonomous vehicle 106 to identify objects, predict actions the objects may take, and/or maneuver in the environment relative to the objects, among other things.” - ¶ [0030])
Fontijne et al. discloses:
using the motion model to transform the time-stamped points (Fontijne et al. “Each camera and radar frame may be timestamped.” - ¶ [0048]) of the moving object cluster to a common reference time (Fontijne et al. latent-space ego-motion compensation technique, Fig. 9; ¶ [0074]-[0078]).
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Zhang et al., Cohen et al. and Fontijne et al. into the invention of Liu et al. to yield the invention of claim 10 above. Liu et al., Zhang et al., Cohen et al. and Fontijne et al. are considered analogous arts to the claimed invention as they disclose vehicle radar systems for object detection. Liu et al. discloses limitations of claim 10 outlined above. However, Liu et al. fails to explicitly disclose the occupancy channel, clustering time-stamped radar points, and determining motion models for the radar point clusters and transforming time-stamped points to a common reference time. These features are disclosed by Zhang et al., Cohen et al. and Fontijne et al. Zhang et al. discloses “The occupancy channel can be set to “1” if at least one point lies within the voxel cell or “0” if there is not a point which lies within the voxel cell. One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” (Zhang et al. ¶ [0025]); Cohen et al. discloses accumulating and clustering time-stamped radar points, and updating the point clusters in order “to identify objects, predict actions the objects may take, and/or maneuver in the environment relative to the objects, among other things.” (Cohen et al. ¶ [0030]); and Fontijne et al. discloses the latent-space ego-motion compensation technique transforms the radar images received at different times to world coordinates using the second input frame as the world frame (Fontijne et al. latent-space ego-motion compensation technique, Fig. 9; ¶ [0074]-[0078]). The combination of Liu et al., Zhang et al., Cohen et al. and Fontijne et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045 ]), more efficiently and accurately detect and characterize objects (Cohen et al. ¶ [0015]), and improve performance for object classification and detection (Fontijne et al. ¶ [0072]).
Regarding claim 11 (Previously Presented), Liu et al. as modified above discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
The computer system of claim 10
Cohen et al. discloses:
wherein the clustering identifies multiple moving object clusters (Cohen et al. “In this example, the process 100 uses multiple sensors with overlapping fields of view to determine point clusters indicative of objects in the environment of the autonomous vehicle.” - ¶ [0018])
Fontijne et al. discloses:
a motion model is determined for each of the multiple moving object clusters and used to transform the respective time-stamped points of each cluster to the common reference time (Fontijne et al. Fig. 9; ¶ [0078])
wherein the transformed point cloud comprises the transformed points of the multiple object clusters (Fontijne et al. “Based on the ego motion from a previous step of the process, each frame’s 902 and 904 feature map 910 and 912 is transformed to a new feature map 914 and 916, respectively, in world coordinates… In the example of FIG. 9, the second frame 904 is chosen as the world frame, and thus, there is no transformation between the feature map 912 and 916. In contrast, the first frame 902 is transformed. In the example of FIG. 9, this transformation is simply a translation on the x-axis.” - ¶ [0076]-[0077]; Fig. 9).
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Cohen et al. into the invention of Liu et al. as modified above to yield the invention of claim 11. Liu et al., Zhang et al., Cohen et al. and Fontijne et al. are considered analogous arts to the claimed invention as they disclose vehicle radar systems for object detection. Liu et al. as modified above discloses computer system of claim 10. However, Liu et al. fails to explicitly disclose wherein the clustering identifies multiple moving object clusters, and a motion model is determined for each of the multiple moving object clusters and used to transform the respective time-stamped points of that cluster to the common reference time; wherein the transformed point cloud comprises the transformed points of the multiple object cluster. These features are disclosed by Cohen et al. and Fontijne et al. where Cohen et al. discloses the clustering identifies multiple objects (Cohen et al. ¶ [0018]) and Fontijne et al. discloses the motion models of the moving object clusters are transformed to a common reference time (Fontijne et al. latent-space ego-motion compensation technique, Fig. 9; ¶ [0074]-[0078]). The combination of Liu et al., Zhang et al., Cohen et al. and Fontijne et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045 ]), more efficiently and accurately detect and characterize objects (Cohen et al. ¶ [0015]), and improve performance for object classification and detection (Fontijne et al. ¶ [0072]).
Regarding claim 12 (Previously Presented), Liu et al. as modified above discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
The computer system of claim 11
Fontijne et al. discloses:
wherein the transformed point cloud additionally comprises untransformed static object points of the radar point cloud (Fontijne et al. “This new tensor is in “world coordinates,” meaning that stationary objects remain in the same location on the feature map over time, and the ego location can change (depending on ego motion).” - ¶ [0076]).
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Fontijne et al. into the invention of Liu et al. as modified above to yield the invention of claim 12. Liu et al., Zhang et al., Cohen et al. and Fontijne et al. are considered analogous arts to the claimed invention as they disclose vehicle radar systems for object detection. Liu et al. as modified above discloses computer system of claim 10. However, Liu et al. fails to explicitly disclose wherein the transformed point cloud additionally comprises untransformed static object points of the radar point cloud. This feature is disclosed by Fontijne et al. where the ego coordinates are converted to world coordinates in which the stationary objects remain in the same location (Fontijne et al. ¶ [0076]). The combination of Liu et al., Zhang et al., Cohen et al. and Fontijne et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045 ]), more efficiently and accurately detect and characterize objects (Cohen et al. ¶ [0015]), and improve performance for object classification and detection (Fontijne et al. ¶ [0072]).
Regarding claim 13 (Previously Presented), Liu et al. as modified above discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
The computer system of claim 10
Cohen et al. discloses:
wherein the clustering is based on timestamps of the time-stamped points (Cohen et al. “Timestamps from the sensor scans may be used to determine whether scans are within the threshold. In some instances, the threshold time may be determined as a part of the clustering (or association) performed. For example, the timestamp may be a single parameter of a sensor measurement used to cluster the one or more points and, in at least some instances, a threshold may be associated with the timestamps.” - ¶ [0034]), and
wherein the clustering in density-based (Cohen et al. “In some implementations, the clustering component may utilize algorithmic processing, e.g., DBSCAN, computer learning processing, e.g., K-means unsupervised learning, and/or additional clustering techniques” - ¶ [0049]) and uses a time threshold to determine whether or not to assign a point to the moving object cluster (Cohen et al. “Timestamps from the sensor scans may be used to determine whether scans are within the threshold. In some instances, the threshold time may be determined as a part of the clustering (or association) performed. For example, the timestamp may be a single parameter of a sensor measurement used to cluster the one or more points and, in at least some instances, a threshold may be associated with the timestamps.” - ¶ [0034]), wherein the point is assigned to the moving object cluster only if a difference between the timestamp of the point and the timestamp of another point assigned to the moving cluster is less than the time threshold (Cohen et al. “Clustering may be performed based on any physical parameters associated with the points (including, but not limited to, velocities, signal strength, location, nearest neighbors, a time stamp the measurement was performed, etc.), as well as corresponding threshold differences in any of the aforementioned parameters. Inclusion may also be based on a combination of these and other information.” - ¶ [0027]).
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Cohen et al. into the invention of Liu et al. as modified above to yield the invention of claim 13. Liu et al., Zhang et al., Cohen et al. and Fontijne et al. are considered analogous arts to the claimed invention as they disclose vehicle radar systems for object detection. Liu et al. as modified above discloses the computer system of claim 10. However, Liu et al. fails to explicitly disclose the clustering is based on the timestamps, wherein the clustering in density-based. These features are disclosed by Cohen et al. where Cohen et al. discloses the “the timestamp may be a single parameter of a sensor measurement used to cluster the one or more points and, in at least some instances, a threshold may be associated with the timestamps.” (Cohen et al. ¶ [0034]). The combination of Liu et al., Zhang et al., Cohen et al. and Fontijne et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045 ]), more efficiently and accurately detect and characterize objects (Cohen et al. ¶ [0015]), and improve performance for object classification and detection (Fontijne et al. ¶ [0072]).
Regarding claim 15 (Previously Presented), Liu et al. as modified above discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
The computer system of claim 10
Cohen et al. discloses:
wherein the clustering is based on the Doppler velocities (Cohen et al. “For example, when the radar sensor is a Doppler-type sensor, velocity of the objects may be used to determine the point cluster 126.” - ¶ [0024]), and
wherein the clustering is density-based (Cohen et al. “In some implementations, the clustering component may utilize algorithmic processing, e.g., DBSCAN, computer learning processing, e.g., K-means unsupervised learning, and/or additional clustering techniques” - ¶ [0049]) and uses a velocity threshold to determine whether or not to assign a point to the moving object cluster, and
wherein the point is assigned to the moving object cluster only if a difference between the Doppler velocity of the point and the Doppler velocity of another point assigned to the moving object cluster is less than the velocity threshold (Cohen et al. “Clustering may be performed based on any physical parameters associated with the points (including, but not limited to, velocities, signal strength, location, nearest neighbors, a time stamp the measurement was performed, etc.), as well as corresponding threshold differences in any of the aforementioned parameters. Inclusion may also be based on a combination of these and other information.” - ¶ [0027]).
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Cohen et al. into the invention of Liu et al. as modified above to yield the invention of claim 15. Liu et al., Zhang et al., Cohen et al. and Fontijne et al. are considered analogous arts to the claimed invention as they disclose vehicle radar systems for object detection. Liu et al. as modified above discloses the computer system of claim 10. However, Liu et al. fails to explicitly disclose the clustering is based on the Doppler velocities, wherein the clustering is density-based. These features are disclosed by Cohen et al. where Cohen et al. discloses “Clustering may be performed based on any physical parameters associated with the points (including, but not limited to, velocities, signal strength, location, nearest neighbors, a time stamp the measurement was performed, etc.), as well as corresponding threshold differences in any of the aforementioned parameters. Inclusion may also be based on a combination of these and other information.” (Cohen et al. ¶ [0027]). The combination of Liu et al., Zhang et al., Cohen et al. and Fontijne et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045 ]), more efficiently and accurately detect and characterize objects (Cohen et al. ¶ [0015]), and improve performance for object classification and detection (Fontijne et al. ¶ [0072]).
Regarding claim 17 (Currently Amended), Liu et al. as modified above discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
The computer system of claim 10
Cohen et al. discloses:
The computer system of claim 10, wherein Doppler velocity of the (or each) moving object cluster is used to determine the motion model for that cluster (Cohen et al. “As described herein, the updated point cluster 136 may be useful to the autonomous vehicle 106 to identify objects, predict actions the objects may take, and/or maneuver in the environment relative to the objects, among other things.” - ¶ [0030]).
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Cohen et al. into the invention of Liu et al. as modified above to yield the invention of claim 17. Liu et al., Zhang et al., Cohen et al. and Fontijne et al. are considered analogous arts to the claimed invention as they disclose vehicle radar systems for object detection. Liu et al. as modified above discloses the computer system of claim 10. However, Liu et al. fails to explicitly disclose wherein Doppler velocities of the (or each) moving object cluster are used to determine the motion model for that cluster. These features are disclosed by Cohen et al. where Cohen et al. discloses the moving object cluster data is used to predict actions that the detected objects will take (Cohen et al. ¶ [0030]). The combination of Liu et al., Zhang et al., Cohen et al. and Fontijne et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045 ]), more efficiently and accurately detect and characterize objects (Cohen et al. ¶ [0015]), and improve performance for object classification and detection (Fontijne et al. ¶ [0072]).
Regarding claim 18 (Previously Presented), Liu et al. as modified above discloses:
The computer system of claim 10, wherein the discretised image representation has one or more motion channels that encode, for each occupied pixel corresponding to a point of (one of) the moving object cluster(s), motion information about that point derived from the motion model of that moving object cluster (Liu et al. “Next, the image generating unit 32 sets the pixel value (values of each RGB channel) of the pixel at the position corresponding to the cell Cj based on the reflection intensity, Doppler velocity, and range of the cell Cj (ST206).” - ¶ [0089]).
Claim(s) 21 and 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2021/0311169 A1, cited by applicant in IDS dated 17 JUL 2023, previously relied upon by the examiner) in view of Zhang et al. (US 2019/0147250 A1, newly cited by the examiner as applied to claim 6 above, and further in view of Insana et al. (US 2019/0154823 A1, newly cited by the examiner).
Regarding claim 21 (Previously Presented), Liu et al. as modified above discloses:
[Note: what is not explicitly taught by Liu et al. has been struck-through]
The method of claim 6
Insana et al. discloses:
wherein the accumulated radar point cloud includes points captured from an object that exhibit smearing effects caused by motion of the object during the multiple radar sweeps, and the discretised image representation retains the smearing effects (Insana et al. “Pixels containing detections from static objects will accumulate the fastest, while pixels having detections from dynamic targets appear smeared as energy will be spread out across more adjacent pixels.” - ¶ [0037]).
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Insana et al. into the invention of Liu et al. as modified above to yield the invention of claim 21. Liu et al., Zhang et al. and Insana et al. are considered analogous arts to the claimed invention as they disclose radar systems for vehicles for object detection. Liu et al. as modified above discloses the invention of claim 6. However, Liu et al. fails to explicitly disclose wherein the accumulated radar point cloud includes points captured from an object that exhibit smearing effects caused by motion of the object during the multiple radar sweeps, and the discretised image representation retains the smearing effects. This feature is disclosed by Insana et al. where “Pixels containing detections from static objects will accumulate the fastest, while pixels having detections from dynamic targets appear smeared as energy will be spread out across more adjacent pixels.” (Insana et al. ¶ [0037]). The combination of Liu et al., Zhang et al. and Insana et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045]) and improve “the signal-to-Noise (SNR) for static targets within the map” in order “to determine a presence of static or substantially static objects within the image.” (Insana et al. ¶ [0009], [0037]).
Regarding claim 23 (Currently Amended), Liu et al. discloses:
A non-transitory computer readable medium storing computer program instructions (Liu et al. “The storage unit 13 stores radar data input from the radar device 2, programs executed by the processor constituting the control unit 12, and the like.” - ¶ [0040]), the computer program instructions configured so as, when executed on one or more hardware processors (Liu et al. “The control unit 12 is configured by a processor, and each unit of the control unit 12 is realized by the processor executing a program stored in the storage unit 13.” - ¶ [0041]), to cause the one or more processors to perform operations comprising:
generating a discretised image representation of a radar point cloud (Liu et al. The image generating unit 43 generates a radar detection image of the entire observation area based on radar data of the entire observation area.” - ¶ [0081]) having;
(i) an occupancy channel indicating whether or not each pixel of the discretised image representation corresponds to a point in the radar point cloud, and
(ii) for each pixel in the discretised image representation that has a non-zero occupancy channel value:
(i) a Doppler channel containing a Doppler velocity of the corresponding point in the radar point cloud (Liu et al. “Next, the image generating unit 32 sets the pixel value (values of each RGB channel) of the pixel at the position corresponding to the cell Cj based on the reflection intensity, Doppler velocity, and range of the cell Cj (ST206).” - ¶ [0089]), or
(ii) a radar cross section (RCS) channel containing an RCS value of the corresponding point in the radar point cloud for use by a machine learning (ML) perception component (Examiner notes that items (ii) and (iii) are alternatives such that only one of (ii) or (iii) is required);
inputting the discretised image representation to the machine learning (ML) perception component (Liu et al. “The object detection and discrimination unit 42 inputs the radar detection image of the entire observation area generated by the image generation unit 43 into a rained deep learning model…” - ¶ [0082]), which has been trained to extract information about structure exhibited in the radar point cloud from (i) the occupancy channel and: (ii) the Doppler channel, or (iii) the RCS channels (Liu et al. “Next, in the object detection and discrimination unit 42, the radar detection image of the entire observation area generated by the image generation unit 43 is input into the trained deep learning model, object detection and object discrimination are performed in the deep learning model, and the object discrimination result output from the deep learning model is obtained (ST112).” - ¶ [0086]; “Next, the object discrimination result and position information for each detected object are output (ST107).” - ¶ [0087]);
wherein the ML perception component comprises a bounding box detector or other object detector (Liu et al. “in this embodiment, in addition to object discrimination, object detection to detect object regions is also performed using a deep learning model.” - ¶ [0079]), the extracted information comprising object position, orientation, and/or size information for at least one detected object (Liu et al. “Then, the reflection intensity, Doppler velocity, and range of the selected cell Cj are obtained from the radar data of the entire observation area (ST205).” - ¶ [0088]);
wherein the radar point cloud is an accumulated radar point cloud comprising points accumulated over multiple radar sweeps (Liu et al. “The radar device 2 outputs radar data for the entire observation area at each time at intervals (e.g., 50 ms) corresponding to the beam scanning period, and when the radar data for the object area at each time extracted from this radar data for the entire observation area at each time is visualized, a radar detection image is generated at a high frame rate (e.g., 20 fps).” - ¶ [0122]; “The data synthesis unit 72 synthesizes (integrates) the radar data of the object area at a plurality of times acquired by the area data extraction unit 31, and generates synthesized radar data of the object area.” - ¶ [0125]; “Specifically, for example, when radar data at four different times is synthesized, it is determined that the timing for output is when the frame order is a multiple of four.” - ¶ [0138]; where radar data for the entire observation area is obtained during a beam scan, or radar sweep, and the radar data for the entire observation area is obtained multiple times, over multiple radar sweeps); and
Zhang et al. discloses:
generating a discretised image representation of a radar point cloud (Zhang et al. “The computing system can generate a two-dimensional voxel representation of the three-dimensional data (e.g., of the three-dimensional point clouds) that can be ingested by ta machine-learned model.” - ¶ [0021]) having;
(i) an occupancy channel indicating whether or not each pixel of the discretised image representation corresponds to a point in the radar point cloud (Zhang et al. “The occupancy channel can be set to “1” if at least one point lies within the voxel cell or “0” if there is not a point which lies within the voxel cell. One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” - ¶ [0025]), and
(ii) for each pixel in the discretised image representation that has anon-zero occupancy channel value (Zhang et al. “One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” - ¶ [0025]):
(i) a Doppler channel containing a Doppler velocity of the corresponding point in the radar point cloud (Zhang et al. “Other sensor modalities that can be encoded into channel(s) include, for example, intensity, speed (e.g., of LIDAR returns), or other image features, etc.” - ¶ [0025]) , or
(ii) a radar cross section (RCS) channel containing an RCS value of the corresponding point in the radar point cloud for use by a machine learning (ML) perception component (Examiner notes that items (i) and (ii) are alternatives such that only one of (i) or (ii) is required);
Insana et al. discloses:
wherein the radar point cloud is an accumulated radar point cloud comprising points accumulated over multiple radar sweeps (Insana et al. “the integrated image is propagated over time based on the host dynamic” - ¶ [0049]); and
wherein the accumulated radar point cloud includes points captured from an object that exhibit smearing effects caused by motion of the object during the multiple radar sweeps, and the discretised image representation retains the smearing effects (Insana et al. “Pixels containing detections from static objects will accumulate the fastest, while pixels having detections from dynamic targets appear smeared as energy will be spread out across more adjacent pixels.” - ¶ [0037]).
It would have been obvious to someone with ordinary skill in the art prior to the effective filing date of the claimed invention to incorporate the features as disclosed by Zhang et al. and Insana et al. into the invention of Liu et al. as modified above to yield the invention of claim 23 above. Liu et al., Zhang et al. and Insana et al. are considered analogous arts to the claimed invention as they disclose radar systems for vehicles for object detection. Liu et al. as discloses the limitations of claim 23 outlined above. However, Liu et al. fails to explicitly disclose an occupancy channel indicating whether or not each pixel of the discretised image representation corresponds to a point in the radar point cloud, and wherein the accumulated radar point cloud includes points captured from an object that exhibit smearing effects caused by motion of the object during the multiple radar sweeps, and the discretised image representation retains the smearing effects. These features are disclosed by Zhang et al. and Insana et al., where Zhang et al. discloses “The occupancy channel can be set to “1” if at least one point lies within the voxel cell or “0” if there is not a point which lies within the voxel cell. One or more second channels can be encoded with one or more sensor modalities associated with the sensor data.” (Zhang et al. ¶ [0025]), and Insana et al. discloses “Pixels containing detections from static objects will accumulate the fastest, while pixels having detections from dynamic targets appear smeared as energy will be spread out across more adjacent pixels.” (Insana et al. ¶ [0037]). The combination of Liu et al., Zhang et al. and Insana et al. would be obvious with a reasonable expectation of success to convert the three-dimensional voxel grid into a two-dimensional representation with a binary occupancy grid in order to allow for sparsity-invariant efficient computation (Zhang et al. ¶ [0045]) and improve “the signal-to-Noise (SNR) for static targets within the map” in order “to determine a presence of static or substantially static objects within the image.” (Insana et al. ¶ [0009], [0037]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAOMI M WOLFORD whose telephone number is (571)272-3929. The examiner can normally be reached Monday - Friday, 8:30 am - 4:30 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Resha Desai can be reached at (571)270-7792. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
NAOMI M. WOLFORD
Examiner
Art Unit 3648
/N.M.W./Examiner, Art Unit 3648
12 AUG 2026
/RESHA DESAI/Supervisory Patent Examiner, Art Unit 3648
Prosecution Insights