Revisiting Radar Perception With Spectral Point Clouds

We present spectral point clouds for radar perception: radar point clouds are compressed representations of the radar spectra that are spectrally grounded and spatially located.

Enriching point clouds with further spectral information improves their performance on deep learning tasks. Spectral point clouds can perform just as well as dense spectra, but they don't suffer from the same sensor-specific differences. They may be the way forward for unified radar perception, and radar foundation models.

Radar perception models are trained with different inputs, from range-Doppler spectra to sparse point clouds. Dense spectra are assumed to outperform sparse point clouds, yet they can vary considerably across sensors and configurations, which hinders transfer. In this paper, we provide alternatives for incorporating spectral information into radar point clouds and show that point clouds need not underperform compared to spectra. We introduce the spectral point cloud paradigm, where point clouds are treated as sparse, compressed representations of the radar spectra, and argue that, when enriched with spectral information, they serve as strong candidates for a unified input representation that is more robust against sensor-specific differences. We develop an experimental framework that compares spectral point cloud (PC) models at varying densities against a dense range-Doppler (RD) benchmark, and report the density levels where the PC configurations meet the performance of the RD benchmark. Furthermore, we experiment with two basic spectral enrichment approaches that inject additional target-relevant information into the point clouds. Contrary to the common belief that the dense RD approach is superior, we show that point clouds can do just as well, and can surpass the RD benchmark when enrichment is applied. Spectral point clouds can therefore serve as strong candidates for unified radar perception, paving the way for future radar foundation models.

We evaluate on the RADIal dataset using two pipelines: an RD benchmark based on FFTRadNet that acts as the baseline, and a PC pipeline based on PointPillars where we vary the density of the spectral point clouds and apply different forms of spectral enrichment. Point cloud density refers to the fraction of radar spectrum peaks retained. The detection head is kept the same in both pipelines to ensure a fair comparison. SpectralPillars is the PointPillars-based configuration with RD neighborhood features (local spectral context around each peak) and angle spectrum descriptors (per-point angular response features).

Adding Doppler and angle amplitude attributes to spatial point cloud coordinates yields average detection F1 gains of +3.6 and +2.0 respectively, with both combined giving +4.6 F1. At the RADIal default density of 5.7%, the full spectral point cloud is within 1.3 F1 of the RD benchmark, and surpasses it at 30.8% density.

SpectralPillars, our enriched detector, reaches the RD benchmark at just 7.2% density (running 89 FPS faster than the RD detector at that operating point) and reaches 93.7% F1 at 56.4% density. As shown in (b), SpectralPillars consistently outperforms sparse RD inputs (RD spectra masked to the same density) by an average of +1.3 F1 across the sweep.

Radar point clouds do not need to be a weaker alternative to dense spectra. When grounded in the spectral basis of the radar signal and enriched with target-relevant cues, they can match and surpass RD-based detectors while being faster, lighter, and more robust to sensor-specific differences. Spectral point clouds are a practical and principled step toward unified radar perception and future radar foundation models.