CMS: Enabling Real-world Cooperative Multi-modal Sensing

Bo Wu     Jerry Li     Ruoshen Mo     Justin Yue     Hang Qiu

University of California, Riverside   

Paper | Code | Demo | Bibtex

Abstract

CMS, is a open-source, cooperative multi-modal sensing platform. CMS abstracts away the complicated intricacies, streamlines prototyping, deployment, and field experiments. Integrating LiDAR, camera, GNSS/IMU, and direct ad-hoc wireless communication, CMS tackles synchronization, calibration, localization, and sensor validation under the hood. This video demonstrates the capabilities of CMS. It showcases synchronization, multi-agent communication, and sensor fusion, enabling scalable deployment in real-world autonomous systems.

Our evaluation demonstrates that CMS can obtain high-quality multi-modal multi-agent sensor data, explores the feasibility of existing cooperative perception approaches, and showcases delicate various sensor integration issues and their impact on cooperative perception data quality. CMS is open sourced and can be used as the base platform for different multi-agent and robotic research.
NOPUSH Intro

CMS Overview

CMS integrates LiDAR, Camera, GNSS with a power-over-ethernet (PoE) switch, which forwards the data to a central ROS node (running on a laptop). The laptop and sensors are synchronized with GNSS time, and all intrinsic and extrinsic parameters are calibrated for all sensors. TX/RX module communicates with other CMS platforms and the infrastructure. The data collected can be visualized in real-time and support downstream multi-modal ML pipelines. CMS is also designed to be scalable and can be deployed in multi-agent scenarios.

CMS Architecture

Evaluations

We evaluate CMS across areas of synchronization, calibration, communication, and localization as they are all necessary for a successful multi agent system. Below are some evaluations while the rest can be found in our paper.

LiDAR to Camera overlays applied after various calibration methods on CMS collected data.
How much data that can be shared between CMS setups when opearting at 10hz
Comparison of point cloud alignment collected from CMS before and after applying Iterative Closest Point (ICP) correction.

Citation

@inbook{10.1145/3715014.3724372,
author = {Wu, Bo and Li, Jerry and Mo, Ruoshen and Yue, Justin and Bharadia, Dinesh and Qiu, Hang},
title = {Demo Abstract: Cooperative Multi-modal Sensing},
year = {2025},
isbn = {9798400714795},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3715014.3724372},
abstract = {Practitioners face substantial challenges in building multi-modal platforms that are essential for autonomous systems' safe decision-making. Those complications, including synchronization, calibration, and tedious sensor validation, hinder user adoption for real-world applications. We present CMS, a Cooperative Multi-modal Sensing Platform. CMS provides one consistent interface, integrating LiDAR, camera, RaDAR, and GNSS/IMU, streamlines these processes and makes the intricacies transparent to users and applications. Our demonstration shows that CMS can obtain high-quality multi-modal sensor data, paving the way toward real-world prototypes of cooperative autonomous systems.},
booktitle = {Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems},
pages = {712–713},
numpages = {2}
}