CMS: Enabling Real-world Cooperative Multi-modal Sensing

Bo Wu     Jerry Li     Ruoshen Mo     Justin Yue     Hang Qiu

University of California, Riverside   

Paper | Code | Demo | Bibtex

CMS, is a open-source, cooperative multi-modal sensing platform. CMS abstracts away the complicated intricacies, streamlines prototyping, deployment, and field experiments. Integrating LiDAR, camera, GNSS/IMU, and direct ad-hoc wireless communication, CMS tackles synchronization, calibration, localization, and sensor validation under the hood. Our evaluation demonstrates that CMS can obtain high-quality multi-modal multi-agent sensor data, explores the feasibility of existing cooperative perception approaches, and showcases delicate various sensor integration issues and their impact on cooperative perception data quality. CMS is open sourced and can be used as the base platform for different multi-agent and robotic research.
NOPUSH Intro

CMS Overview

CMS integrates LiDAR, Camera, GNSS with a power-over-ethernet (PoE) switch, which forwards the data to a central ROS node (running on a laptop). The laptop and sensors are synchronized with GNSS time, and all intrinsic and extrinsic parameters are calibrated for all sensors. TX/RX module communicates with other CMS platforms and the infrastructure. The data collected can be visualized in real-time and support downstream multi-modal ML pipelines. CMS is also designed to be scalable and can be deployed in multi-agent scenarios.

CMS Architecture

Evaluations

We evaluate CMS across areas of synchronization, calibration, communication, and localization as they are all necessary for a successful multi agent system. Below are some evaluations while the rest can be found in our paper.

LiDAR to Camera overlays applied after various calibration methods on CMS collected data.
How much data that can be shared between CMS setups when opearting at 10hz
Comparison of point cloud alignment collected from CMS before and after applying Iterative Closest Point (ICP) correction.

Demo of CMS

This video demonstrates the capabilities of CMS. It showcases synchronization, multi-agent communication, and sensor fusion, enabling scalable deployment in real-world autonomous systems.