This page summarizes the spring-semester performance results for FireSense, evaluated against the requirements targeted for the Spring Validation Demonstration (SVD). Tests were conducted at the Allegheny County Fire Academy (ACFA) concrete building and at the Robolounge in Newell-Simon Hall (NSH) for the SVD encore.
Spring Targeted Requirements
The four core capabilities prioritized this semester were: mapping the building, sensing physical barriers, localizing the payloads, and producing an accurate global map, along with the deployability and simple-power-supply non-functional requirements.
| Requirement | Target | Result |
|---|---|---|
| Map the building | 5 m Chamfer Distance, 0.2 Hz update | ✅ Met |
| Sense physical barriers | 0.8–9 m range | ✅ Met |
| Localize payloads | 1 m ATE, 45° AOE | ✅ Met (0.18 m ATE) |
| Create global map | 1 m ATE, 45° AOE | ✅ Met |
| Deployable system | Field-ready, untethered | ✅ Met (915 MHz HaLow) |
| Simple power supply | Standard battery | ✅ Met (Makita 18V) |
SLAM Frontend Performance
The MAC-VIO frontend was evaluated against an RGB-based MAC-VO baseline. Key results:
- Relative Translation Error (RTE): 0.0066 m — strong translational consistency
- Relative Orientation Error (ROE): 0.2302 rad — higher rotational drift due to limited and noisy thermal features
- Runtime: ~4 Hz on Jetson AGX Orin with TensorRT acceleration
The frontend provides reliable short-term tracking, while accumulated drift is corrected by the backend’s loop closure for global consistency.
SLAM Backend & Trajectory Accuracy
After loop closure, the optimized pose graph dramatically reduces drift compared to raw odometry. The table below summarizes alignment RMSE (Umeyama-aligned, evaluated with evo) for both demonstration runs:
| Run | ATE (m) | Orientation RMSE (°) |
|---|---|---|
| SVD — ACFA (concrete building) | 0.18 | 15 |
| SVD Encore — NSH (Robolounge) | 0.21 | 18 |
Both results comfortably meet the 1 m ATE and 45° AOE targets.
[Sequential odometry/VIO (red) vs. optimized pose graph after loop closures (yellow) — shows how loop closure pulls the drifted trajectory back into a globally consistent map]
Map Quality Comparison
The FireSense thermal SLAM map was compared against a baseline reconstruction generated by NVIDIA cuVSLAM using the ZED-X RGB stereo camera. Chamfer-distance residuals between the aligned point clouds are color-coded to show spatial agreement.
[Map comparison: cuVSLAM RGB dense point cloud vs. our stereo-thermal map from MAC-VIO/SLAM, with Chamfer residuals false-colored after spatial alignment]
The thermal-based reconstruction successfully captures the structural layout of the test environment, demonstrating that thermal SLAM can produce maps comparable to RGB-based methods — a critical capability for smoke-filled environments where RGB cameras fail.
Communication Performance
Migrating the middleware from FastDDS multicast to CycloneDDS + Zenoh bridge unicast produced major bandwidth improvements, eliminating the network jamming caused by ROS topic discovery traffic during multi-payload operation:
| Traffic Metric | Old (FastDDS) | New (Zenoh Bridge) | Reduction |
|---|---|---|---|
| Incoming AVG | 0.49 MB/s | 0.09 MB/s | 81% |
| Incoming MAX | 2.09 MB/s | 0.16 MB/s | 92% |
| Outgoing AVG | 0.03 MB/s | 0.00 MB/s | ~100% |
| Outgoing MAX | 4.30 MB/s | 0.01 MB/s | 99.7% |
These improvements directly support multi-payload C-SLAM scalability and prepared the system for two-payload operation in the fall.
Summary
By the end of the spring semester, FireSense successfully demonstrated:
- A working end-to-end thermal SLAM pipeline from sensing through global optimization
- Trajectory accuracy well within targeted specifications (0.18 m ATE vs. 1 m target)
- Reliable wireless operation through walls via 915 MHz HaLow
- A modular, field-deployable wearable payload with simple power supply
- Substantial bandwidth headroom for scaling to two-payload collaborative SLAM in the fall