FireSense’s command-side mapping stack fuses thermal keyframes from a visual–inertial front end into an incrementally optimized SLAM solution, using learned global descriptors to propose loop closures and geometric verification to accept or reject them. It supports compressed video on the link, session-based start/stop with gravity-aligned world initialization, and live visualization of the optimized map and trajectory to aid firefighters in search-relevant indoor scenarios.
Mapping backend (command stack)
The command computer on FireSense runs the thermal SLAM backend: the part of the system that turns keyframes from the visual–inertial front end into a consistent trajectory and map over time.
Each keyframe bundles what the robot knew at one instant: thermal imagery, 3D landmarks in the camera frame, and motion-related pose information so the backend can chain poses and uncertainties. The backend incrementally optimizes a factor graph (using GTSAM / iSAM2-style smoothing): it adds odometry-style constraints from the front end and, when appropriate, loop-closure constraints when the robot revisits a place.
Loop closure and verification
Revisits are found using global image descriptors from a learned visual place-recognition model suited to thermal imagery (high-dimensional, normalized embeddings). Candidate matches are not accepted on appearance alone: a geometric verification stage checks consistency using 3D structure and pose information, optionally with dense alignment-style checks, so false positives from repeated texture are suppressed.
Perception pipeline on the command side
Because bandwidth matters between computers, thermal video may arrive compressed on the command machine. The stack can decode it before descriptor extraction so the same appearance model sees a full image. An optional visualization path turns optimized poses and map updates into point clouds, trajectories, and loop markers for operators and debugging (e.g. via Foxglove-style tools).
Session lifecycle
Mapping is treated as a session: the operator starts with the vehicle stationary so a short IMU sample can define gravity and a consistent world frame (Z-up). While mapping, the backend consumes keyframes and publishes refined poses and map updates. Stopping the session can trigger an optional final global refinement before exporting results; the design targets standard trajectory/map artifacts for evaluation and field use.
Role in FireSense
FireSense splits work across machines: the payload focuses on sensing and the VIO front end; the command side focuses on heavy optimization, thermal place recognition, and operator-facing mapping outputs. Shared message and service definitions keep both sides aligned so one keyframe stream and one session model drive the full stack.