System Details
Subsystem Details
Problem Description
Residential building fires pose a severe threat to public safety, causing thousands of deaths and billions of dollars in losses each year. During a primary search, smoke and heat can remove the visual and spatial cues that firefighters and incident commanders normally rely on.
Inside the building, firefighters may have only a short thermal view and verbal updates to maintain orientation. Outside, the incident commander often lacks a live picture of the interior, the route already searched, and the firefighter’s current position.
FireSense is a human-centered sensing system built to close that gap. A firefighter carries a sensor payload with thermal cameras and radar through the structure while a command station outside turns the sensor stream into a live position, path, heat view, and structural map. The current demonstration uses one handheld sensor payload and one command station.
Thermal sensing works in smoke, radar contributes room structure, and loop closure corrects accumulated motion error when the firefighter returns to a familiar place. The result is a map that remains useful over a longer search. A smaller configuration can use one thermal camera with radar; another configuration can use radar for both mapping and recognizing a return. Multi-firefighter shared mapping, automated victim detection, and route recommendations remain future work.
Use Case
After incident command establishes the entry plan, one firefighter carries the FireSense sensor payload through rooms, hallways, and stairs while an operator monitors the search from the command station. The current system demonstrates the complete sensing-to-map workflow with a single payload; it does not yet claim collaborative operation across multiple firefighters.
Current Sensor Payload
The current sensor payload is handheld and combines stereo thermal cameras with radar. Thermal imagery supports motion tracking and reveals heat through smoke, while radar measures nearby surfaces such as walls, floors, and openings. A single-thermal-camera-plus-radar configuration is also possible for a smaller form factor.
Edge Processing & Data Link
The sensor payload performs time-critical processing locally and sends a compact keyframe stream to the command station. FireSense uses a 915 MHz Wi-Fi HaLow link for long-range, through-wall communication while reducing the amount of data that must cross the link. The mapping system works from selected observations instead of relying on continuous raw video, helping it tolerate limited or intermittent connectivity.
Command Station & Live SLAM
At the command station, the system decodes the incoming keyframes, estimates motion, recognizes revisited places, and updates the map. The current fall-2026 demonstration operates one payload and shows:
- A live position and the path already searched
- Enhanced thermal imagery for the current view
- A sparse 3D map that is corrected when loop closure recognizes a return
- Radar-derived structure and a map that can be saved after the search
Operational Awareness
The command station gives the incident commander a spatial record of the search: where the firefighter is, where they have been, what the thermal camera sees, and how the surrounding structure is arranged. That can support faster, better-informed coordination than verbal location reports alone. Automated victim detection, egress recommendations, and a shared map for multiple firefighters are longer-term system goals, not capabilities claimed by the current demonstration.
