{"id":140,"date":"2026-04-03T17:39:44","date_gmt":"2026-04-03T17:39:44","guid":{"rendered":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamf\/?page_id=140"},"modified":"2026-04-04T04:25:48","modified_gmt":"2026-04-04T04:25:48","slug":"slam","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamf\/system-summary\/slam\/","title":{"rendered":"SLAM"},"content":{"rendered":"\n<p><em>FireSense\u2019s&nbsp;command-side mapping stack&nbsp;fuses&nbsp;thermal keyframes&nbsp;from a visual\u2013inertial front end into an&nbsp;incrementally optimized SLAM solution, using&nbsp;learned global descriptors&nbsp;to propose&nbsp;loop closures&nbsp;and&nbsp;geometric verification&nbsp;to accept or reject them. It supports&nbsp;compressed video&nbsp;on the link,&nbsp;session-based&nbsp;start\/stop with&nbsp;gravity-aligned world initialization, and&nbsp;live visualization&nbsp;of the optimized map and trajectory to aid firefighters in search-relevant indoor scenarios.<\/em><\/p>\n\n\n\n<iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/XzgGy22tvHU\" frameborder=\"0\" allowfullscreen><\/iframe>\n\n\n\n<h3 class=\"wp-block-heading\">Mapping backend (command stack)<\/h3>\n\n\n\n<p>The&nbsp;command computer&nbsp;on FireSense runs the&nbsp;thermal SLAM backend: the part of the system that turns&nbsp;keyframes&nbsp;from the visual\u2013inertial front end into a&nbsp;consistent trajectory and map&nbsp;over time.<\/p>\n\n\n\n<p>Each keyframe bundles what the robot knew at one instant:&nbsp;thermal imagery,&nbsp;3D landmarks&nbsp;in the camera frame, and&nbsp;motion-related pose information&nbsp;so the backend can chain poses and uncertainties. The backend&nbsp;incrementally optimizes&nbsp;a factor graph (using&nbsp;GTSAM \/ iSAM2-style smoothing): it adds&nbsp;odometry-style constraints&nbsp;from the front end and, when appropriate,&nbsp;loop-closure constraints&nbsp;when the robot revisits a place.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Loop closure and verification<\/h3>\n\n\n\n<p>Revisits are found using&nbsp;global image descriptors&nbsp;from a&nbsp;learned visual place-recognition&nbsp;model suited to&nbsp;thermal&nbsp;imagery (high-dimensional, normalized embeddings). Candidate matches are not accepted on appearance alone: a&nbsp;geometric verification&nbsp;stage checks consistency using&nbsp;3D structure&nbsp;and pose information, optionally with&nbsp;dense alignment-style checks, so false positives from repeated texture are suppressed.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Perception pipeline on the command side<\/h3>\n\n\n\n<p>Because bandwidth matters between computers, thermal video may arrive&nbsp;compressed&nbsp;on the command machine. The stack can&nbsp;decode&nbsp;it before descriptor extraction so the same appearance model sees a full image. An optional&nbsp;visualization&nbsp;path turns optimized poses and map updates into&nbsp;point clouds, trajectories, and loop markers&nbsp;for operators and debugging (e.g. via Foxglove-style tools).<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Session lifecycle<\/h3>\n\n\n\n<p>Mapping is treated as a&nbsp;session: the operator starts with the vehicle&nbsp;stationary&nbsp;so a short&nbsp;IMU&nbsp;sample can define&nbsp;gravity and a consistent world frame&nbsp;(Z-up). While mapping, the backend consumes keyframes and publishes&nbsp;refined poses&nbsp;and&nbsp;map updates. Stopping the session can trigger an optional&nbsp;final global refinement&nbsp;before exporting results; the design targets standard trajectory\/map artifacts for evaluation and field use.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Role in FireSense<\/h3>\n\n\n\n<p>FireSense splits work across machines: the&nbsp;payload&nbsp;focuses on&nbsp;sensing and the VIO front end; the&nbsp;command&nbsp;side focuses on&nbsp;heavy optimization, thermal place recognition, and operator-facing mapping outputs. Shared&nbsp;message and service definitions&nbsp;keep both sides aligned so one keyframe stream and one session model drive the full stack.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>FireSense\u2019s&nbsp;command-side mapping stack&nbsp;fuses&nbsp;thermal keyframes&nbsp;from a visual\u2013inertial front end into an&nbsp;incrementally optimized SLAM solution, using&nbsp;learned global descriptors&nbsp;to propose&nbsp;loop closures&nbsp;and&nbsp;geometric verification&nbsp;to accept or reject them. It supports&nbsp;compressed video&nbsp;on the link,&nbsp;session-based&nbsp;start\/stop with&nbsp;gravity-aligned world initialization, and&nbsp;live visualization&nbsp;of the optimized map and trajectory to aid firefighters in search-relevant indoor scenarios. Mapping backend (command stack) The&nbsp;command computer&nbsp;on FireSense runs the&nbsp;thermal SLAM [&hellip;]<\/p>\n","protected":false},"author":458,"featured_media":0,"parent":2,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"page-with-sidebar","meta":{"footnotes":""},"class_list":["post-140","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamf\/wp-json\/wp\/v2\/pages\/140","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamf\/wp-json\/wp\/v2\/users\/458"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamf\/wp-json\/wp\/v2\/comments?post=140"}],"version-history":[{"count":6,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamf\/wp-json\/wp\/v2\/pages\/140\/revisions"}],"predecessor-version":[{"id":218,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamf\/wp-json\/wp\/v2\/pages\/140\/revisions\/218"}],"up":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamf\/wp-json\/wp\/v2\/pages\/2"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamf\/wp-json\/wp\/v2\/media?parent=140"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}