{"id":28,"date":"2020-09-28T20:01:17","date_gmt":"2020-09-28T20:01:17","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/?page_id=28"},"modified":"2021-12-08T15:50:03","modified_gmt":"2021-12-08T20:50:03","slug":"implementation","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/implementation\/","title":{"rendered":"Implementation"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Full System Development<\/h2>\n\n\n\n<p>All hardware and software development on the system has been completed. We have also tuned the capabilities of Dragoon so that it is able to perform its tasks in post-disaster scenarios, as demonstrated in our Fall Validation Demonstration (FVD). Below is more information regarding the different subsystems implemented on Dragoon.<\/p>\n\n\n\n<div class=\"iframe-container\"><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/LCBIm3sdQv4\" allowfullscreen=\"\" width=\"100%\" height=\"315\" frameborder=\"0\"><\/iframe><\/div>\n\n\n\n<p class=\"has-text-align-center\">Dragoon&#8217;s Full Autonomy Test (10\/27)<\/p>\n\n\n\n<p class=\"has-text-align-center\"><a href=\"https:\/\/drive.google.com\/file\/d\/1Zxf3SAd-vvV8yrWp7G4z9y2sGT1aeHWQ\/view?usp=sharing\">Dragoon&#8217;s Autonomy Test in FVD setup (11\/10)<\/a>  (external video)<\/p>\n\n\n\n<p class=\"has-text-align-center\"><a href=\"https:\/\/drive.google.com\/file\/d\/1xt2HdNHz3WkwOEEgM1EyGYKwVthdZjXR\/view?usp=sharing\">Dragoon&#8217;s 7m Human Detection Test with Custom-trained Networks<\/a>  (external video)<\/p>\n\n\n\n<p class=\"has-text-align-center\"><a href=\"https:\/\/drive.google.com\/file\/d\/12qkWIapSGjiVxcnYyUWW5XA9V9L50vk1\/view?usp=sharing\">Dragoon&#8217;s 3m Human Detection Test with Custom-trained Networks<\/a>  (external video)<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"768\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/IMG_3076-1024x768.jpg\" alt=\"\" class=\"wp-image-543\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/IMG_3076-1024x768.jpg 1024w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/IMG_3076-300x225.jpg 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/IMG_3076-768x576.jpg 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/IMG_3076-1536x1152.jpg 1536w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/IMG_3076-2048x1536.jpg 2048w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Dragoon&#8217;s first outing after completion of hardware development<\/p>\n\n\n\n<p class=\"has-text-align-center\"><a href=\"https:\/\/drive.google.com\/file\/d\/1yBJvyvRQ6H9c8qovuxfhnBYwHQh4TYa9\/view?usp=sharing\">Dragoon&#8217;s first outing<\/a> (external video)<\/p>\n\n\n\n<p class=\"has-text-align-center\"><a href=\"https:\/\/drive.google.com\/file\/d\/1bC43B3CrDMJH0h3fH652fKWhqa_rcKVk\/view?usp=sharing\">Visualization of software stack<\/a> (external video)<\/p>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">Subsystem Development<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Robot Base<\/h3>\n\n\n\n<p>As planned, hardware development was completed during the spring semester. We ordered and received our SuperDroid robot base, which came with a remote controller. We then proceeded to design the layout of the robot and fabricate, either using the 3D printer or the machine shop, the parts needed to put all our parts on the robot. Prior to fabrication, we did load analysis on all our mounts to ensure that they will be sturdy. Alongside the fabrication of parts was the development of the power distribution board of the robot. Each iteration of the board was designed, received from the manufacturer, and then put together using the tools and the reflow oven in the lab.<\/p>\n\n\n\n<p class=\"has-text-align-center\"><a href=\"https:\/\/drive.google.com\/file\/d\/11XX20HwvL5tcdJKlSaEYnn1ZZMJOyot_\/view?usp=sharing\">Simulation of full hardware stack in Rviz<\/a> (external video)<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"675\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.10.55-PM-1-1024x675.png\" alt=\"\" class=\"wp-image-653\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.10.55-PM-1-1024x675.png 1024w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.10.55-PM-1-300x198.png 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.10.55-PM-1-768x506.png 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.10.55-PM-1.png 1050w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Complete hardware stack<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"762\" height=\"564\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.11.08-PM.png\" alt=\"\" class=\"wp-image-680\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.11.08-PM.png 762w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.11.08-PM-300x222.png 300w\" sizes=\"auto, (max-width: 762px) 100vw, 762px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">CAD model of  complete hardware stack<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"584\" height=\"784\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-12.49.37-AM.png\" alt=\"\" class=\"wp-image-550\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-12.49.37-AM.png 584w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-12.49.37-AM-223x300.png 223w\" sizes=\"auto, (max-width: 584px) 100vw, 584px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">RealSense D435i and Seek Thermal sensors on custom-made sensor mount and casing<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"768\" height=\"1024\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/PDB2-2-768x1024.jpeg\" alt=\"\" class=\"wp-image-555\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/PDB2-2-768x1024.jpeg 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/PDB2-2-225x300.jpeg 225w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/PDB2-2-1152x1536.jpeg 1152w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/PDB2-2.jpeg 1200w\" sizes=\"auto, (max-width: 768px) 100vw, 768px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Version 2 of PDB<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"768\" height=\"1024\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/hardware1-768x1024.jpeg\" alt=\"\" class=\"wp-image-556\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/hardware1-768x1024.jpeg 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/hardware1-225x300.jpeg 225w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/hardware1-1152x1536.jpeg 1152w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/hardware1.jpeg 1200w\" sizes=\"auto, (max-width: 768px) 100vw, 768px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Custom-made LiDAR mount, RealSense\/Seek mount, and AGX mount on robot base <\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"736\" height=\"552\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-12.48.17-AM.png\" alt=\"\" class=\"wp-image-549\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-12.48.17-AM.png 736w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-12.48.17-AM-300x225.png 300w\" sizes=\"auto, (max-width: 736px) 100vw, 736px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">3D printing fabrication of battery holder<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"643\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/lidarmount-1024x643.jpeg\" alt=\"\" class=\"wp-image-560\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/lidarmount-1024x643.jpeg 1024w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/lidarmount-300x188.jpeg 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/lidarmount-768x482.jpeg 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/lidarmount.jpeg 1302w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Load analysis of LiDAR mount<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"855\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-12.49.01-AM-1024x855.png\" alt=\"\" class=\"wp-image-548\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-12.49.01-AM-1024x855.png 1024w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-12.49.01-AM-300x250.png 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-12.49.01-AM-768x641.png 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-12.49.01-AM.png 1102w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Version 1 of PDB<\/p>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Sensing<\/h3>\n\n\n\n<p>Our sensing subsystem consists of 3 major sensors: the RealSense RGB sensor, the RealSense depth sensor, and the Seek Thermal sensor. All sensors have been calibrated to get their intrinsic parameters, and the extrinsic parameters between the two RealSense sensors have also been obtained. For SVD, we used the registration between the two RealSense sensors in order to detect humans. For FVD, we registered both the RealSense RGB and the Seek Thermal to the RealSense depth in order to extract the location of the human in the frame. <\/p>\n\n\n\n<p class=\"has-text-align-center\"><a href=\"https:\/\/drive.google.com\/file\/d\/1TQso_bIFQ-b9bhlWj2Q4jz4WXOWcRoS_\/view?usp=sharing\">RealSense RGB and Depth and Seek Thermal run on robot&#8217;s AGX<\/a> (external video)<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"330\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-03-31-at-12.50.32-PM-1024x330.png\" alt=\"\" class=\"wp-image-558\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-03-31-at-12.50.32-PM-1024x330.png 1024w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-03-31-at-12.50.32-PM-300x97.png 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-03-31-at-12.50.32-PM-768x248.png 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-03-31-at-12.50.32-PM.png 1116w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Registration of points between the RealSense RGB sensor and the RealSense depth sensor<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"746\" height=\"564\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-1.27.30-AM.png\" alt=\"\" class=\"wp-image-575\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-1.27.30-AM.png 746w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/Screen-Shot-2021-04-10-at-1.27.30-AM-300x227.png 300w\" sizes=\"auto, (max-width: 746px) 100vw, 746px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Calibration of RealSense<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"320\" height=\"240\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/SeekCalib.jpeg\" alt=\"\" class=\"wp-image-563\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/SeekCalib.jpeg 320w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/04\/SeekCalib-300x225.jpeg 300w\" sizes=\"auto, (max-width: 320px) 100vw, 320px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Result of calibration of Seek Thermal sensor<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Software<\/h3>\n\n\n\n<p>Our software pipeline consists of 3 major components: SLAM, human detection, and autonomy. We are using our Velodyne VLP-16 and the onboard RealSense IMU as inputs to the Cartographer SLAM system, which we have implemented on Dragoon. We are using YOLO to detect humans using both the RealSense RGB and the Seek Thermal; calibration done during the spring semester between the RealSense RGB and the RealSense depth streams allowed us to take the location of the bounding box from the RGB image to the depth image and extract the depth of the human, giving us a 3D pose(s) of the human(s) in the frame. The global poses of humans on the visualizer is then determined by a Kalman filter, which receives information on each human detection seen on each individual frame.<\/p>\n\n\n\n<p>During the fall semester, we completed registration between the Seek Thermal and RealSense, and now feed global poses of humans from both RealSense RGB as well as Seek Thermal into the Kalman filter. Furthermore, we collected custom data and retrained both our RGB and IR YOLO networks to be better able to detect supine humans in dim and occluded conditions. Another major software component we have worked on during the fall semester is Dragoon&#8217;s autonomy platform. Dragoon is now capable of autonomously navigating a space such that it does not require human input in order to find survivors. This was done by implementing both a global planner that determines exploration points in previously unexplored spaces, and a local path following controller that guides Dragoon between each exploration point, and a behavioral executive that determines whether Dragoon should explore more of its current area or proceed to the next unseen frontier. <\/p>\n\n\n\n<p class=\"has-text-align-center\"><a href=\"https:\/\/drive.google.com\/file\/d\/1rxxvvSTffSr3SHKrGsq_iYIJ4kxhGzDx\/view?usp=sharing\">SLAM system working on robot base<\/a> (external video)<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"765\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-12-at-12.56.05-AM-1-1024x765.png\" alt=\"\" class=\"wp-image-682\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-12-at-12.56.05-AM-1-1024x765.png 1024w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-12-at-12.56.05-AM-1-300x224.png 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-12-at-12.56.05-AM-1-768x574.png 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-12-at-12.56.05-AM-1.png 1536w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Overview of Cartographer SLAM system<\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"762\" height=\"562\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/Screen-Shot-2021-12-07-at-5.26.55-PM-1.png\" alt=\"\" class=\"wp-image-775\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/Screen-Shot-2021-12-07-at-5.26.55-PM-1.png 762w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/Screen-Shot-2021-12-07-at-5.26.55-PM-1-300x221.png 300w\" sizes=\"auto, (max-width: 762px) 100vw, 762px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"680\" height=\"510\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/Screen-Shot-2021-12-07-at-5.26.44-PM.png\" alt=\"\" class=\"wp-image-776\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/Screen-Shot-2021-12-07-at-5.26.44-PM.png 680w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/Screen-Shot-2021-12-07-at-5.26.44-PM-300x225.png 300w\" sizes=\"auto, (max-width: 680px) 100vw, 680px\" \/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<p class=\"has-text-align-center\">Custom data collection (on left) and occlusion augmentation for retraining (on right)<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"948\" height=\"750\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/Screen-Shot-2021-12-07-at-5.24.21-PM.png\" alt=\"\" class=\"wp-image-773\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/Screen-Shot-2021-12-07-at-5.24.21-PM.png 948w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/Screen-Shot-2021-12-07-at-5.24.21-PM-300x237.png 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/Screen-Shot-2021-12-07-at-5.24.21-PM-768x608.png 768w\" sizes=\"auto, (max-width: 948px) 100vw, 948px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Diagram of Dragoon&#8217;s Autonomy Behavior<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Vehicle UI<\/h3>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"673\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.29.33-PM-1024x673.png\" alt=\"\" class=\"wp-image-656\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.29.33-PM-1024x673.png 1024w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.29.33-PM-300x197.png 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.29.33-PM-768x505.png 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/05\/Screen-Shot-2021-05-11-at-10.29.33-PM.png 1102w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Vehicle UI architecture <\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"609\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/FULL_GUI-1024x609.jpg\" alt=\"\" class=\"wp-image-785\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/FULL_GUI-1024x609.jpg 1024w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/FULL_GUI-300x178.jpg 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/FULL_GUI-768x457.jpg 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/FULL_GUI-1536x914.jpg 1536w, https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-content\/uploads\/sites\/51\/2021\/12\/FULL_GUI.jpg 1594w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Final visualizer<\/p>\n\n\n\n<p>As seen above, our Vehicle UI displays the Cartographer map (with the location(s) of human(s) drawn on it) as well as data from our sensors through Rviz onto a screen in front of the operator. In the top left corner of the visualizer, the operator can see a watchdog module that indicates the heartbeats of the different sensors. On the top middle part of the screen, the operator is able to see if any humans have been detected and the locations of the humans detected. Red dots on the map indicate &#8220;evidences&#8221; of human detection as seen on each individual RGB frame, while green dots on the map are the positions of the humans as determined by the Kalman filter using the evidences. The operator can also see the path taken by the robot on the screen. Finally, on the right side, the operator can see the current Seek Thermal and RealSense RGB (both with YOLO running on it) streams.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Full System Development All hardware and software development on the system has been completed. We have also tuned the capabilities of Dragoon so that it is able to perform its tasks in post-disaster scenarios, as demonstrated in our Fall Validation Demonstration (FVD). Below is more information regarding the different subsystems implemented on Dragoon. Dragoon&#8217;s Full<br \/><a class=\"moretag\" href=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/implementation\/\">+ Read More<\/a><\/p>\n","protected":false},"author":236,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-28","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/pages\/28","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/users\/236"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/comments?post=28"}],"version-history":[{"count":50,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/pages\/28\/revisions"}],"predecessor-version":[{"id":788,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/pages\/28\/revisions\/788"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamb\/wp-json\/wp\/v2\/media?parent=28"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}