{"id":16,"date":"2022-11-13T01:42:55","date_gmt":"2022-11-13T01:42:55","guid":{"rendered":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/?page_id=16"},"modified":"2023-05-07T03:51:56","modified_gmt":"2023-05-07T03:51:56","slug":"system-design","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/system-design\/","title":{"rendered":"System Design"},"content":{"rendered":"\n<p>The system-level requirements of DarkBot are derived from the use case and objective tree. The requirements include functional, performance, and nonfunctional ones, and each of them is divided into mandatory and desirable requirements. By the end of the fall semester, we would achieve all mandatory requirements. The desirable requirements are initially out of scope, but they will significantly enhance the performance of the system. The team will try its best to achieve them as much as possible.<\/p>\n\n\n\n<p>All requirements have been discussed and reviewed by stakeholders to make sure that they are realistic and doable for the project. Note that those requirements are not finalized and will be changed according to future research and tests.<\/p>\n\n\n\n<p class=\"has-text-align-center\" style=\"font-size:30px\"><strong>System Requirements<\/strong><\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<p class=\"has-text-align-center\">Mandatory Functional Requirements<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"786\" height=\"326\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150017-1.jpg\" alt=\"\" class=\"wp-image-234\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150017-1.jpg 786w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150017-1-300x124.jpg 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150017-1-768x319.jpg 768w\" sizes=\"auto, (max-width: 786px) 100vw, 786px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<p class=\"has-text-align-center\">Desirable functional requirements<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"786\" height=\"257\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150544.jpg\" alt=\"\" class=\"wp-image-235\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150544.jpg 786w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150544-300x98.jpg 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150544-768x251.jpg 768w\" sizes=\"auto, (max-width: 786px) 100vw, 786px\" \/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<p class=\"has-text-align-center\" style=\"font-size:30px\"><strong>Performance Requirements<\/strong><\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<p class=\"has-text-align-center\">Mandatory Performance Requirements<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"792\" height=\"446\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150856.jpg\" alt=\"\" class=\"wp-image-236\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150856.jpg 792w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150856-300x169.jpg 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150856-768x432.jpg 768w\" sizes=\"auto, (max-width: 792px) 100vw, 792px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<p class=\"has-text-align-center\">Desirable Performance Requirements<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"793\" height=\"443\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150914.jpg\" alt=\"\" class=\"wp-image-237\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150914.jpg 793w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150914-300x168.jpg 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-150914-768x429.jpg 768w\" sizes=\"auto, (max-width: 793px) 100vw, 793px\" \/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<p class=\"has-text-align-center\" style=\"font-size:30px\"><strong>Non-functional Requirements<\/strong><\/p>\n\n\n\n<p><\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<p class=\"has-text-align-center\">Mandatory Nonfunctional Requirements<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"785\" height=\"408\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-151339.jpg\" alt=\"\" class=\"wp-image-239\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-151339.jpg 785w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-151339-300x156.jpg 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-151339-768x399.jpg 768w\" sizes=\"auto, (max-width: 785px) 100vw, 785px\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<p class=\"has-text-align-center\">Desirable Nonfunctional Requirements<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"786\" height=\"322\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-151354.jpg\" alt=\"\" class=\"wp-image-240\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-151354.jpg 786w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-151354-300x123.jpg 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Screenshot-2023-02-21-151354-768x315.jpg 768w\" sizes=\"auto, (max-width: 786px) 100vw, 786px\" \/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<p class=\"has-text-align-center\" style=\"font-size:30px\"><strong>Functional Architecture<\/strong><\/p>\n\n\n\n<p>Autonomous exploration and human localization are the system&#8217;s core high functions to solve our use case. To accomplish such functions, we compiled the functional architecture from function requirements accordingly. The functional architecture diagram is shown in Figure 1. From a black-box perspective, the system takes in the environment information, operator inputs, and electricity, and outputs robot terrain traversal, a 3D map with robot and detected human poses, etc. This architecture can be broken down into three parallel pipelines: perception-based planning (top), SLAM (middle), and human detection and localization (bottom).<\/p>\n\n\n\n<p>The perception-based planning pipeline (top) first performs a terrain analysis about the surrounding environment of the robot to generate a 2.5D elevation map. This map is used as a prior to a more efficient motion planning that executes the local trajectories and global exploration route generated by the Navigation module, given the 3D map and robot pose from the second pipeline. Second, the SLAM pipeline (middle) continuously builds a 3D environment map with a robot pose estimation from collected sensor data. The map and robot pose serves as a foundation for explorations (top) and human localization (bottom). Third, the human detection and localization pipeline uses color-and-depth image stream data to detect humans in robot&#8217;s field of view and localize them in the global 3D map. This end-user will see human location pins on the 3D map, which is achieved with robot pose, human pose estimation relative to the robot, and the visualized 3D map. Last but not least, an operator would have overriding teleoperation access to intervene and control the robot during any circumstances.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"412\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/functional_architecture-version1203.drawio-1024x412.png\" alt=\"\" class=\"wp-image-243\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/functional_architecture-version1203.drawio-1024x412.png 1024w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/functional_architecture-version1203.drawio-300x121.png 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/functional_architecture-version1203.drawio-768x309.png 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/functional_architecture-version1203.drawio-1536x619.png 1536w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/functional_architecture-version1203.drawio.png 1641w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Figure 1. Functional Architecture<\/figcaption><\/figure>\n<\/div>\n\n\n<p><\/p>\n\n\n\n<p class=\"has-text-align-center\" style=\"font-size:30px\"><strong>Cyberphysical Architecture<\/strong><\/p>\n\n\n\n<p>Building upon the conclusions in our trade studies, the cyber-physical architecture as shown in Figure 2 realizes our requirements by combining the functional architecture with the Unitree Go1\u2019s hardware architecture and the sensor payload. The main components of the system are divided into the eight parts: Safety, Simultaneous Localization and Mapping, State Estimator, Human Detection, Video Feedback, Power, Plan and Traverse, Information Process, and User Command.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"692\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Cyberphysical_architecture-version1212.drawio-1024x692.png\" alt=\"\" class=\"wp-image-244\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Cyberphysical_architecture-version1212.drawio-1024x692.png 1024w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Cyberphysical_architecture-version1212.drawio-300x203.png 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Cyberphysical_architecture-version1212.drawio-768x519.png 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Cyberphysical_architecture-version1212.drawio-1536x1038.png 1536w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Cyberphysical_architecture-version1212.drawio-400x270.png 400w, https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-content\/uploads\/sites\/66\/2023\/02\/Cyberphysical_architecture-version1212.drawio.png 1691w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Figure 2: Cyberphysical Architecture<\/figcaption><\/figure>\n<\/div>\n\n\n<p class=\"has-text-align-center\" style=\"font-size:30px\"><strong>System Design<\/strong><\/p>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">Elevation Mapping and State Estimation Subsystem<\/h3>\n\n\n\n<p>Following the cyber-physical architecture, the simultaneous localization and mapping (SLAM) module should take 3D LiDAR scans, IMU reading, and Stereo camera data as input and maintain a 3D world map with estimated robot pose. It serves as the foundation for search and rescue exploration through narrow spaces and upon terrains with elevation changes. The SLAM architecture should be robust and reliable as the map grows larger, as well as consistent on runtime and memory consumption to guarantee real-timeness. Further based on the trade study, our SLAM architecture will likely follow the standard formulation proposed in LIO-SAM, which stands for Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping.&nbsp;<\/p>\n\n\n\n<p>LIO-SAM formulates lidar-inertial odometry on top of a factor graph, enabling both relative and absolute measurements, to perform pose graph joint optimizations and loop closures. The tightly coupled IMU provides nonlinear motion pre-integration to de-skew point clouds, initialization of odometry optimization, and most importantly contributes to part of the factor graph for bundle adjustments. The result of lidar odometry optimization can conversely be utilized to estimate the bias of IMU in the factor graph. To ensure the real-timeliness without sacrificing accuracies, LIO-SAM uses edge and plane features as descriptors proposed by LOAM for 3D lidar scans and achieves critical efficiency in the following:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Computing Lidar Odometry Factor:<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Maintain a fixed-lag smoother (i.e. sliding window) consisting of sub-keyframes for voxel map in the world frame to prepare for scan matching<\/li>\n\n\n\n<li>Fast data association during local scan-matching whose motion prior is initialized from IMU.<\/li>\n\n\n\n<li>Non-linear least square optimization (GaussNewton) to compute lidar odometry transformation factors<\/li>\n\n\n\n<li>Keyframes registrations<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Localization:<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Marginalize old lidar scans for pose and trajectory optimization<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Computing Loop Closure Factor:<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Place recognition based on baseline metric\u2014Euclidean distance-based detection<\/li>\n\n\n\n<li>Place recognition mechanism based on more advanced point cloud descriptors<\/li>\n\n\n\n<li>Fast data association when matching new key-frame to corresponding past sub-keyframes<\/li>\n\n\n\n<li>Correcting drift in translation and most importantly altitude<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading has-medium-font-size\">Plan and Traverse Subsystem<\/h3>\n\n\n\n<p>Building on top of the Elevation Mapping and State Estimation subsystem, the goal of the Plan and Traverse subsystem is to efficiently explore the environment. The exploration strategy can be information-theoretic, frontier-based, topological, random sampling-based, or a hierarchy of these methods.&nbsp;<\/p>\n\n\n\n<p>The TARE exploration strategy is a candidate approach that performs frontier exploration with a minimum set of viewpoints by recursively and randomly sampling the viewpoints. Using the estimated state of the robot and the elevation map (from the previous subsystem), we obtain the traversable configuration space within the robot\u2019s local planning horizon, current viewpoint location, and boundary viewpoints. These are provided as inputs to the planner, which recursively computes viewpoints, concatenates local paths between them, and outputs the exploration trajectory plan. This trajectory plan is provided as input to the traversal APIs of the Unitree quadruped robot.<\/p>\n\n\n\n<p>For a legged robot, the best local path should be determined by the highest foothold score. The foothold score is a function of the amount of time taken to traverse the path and the robot\u2019s ability to find stable footholds in the path. We will use this as the local cost function for the TARE exploration planner.<\/p>\n\n\n\n<h3 class=\"wp-block-heading has-text-align-left has-medium-font-size\">Human Detection Subsystem<\/h3>\n\n\n\n<p>The human detection subsystem makes use of the RGB and depth camera. YOLOv4 processes the RGB image, and marks humans on the image. Human locations on RGB images will be used to locate humans on the depth image. Last, humans on the depth image will be mapped to the 2d world map as points of rescue interest.&nbsp;<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The system-level requirements of DarkBot are derived from the use case and objective tree. The requirements include functional, performance, and nonfunctional ones, and each of them is divided into mandatory and desirable requirements. By the end of the fall semester, we would achieve all mandatory requirements. The desirable requirements are initially out of scope, but<br \/><a class=\"moretag\" href=\"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/system-design\/\">+ Read More<\/a><\/p>\n","protected":false},"author":305,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-16","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-json\/wp\/v2\/pages\/16","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-json\/wp\/v2\/users\/305"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-json\/wp\/v2\/comments?post=16"}],"version-history":[{"count":15,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-json\/wp\/v2\/pages\/16\/revisions"}],"predecessor-version":[{"id":439,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-json\/wp\/v2\/pages\/16\/revisions\/439"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teamb\/wp-json\/wp\/v2\/media?parent=16"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}