{"id":67,"date":"2020-10-20T14:43:03","date_gmt":"2020-10-20T14:43:03","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/?page_id=67"},"modified":"2021-10-04T19:19:22","modified_gmt":"2021-10-04T19:19:22","slug":"test-plan","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/project-management\/test-plan\/","title":{"rendered":"Test Plan"},"content":{"rendered":"\n<h3 class=\"wp-block-heading\">Fall 2021 (Up-to-date, Oct 4th)<\/h3>\n\n\n\n<p>Our test plan until FVD can be found <a href=\"https:\/\/drive.google.com\/file\/d\/1Ng9KN3NSSPjxnpcxN_KFpAinN9fy5S8S\/view?usp=sharing\" data-type=\"URL\" data-id=\"https:\/\/drive.google.com\/file\/d\/1Ng9KN3NSSPjxnpcxN_KFpAinN9fy5S8S\/view?usp=sharing\">here<\/a> and 1-pager for FVD <a href=\"https:\/\/drive.google.com\/file\/d\/1kB1U6_Rwf8-93c_pPkLBXTomoqVmN45h\/view?usp=sharing\" data-type=\"URL\" data-id=\"https:\/\/drive.google.com\/file\/d\/1kB1U6_Rwf8-93c_pPkLBXTomoqVmN45h\/view?usp=sharing\">here<\/a><\/p>\n\n\n\n<p><\/p>\n\n\n\n<hr class=\"wp-block-separator is-style-default\" \/>\n\n\n\n<p>1-pager for SVD can be found <a href=\"https:\/\/drive.google.com\/file\/d\/1TclVWGvL2NmGR6ctAUaF7hAy60ycIUXA\/view?usp=sharing\" data-type=\"URL\" data-id=\"https:\/\/drive.google.com\/file\/d\/1TclVWGvL2NmGR6ctAUaF7hAy60ycIUXA\/view?usp=sharing\">here<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Spring 2021 (Updated)<\/h3>\n\n\n\n<div align=\"center\"><figure class=\"aligncenter size-medium\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/53\/2021\/04\/springvrs2.png\" alt=\"\" class=\"wp-image-466 img-responsive\" width=\"800\" height=\"640\"><\/figure><\/div>\n\n\n\n<h3 class=\"wp-block-heading\">Fall 2021 (Updated)<\/h3>\n\n\n\n<div align=\"center\"><figure class=\"aligncenter size-medium\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/53\/2021\/05\/FVD_VRS.png\" alt=\"\" class=\"wp-image-466 img-responsive\" width=\"800\" height=\"640\"><\/figure><\/div>\n\n\n\n<div align=\"center\"><figure class=\"aligncenter size-medium\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/53\/2021\/05\/FVD_RLS.png\" alt=\"\" class=\"wp-image-466 img-responsive\" width=\"800\" height=\"640\"><\/figure><\/div>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h3 class=\"wp-block-heading\">Spring 2021<\/h3>\n\n\n\n<p><\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Virtual System Validation:<\/h4>\n\n\n\n<h5 class=\"wp-block-heading\">A. Demo Conditions:<\/h5>\n\n\n\n<ul class=\"wp-block-list\"><li>Location: In simulation<\/li><li>Equipment: A computer system or cloud server with Gazebo, ROS and COLA predictive avoidance packages installed<\/li><li>Operating area: Simulated 92*64 m factory environment (virtual)<\/li><\/ul>\n\n\n\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<div align=\"center\"><figure class=\"aligncenter size-medium\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-content\/uploads\/sites\/53\/2021\/03\/Floor-Plan.png\" alt=\"\" class=\"wp-image-466 img-responsive\" width=\"600\" height=\"480\"><\/figure><\/div>\n\n\n\n<p class=\"has-text-align-center\">Figure 1: Floor plan for the virtual factory environment<\/p>\n<\/div><\/div>\n\n\n\n<p class=\"has-text-align-center\"><\/p>\n\n\n\n<h5 class=\"wp-block-heading\">B. Procedure:<\/h5>\n\n\n\n<ol class=\"wp-block-list\"><li>The user starts ROS core and simulation<\/li><li>Simulator loads the floor plan for the factory, and launches the visualization tool.<\/li><li>Simulator launches pedestrians and forklifts, following their fixed paths and routines<\/li><li>Simulator launches one robot, and publish waypoints and the static map to robot<\/li><li>Robots plan global paths and start moving<\/li><li>Robots continuously receive localization and noisy obstacle observation from simulator<\/li><li>Robots avoid obstacles according to its predicted trajectories of obstacles<\/li><li>System stops after a certain duration of time<\/li><\/ol>\n\n\n\n<p><\/p>\n\n\n\n<h5 class=\"wp-block-heading\">C. Objective &amp; Requirements to Demo:<\/h5>\n\n\n\n<ol class=\"wp-block-list\"><li>To demonstrate the simulation system and simulated environment with obstacles<\/li><li>To demonstrate the robot\u2019s ability to plan paths and follow them<\/li><li>To demonstrate the predictive avoidance algorithm (showing robots dodging obstacles using predicted trajectories)<\/li><li>To validate M.P.1-3: Robot speed will be above 0.5m\/s and below 1.8m\/s if no obstacle is detected<\/li><li>To validate M.P.5: Robots should receive localization and observation in at least 10 Hz, and send back control signals to the simulator in at least 10 Hz<\/li><\/ol>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Fall 2021<\/h3>\n\n\n\n<p><\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Real-Life System Validation:<\/h4>\n\n\n\n<h5 class=\"wp-block-heading\">A. Demo Conditions:<\/h5>\n\n\n\n<ul class=\"wp-block-list\"><li>Location: Any location with flat ground, sufficient lighting and high enough ceiling (over 2.5 m).<\/li><li>Equipment: TurtleBot, obstacle action figures\/toys, overhead camera, lights<\/li><li>Operating area: 2*2 m flat ground<\/li><\/ul>\n\n\n\n<p><\/p>\n\n\n\n<h5 class=\"wp-block-heading\">B. Procedure:<\/h5>\n\n\n\n<ol class=\"wp-block-list\"><li>Setup overhead camera and place markers on top of robot\/obstacles for ground truth gathering<\/li><li>Place robot and one or more obstacles in testing ground<\/li><li>Manually drive robot and\/or obstacles<\/li><li>Obtain robot estimation of obstacle positions and classifications for demo duration<\/li><li>Obtain ground truth position data of robot and all obstacles from overhead camera readings for demo duration<\/li><li>Use measurement and ground truth data for visualization\/plotting<\/li><li>Evaluate detection (position) and classification accuracy<\/li><\/ol>\n\n\n\n<p><\/p>\n\n\n\n<h5 class=\"wp-block-heading\">C. Objective &amp; Requirements to Demo:<\/h5>\n\n\n\n<ol class=\"wp-block-list\"><li>To Validate M.P.6: Classify obstacles of interest with mAP of at least 60%<\/li><li>To Validate M.P.7: Detect positions of obstacles of interest within 0.1 m accuracy<\/li><li>To Validate M.P.8: Detect obstacles of interest within a range of 3 m<\/li><li>To Validate: M.P.9 Output results of positioning and classification within 100 ms per frame<\/li><\/ol>\n\n\n\n<p><\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Virtual System Validation:<\/h4>\n\n\n\n<h5 class=\"wp-block-heading\">A. Demo Conditions:<\/h5>\n\n\n\n<ul class=\"wp-block-list\"><li>Location: In simulation<\/li><li>Equipment: A computer system or cloud server with Gazebo, ROS and COLA predictive avoidance packages installed<\/li><li>Operating area: Simulated 92*64 m factory environment (virtual)<\/li><\/ul>\n\n\n\n<p><\/p>\n\n\n\n<h5 class=\"wp-block-heading\">B. Procedure:<\/h5>\n\n\n\n<ol class=\"wp-block-list\"><li>The user starts ROS core and simulation<\/li><li>Simulator loads the floor plan for the factory, and launches the visualization tool.<\/li><li>Simulator launches pedestrians and forklifts, following their fixed paths and routines<\/li><li>Simulator launches robot fleet, and publish waypoints and the static map to robots<\/li><li>Robots plan global paths and start moving<\/li><li>Robots continuously receive localization and noisy obstacle observation from simulator<\/li><li>Robots avoid obstacles using all three avoidance algorithms<\/li><li>System stops after a certain duration of time and calculates resulting productivity<\/li><li>System repeats the previous steps in naive avoidance mode without any classification and prediction, and computes nominal productivity<\/li><\/ol>\n\n\n\n<p><\/p>\n\n\n\n<h5 class=\"wp-block-heading\">C. Objective &amp; Requirements to Demo:<\/h5>\n\n\n\n<ol class=\"wp-block-list\"><li>To demonstrate all three avoidance algorithms (predictive, conservative, reciprocal)<\/li><li>To validate M.P.4: Robot fleet should have productivity increased by &gt;5% when using classification-based predictive\/reciprocal avoidance compared to nominal productivity using only naive avoidance<\/li><li>To validate M.P.5: Robots should receive localization and observation in at least 10 Hz, and send back control signals to the simulator in at least 10 Hz<\/li><\/ol>\n","protected":false},"excerpt":{"rendered":"<p>Fall 2021 (Up-to-date, Oct 4th) Our test plan until FVD can be found here and 1-pager for FVD here 1-pager for SVD can be found here Spring 2021 (Updated) Fall 2021 (Updated) Spring 2021 Virtual System Validation: A. Demo Conditions: Location: In simulation Equipment: A computer system or cloud server with Gazebo, ROS and COLA &#8230; <a href=\"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/project-management\/test-plan\/\" class=\"more-link text-uppercase small\"><strong>Continue Reading<\/strong> <i class=\"fa fa-angle-double-right\" aria-hidden=\"true\"><\/i><\/a><\/p>\n","protected":false},"author":242,"featured_media":0,"parent":44,"menu_order":2,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-67","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages\/67","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/users\/242"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/comments?post=67"}],"version-history":[{"count":27,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages\/67\/revisions"}],"predecessor-version":[{"id":638,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages\/67\/revisions\/638"}],"up":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/pages\/44"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2021teamd\/wp-json\/wp\/v2\/media?parent=67"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}