{"id":171,"date":"2020-02-19T21:15:10","date_gmt":"2020-02-19T21:15:10","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/?p=171"},"modified":"2020-10-10T02:09:21","modified_gmt":"2020-10-10T02:09:21","slug":"test-plan","status":"publish","type":"post","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/2020\/02\/19\/test-plan\/","title":{"rendered":"Test Plan"},"content":{"rendered":"<h1><span style=\"color: #000000\">Spring Validation Demo<\/span><\/h1>\n<h1><span style=\"font-weight: 400;color: #000000\">Spring Validation Demo-1<\/span><\/h1>\n<h2><span style=\"font-weight: 400;color: #000000\">Objective<\/span><\/h2>\n<p><span style=\"font-weight: 400;color: #000000\">To show that the system can perform successful lane changes in straight roads.&nbsp;<\/span><\/p>\n<h2><span style=\"font-weight: 400;color: #000000\">Validation Criteria<\/span><\/h2>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">The vehicle\u2019s speed remains in the range of 10-20mph throughout the whole manoeuvre. (PFR.3, FR.3)<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">The system can perform collision-free lane changes 80% of the time. (PFR.4, FR.4)<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">The vehicle will remain at least 0.3m away from lane markings and other actors at all times. (PFR.1, PFR.2, FR.2)<\/span><\/li>\n<\/ul>\n<h2><span style=\"font-weight: 400;color: #000000\">Experiment Procedure<\/span><\/h2>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">A Carla scenario with a straight road containing several lanes is created.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">Several actors, moving in a straight line on different lanes and at varying speeds in the range of 10-20 mph, are spawned.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">An ego vehicle in the adjacent lane is spawned which is moving at 20mph.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">The ego vehicle shall perform a lane change when a proper opportunity presents.<\/span><\/li>\n<\/ul>\n<h1><span style=\"font-weight: 400;color: #000000\">Spring Validation Demo-2<\/span><\/h1>\n<h2><span style=\"font-weight: 400;color: #000000\">Objective<\/span><\/h2>\n<p><span style=\"font-weight: 400;color: #000000\">To show that the system can detect and stop when pedestrians are encountered<\/span><\/p>\n<h2><span style=\"font-weight: 400;color: #000000\">Validation Criteria<\/span><\/h2>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">Ego vehicle will stop at least 0.3 metres away from the pedestrian if the pedestrian is at least 14 metres away. (PFR.2, FR.2, FR.5)<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">The ego vehicle can detect pedestrians and safely come to a halt 80% of the time. (PFR.5, FR.5)<\/span><\/li>\n<\/ul>\n<h2><span style=\"font-weight: 400;color: #000000\">Experiment Procedure<\/span><\/h2>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">A Carla scenario of a straight lane is created and a pedestrian is spawned at a random position, at least 14 metres ahead of the ego vehicle.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">An ego vehicle with an initial speed of 20mph is spawned.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400;color: #000000\">The ego vehicle detects the pedestrian and decelerates to a stop.<\/span><\/li>\n<\/ul>\n<h1><span style=\"color: #000000\">Fall Validation Demo<\/span><\/h1>\n<h2 id=\"tablepress-4-name\" class=\"tablepress-table-name tablepress-table-name-id-4\">Fall Validation Demo<\/h2>\n\n<table id=\"tablepress-4\" class=\"tablepress tablepress-id-4\" aria-labelledby=\"tablepress-4-name\">\n<tbody class=\"row-striping row-hover\">\n<tr class=\"row-1\">\n\t<td class=\"column-1\">Objective<\/td><td class=\"column-2\">This test is designed to verify the full system performance ensuring the accumulation of errors from each sub-system satisfies the full system requirement. Additionally, it tests for the proper integration between all the sub-systems.<\/td>\n<\/tr>\n<tr class=\"row-2\">\n\t<td class=\"column-1\">Personnel<\/td><td class=\"column-2\">Full Team<\/td>\n<\/tr>\n<tr class=\"row-3\">\n\t<td class=\"column-1\">Equipment<\/td><td class=\"column-2\">A Computer with<br \/>\n1. Appropriate version of CARLA installed.<br \/>\n2. CARLA testing maps stored.<br \/>\n3. ROS installed.<br \/>\nA mouse with a scroll. (For visualization view point movements)<\/td>\n<\/tr>\n<tr class=\"row-4\">\n\t<td class=\"column-1\">Location<\/td><td class=\"column-2\">Remote<\/td>\n<\/tr>\n<tr class=\"row-5\">\n\t<td class=\"column-1\">Elements<\/td><td class=\"column-2\">Utility Classes, Simulation, Trajectory Generator, Reinforcement Learning Agents, Neural Network Selector, State Manager<\/td>\n<\/tr>\n<tr class=\"row-6\">\n\t<td class=\"column-1\">Procedure<\/td><td class=\"column-2\">This test needs to be conducted for different combination of scenarios, the procedure for testing each combination is as follows:<br \/>\n1. Start the CARLA Simulator.<br \/>\n2. Launch the Path Planner Node and CARLA Interface Node.<br \/>\n3. Verify the connection between all ROS nodes and CARLA simulator.<br \/>\n4. Initialize the test case with a pre-defined parameter file that specifies: map name, starting pose, destination pose, initial state of other actors, and max allowed time.<br \/>\n5. The controlled vehicle is expected to travel safely from the starting pose to the destination pose within the max allowed time.<br \/>\n<br \/>\nThe procedure above will be repeated for the following cases:<br \/>\n1. A case that involves at least 1 lane change, 1 left-turn, and 1 straight intersection.<br \/>\n2. A case that involves at least 1 lane change and 1 right-turn.<br \/>\n3. A case that involves pedestrian jay-walking.<br \/>\n4. A case that involves at least 1 left-turn that has pedestrians crossing the road.<br \/>\n5. A case that involves at least 1 right-turn that has pedestrians crossing the road.<\/td>\n<\/tr>\n<tr class=\"row-7\">\n\t<td class=\"column-1\">Verification Criteria<\/td><td class=\"column-2\">We require an overall success rate of at least 70%, and a successful result is evaluated by all of the following criteria:<br \/>\n1.The vehicle stops within 2 meters of the goal pose and signals a job done.<br \/>\n2.Goal pose is reached within the max allowed time.<br \/>\n3.The vehicle does not collide with any other object. (including pedestrians, other vehicles, and any static object)<br \/>\n4.The vehicle only takes a left turn in the left-most lane.<br \/>\n5.The vehicle only arrives at the left-most lane after a left turn.<br \/>\n6.The vehicle only takes a right turn in the right-most lane.<br \/>\n7.The vehicle only arrives at the right-most lane after a right turn.<br \/>\n8.While proceeding straight in an intersection, the vehicle only crosses the stopline in green light, yellow light, or within the first 0.3 seconds of the red light.<br \/>\n9.While taking left-turn in an intersection, the vehicle only crosses the stopline in green light, yellow light, or within the first 0.3 seconds of the red light.<br \/>\n10.The vehicle does not exceed the speed limit by more than 10%.<br \/>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<!-- #tablepress-4 from cache -->\n\n\n<p><span class=\"has-inline-color has-black-color\">For a detailed test plan please checkout <a href=\"https:\/\/drive.google.com\/file\/d\/1gnjYJnw9DVU00aIDvyqfF-b-iwYHZAkq\/view?usp=sharing\">this report.<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Spring Validation Demo Spring Validation Demo-1 Objective To show that the system can perform successful lane changes in straight roads.&nbsp; Validation Criteria The vehicle\u2019s speed remains in the range of 10-20mph throughout the whole manoeuvre. (PFR.3, FR.3) The system can perform collision-free lane changes 80% of the time. (PFR.4, FR.4) The vehicle will remain at<br \/><a class=\"moretag\" href=\"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/2020\/02\/19\/test-plan\/\">+ Read More<\/a><\/p>\n","protected":false},"author":196,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-171","post","type-post","status-publish","format-standard","hentry","category-project-management"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/wp-json\/wp\/v2\/posts\/171","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/wp-json\/wp\/v2\/users\/196"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/wp-json\/wp\/v2\/comments?post=171"}],"version-history":[{"count":16,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/wp-json\/wp\/v2\/posts\/171\/revisions"}],"predecessor-version":[{"id":583,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/wp-json\/wp\/v2\/posts\/171\/revisions\/583"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/wp-json\/wp\/v2\/media?parent=171"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/wp-json\/wp\/v2\/categories?post=171"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2020teamd\/wp-json\/wp\/v2\/tags?post=171"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}