{"id":64,"date":"2019-01-19T06:01:56","date_gmt":"2019-01-19T06:01:56","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2018teama\/?page_id=64"},"modified":"2019-12-14T13:00:21","modified_gmt":"2019-12-14T13:00:21","slug":"test-plan","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teama\/test-plan\/","title":{"rendered":"Test Plan"},"content":{"rendered":"

Spring Test Plan<\/span><\/h4>\n

Objective:<\/span><\/p>\n

1. To demonstrate the Perception and Prediction pipeline of the Collision Avoidance System (CAS).
\n2.\u00a0To demonstrate the data collected from the sensor rig.<\/span><\/p>\n

Equipment:<\/span><\/p>\n

1.\u00a0Computer system with Collision Avoidance System (CAS) framework and CARLA Simulator installed.
\n2.\u00a0Sensor Rig with sensor modules mounted (Point Grey Camera and Ti mmWave RADAR).<\/span>
\n<\/span><\/p>\n

Procedure:<\/span><\/p>\n

Demonstration 1<\/span>
\n1.\u00a0The user starts the simulator and chooses one of the scenarios in which the truck runs and collects sensor data.
\n2.\u00a0The user starts the CAS visualization GUI (Rviz).
\n\"\"\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0Rviz GUI with Perception and Prediction<\/span><\/span><\/p>\n

3.\u00a0The truck traverses on the road and CAS detects and tracks vehicles and road lane markings.
\n4.\u00a0In the visualization GUI, position and velocity of all detected vehicles along with the predicted ego trajectories are shown.
\n\"\"
\n<\/span><\/span><\/p>\n

Demonstration 2<\/span>
\n1.\u00a0The user switches on the power to the sensor rig.
\n2.\u00a0The user then launches Rviz and demonstrates the data collection procedure (Point Grey Camera and Ti mmWave RADAR).
\n3. The team will also demonstrate pre-collected data from outdoor road environment.<\/span><\/span><\/p>\n

Validation:<\/span><\/p>\n

Demonstration 1<\/span>
\n1. Simulator starts and begins to record sensor data. Also, pre-recorded 10 Km sequences are shown<\/span>
\n<\/span>2. Detected vehicles (within 100m) with an mAP-50 of 0.4 are displayed in the GUI<\/span>\u00a0(mAP value is chosen based on the results from the original <\/span>YOLOv3 model and other baseline models mentioned in the paper<\/span><\/a>)
\n<\/span>3. Road lane markings are detected in at least 75% of the frames within a maximum lateral offset of 5% image width (Derived from half the vehicle’s width), for each lane<\/span>
\n<\/span>4. Ego vehicle trajectory is predicted 3 seconds into the future with a RMSE of less than 2 metres w.r.t. the ground truth trajectory<\/span>
\n<\/span>5. Rviz GUI displays 2D visualization of objects, detections, and trajectories at a minimum of 3 FPS<\/span>
\n<\/b>6. The entire perception pipeline runs at a minimum of 10 FPS<\/span><\/span><\/p>\n

Demonstration 2<\/span>
\n1. The collected data from the sensor rig is visualized in Rviz (30 FPS for Point Grey) with a playback drop rate of maximum 10% <\/span><\/p>\n

Fall Test Plan<\/span><\/h4>\n

Objective:<\/span>
\n1. Demonstrate the fully developed prediction<\/span>
\n2. Demonstrate the fully developed Collision Avoidance System on a RC car<\/span>
\n3. Demonstrate the detection of faults and failure in the system<\/span><\/p>\n

Equipment:<\/span><\/p>\n

1.\u00a0Computer system with Collision Avoidance System (CAS) framework and CARLA Simulator installed.
\n2. RC car with Hokuyo LiDAR and Sparkfun IMU and Point Grey cameras mounted at the ceiling of NSH B level.<\/span><\/span><\/p>\n

Procedure<\/span><\/p>\n

Demonstration 1<\/span>
\n1.\u00a0User starts the simulator and chooses one of the 3 scenarios in which the truck runs and collects sensor data along with the ground truth
\n2.\u00a0The user starts the visualization GUI which displays the output of the prediction algorithm for oncoming vehicles 2 seconds into the future
\n3.\u00a0In each scenario, the system predicts possible collisions where a collision is defined by a vehicle overlap of greater than 40%. The system predicts possibility of collisions in at least 6 cases of the 10 possible collision situations
\n4.\u00a0A alarm beeps within 0.2 seconds of collision prediction
\n<\/span><\/span><\/p>\n

Demonstration 2<\/span>
\n1. The operator starts the fully powered RC car placed in the test environment.
\n<\/span>2. The scaled trajectory data is received by the RC car within 1 second of the simulator step\u00a0<\/span>
\n<\/span>3. The RC car starts executing the evasive trajectory within 0.7 seconds of prediction of a collision in the simulator\u00a0<\/span>
\n<\/span>4. The maneuver is executed with the help of closed-loop controls strategy and the RC car localizes itself within<\/span> and tracks the desired trajectory\u00a0<\/span><\/span><\/p>\n

\"\" \"\" \"\" \"\"<\/span><\/p>\n

Monocular Camera at ceiling\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 Scenario at t = t\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0Scenario at t = t +\u00a0\u0394t1\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0\u00a0Scenario at t = t +\u00a0\u0394t2<\/span><\/p>\n

Demonstration 3<\/span>
\n1. User starts the simulator and chooses one of the 3 scenarios in which the truck runs and collects sensor data along with the ground truth.
\n<\/span>2. The operator interrupts the CAS software (signifying system software failure) randomly as the user wishes, thereby introducing a system failure <\/span>(OR)<\/span><\/i> The operator interrupts the simulator software (signifying sensor\/hardware failure) randomly as the user wishes, thereby introducing a sensor failure.
\n<\/span>3. The system fault detection software detects either of them and raises a notification\u00a0<\/span><\/span><\/p>\n

Validation:<\/span><\/p>\n

Demonstration 1<\/span>
\n1. System predicts oncoming vehicle trajectory <\/span>2 sec into the future with an of RMSE of less than 3m
\n2.\u00a0<\/span>Predict the possibility\u00a0<\/span>of collision with 60% accuracy
\n3.\u00a0Send alert signal to driver within 0.2s of collision detection<\/span><\/span><\/p>\n

Demonstration 2<\/span><\/p>\n

1. RC car tracks the scaled trajectory within <\/span>1s of simulation step<\/span>.
\n<\/b>2.\u00a0The RC car starts executing the maneuver <\/span>within 1.2s of prediction<\/span><\/span>
\n3.\u00a0RC car tracks the desired \u00a0trajectory <\/span>within \u00a0\u00b15 cm (X, Y) and<\/span> \u00b110 deg (Yaw)<\/span><\/p>\n

Demonstration 3<\/span>
\n1.\u00a0Warn driver in case\u00a0<\/span><\/span>of system failure<\/span> <\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Spring Test Plan Objective: 1. To demonstrate the Perception and Prediction pipeline of the Collision Avoidance System (CAS). 2.\u00a0To demonstrate the data […]<\/p>\n","protected":false},"author":178,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-64","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teama\/wp-json\/wp\/v2\/pages\/64","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teama\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teama\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teama\/wp-json\/wp\/v2\/users\/178"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teama\/wp-json\/wp\/v2\/comments?post=64"}],"version-history":[{"count":24,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teama\/wp-json\/wp\/v2\/pages\/64\/revisions"}],"predecessor-version":[{"id":825,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teama\/wp-json\/wp\/v2\/pages\/64\/revisions\/825"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teama\/wp-json\/wp\/v2\/media?parent=64"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}