Spring Test Plan
Objective:
1. To demonstrate the Perception and Prediction pipeline of the Collision Avoidance System (CAS).
2. To demonstrate the data collected from the sensor rig.
Equipment:
1. Computer system with Collision Avoidance System (CAS) framework and CARLA Simulator installed.
2. Sensor Rig with sensor modules mounted (Point Grey Camera and Ti mmWave RADAR).
Procedure:
Demonstration 1
1. The user starts the simulator and chooses one of the scenarios in which the truck runs and collects sensor data.
2. The user starts the CAS visualization GUI (Rviz).
Rviz GUI with Perception and Prediction
3. The truck traverses on the road and CAS detects and tracks vehicles and road lane markings.
4. In the visualization GUI, position and velocity of all detected vehicles along with the predicted ego trajectories are shown.
Demonstration 2
1. The user switches on the power to the sensor rig.
2. The user then launches Rviz and demonstrates the data collection procedure (Point Grey Camera and Ti mmWave RADAR).
3. The team will also demonstrate pre-collected data from outdoor road environment.
Validation:
Demonstration 1
1. Simulator starts and begins to record sensor data. Also, pre-recorded 10 Km sequences are shown
2. Detected vehicles (within 100m) with an mAP-50 of 0.4 are displayed in the GUI (mAP value is chosen based on the results from the original YOLOv3 model and other baseline models mentioned in the paper)
3. Road lane markings are detected in at least 75% of the frames within a maximum lateral offset of 5% image width (Derived from half the vehicle’s width), for each lane
4. Ego vehicle trajectory is predicted 3 seconds into the future with a RMSE of less than 2 metres w.r.t. the ground truth trajectory
5. Rviz GUI displays 2D visualization of objects, detections, and trajectories at a minimum of 3 FPS
6. The entire perception pipeline runs at a minimum of 10 FPS
Demonstration 2
1. The collected data from the sensor rig is visualized in Rviz (30 FPS for Point Grey) with a playback drop rate of maximum 10%
Fall Test Plan
Objective:
1. Demonstrate the fully developed prediction
2. Demonstrate the fully developed Collision Avoidance System on a RC car
3. Demonstrate the detection of faults and failure in the system
Equipment:
1. Computer system with Collision Avoidance System (CAS) framework and CARLA Simulator installed.
2. RC car with Hokuyo LiDAR and Sparkfun IMU and Point Grey cameras mounted at the ceiling of NSH B level.
Procedure
Demonstration 1
1. User starts the simulator and chooses one of the 3 scenarios in which the truck runs and collects sensor data along with the ground truth
2. The user starts the visualization GUI which displays the output of the prediction algorithm for oncoming vehicles 2 seconds into the future
3. In each scenario, the system predicts possible collisions where a collision is defined by a vehicle overlap of greater than 40%. The system predicts possibility of collisions in at least 6 cases of the 10 possible collision situations
4. A alarm beeps within 0.2 seconds of collision prediction
Demonstration 2
1. The operator starts the fully powered RC car placed in the test environment.
2. The scaled trajectory data is received by the RC car within 1 second of the simulator step
3. The RC car starts executing the evasive trajectory within 0.7 seconds of prediction of a collision in the simulator
4. The maneuver is executed with the help of closed-loop controls strategy and the RC car localizes itself within and tracks the desired trajectory
Monocular Camera at ceiling Scenario at t = t Scenario at t = t + Δt1 Scenario at t = t + Δt2
Demonstration 3
1. User starts the simulator and chooses one of the 3 scenarios in which the truck runs and collects sensor data along with the ground truth.
2. The operator interrupts the CAS software (signifying system software failure) randomly as the user wishes, thereby introducing a system failure (OR) The operator interrupts the simulator software (signifying sensor/hardware failure) randomly as the user wishes, thereby introducing a sensor failure.
3. The system fault detection software detects either of them and raises a notification
Validation:
Demonstration 1
1. System predicts oncoming vehicle trajectory 2 sec into the future with an of RMSE of less than 3m
2. Predict the possibility of collision with 60% accuracy
3. Send alert signal to driver within 0.2s of collision detection
Demonstration 2
1. RC car tracks the scaled trajectory within 1s of simulation step.
2. The RC car starts executing the maneuver within 1.2s of prediction
3. RC car tracks the desired trajectory within ±5 cm (X, Y) and ±10 deg (Yaw)
Demonstration 3
1. Warn driver in case of system failure