Perception
The goal of the perception system is to detect multiple pedestrians reliably and accurately in near real time. The perception pipeline includes 2D and 3D pedestrian detection modules. 3D module uses Velodyne 16 beams lidar as the point cloud source. 2D module uses the ZED camera as the image source. We implemented the background subtraction module to detect and track the moving objects. We used convolutional neural network to classify and segment the target objects. And then we pass the targets’ poses information to the prediction module.
Prediction
The prediction subsystem subscribes to input from the tracking module in the form of a list of pedestrian centroids for each pedestrian in the past 12 frames. These 12 frames are defined as the observation length.
The prediction subsystem then publishes the predicted trajectory of all pedestrians for the next 1.2 seconds which is equivalent to 12 frames (since we have chosen the frame rate to be 10 Hz).
Sensor Mount
The sensor mount is our mechanical subsystem designed to provide flexibility and stability in gathering pedestrian data. The mount is composed of a telescoping pole, mobile cart, and mounting bracket.
The mounting bracket gives the LiDAR and ZED camera a stable mount to fix to, while maintiaining the 27cm x 8cm difference between those two camera’s origins that is present in the KITTI dataset.
The telescoping pole adds two degrees of freedom for the pose of the cameras, while providing stability at height. The pole allows the LiDAR to rest between 6 to 10 feet above the ground. It also allows a +/- 90 degree tilt to look up or down with the cameras.
Finally, the cart provides a steady base that can be moved easily for testing. It also has the benefit of multiple surfaces to store things on, which we just about always do!
Vehicle
We are using last year’s Team Lo-Co vehicle, the details of which can be found here.
For our purposes, we are simply running the vehicle in a straight line and demonstrating that it will stop when a pedestrian is approaching from out of sight. The vehicle can be seen below, including the “bumper” we have attached to make the vehicle the width of a Toyota Prius, in order to mimic the scale of a real car.
