Simulation Subsystem

The system needs to simulate the environment and external agent behaviors realistically. This
is an important requirement if the system needs to work in real-world scenarios. Therefore,
CARLA was the best choice, one of the most realistic simulators out there. Additionally, it also
has a good developer community for support.

CARLA can provide simulated sensor data streams of cameras, lidars, and radars. The sensors
can be virtually placed anywhere in the environment or agent. This sensor placement will be
made realistically which can be replicated in the real world later. Additionally, in order to
accelerate development, we plan on initially getting data abstractions like segmentation maps
and bounding boxes directly from the simulator.

There are two main inputs to this subsystem one from the user and one from the system. The
user inputs the types of behaviors of traffic that need to be generated. The simulation
subsystem must produce a diverse set of scenarios in order to ensure our RL agent doesn’t
overfit to just a few scenarios. Furthermore, the system provides continuous trajectories
that need to be tracked by the agent. The input trajectory is tracked by the simulator and
updated sensor data streams produced from the updated environment will be sent to the next
phase.

Figure: Camera Stream in Carla Simulation environment

Pedestrian Modelling

Figure: Modelling pedestrian crossing behaviour in CARLA

We are also utilizing CARLA to train pedestrian detection and avoidance to our RL agent. We spawn a pedestrian in simulation and train our agent to avoid colliding with it while it crosses the road.

Improving Simulation Performance:

The ‘performance’ of CARLA is defined as the maximum simulated frames per second (fps) that can be obtained from CARLA during training runs, while minimizing the impact on computing resources so they may be used by other processes. There are several ways CARLA’s actual and perceived performance can be improved that are listed below. 

• Reducing graphics overhead (No-rendering mode, graphics API)
  • Take a look at the Rendering options. Rendering only serves a visual purpose. We should be disabling it during training. It can be enabled either during carla initialization or during runtime with the config file

Figure: Carla’s no render mode rendering only basic graphics.

• Graphics opengl vs Vulcan: Performance can also be increased by using open-gl mode rather than the vulcan graphics mode
• Hybrid Physics Mode:
  • Disable physics calculations of vehicles outside of a certain distance away from the vehicle
  • Faster computation of Traffic Manager
  • May be able to disable physics altogether, only rely on collisions