Spring Validation Demonstrations

Perception Demonstration

Targeted Performance Requirements:

M.P.6: Detects objects up to 5 meters away. (< 20 cm error)
M.P.7: Detect objects with 70% accuracy.

Demonstration Results

In this part of the demonstration, a 5×4 meter grid measures the position inside a team member’s apartment. Constructing the grid involved placing visual marks one meter apart with a tape measure. The sensor pod sat at the origin of the grid. The sensor pod then detected and localized a static object at different locations on the grid. ROS bags were collected during the demonstration to analyze the data after testing. The demonstration set up, as well as system outputs, are shown in Fig. 10. On the right, the bounding box around the object shows the output from the detector and tracker. On the left, the red lidar points show the corresponding filtered point cloud of the tracked object. 

Planning Demonstration

Targeted Performance Requirements:

M.P.10: The system will avoid 100% of potential collisions.
M.P.11: The system will not perform unnecessary avoidance maneuvers.

Demonstration Results

The original navigation demonstration involved autonomously piloting the drone in open airspace to validate the flight performance of the system. Due to COVID-19, the physical flight test became infeasible. In response to the pandemic, the navigation SVD became a simulated obstacle avoidance test. To make the simulation less deterministic, the simulated object detector reported the locations of the obstacles at a 70% detection rate with 0.5 m gaussian noise in the X, Y, and Z directions independently.

Four simulated flight tests validated the system performance on avoiding stationary obstacles: obstacle far left (no-avoidance), obstacle skewed left (avoid right), obstacle on flight path (mixed avoidance), and obstacle skewed right (avoid left). This image shows 10 minutes of simulated flight to accurately portray the distribution of avoidance maneuvers.

Scenarios 2-4 show the planner successfully avoided the obstacle near the flight path. There was only one near miss (see Scenario 3) which occurred due to an adverse combination of noisy position and consecutively failed detections. Not unnecessarily performing avoidance (M.P.11) is shown in Scenario 1 by the mostly linear flight path. The small deviations from a perfect line come from noisy detections on the object near the drone’s safety radius. The deviations are minimal when compared to expected deviations from the flight path in real life.

For the SVD encore, we tested the system in scenarios with multiple obstacles, or with a single obstacle.