Test Plan

Our test plan until FVD can be found here and 1-pager for FVD here

1-pager for SVD can be found here

Location: In simulation
Equipment: A computer system or cloud server with Gazebo, ROS and COLA predictive avoidance packages installed
Operating area: Simulated 92*64 m factory environment (virtual)

Figure 1: Floor plan for the virtual factory environment

The user starts ROS core and simulation
Simulator loads the floor plan for the factory, and launches the visualization tool.
Simulator launches pedestrians and forklifts, following their fixed paths and routines
Simulator launches one robot, and publish waypoints and the static map to robot
Robots plan global paths and start moving
Robots continuously receive localization and noisy obstacle observation from simulator
Robots avoid obstacles according to its predicted trajectories of obstacles
System stops after a certain duration of time

To demonstrate the simulation system and simulated environment with obstacles
To demonstrate the robot’s ability to plan paths and follow them
To demonstrate the predictive avoidance algorithm (showing robots dodging obstacles using predicted trajectories)
To validate M.P.1-3: Robot speed will be above 0.5m/s and below 1.8m/s if no obstacle is detected
To validate M.P.5: Robots should receive localization and observation in at least 10 Hz, and send back control signals to the simulator in at least 10 Hz

Location: Any location with flat ground, sufficient lighting and high enough ceiling (over 2.5 m).
Equipment: TurtleBot, obstacle action figures/toys, overhead camera, lights
Operating area: 2*2 m flat ground

Setup overhead camera and place markers on top of robot/obstacles for ground truth gathering
Place robot and one or more obstacles in testing ground
Manually drive robot and/or obstacles
Obtain robot estimation of obstacle positions and classifications for demo duration
Obtain ground truth position data of robot and all obstacles from overhead camera readings for demo duration
Use measurement and ground truth data for visualization/plotting
Evaluate detection (position) and classification accuracy

To Validate M.P.6: Classify obstacles of interest with mAP of at least 60%
To Validate M.P.7: Detect positions of obstacles of interest within 0.1 m accuracy
To Validate M.P.8: Detect obstacles of interest within a range of 3 m
To Validate: M.P.9 Output results of positioning and classification within 100 ms per frame

Location: In simulation
Equipment: A computer system or cloud server with Gazebo, ROS and COLA predictive avoidance packages installed
Operating area: Simulated 92*64 m factory environment (virtual)

The user starts ROS core and simulation
Simulator loads the floor plan for the factory, and launches the visualization tool.
Simulator launches pedestrians and forklifts, following their fixed paths and routines
Simulator launches robot fleet, and publish waypoints and the static map to robots
Robots plan global paths and start moving
Robots continuously receive localization and noisy obstacle observation from simulator
Robots avoid obstacles using all three avoidance algorithms
System stops after a certain duration of time and calculates resulting productivity
System repeats the previous steps in naive avoidance mode without any classification and prediction, and computes nominal productivity

To demonstrate all three avoidance algorithms (predictive, conservative, reciprocal)
To validate M.P.4: Robot fleet should have productivity increased by >5% when using classification-based predictive/reciprocal avoidance compared to nominal productivity using only naive avoidance
To validate M.P.5: Robots should receive localization and observation in at least 10 Hz, and send back control signals to the simulator in at least 10 Hz