Test Equipment:
FlySense System: is a collection of subsystems that senses an environment and provides warnings to a pilot
The flight system: consists of a DJI Matrice 100 Quadrotor with a Velodyne VLP-16 attached, with the entire assembly mounted on a rolling utility cart.
The user system: consists of the Epson BT300 AR headset and headphones
The environment: Consists of the path shown below with human obstacles
Personnel:
The pilot: person who is demonstrating the user system
The operators: people who are positioning the flight system and setting up the headset demo
The explainer: person who is narrating the demonstration
HUD, Bird’s Eye View, and Sound test:
These tests were conducted outside the NSH entrance on the system mounted on a cart.
The map on the left shows the start/stop markers, obstacles in red and the arrows indicating the direction of movement.
Voice Command Test:
Since it was difficult to quantitatively prove that we could satisfy the listed performance requirement in the limited time, we generated some statistics that helped us analyze the accuracy of the speech recognition system.
Performed tests for 50 iterations in noiseless and noisy environments to check the accuracy of the recognition system.
Tested for:
– Recognizing the words and performing suitable action
– Recognizing incorrect word
– Rejecting Garbage before activation
– Rejecting Garbage after activation
FlySense Voice Commands Test Procedure
List of words:
FlySense Dictionary
– Computer: Activation Word
– Alpha: Home Screen Command
– Bravo: HUD Command
– Charlie: Bird’s Eye View Command
Test Garbage
– Beta, Gamma, Jarvis, Jetson, Delta, Startup (we can test any garbage word)
| Req | Test Procedure 0 | Metric |
| NF.03 | 1. Explainer reports weight of the system | Weight < 1 lb. (453g) |
| NF.04 | 2. Explainer provides Bill of Materials of FlySense System | Cost <$5000 |
| NF.01 | 3. Explainer provides documentation on the projection range of the user system | 20m spec from datasheet |
| Req | Test Procedure 1 (Location: Outside NSH 3rd Level Entrance) | Metric |
| NF.02 | 1. Pilot picks up the user system from an adjacent table and puts it on | |
| NF.06 | 3. Pilot starts up system and enters the home screen | 1 minute setup time |
| F.07 | 4. Pilot gives garbage command. Nothing changes on user system.
5. Pilot gives voice commands to activate the HUD. HUD is shown on user system. 6. Pilot gives voice commands to activate the Birds Eye View. Bird’s Eye View is shown on user system. 7. Give 5 valid voice commands total |
7. 80% correct ID
|
| Req | Test Procedure 2 (Location: Outside NSH 3rd Level Entrance) | Metric |
| NF.07
|
1. Pilot enters into HUD mode.
2. Operators tilt flight system changing roll, pitch and heading. |
Roll (right roll positive, left roll negative), pitch (pitch up positive, pitch down negative) and heading (CCW positive) values update correctly |
| NF.05 | 5. Operator holds a visual in front of pilot at 20 ft. away
6. Pilot reads visual while wearing user system |
Pilot correctly IDs randomly selected visual |
| F.01
F.02 F.04 F.06 F.07 |
7. Cart starts > 5.5s away from obstacle
8. Cart starts moving towards obstacle at ~1.5m/s 9. Cart reaches 5.5s from obstacle, obstacle(s) on left 10. Cart gets closer to obstacle 11. Cart passes obstacle 12. Cart turns around and moves towards obstacle(s) at ~1.5m/s 13. Cart <5.5s from obstacle (left hand side) 14. Cart gets closer to obstacle 15. Carts moves away from obstacle on right hand side See provided diagram for more specific details on the motion of the flight system |
7. No sound warnings, surroundings are shown
8. Display window tracks with cart 9. Sound warnings heard in left ear within .5s of reaching 5.5s threshold 10. Frequency of chimes increase 11. Sound warnings go away 12. Display window tracks with cart 13. Sound warnings start in right ear within .5s of reaching 5.5s threshold, no warnings on right 14. Frequency of chimes increase 15. Sound warnings go away |
