{"id":268,"date":"2017-11-18T01:21:22","date_gmt":"2017-11-18T01:21:22","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2017teamc\/?page_id=268"},"modified":"2018-02-22T03:34:46","modified_gmt":"2018-02-22T03:34:46","slug":"fall-validation-experiment-test-plan","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamc\/demonstrations\/fall-validation-experiment\/fall-validation-experiment-test-plan\/","title":{"rendered":"FVE Test Plan"},"content":{"rendered":"

Test Equipment:<\/b><\/p>\n

FlySense System: <\/i>is a collection of subsystems that senses an environment and provides warnings to a pilot<\/p>\n

The flight system<\/i>: consists of a DJI Matrice 100 Quadrotor with a Velodyne VLP-16 attached, with the entire assembly mounted on a rolling utility cart.<\/p>\n

The user system<\/i>: consists of the Epson BT300 AR headset and headphones<\/p>\n

The environment:<\/i> Consists of the path shown below with human obstacles<\/p>\n

Personnel:\u00a0<\/b><\/p>\n

The pilot: <\/i>person who is demonstrating the user system<\/p>\n

The operators<\/em>: people who are positioning the flight system and setting up the headset demo<\/p>\n

The explainer<\/em>: person who is narrating the demonstration<\/p>\n

 <\/p>\n

HUD, Bird’s Eye View, and\u00a0Sound test:<\/strong><\/p>\n

These\u00a0tests were conducted outside the NSH entrance on the system mounted on a cart.<\/p>\n

\"\" \"\"<\/p>\n

The map on the left shows the start\/stop markers, obstacles in red and the arrows indicating the direction of movement.<\/p>\n

Voice Command Test:<\/strong><\/p>\n

Since it was difficult to quantitatively prove that we could satisfy the listed performance requirement in the limited time, we generated some statistics that helped us analyze the accuracy of the speech recognition system.<\/p>\n

Performed tests for 50 iterations in noiseless and noisy environments to check the accuracy of the recognition system.
\nTested for:
\n– Recognizing the words and performing suitable action
\n– Recognizing incorrect word
\n– Rejecting Garbage before activation
\n– Rejecting Garbage after activation<\/p>\n

FlySense Voice Commands Test Procedure<\/strong>
\nList of words:<\/strong><\/p>\n

FlySense Dictionary
\n– Computer: Activation Word
\n– Alpha: Home Screen Command
\n– Bravo: HUD Command
\n– Charlie: Bird\u2019s Eye View Command<\/p>\n

Test Garbage
\n– Beta, Gamma, Jarvis, Jetson, Delta, Startup (we can test any garbage word)<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n\n\n\n\n\n\n
Req<\/strong><\/td>\nTest Procedure 0<\/strong><\/td>\nMetric<\/strong><\/td>\n<\/tr>\n
NF.03<\/strong><\/td>\n1.\u00a0\u00a0\u00a0 Explainer reports weight of the system<\/td>\nWeight < 1 lb. (453g)<\/td>\n<\/tr>\n
\u00a0NF.04<\/strong><\/td>\n2.\u00a0\u00a0\u00a0 Explainer provides Bill of Materials of FlySense System<\/td>\nCost <$5000<\/td>\n<\/tr>\n
NF.01<\/strong><\/td>\n3.\u00a0\u00a0\u00a0 Explainer provides documentation on the projection range of the user system<\/td>\n20m spec from datasheet<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n\n\n\n\n
Req <\/strong><\/td>\nTest Procedure 1 (Location: Outside NSH 3rd Level Entrance)<\/strong><\/td>\nMetric<\/strong><\/td>\n<\/tr>\n
NF.02<\/strong><\/td>\n1.\u00a0\u00a0\u00a0 Pilot picks up the user system from an adjacent table and puts it on<\/td>\n<\/td>\n<\/tr>\n
NF.06<\/strong><\/td>\n3.\u00a0\u00a0\u00a0 Pilot starts up system and enters the home screen<\/td>\n1 minute setup time<\/td>\n<\/tr>\n
F.07<\/strong><\/td>\n4.\u00a0\u00a0\u00a0 Pilot gives garbage command. Nothing changes on user system.<\/p>\n

5.\u00a0\u00a0\u00a0 Pilot gives voice commands to activate the HUD. HUD is shown on user system.<\/p>\n

6.\u00a0\u00a0\u00a0 Pilot gives voice commands to activate the Birds Eye View. Bird\u2019s Eye View is shown on user system.<\/p>\n

7.\u00a0\u00a0\u00a0 Give 5 valid voice commands total<\/td>\n

7.\u00a0\u00a0\u00a0 80% correct ID<\/p>\n

 <\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n\n\n\n\n
Req<\/strong><\/td>\nTest Procedure 2 (Location: Outside NSH 3rd Level Entrance)<\/strong><\/td>\nMetric<\/strong><\/td>\n<\/tr>\n
NF.07<\/strong><\/p>\n

 <\/td>\n

1.\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Pilot enters into HUD mode.<\/p>\n

2.\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Operators tilt flight system changing roll, pitch and heading.<\/td>\n

Roll (right roll positive, left roll negative), pitch (pitch up positive, pitch down negative) and heading (CCW positive) values update correctly<\/td>\n<\/tr>\n
NF.05<\/strong><\/td>\n5.\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Operator holds a visual in front of pilot at 20 ft. away<\/p>\n

6.\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Pilot reads visual while wearing user system<\/td>\n

Pilot correctly IDs randomly selected visual<\/td>\n<\/tr>\n
F.01<\/strong><\/p>\n

F.02
\nF.03<\/strong><\/p>\n

F.04<\/strong><\/p>\n

F.06<\/strong><\/p>\n

F.07<\/strong><\/td>\n

7.\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Cart starts > 5.5s away from obstacle<\/p>\n

8.\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Cart starts moving towards obstacle at ~1.5m\/s<\/p>\n

9.\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Cart reaches 5.5s from obstacle, obstacle(s) on left<\/p>\n

10.\u00a0\u00a0\u00a0 Cart gets closer to obstacle<\/p>\n

11.\u00a0\u00a0\u00a0 Cart passes obstacle<\/p>\n

12.\u00a0\u00a0\u00a0 Cart turns around and moves towards obstacle(s) at ~1.5m\/s<\/p>\n

13.\u00a0\u00a0\u00a0 Cart <5.5s from obstacle (left hand side)<\/p>\n

14.\u00a0\u00a0\u00a0 Cart gets closer to obstacle<\/p>\n

15.\u00a0\u00a0\u00a0 Carts moves away from obstacle on right hand side<\/p>\n

See provided diagram for more specific details on the motion of the flight system<\/em><\/td>\n

7.\u00a0\u00a0\u00a0 No sound warnings, surroundings are shown<\/p>\n

8.\u00a0\u00a0\u00a0 Display window tracks with cart<\/p>\n

9.\u00a0\u00a0\u00a0 Sound warnings heard in left ear within .5s of reaching 5.5s threshold<\/p>\n

10. Frequency of chimes increase<\/p>\n

11. Sound warnings go away<\/p>\n

12. Display window tracks with cart<\/p>\n

13. Sound warnings start in right ear within .5s of reaching 5.5s threshold, no warnings on right<\/p>\n

14. Frequency of chimes increase<\/p>\n

15. Sound warnings go away<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n","protected":false},"excerpt":{"rendered":"

Test Equipment: FlySense System: is a collection of subsystems that senses an environment and provides warnings to a pilot The flight system: consists of a DJI Matrice 100 Quadrotor with a Velodyne VLP-16 attached, with the entire assembly mounted on a rolling utility cart. The user system: consists of the Epson BT300 AR headset and … Continue Reading<\/strong> <\/i><\/a><\/p>\n","protected":false},"author":105,"featured_media":0,"parent":342,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-268","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamc\/wp-json\/wp\/v2\/pages\/268","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamc\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamc\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamc\/wp-json\/wp\/v2\/users\/105"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamc\/wp-json\/wp\/v2\/comments?post=268"}],"version-history":[{"count":11,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamc\/wp-json\/wp\/v2\/pages\/268\/revisions"}],"predecessor-version":[{"id":521,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamc\/wp-json\/wp\/v2\/pages\/268\/revisions\/521"}],"up":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamc\/wp-json\/wp\/v2\/pages\/342"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamc\/wp-json\/wp\/v2\/media?parent=268"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}