{"id":128,"date":"2016-09-15T01:48:07","date_gmt":"2016-09-15T01:48:07","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2016teamf\/?page_id=128"},"modified":"2017-07-04T04:09:53","modified_gmt":"2017-07-04T04:09:53","slug":"system-performance","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2016teamf\/system-performance\/","title":{"rendered":"System performance"},"content":{"rendered":"<p><strong>Spring videos:<\/strong><\/p>\n<p>Full system demo:<\/p>\n<p><iframe loading=\"lazy\" width=\"1170\" height=\"658\" src=\"https:\/\/www.youtube.com\/embed\/PSEruHejoz4?feature=oembed\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<p>&nbsp;<\/p>\n<p>Signature Detection Result in RGB Video :<\/p>\n<p><iframe loading=\"lazy\" width=\"1170\" height=\"878\" src=\"https:\/\/www.youtube.com\/embed\/4sayaqmbSK4?feature=oembed\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<p>Signature Detection Result in Thermal\u00a0Video :<\/p>\n<p><iframe loading=\"lazy\" width=\"1170\" height=\"878\" src=\"https:\/\/www.youtube.com\/embed\/srCRvJt1i6Y?feature=oembed\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<p>Bright-colored object detection:<\/p>\n<p><iframe loading=\"lazy\" width=\"1170\" height=\"658\" src=\"https:\/\/www.youtube.com\/embed\/dsm2PwdwgPQ?feature=oembed\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<p>&nbsp;<\/p>\n<p>Hot objects detection using IR imagery:<\/p>\n<p><iframe loading=\"lazy\" width=\"1170\" height=\"878\" src=\"https:\/\/www.youtube.com\/embed\/tXwmk_WfthI?feature=oembed\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<p>&nbsp;<\/p>\n<p><strong>Spring Validation Experiment<\/strong><\/p>\n<p>During the SVE, our system&#8217;s performance was validated as per the designed test sequences.<\/p>\n<p><em><strong>All our subsystems were successful on all of their respective performance measures.<\/strong><\/em><\/p>\n<p>This section summarizes the overall system performance during the SVE and SVE Encore. For detailed analysis, refer to the\u00a0analysis done for each of the subsystems.<\/p>\n\n<table id=\"tablepress-2\" class=\"tablepress tablepress-id-2\">\n<thead>\n<tr class=\"row-1\">\n\t<th class=\"column-1\">Procedure<br \/>\n<\/th><th class=\"column-2\">Procedure<br \/>\n<\/th><th class=\"column-3\">Verification Criteria<\/th><th class=\"column-4\">SVE Performance<\/th><th class=\"column-5\">SVE Encore Performance<\/th>\n<\/tr>\n<\/thead>\n<tbody class=\"row-striping row-hover\">\n<tr class=\"row-2\">\n\t<td class=\"column-1\">1.<br \/>\n<\/td><td class=\"column-2\">Place seven signatures specified above, at various locations in a 50m x 50m area.waypoints<br \/>\n<\/td><td class=\"column-3\"><\/td><td class=\"column-4\"><\/td><td class=\"column-5\"><\/td>\n<\/tr>\n<tr class=\"row-3\">\n\t<td class=\"column-1\">2.<br \/>\n<\/td><td class=\"column-2\">Place UAV on the ground, turn it and the payload power ON. Create and launch a search mission through the Mobile app.<\/td><td class=\"column-3\"><\/td><td class=\"column-4\"><\/td><td class=\"column-5\"><\/td>\n<\/tr>\n<tr class=\"row-4\">\n\t<td class=\"column-1\">3.<\/td><td class=\"column-2\">UAV sweeps the area (50m x 50m) collecting sensor data.<br \/>\n<\/td><td class=\"column-3\"><\/td><td class=\"column-4\"><\/td><td class=\"column-5\"><\/td>\n<\/tr>\n<tr class=\"row-5\">\n\t<td class=\"column-1\">4.<\/td><td class=\"column-2\">Transfer data from the payload to the laptop.<\/td><td class=\"column-3\"><\/td><td class=\"column-4\"><\/td><td class=\"column-5\"><\/td>\n<\/tr>\n<tr class=\"row-6\">\n\t<td class=\"column-1\">5.<\/td><td class=\"column-2\">Run integrated human detection software to detect human signatures.<\/td><td class=\"column-3\">The system should be able to detect at least 5\/7 human signatures planted<br \/>\n<\/td><td class=\"column-4\">6\/7 signatures detected.<br \/>\nThe system was able to detect 1 out of 2 humans and all the other signatures<br \/>\n<\/td><td class=\"column-5\">7\/7 signatures detected.<br \/>\nThe system was able to detect all the signatures<br \/>\n<\/td>\n<\/tr>\n<tr class=\"row-7\">\n\t<td class=\"column-1\">6.<\/td><td class=\"column-2\">Run software to report GPS coordinates of the probable rescue locations.<\/td><td class=\"column-3\">The system should be able to report GPS coordinates of the rescue locations with a margin of error of +-8m<br \/>\n<\/td><td class=\"column-4\">We got likely locations within +\/- 5m of all signatures just except for the human. Human was reported to be 8.65 m away from actual<br \/>\n<\/td><td class=\"column-5\">We got likely locations within +\/- 3m of all the signatures<br \/>\n<\/td>\n<\/tr>\n<tr class=\"row-8\">\n\t<td class=\"column-1\">7.<\/td><td class=\"column-2\">Select one location out of the reported rescue locations for the package drop<br \/>\n<\/td><td class=\"column-3\"><\/td><td class=\"column-4\"><\/td><td class=\"column-5\"><\/td>\n<\/tr>\n<tr class=\"row-9\">\n\t<td class=\"column-1\">8.<\/td><td class=\"column-2\">Detach the microphone from the payload, attach the rescue package. Launch rescue package drop mission through the Mobile app.<br \/>\n<\/td><td class=\"column-3\"><\/td><td class=\"column-4\"><\/td><td class=\"column-5\"><\/td>\n<\/tr>\n<tr class=\"row-10\">\n\t<td class=\"column-1\">9.<\/td><td class=\"column-2\">UAV flies to the provided GPS location, lands, and autonomously drops the package.<br \/>\n<\/td><td class=\"column-3\">Rescue package should be dropped with distance less than 8m from the signature. (assuming the signature has not moved)<br \/>\n<\/td><td class=\"column-4\">Autonomous package drop did not work, but the drone landed 2 m away while testing for a bright mattress, and 9 m away for the human<br \/>\n<\/td><td class=\"column-5\">Autonomously dropped the package 2.3 m away from a human location<br \/>\n<\/td>\n<\/tr>\n<tr class=\"row-11\">\n\t<td class=\"column-1\">10.<\/td><td class=\"column-2\">UAV flies back to the home location and lands.<br \/>\n<\/td><td class=\"column-3\">Total time &lt; 25 minutes<br \/>\n<\/td><td class=\"column-4\">Total time &gt; 25 minutes<br \/>\n<\/td><td class=\"column-5\">Total time &lt; 25 minutes<br \/>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<!-- #tablepress-2 from cache -->\n<p>&nbsp;<\/p>\n<p><strong>Fall Validation Experiment<\/strong><\/p>\n<p>During the FVE, our system&#8217;s performance was validated as per the designed test sequences.<\/p>\n<p><em><strong>All our subsystems were successful on all of their respective performance measures.<\/strong><\/em><\/p>\n<p>Following table details out the requirements for the various test steps for each of our subsystems and our subsystems&#8217; performance on them.<\/p>\n\n<table id=\"tablepress-1\" class=\"tablepress tablepress-id-1\">\n<thead>\n<tr class=\"row-1\">\n\t<th class=\"column-1\">Steps<br \/>\n<\/th><th class=\"column-2\">Requirements<\/th><th class=\"column-3\">Subsystem<\/th><th class=\"column-4\">Performance<\/th>\n<\/tr>\n<\/thead>\n<tbody class=\"row-hover\">\n<tr class=\"row-2\">\n\t<td class=\"column-1\">A.1.<\/td><td class=\"column-2\"><\/td><td class=\"column-3\">Autonomous Flight System<\/td><td class=\"column-4\"> iOS App to provide input GPS <\/td>\n<\/tr>\n<tr class=\"row-3\">\n\t<td class=\"column-1\">A.2.<\/td><td class=\"column-2\">Accuracy in reaching desired height (+-1m tolerance)<br \/>\n<\/td><td class=\"column-3\">Autonomous Flight System<\/td><td class=\"column-4\">Height provided = 5M. Min height = 4.8M, Max height = 5.3M<\/td>\n<\/tr>\n<tr class=\"row-4\">\n\t<td class=\"column-1\">A.3.<\/td><td class=\"column-2\">Accuracy in reaching the waypoints (+-5m tolerance)<br \/>\n<\/td><td class=\"column-3\">Autonomous Flight System<\/td><td class=\"column-4\">Accuracy : &lt;3M from each waypoint<br \/>\n<\/td>\n<\/tr>\n<tr class=\"row-5\">\n\t<td class=\"column-1\">A.4.<\/td><td class=\"column-2\">Accuracy in reaching the starting location (+-5m tolerance)<br \/>\n<\/td><td class=\"column-3\">Autonomous Flight System<\/td><td class=\"column-4\">Back to start location : &lt;3M <\/td>\n<\/tr>\n<tr class=\"row-6\">\n\t<td class=\"column-1\">B.1.<\/td><td class=\"column-2\">Ability to detect at least 60% of valid human signatures in images<\/td><td class=\"column-3\">Signature Detection<\/td><td class=\"column-4\">Overall accuracy for 40 test images: 65.1%<\/td>\n<\/tr>\n<tr class=\"row-7\">\n\t<td class=\"column-1\">C.1.<\/td><td class=\"column-2\">Should be able to hold package of weight 100g, and size 10cmx10cm<br \/>\n<\/td><td class=\"column-3\">SDPD (Package Drop Mechanism)<\/td><td class=\"column-4\">Successful (package weight ~160g)<br \/>\n<\/td>\n<\/tr>\n<tr class=\"row-8\">\n\t<td class=\"column-1\">C.2.<\/td><td class=\"column-2\">Should not lose grip of the package for drone velocities under 2 m\/s<\/td><td class=\"column-3\">SDPD (Package Drop Mechanism)<\/td><td class=\"column-4\">Successful <\/td>\n<\/tr>\n<tr class=\"row-9\">\n\t<td class=\"column-1\">C.3.<\/td><td class=\"column-2\">Manually demonstrate the mechanism\u2019s ability to release the package<br \/>\n<\/td><td class=\"column-3\">SDPD (Package Drop Mechanism)<\/td><td class=\"column-4\">Successful (demonstrated switching through commands sent from phone)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<!-- #tablepress-1 from cache -->\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Spring videos: Full system demo: &nbsp; Signature Detection Result in RGB Video : Signature Detection Result in Thermal\u00a0Video : Bright-colored object detection: [&hellip;]<\/p>\n","protected":false},"author":83,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-128","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2016teamf\/wp-json\/wp\/v2\/pages\/128","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2016teamf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2016teamf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2016teamf\/wp-json\/wp\/v2\/users\/83"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2016teamf\/wp-json\/wp\/v2\/comments?post=128"}],"version-history":[{"count":24,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2016teamf\/wp-json\/wp\/v2\/pages\/128\/revisions"}],"predecessor-version":[{"id":1115,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2016teamf\/wp-json\/wp\/v2\/pages\/128\/revisions\/1115"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2016teamf\/wp-json\/wp\/v2\/media?parent=128"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}