{"id":449,"date":"2018-05-07T17:48:53","date_gmt":"2018-05-07T17:48:53","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2017teame\/?page_id=449"},"modified":"2018-05-07T17:49:42","modified_gmt":"2018-05-07T17:49:42","slug":"fall-validation-performance","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teame\/fall-validation-performance\/","title":{"rendered":"Fall Validation Performance"},"content":{"rendered":"

Fall System Requirements<\/strong><\/h2>\n

Fall System Requirements<\/b><\/p>\n\n\n\n\n\n\n\n\n\n\n
Test ID<\/b><\/td>\nDescription<\/b><\/td>\nPeformance<\/b><\/p>\n

(See Details Below)<\/td>\n<\/tr>\n

<\/td>\nINFRASTRUCTURE<\/b><\/td>\n<\/td>\n<\/tr>\n
Test 1<\/span><\/td>\nDetect single pedestrian centroid with Euclidean distance error < 0.3m <\/span><\/td>\nDistance error < 0.09m<\/td>\n<\/tr>\n
Test 2<\/span><\/td>\nTrack single pedestrian within 20m of the infrastructure<\/span><\/td>\nSuccess<\/td>\n<\/tr>\n
Test 3<\/span><\/td>\nPredict single pedestrian trajectory 1.2 seconds into the future with an average error of 0.5m<\/span><\/td>\nAverage error = 0.46m.<\/td>\n<\/tr>\n
Test 4<\/span><\/td>\nTime between first frame with a pedestrian to first published trajectory should be less than 1 second.<\/span><\/td>\nAverage time = 0.15 second.<\/td>\n<\/tr>\n
<\/td>\nVEHICLE<\/b><\/td>\n<\/td>\n<\/tr>\n
Test 5<\/span><\/td>\nStop short of single pedestrian before he\/she enters the field of view<\/span><\/td>\nSuccess<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Test 1: Detection Performance Test<\/b><\/p>\n

Analysis testing: the unit test for detection module is conducted by assigning human agent to 16 of the pre-measured points. The errors are calculated as the Euclidean distance between the ground truth and detected coordinate. All the testing data are shown below:<\/span><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Point No.<\/span><\/td>\nPoint Coordinate (m)<\/span><\/td>\nDetected Coordinate (m)<\/span><\/td>\nError (m)<\/span><\/td>\n<\/tr>\n
1<\/span><\/td>\n2,1<\/span><\/td>\n2.00,1.00<\/span><\/td>\n<0.0001<\/span><\/td>\n<\/tr>\n
2<\/span><\/td>\n4,1<\/span><\/td>\n4.00,1.02<\/span><\/td>\n0.0200<\/span><\/td>\n<\/tr>\n
3<\/span><\/td>\n6,1<\/span><\/td>\n5.99,1.02<\/span><\/td>\n0.0224<\/span><\/td>\n<\/tr>\n
4<\/span><\/td>\n10,1<\/span><\/td>\n10.02,0.99<\/span><\/td>\n0.0224<\/span><\/td>\n<\/tr>\n
5<\/span><\/td>\n15,1<\/span><\/td>\n15,0.98<\/span><\/td>\n0.0200<\/span><\/td>\n<\/tr>\n
6<\/span><\/td>\n21,1<\/span><\/td>\n20.92,1.00<\/span><\/td>\n0.0800<\/span><\/td>\n<\/tr>\n
7<\/span><\/td>\n21,3.3<\/span><\/td>\n20.85,3.34<\/span><\/td>\n0.1552<\/span><\/td>\n<\/tr>\n
8<\/span><\/td>\n9,3.8<\/span><\/td>\n8.98,3.57<\/span><\/td>\n0.2309<\/span><\/td>\n<\/tr>\n
9<\/span><\/td>\n8,3.8<\/span><\/td>\n8.04,3.72<\/span><\/td>\n0.0894<\/span><\/td>\n<\/tr>\n
10<\/span><\/td>\n3,8<\/span><\/td>\n3.08,7.93<\/span><\/td>\n0.1063<\/span><\/td>\n<\/tr>\n
11<\/span><\/td>\n3,6<\/span><\/td>\n3.05,5.94<\/span><\/td>\n0.0781<\/span><\/td>\n<\/tr>\n
12<\/span><\/td>\n3,4<\/span><\/td>\n3.06,3.95<\/span><\/td>\n0.0781<\/span><\/td>\n<\/tr>\n
13<\/span><\/td>\n3,2<\/span><\/td>\n3.03,1.98<\/span><\/td>\n0.0361<\/span><\/td>\n<\/tr>\n
14<\/span><\/td>\n0,1<\/span><\/td>\n0.03,1<\/span><\/td>\n0.0300<\/span><\/td>\n<\/tr>\n
15<\/span><\/td>\n-7,4<\/span><\/td>\n-6.86,4.09<\/span><\/td>\n0.1664<\/span><\/td>\n<\/tr>\n
16<\/span><\/td>\n4,8<\/span><\/td>\n4.05,7.90<\/span><\/td>\n0.1118<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

The mean of Euclidean distance error is 0.0779m which proves that the detection module achieves a decent result and meets the FVE requirement.<\/span><\/p>\n

 <\/p>\n

Test 2: Detection Performance Test<\/b><\/p>\n

Analysis testing: the unit test for tracking module is conducted by assigning human agent to walk between coordinate (20, 0) to (20, 3) repeatedly. The detected points\u2019 y-coordinates versus time frame are stored and visualized. The optimal shape of the graph should be sine-wave like and y-coordinate value should be within the range of (-0.3, 3.3). The tracking test graph is shown below:<\/span><\/p>\n

\"\"<\/p>\n

Figure 1. Tracking Test<\/b><\/p>\n

 <\/p>\n

Test 3: Prediction Performance Test<\/b><\/p>\n

For the fall semester, we proceeded with a second degree polynomial fit. We tested this algorithm on live pedestrian data and were satisfied with the results that we were getting. The graphs below show the performance for some specific pedestrian trajectories.<\/span><\/p>\n

\"\"<\/p>\n

Figure 2. Graphs of trajectory prediction for trajectories of different radii of curvature (r).<\/b>
\n<\/b>(a) r = \u221e, (b) r = 3m, (c) r = 2m, (d) r = 1m, (e) r = 0m (right angle) <\/b><\/p>\n

 <\/p>\n

Test 4: Cycle Time Performance Test<\/b><\/p>\n

The system takes 0.15 seconds between the first frame with a pedestrian to first published trajectory to the vehicle. Our requirement is <1 seconds.<\/p>\n

 <\/p>\n

Test 5: Single Pedestrian Collision Prevention Performance Test<\/b><\/p>\n

https:\/\/www.youtube.com\/watch?v=_3lJCjnGp6c<\/a><\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Fall System Requirements Fall System Requirements Test ID Description Peformance (See Details Below) INFRASTRUCTURE Test 1 Detect single pedestrian centroid with Euclidean distance error < 0.3m Distance error < 0.09m Test 2 Track single pedestrian within 20m of the infrastructure Success Test 3 Predict single pedestrian trajectory 1.2 seconds into the future with an average
+ Read More<\/a><\/p>\n","protected":false},"author":116,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-449","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teame\/wp-json\/wp\/v2\/pages\/449","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teame\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teame\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teame\/wp-json\/wp\/v2\/users\/116"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teame\/wp-json\/wp\/v2\/comments?post=449"}],"version-history":[{"count":1,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teame\/wp-json\/wp\/v2\/pages\/449\/revisions"}],"predecessor-version":[{"id":450,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teame\/wp-json\/wp\/v2\/pages\/449\/revisions\/450"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teame\/wp-json\/wp\/v2\/media?parent=449"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}