{"id":328,"date":"2016-12-16T22:25:48","date_gmt":"2016-12-16T22:25:48","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2016teama\/?page_id=328"},"modified":"2017-05-13T04:50:04","modified_gmt":"2017-05-13T04:50:04","slug":"spring-validation-experiments","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2016teama\/system-performance\/spring-validation-experiments\/","title":{"rendered":"Spring Validation Experiments"},"content":{"rendered":"

SVE Document<\/strong><\/p>\n\n\n\n\n\n\n\n\n
Experiment<\/td>\nG (Spring Validation Experiment)<\/td>\n<\/tr>\n
Objective<\/td>\nDemonstrate and verify functionality of the fully integrated system<\/td>\n<\/tr>\n
Location<\/td>\nMajor streets around campus with moderate traffic flow in favorable weather conditions (daytime)<\/td>\n<\/tr>\n
Elements<\/td>\nIntegrated perception system with the stereo cameras, radar, power devices, and computer on the testing vehicle (Volvo S60)<\/td>\n<\/tr>\n
Procedure<\/td>\n\u00b7\u00a0\u00a0\u00a0\u00a0\u00a0 Mount all sensors and place power and computing devices properly on the testing vehicle.<\/p>\n

\u00b7\u00a0\u00a0\u00a0\u00a0\u00a0 Drive the car around the campus for about 15 minutes.<\/p>\n

\u00b7\u00a0\u00a0\u00a0\u00a0\u00a0 Detect and classify pedestrians and vehicles on road. Display their relative positions and velocities in the customized GUI.<\/p>\n

\u00b7\u00a0\u00a0\u00a0\u00a0\u00a0 Track pedestrians and vehicles on road continuously with label.<\/td>\n<\/tr>\n

Verification Criteria<\/td>\n\u00b7\u00a0\u00a0\u00a0\u00a0\u00a0 The ID, relative position, velocity, and class (pedestrian or vehicle) of each detected object should be clearly displayed in the GUI in real time (shown in Figure 1).<\/p>\n

\u00b7\u00a0\u00a0\u00a0\u00a0\u00a0 The absolute position (longitude and latitude) and velocity of the testing vehicle should also be displayed clearly in the GUI in real time.<\/p>\n

\u00b7\u00a0\u00a0\u00a0\u00a0\u00a0 Criteria for Experiment A, B, C, D should be all be met successfully.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Evaluation<\/strong><\/p>\n

Although we were able to successfully complete most of the planned tasks we had set out to do by the SVE (see Table 1), we were unable to meet all the criteria. In the SVE, we did not include the velocities for the detected objects. This was due to simple oversight; we did have the system capabilities to calculate and display the velocities at the time. For the SVE Encore, we were more careful about presenting all the features of our perception system, and so we made sure to demonstrate them.<\/span><\/p>\n

Even though we can claim that we have working implementations for a variety of advanced perception methods, many of our implementations need further work and tuning in order to perform consistently. We need to modify the algorithms and methods we currently use to be better suited to our system and applications.<\/span><\/p>\n

Table 1. SVE performance checklist<\/b><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n
Success Criteria<\/span><\/td>\nSVE<\/span><\/td>\nSVE Encore<\/span><\/td>\n<\/tr>\n
Tracking ID of each object displayed in GUI<\/span><\/td>\nNo<\/span><\/td>\nYes<\/span><\/td>\n<\/tr>\n
Relative position of each object displayed in GUI<\/span><\/td>\nYes<\/span><\/td>\nYes<\/span><\/td>\n<\/tr>\n
Velocity of each object displayed in GUI<\/span><\/td>\nNo<\/span><\/td>\nYes<\/span><\/td>\n<\/tr>\n
Classification of each object displayed in GUI<\/span><\/td>\nYes<\/span><\/td>\nYes<\/span><\/td>\n<\/tr>\n
Absolute position (Lat. & Long. ) of host vehicle in GUI<\/span><\/td>\nYes<\/span><\/td>\nYes<\/span><\/td>\n<\/tr>\n
Absolute velocity of host vehicle in GUI<\/span><\/td>\nYes<\/span><\/td>\nYes<\/span><\/td>\n<\/tr>\n
Accuracy of filtered depth value of objects > 70%<\/span><\/td>\nNo<\/span><\/td>\nYes<\/span><\/td>\n<\/tr>\n
Accuracy of detection and classification by vision > 60%<\/span><\/td>\nYes<\/span><\/td>\nYes<\/span><\/td>\n<\/tr>\n
Accuracy of detection by integrated system > 70%<\/span><\/td>\nNo<\/span><\/td>\n\n

Yes<\/span><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

\u00a0 Strong and weak points<\/b><\/p>\n

The following strengths and weaknesses of our perception system were noted by our team over the course of this project. The strong points are what we depend on for our system\u2019s performance, whereas weak points are potential issues we might be able to fix in the future.<\/p>\n

Strong points:<\/i><\/strong><\/p>\n