{"id":409,"date":"2023-05-06T06:12:07","date_gmt":"2023-05-06T06:12:07","guid":{"rendered":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teami\/?page_id=409"},"modified":"2023-12-14T07:14:21","modified_gmt":"2023-12-14T07:14:21","slug":"svd-system-performance","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teami\/svd-system-performance\/","title":{"rendered":"SVD System Performance"},"content":{"rendered":"\n

SVD System Performance<\/strong><\/p>\n\n\n\n

SLAM Subsystem<\/strong><\/p>\n\n\n\n

Mandatory<\/td>Mandatory<\/td>Mandatory<\/td><\/tr>
M.F.R.2<\/td>Simultaneously Map and Localize the Environment<\/td>Relative pose error(T) <= 15cm
Relative pose error(R) <= 10 deg<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

<\/p>\n\n\n\n

During the Spring Validation Demonstrations, our system successfully met the performance requirements for the Simultaneous Localization and Mapping (SLAM) subsystem. The testing environment was provided by the Outersense team, and we evaluated the SLAM performance using their layout. Both rotational and translational errors were assessed by comparing the system’s performance to the available ground truth data.<\/p>\n\n\n\n

The translational error remained below the threshold of 50 millimeters. However, we observed that the rotational error exhibited occasional overshoots when executing turns. Upon further investigation, we discovered that the ground truth data we were using for comparison was not entirely accurate, as it was not obtained using high-precision equipment such as a total station.<\/p>\n\n\n\n

<\/p>\n\n\n\n

\"\"
SLAM – Translational and Rotational Error<\/em><\/figcaption><\/figure>\n\n\n\n

To obtain a more reliable evaluation of the rotational error, we conducted a loop closure test. The results revealed that our system demonstrated an error of approximately 6 degrees, as depicted in the corresponding figure. In light of these findings, we have determined that a different approach is necessary for accurately measuring the SLAM metrics during the Fall Validation Demonstration.<\/p>\n\n\n\n

For the upcoming demonstration, we plan to employ a more precise method for determining ground truth data, which will enable a more accurate assessment of our system’s performance in relation to the SLAM metrics. By implementing this enhanced approach, we hope to further refine the system’s capabilities and ensure the achievement of our performance objectives during the Fall Validation Demonstration.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

SLAM – Loop Closure Test <\/em><\/p>\n\n\n\n

Planning Subsystem<\/strong><\/p>\n\n\n\n

For testing the planner thoroughly, we developed a Qt GUI interface (see below figure) that could allow the tester to add or modify obstacles on a simulated 2.5D grid map. The tester could control the location, dimensions, and slopes on each side of rectangular obstacles.

A total of 7 test cases across 3 layouts were developed, to thoroughly test each aspect of the planner. Each had walls and obstacles of varying positions, locations, and dimensions. In each case, the planner was run and the path generated was visually verified.
<\/p>\n\n\n\n

The Qt GUI interface could also change all 4 robot parameters for two robots. This allowed the tester to see the changes in path when the robot parameters change. Additionally, we also developed a RQT GUI plugin interface (see Figure 17b) that controlled planner parameters such as the number of iterations, the random seed, the neighborhood radius, etc. and allowed changing the start and goal location as well.<\/p>\n\n\n\n

\n
\"\"
Obstacle Control Panel<\/strong><\/figcaption><\/figure>\n\n\n\n
\"\"
Planner Control Panel<\/strong><\/figcaption><\/figure>\n<\/figure>\n\n\n\n

<\/p>\n\n\n\n

In below table, the results are shown for different test cases upon which the planner was initially developed. These test cases allowed for the implementation to include edge cases where the planner could potentially fail, and apply mitigation strategies (such as running the planner for longer iterations). Lastly, the unit tests served the purposed of debugging minor issues in the planner that were not easily detected during implementation and in overall subsystem tests.<\/p>\n\n\n\n

<\/p>\n\n\n\n

Test Case\/Layout No.
<\/td>		Summary<\/td>Result<\/td><\/tr>
0<\/td>Finding a feasible path<\/td>Success<\/td><\/tr>
1<\/td>Finding an optimal path<\/td>Success<\/td><\/tr>
2<\/td>Effect of FSOH on optimal path<\/td>Success<\/td><\/tr>
3<\/td>Effect of MTS on optimal path<\/td>Success<\/td><\/tr>
4<\/td>Effect of Gradient Influence on optimal path<\/td>Success<\/td><\/tr>
5<\/td>Effect of Robot Radius on optimal path<\/td>Success<\/td><\/tr>
6<\/td>Effect of Robot Radius, FSOH and MTS <\/td>Success<\/td><\/tr>
7<\/td>Feasible\/Optimal path in a difficult layout<\/td>Success but only with high number of iterations<\/td><\/tr><\/tbody><\/table>
Unit Tests<\/strong><\/figcaption><\/figure>\n\n\n\n

<\/p>\n\n\n\n

For analysis, we included an option for verbose output in the planner to get the details of each step the planning algorithm was taking. Additionally, the planner also outputs the total time taken as well as the cost of the generated plan for analysis (see Figure 18). In our analysis, we found that the planner starts slowing down significantly if the number of iterations go over 2000. However, keeping the iterations below 1200 gave a lot of sub-optimal paths. Hence we decided the number of iterations to be 1500. We also tested the effect of neighborhood size, and the results were similar to the number of iterations\u037e very slow planning for higher neighborhood sizes and sub-optimal results in case of low neighborhood sizes. The sweet spot in this case was 4.0 meters.<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

SVD System Performance SLAM Subsystem Mandatory Mandatory Mandatory M.F.R.2 Simultaneously Map and Localize the Environment Relative pose error(T) <= 15cmRelative pose error(R) <= 10 deg During the Spring Validation Demonstrations, our system successfully met the performance requirements for the Simultaneous Localization and Mapping (SLAM) subsystem. The testing environment was provided by the Outersense team, and … Continue Reading<\/strong> <\/i><\/a><\/p>\n","protected":false},"author":339,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-409","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teami\/wp-json\/wp\/v2\/pages\/409","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teami\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teami\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teami\/wp-json\/wp\/v2\/users\/339"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teami\/wp-json\/wp\/v2\/comments?post=409"}],"version-history":[{"count":18,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teami\/wp-json\/wp\/v2\/pages\/409\/revisions"}],"predecessor-version":[{"id":593,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teami\/wp-json\/wp\/v2\/pages\/409\/revisions\/593"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2023teami\/wp-json\/wp\/v2\/media?parent=409"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}