Final System Performance
The Fall Validation Demonstration (FVD) and the FVD Encore were held on the 20th & 29th of November at the Moonyard, a sandbox commonly used to test space robotic systems, situated in the Planetary Robotics Lab at CMU. In the demonstration, the system was tested to evaluate the performance. The table below details the achieved performance of the system as compared to the system requirements. The system successfully met all of the mandatory performance and non functional requirements.
To evaluate the system, it was given a desired berm configuration that was 15 cm high and 75 cm long. The rover was then left to operate autonomously in-order to construct the desired berm. There was no human intervention required throughout the autonomous construction. The system even met some of the desirable performance requirements.
| ID | Promised Metric | Achieved Performance |
| M.P.1 | Operate with 2 types of user command | Operated with 2 types of user command |
| M.P.2 | Update worksite map at ≥ 0.1 Hz | Updated worksite map at 1.0 Hz |
| M.P.3 | Output ≥ 2 data products of worksite map | Output 2 data products of worksite map |
| M.P.4 | Localize with a positional accuracy of ≤ 30 cm | Localized with a positional accuracy of ≤ 2 cm |
| M.P.5 | Operate autonomously with 0 manual intervention | Operated autonomously with 0 manual intervention |
| M.P.6 | Traverse a site terrain with ≤ 10 deg incline | Traversed a site terrain with an incline up to 15 deg |
| M.P.7 | Excavate ≥ 3 kg material per cycle | Excavated 8 kg material per cycle |
| M.P.8 | Build a berm that has minimum dimensions of 15 cm in height and 30 cm in length | Built a berm that has dimensions of 15 cm in height and 76 cm in length |
| M.P.9 | Build a berm with error tolerance of ± 3 cm in height and ± 5 cm in length | Built a berm with an error of 2.1 cm in height and 1.3 cm in length |
SVD System Performance
In the Spring Validation Demonstration (SVD) and the SVD Encore, held on the 19th \& 26th of April, respectively, the system successfully fulfilled all the required metrics. The performance of the system in relation to the targeted requirements has been detailed in the table below.
| Requirement ID | Promised Metric | Achieved Metric |
| M.P.4 | Localize with a positional accuracy of ≤ 30 cm | Localized with a positional accuracy of ≤ 2 cm |
| M.P.7 | Excavate ≥ 3 kg material per cycle | Excavated 8-9 kg material per cycle |
| M.P.8 | Build a berm that has minimum dimensions of 15 cm in height and 30 cm in length | Built a berm that has dimensions of 17 cm in height and > 60 cm in length |
To test the first performance metric of localization, a loop-closure test was conducted. In the loop-closure test, the system is teleoperated around the worksite in a loop, stopping back at the starting position. If the localization module also shows that the system followed a loop back to the originating position and orientation, that validates the accuracy of the localization. The exact accuracy metric is calculated by the difference in the starting and stopping position as estimated by the localization module. The system achieved a positional accuracy of approximately 2 cm, which is a higher accuracy than both the mandatory and the desired performance requirements of 30 cm and 15 cm, respectively. This positional accuracy was achieved with the help of the total station that tracks the reflector prism mounted at the highest point of the robot. The system also had accurate orientation due to the mounted IMU.
The system was teleoperated to test the second and third performance metrics of excavation and berm construction. Three excavation-deposition cycles were executed that resulted in a berm that was 17 cm high and 60 cm long, which satisfies the minimum performance metric M.P.8. These dimensions bring the volume of the berm to approximately 0.015 cubic meter. The density of the sand used to test the system is 1604 kg per cubic meter. Using the volume and density, the weight of the total sand excavated and deposited is estimated to be 26.04 kg. Therefore, the system was able to excavate approximately 8 kg of material per cycle, more than 2.5 times the promised performance metric M.P.7.