Spring Validation Experiment
Test Results
During our Spring Validation Experiment, we tested most of the key subsystems and their requirements. Our experiments were for the most part successful, with reliability being the only metric that succeeded in all tests but was not shown in the SVE.
| Requirements | Expectations | FVE | FVE-ENCORE |
| MP1.1, MP1.2, MP1.3 | Docking platform shall move according to the given input frequency in Z-direction | Successful
Dock changed to given frequency as detected by IR |
Successful |
| MP1.4 | Docking platform shall have a locking mechanism which supports weight of 5 kg | Successful
Dock held quadcopter for 30 seconds without external support |
Successful |
| MP1.2 | Sensor gathers data from the motion of the platform and outputs the frequency within an accuracy of 0.05 Hz | Successful
|
Successful |
| MP2.1 | Quadcopter shall localize w.r.t. platform within 50mm accuracy | Successful
Quadcopter maintained XY centered hover with a distance of less than 50mm between center of quadcopter and center of platform |
Successful |
| MP2.2 | Quadcopter shall dock to the platform autonomously and without colliding within 10 minutes | Successful
Quadcopter docked 4 times and rejected once, with an average relative velocity of 38 cm/s and an average time of TIME
|
Successful
|
| MP2.2 | Quadcopter shall dock to the platform autonomously and without colliding in 80% of tests | Not Successful
Quadcopter docked 4 times and rejected once out of 6 attempts, a rate of 67% However, exhaustive testing outside of demo showed a failure rate of 3 out of 30, 90%
|
Not Successful
|
| DP2.1 | Quadcopter shall localize w.r.t. platform within 30mm accuracy | Sucessful | Sucessful |
| DP2.2. | Quadcopter shall dock to the platform within 5 minutes | Successful
Docking occurred in an average of TIME |
Successful |
Table 1 – SVE Results with FVE goals included
Fall Validation Experiment
Test Results
During our Fall Validation Experiment, we tested most of the key subsystems and their requirements. Our experiments were for the most part successful, with navigation being the only subsystem that needed to have further development scheduled for the Spring semester in addition to all other planned implementation.
| REQUIREMENTS | EXPECTATIONS | FVE | FVE-ENCORE |
| MFR1.1 | Docking platform shall move according to the given input frequency | Successful (0.15 to 0.3 achieved with 10% error) |
Successful (added qualitative functionality, no quantitative improvement) |
| MFR2 | Sensor gathers data from the motion of the platform and outputs the frequency | Successful (output with less than 5% error) |
Successful (no change) |
| MFR2.1 | Quadcopter shall be able to autonomously hover | Successful (quadcopter successfully hovered for 20 seconds) |
Successful (simulated, quadcopter unavailable during encore) |
| MFR2.2 | Quadcopter shall move autonomously from point A to point B | Failure due to lack of spare parts | Failure due to lack of spare parts |
| MFR2.1 | AprilTag should detect the camera and ascertain the distance moved within 5% error | Successful (AprilTag system output within 5% error) |
Successful (AprilTag system output within 5% error) |
| MFR2.1 | AprilTag should detect the camera moving and therefore make the quadcopter move accordingly in the simulation | Not planned initially | Successful (simulated quadcopter moved correct amount based on motion of April Tag) |
Table 2 – FVE Results
Strengths and Weaknesses
Our system and subsystems are also for the most part acceptable. Our strengths are in the robustness of our dock and vision, and most of our weaknesses are solvable with further improvement to control of the quadcopter. Our dock’s robustness, however, comes at the cost of being large and heavy, necessitating special arrangements to find a location for it that will facilitate testing and evaluation of the system as a whole.
| STRENGTHS | WEAKNESSES |
| Docking Platform is robust | Velocity control not stable in Matrice |
| Motor can withstand the desired load | Flight controller code is not accessible |
| April Tag works suitably even in low lighting | Cannot provide state estimation values to flight controller |
| IMU is giving accurate readings | Platform is heavy. Not able to find place to mount it permanently |
| Indoor hovering is stable using guidance |
Table 3 – Strengths and Weaknesses of System
The motor and platform are strong enough to withstand the weight of the quadcopter after it has docked. A 100% margin was used while selecting the motor. Currently, it can withstand weight of 5 kg at 60V and the maximum take-off weight of the quadcopter is 3.4 kg. Higher torques can be achieved at higher voltages. The accuracy of IMU readings is critical to our project because these values will be transmitted to the quadcopter and used to determine the suitable moment to initiate the docking operation. The April Tag detection is working suitably in low light conditions as well as when the resolution is reduced to one-fourth of the original resolution. Also, the tag detection rate is 20 Hz in poor lighting conditions. We are able to achieve stable hover of quadcopter indoors using the Guidance. However, when we try to give velocity commands for position control, it drifts significantly. This is because the default position control in Matrice 100 is done using GPS coordinates.
Another weakness is that the flight controller’s code is not accessible to us and we cannot provide state estimation values to it. Thus, the only possible option to do position control is to provide velocity commands. The loop is closed using information provided from the IMU sensors and the Guidance’s cameras. The IMU does not provide accurate position estimates due to high noise. The Guidance uses optical flow to calculate velocity. This is effective only if there are significant non-repeating features available for tracking.
To overcome the weaknesses of our quadcopter we first need to set safe limits to yaw, pitch and roll commands. Further, we need to confirm that the manual override is functional by testing it at least ten times. For CV we need to increase the detection rate further. Implementing Lucas-Kanade tracking should help us in achieving this. Further, using a camera with higher frame rate should also contribute to an increased detection rate.
Unit Tests
Our Unit Tests for Fall are tracked on a separate page.
Our Unit Tests for Spring are tracked in our Test Plan.