Fall Validation Demonstration – Verification Criteria
- VTOL is able to takeoff and reach target location autonomously in fixed wing mode.
- System demonstrates VTOL transitions without any errors.
- The marker is detected by the VTOL from at least 40 ft above the ground.
- The package is delivered from at least 4 ft above the ground.
- Package is dropped within 2 ft of the target location
- All eggs in the package are delivered intact, with no breakage.
In a succesful FVD Encore, we delivered 3 times while meeting all Verification criteria in each of those runs!
Flight Subsystem
MPF.1.1 Complete a delivery up to 8 miles away within 30 minutes.
In the fall semester, the VTOL was rigorously tested in Fixed wing mode with transitions and autonomous capabilities were developed to meet the performance requirement defined by MPF.1.1. A mission was set up for the VTOL system to travel a distance of ~8 miles(travel to location), perform delivery and travel a distance of ~8 miles again(travel back from location). The system was able to meet MPF.1.1 and below are the metrics of the test.
- Total Time: 26 minutes
- Total Distance: 27.3km (~17 miles)
| Travel to Delivery Location | 14 minutes |
| Delivery | 2 minutes |
| Return Home | 10 minutes |
Check out what a delivery mission would look like, with complete transitions into and out of fixed wing flight, even in windy conditions!
Perception Subsystem
MPF.3.1 Detect marker from 40 ft above ground 95% of the time.
The performance requirement was verified by measuring and logging the height of the VTOL during the first instance of marker detection. The plot below shows the performance of the system across 18 different flights with height measurements from the onboard GPS and the marker pose. The system is able to meet the requirement succesfully.
MPF.3.2 Drop package within 2 ft of the target.
To verify MPF.3.2, the package position is measured from the outer boundary of the marker. This was logged for 14 flights and the system failed to meet the requirement only during one trial. The trials that show distance as 0 mean that the package was dropped on the marker.
Delivery Subsystem
There were 5 mandatory performance requirements for the delivery subsystem requirements that if met, would ensure that we are able to reliably deliver small high-value packages without landing. We encountered great success in meeting all performance requirements, as highlighted below.
MPF.4.1 Ensure we can deliver eggs without breaking them.
MPF.4.3 Not drop the package mid-flight 100% of the time.
MPF.4.5 Hold, lower, and release a package of at least 500 grams.
In validation testing of over 25 flights and 19 deliveries, we are proud to report 0 breakages and 0 accidental drops.
MPF.4.2 Release the package successfully 90% of the time.
MPF.4.4 Deliver package from at least 4 ft above the ground.
In 20 delivery validation attempts, we separated 19 times, yielding a success rate of 95%.
Our average delivery height was above 10.5ft, well above our performance requirement. We delivered from as high as 20ft in the air!
GPSND
MPF.2.1 Achieve a relative trajectory error of 3% or less during fixed wing modes.
To determine if we achieved MPF.2.1, we collected data from several different runs at both the Hawkins and Nardo flight locations, which represent urban and forested areas respectively. We then ran our GPSND system on the collected data and used the onboard GPS data as a ground truth to compare the output. The figure below shows the average relative trajectory error results from these runs for various segment lengths. The dotted red line represents the 3% bar, proving that for both urban and forested environments we are able to achieve the requirement we set.
