Steward, along with its web-based user interface, Canopy, met nearly all of our FVD expectations.
For example, our goal was for foresters to plan a forest and operate Steward with less than a minute of training. During our FVD, we were able to give the controls directly to a forester, who was unacquainted with our system, and observe them as they planned their own forest within our test site.
Another goal was for Steward to plant autonomously, including navigation through the field, following a forest plan. We met this goal during FVD, where Steward was able to plant multiple seedlings without any human control input.
While Steward’s seedlings were unable to pass the so-called “tug test,” they were planted at an appropriate height in the ground, and both on a flat surface and on a slight slope.
At a high level, Steward did exactly what we designed it to: Act as a convenient assistant to foresters in their planning and planting tasks.
- Alignment with planned procedure
Our original procedure, as outlined in our FVD one pager, was:
- Ask a non-roboticist to use the UI to set the planting bounds and generate a Forest Plan.
- Load the robot with seedlings before planting sequence.
- Observe the robot as it plants the seedlings following the Forest Plan.
- Once the robot is finished planting, 5 seedlings should be manually verified by forestry experts.
During our demonstration, we performed items 1-3 exactly as outlined. While observing the robot, guests from the PA Bureau of Forestry were able to inspect each seedling that was planted. In other words, the manual inspection outlined in item 4 was performed synchronously with item 3, not after.
- Alignment with verification criteria
General planting success. Our primary goal during FVD was to test the behavior of the planting mechanism in a real field environment. Our primary verification criterion was therefore that each planting location within a forest plan should be planted with a success rate of 60% or higher. We defined “success” to mean that a hole should be made, and a seedling added to the hole, autonomously. Under these conditions, our system achieved a 100% success rate, all while following a forest plan that was generated by the forester during the demo.
Tug test. Our second verification criterion was that at least 40% of the manually verified seedlings should be deemed by forestry experts to be correctly planting, using the “tug test” as the primary indicator of correctness.
Usability of user interface. Our third verification criterion specified that the user interface, Canopy, should be usable by a non-roboticist after no more than one minute of training. We met this completely. One major reason was “Druid,” a virtual planning wizard included in Canopy that guided the user through the forest planning process step by step. Druid used a combination of spoken and written dialog to mimic conversation, eliminating the need for any training.
Connection update between Canopy and Steward. During the demonstration, we experienced no communication loss between the robot and the user’s laptop, meaning that we met the uptime criterion.
Trajectory planner update rate. Our trajectory planning node updated at 10 Hz during the demo, meeting this criterion.
Minimum planting distance. During our demonstration, Steward drove to, and planted, within one meter of each planting site in the generated forest plan, meeting this condition.
Zero uncaught/unhandled software failures. This was a challenging criterion to meet, especially given the complexity and interconnectedness of our core software. However, neither Steward nor Canopy encountered any software issues that affected their performance, and so the health indicator displayed to the user remained a green checkmark throughout the demonstration.
The table below highlights the performance requirements set for the Steward:
| No. | Requirement | Metric |
|---|---|---|
| 1 | Identifies locations of seedlings | Across 10 acres w/ 1m accuracy |
| 2 | Plant Seedlings | 30 seedlings / hour |
| 3 | Traverse Land | Up to 30 degree slopes |
| 4 | Avoid Obstacles | 10 hours without collision |
| 5 | Meet Forestry Standards | For 75% of planted seedlings |
| 6 | Operate with seedlings | Of height 12-38 cm |
| 7 | Returns planting locations | At least every 30 minutes |
Of these, during the spring Semester we targeted 2, 4, and 7. We also loosely targeted 1 and 3.
The following table shows the results from our Spring Testing and Spring Validation Demonstration:
The table below highlights the performance requirements set for the Steward:
| No. | Requirement | Metric | Results |
|---|---|---|---|
| 1 | Identifies locations of seedlings | Across 10 acres w/ 1m accuracy | Ground truth, 100% |
| 2 | Plant Seedlings | 30 seedlings / hour | Avg 40 seedlings / hour |
| 3 | Traverse Land | Up to 30 degree slopes | Max slop of 40 degrees |
| 4 | Avoid Obstacles | 10 hours without collision | Never collided (in sim) |
| 5 | Meet Forestry Standards | For 75% of planted seedlings | N/A |
| 6 | Operate with seedlings | Of height 12-38 cm | N/A |
| 7 | Returns planting locations | At least every 30 minutes | Instantaneously updates |