Fall Validation Demonstrations

 

Test Name:

Fall Validation Demonstration

 Test Date: 11/25
Objective: Verify end-to-end robot system functionality
Elements Tested: Robot Platform, Navigation, Plant Health Monitoring, GUI
Location: Newell-Simon Hall
Equipment: Newell-Simon Hall

  • Server with GPU
  • ROS Bag from field visit
 Personnel:
  • Team E 
  • MRSD Advisors
Procedure:
  1. View the video of the Rivendale field test
  2. Use the exposure testing script to process the ROS Bag and find the percentage of images which pass an over/under exposed test. 
  3. Run plant health model inference on ROS Bag
  4. Software loads test images and computes performance metric
  5. Run visualization pipeline
  6. Software loads test images and visualizes the results in GUI
  7. Visually demonstrate the field’s layout
  8. Select a datapoint and change the classification, to demonstrate change in the stored data
  9. End the visualizer and relaunch it to show the same data
Verification Criteria:
  • Video correctly shows criteria for Test 1, Autonomous Navigation passed
  • The percentage of images that pass the exposure test should be > 75%
  • Robot successfully processes data at a rate faster

than one field per 24 hours (MR 11)

  • Robot successfully identifies fungus and holes

with greater than 80% precision and recall with unhealthy defined as positive 

            * precision = Tp / (Fp + Tp), recall = Tp / (Fn + Tp)

  • Clear depiction of the field layout and data presentation (subjective)
  • Successful use of interactive portions of the GUI
  • Successful data preservation on relaunch

 

Results

  • Navigation
    • Robot does not hit plant stems while navigating – view here
    • Robot traverses till the end of the first row – view here
    • Robot successfully switches between two navigable rows 4 out of 5 times – view here
  • Monitoring 
    • Robot successfully processes data at a rate faster than one field per 24 hours (MR 11)  – 11.25 hours max view here
    • Robot successfully identifies fungus with greater than 80% precision and recall – see below
    • Robot successfully identifies holes with greater than 80% precision and recall – see below

 

Binary Classification Performance
Plant Type Category Precision Recall
Broccolini Pest Hole 79% 100%
Fungus 95% 95%
Cabbage Pest Hole 78% 64%
Fungus 100% 91%
Curly Kale Fungus 91% 83%
  • User Interface
    • Clear depiction of the field layout and data presentation (subjective) – cleared with farmers
    • Successful use of interactive portions of the GUI – demonstrated live
    • Successful data preservation on relaunch of GUI – demonstrated live