Fall Semester Performance

Our Fall Validation Demonstration (FVD) was highly successful. We were able to satisfy all the expected criteria and meet all of our performance requirements as well. The results are summarized in Table 3 below.

The criteria for our FVD are as follows:

1. Distinguish 75% of visible peppers from plants
2. Ensure 75% success in gripping and retrieving easily accessible peppers without dropping them.
3. Succeed in cutting the peduncle 90% of the time, given proper cutting mechanism placement.
4. Extract each pepper in under 5 minutes

The system succeeded in picking 8 out of 8 peppers with an average harvesting time of about 90 seconds per pepper. Therefore, the system fell well within the requirements we determined beforehand. Additionally, during the encore demonstration, we had the opportunity to test the full limitations and capabilities of our system. We were able to demonstrate handling clustered peppers, oriented peppers, and occluded peppers, which were out of scope for our requirements. Additionally, we were able to show that the system could avoid false positive actuated cuts by adding gripping feedback. The final results of our system are shown in the table below.

Spring Semester Performance

The aim of our spring validation demo (SVD) was to demonstrate the integrated perception, motion planning, and end effector subsystems autonomously harvesting 4 green bell peppers while mounted to a stationary workbench. A successful demonstration would thus position the team well to develop and integrate mobility into the already working system. The primary requirements to be satisfied by this demo would include:

1. Each successful harvest has a duration less than 5 minutes.
2. 75% of visible peppers are distinguished from plant
3. Visible damage is avoided for 90% of the picked peppers (cuts/bruises)
4. The peduncle of the desired pepper is successfully cut within 90% of attempts
5. Gripping and retrieving peppers is 90% successful without drops

The required equipment of the demo includes the Kinova Gen3 6 DOF arm, the end-effector subsystem and associate electrical components including our power supply, custom power distribution board (PDB), and the realsense RGBD camera, and our testbed setup.  The demo took place in the JPMorgan Chase & Co. AI Maker Space within the Tepper Quad.

Our system experienced a few shortcomings during both the SVD and Encore. During SVD, a bug in our software stack caused our visual servoing to falsely report failures. Thus, the determined point of interaction with the peduncle was not fine-tuned and the motion planning subsystem placed the end effector at a non-ideal position in the plant. The cutter mechanism directly contacted metal wire which was used to attach the peppers to the fake plant setup, and the motors stalled due to over-torquing. The overall process of pepper harvest was also found to be slower than the set requirements, with an average time of nearly 6 minutes.

To address these concerns, we made substantial optimization efforts to our software stack and were able to modify the set speed limitation of the Kinova arm to reduce harvesting times down to 2.5 minutes under ideal circumstances. A decision was also made to attach the green peppers to the fake plant setup via thread which is easily cuttable and does impose a threat of harm to the motors of the end effector. During the encore demonstration, however, the motors failed to receive communications due to an unrelated, undiagnosed issue that had not been seen in our extensive prior testing. Factory resetting the motors, which had previously been a reliable remedying action and was integrated into our system level state machine, had no effect during the demonstration. This issue propagated failures to three of our performance requirements by inflating harvesting times, causing failures in gripping, and causing damage to the pepper.  Still, while the integrated system performance had its shortcomings, our subsystem-level demonstrations and prior testing were able to satisfy a few of our requirements. The results are summarized in the table below.