February Progress Update

Dataset collection and pre-processing

Relevant datasets for training object detection models were sourced from the Internet. All the datasets were inspected for any wrong labels and were converted into the MS-COCO format. Following are sample images from the datasets found:

1. Kaggle Dataset (Fig 1, Fig 2)

2. TomatOD Dataset (Fig 3, Fig 4) [1]

3. Rob2Pheno Dataset (Fig 5, Fig 6) [2]

4. Tomato Flower Dataset (Fig 7, Fig 8) [3]

March Progress Update

Training pipeline

As shown in Fig 9 and 10, a model was used and modified to enable the detection of tomato flowers and fruits. The bounding box detection was first performed using a laptop and then deployed to an NVIDIA Jetson.

Pose estimation

The pose of the fruits and flowers must be estimated so as to find the location at which the robot should stop and perform the action.

The bounding boxes have been clustered and the centroid has been calculated.

References:

[1] ”Tsironis V., Bourou S., Stentoumis C. (2020). tomatOD: Evaluation of object detection
algorithms on a new real-world tomato dataset. In ISPRS – International Archives of
the Photogrammetry, Remote Sensing and Spatial Information Sciences. Available from
https://github.com/up2metric/tomatOD ”

[2] Afonso, Manya; Fonteijn, Hubert; Polder, G. (Gerrit); Wehrens, Ron; Lensink,
Dick; Faber, Nanne; et al. (2021): Rob2Pheno Annotated Tomato Image Dataset.
4TU.ResearchData. Dataset. https://doi.org/10.4121/13173422.v3

[3] Oppenheim, Dor Shani, Guy Edan, Yael. (2020). Tomato Flower Detection Using Deep
Learning. 10.13140/RG.2.2.19486.56647.

Fall update-

• Improved Clustering

• Point Cloud-based Stem Localization