{"id":154,"date":"2022-02-20T17:44:05","date_gmt":"2022-02-20T17:44:05","guid":{"rendered":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamg\/?page_id=154"},"modified":"2022-12-12T09:49:28","modified_gmt":"2022-12-12T09:49:28","slug":"perception","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamg\/implementation\/perception\/","title":{"rendered":"Perception"},"content":{"rendered":"\n

February Progress Update<\/strong><\/p>\n\n\n\n

Dataset collection and pre-processing<\/p>\n\n\n\n

Relevant datasets for training object detection models were sourced from the Internet. All the datasets were inspected for any wrong labels and were converted into the MS-COCO format. Following are sample images from the datasets found:<\/p>\n\n\n\n

1. Kaggle Dataset (Fig 1, Fig 2)<\/p>\n\n\n\n

\n
\"\"
Fig 1. Kaggle Dataset Images<\/figcaption><\/figure>\n\n\n\n
\"\"
Fig 2. Kaggle Dataset Images<\/figcaption><\/figure>\n<\/figure>\n\n\n\n

2. TomatOD Dataset (Fig 3, Fig 4) [1]<\/p>\n\n\n\n

\n
\"\"
Fig 3. TomatOD Dataset Images<\/figcaption><\/figure>\n\n\n\n
\"\"
Fig 4. TomatOD Dataset Images<\/figcaption><\/figure>\n<\/figure>\n\n\n\n

3. Rob2Pheno Dataset (Fig 5, Fig 6) [2]<\/p>\n\n\n\n

\n
\"\"
Fig 5. Rob2Pheno Dataset Images<\/figcaption><\/figure>\n\n\n\n
\"\"
Fig 6. Rob2Pheno Dataset Images<\/figcaption><\/figure>\n<\/figure>\n\n\n\n

4. Tomato Flower Dataset (Fig 7, Fig 8) [3]<\/p>\n\n\n\n

\n
\"\"
Fig 7. Tomato Flower Dataset Images<\/figcaption><\/figure>\n\n\n\n
\"\"
Fig 8. Tomato Flower Dataset Images<\/figcaption><\/figure>\n<\/figure>\n\n\n\n

March Progress Update<\/strong><\/p>\n\n\n\n

Training pipeline<\/p>\n\n\n\n

As shown in Fig 9 and 10, a model was used and modified to enable the detection of tomato flowers and fruits. The bounding box detection was first performed using a laptop and then deployed to an NVIDIA Jetson.<\/p>\n\n\n\n

\"\"
Fig 9. Fruit detection via head camera<\/figcaption><\/figure><\/div>\n\n\n\n
\"\"
Fig 10. Flower detection via head camera<\/figcaption><\/figure><\/div>\n\n\n\n

Pose estimation<\/p>\n\n\n\n

The pose of the fruits and flowers must be estimated so as to find the location at which the robot should stop and perform the action.<\/p>\n\n\n\n

The bounding boxes have been clustered and the centroid has been calculated.<\/p>\n\n\n\n

\"\"
Fig 11. Flower bounding box clustering<\/figcaption><\/figure><\/div>\n\n\n\n

References:<\/p>\n\n\n\n

[1] \u201dTsironis V., Bourou S., Stentoumis C. (2020). tomatOD: Evaluation of object detection
algorithms on a new real-world tomato dataset. In ISPRS – International Archives of
the Photogrammetry, Remote Sensing and Spatial Information Sciences. Available from
https:\/\/github.com\/up2metric\/tomatOD \u201d<\/p>\n\n\n\n

[2] Afonso, Manya\u037e Fonteijn, Hubert\u037e Polder, G. (Gerrit)\u037e Wehrens, Ron\u037e Lensink,
Dick\u037e Faber, Nanne\u037e et al. (2021): Rob2Pheno Annotated Tomato Image Dataset.
4TU.ResearchData. Dataset. https:\/\/doi.org\/10.4121\/13173422.v3<\/p>\n\n\n\n

[3] Oppenheim, Dor Shani, Guy Edan, Yael. (2020). Tomato Flower Detection Using Deep
Learning. 10.13140\/RG.2.2.19486.56647.<\/p>\n\n\n\n

Fall update-<\/p>\n\n\n\n

  • Improved Clustering<\/li><\/ul>\n\n\n\n
    • Point Cloud-based Stem Localization<\/li><\/ul>\n\n\n\n
      \"\"
      Stem localization<\/figcaption><\/figure>\n","protected":false},"excerpt":{"rendered":"

      February Progress Update Dataset collection and pre-processing Relevant datasets for training object detection models were sourced from the Internet. All the datasets […]<\/p>\n","protected":false},"author":281,"featured_media":0,"parent":146,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-154","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamg\/wp-json\/wp\/v2\/pages\/154","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamg\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamg\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamg\/wp-json\/wp\/v2\/users\/281"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamg\/wp-json\/wp\/v2\/comments?post=154"}],"version-history":[{"count":12,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamg\/wp-json\/wp\/v2\/pages\/154\/revisions"}],"predecessor-version":[{"id":727,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamg\/wp-json\/wp\/v2\/pages\/154\/revisions\/727"}],"up":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamg\/wp-json\/wp\/v2\/pages\/146"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamg\/wp-json\/wp\/v2\/media?parent=154"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}