{"id":457,"date":"2019-12-11T18:02:34","date_gmt":"2019-12-11T18:02:34","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/?page_id=457"},"modified":"2019-12-11T18:43:07","modified_gmt":"2019-12-11T18:43:07","slug":"litter-localization","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/litter-localization\/","title":{"rendered":"Litter Localization"},"content":{"rendered":"<p><strong>Fall 2019 Implementation<\/strong><\/p>\n<p>The flight data \u2013 images, metadata (RTK GPS of the UAV, attitude, and altitude) is stored on the UAV onboard computer in a time-stamped folder via trigger of a ROS service call on the UAV onboard computer from the server. After the UAV scout operation, the rosbags are transferred to the server via wireless secure copy (scp).<\/p>\n<p>The figure below shows the localization pipeline. The rosbags from the UAV are processed to extract images, and metadata. The images are passed to the computer vision subsystem and corresponding litter pixel coordinates are obtained. Those pixel coordinates are then used to compute the world coordinates of the litter items using the formula in the second figure below.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-424\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/localization-1024x670.png\" alt=\"\" width=\"840\" height=\"550\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/localization-1024x670.png 1024w, https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/localization-300x196.png 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/localization-768x502.png 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/localization-1200x785.png 1200w, https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/localization.png 1483w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-435\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/equation.png\" alt=\"\" width=\"612\" height=\"181\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/equation.png 612w, https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/equation-300x89.png 300w\" sizes=\"auto, (max-width: 612px) 85vw, 612px\" \/><\/p>\n<ul>\n<li>K &#8211; camera intrinsics, which are obtained by calibrating the UAV onboard camera<\/li>\n<li>R \u2013 Rotation matrix of the camera on the UAV<\/li>\n<li>T &#8211; The x, y, z translation of the camera from the UAV center<\/li>\n<li>u, v &#8211; Litter pixels in an image<\/li>\n<li>X, Y, Z &#8211; Litter position in world coordinates (m)<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p>This process is repeated for every litter object detected in every image. This results in a series of localized points in the world coordinate frame (denoted by the gray \u2018+\u2019 marks in the figure below). As you can see, distinct clusters start to form as the same item in multiple images is localized to roughly the same real-world coordinate. Using this aggregated data, we use K-means clustering to determine how many clusters there are in the environment and compute their cluster centers. This gives us an estimated value of each litter object\u2019s actual location by taking the mean across multiple observations, thus reducing our error. The red marks in the figure denote the estimate litter locations, and as you can see, they are roughly in the center of the clusters.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-427\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/cluster.png\" alt=\"\" width=\"768\" height=\"899\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/cluster.png 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-content\/uploads\/sites\/36\/2019\/12\/cluster-256x300.png 256w\" sizes=\"auto, (max-width: 768px) 85vw, 768px\" \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Fall 2019 Implementation The flight data \u2013 images, metadata (RTK GPS of the UAV, attitude, and altitude) is stored on the UAV onboard computer in a time-stamped folder via trigger of a ROS service call on the UAV onboard computer from the server. After the UAV scout operation, the rosbags are transferred to the server &hellip; <a href=\"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/litter-localization\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Litter Localization&#8221;<\/span><\/a><\/p>\n","protected":false},"author":161,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-457","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-json\/wp\/v2\/pages\/457","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-json\/wp\/v2\/users\/161"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-json\/wp\/v2\/comments?post=457"}],"version-history":[{"count":3,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-json\/wp\/v2\/pages\/457\/revisions"}],"predecessor-version":[{"id":471,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-json\/wp\/v2\/pages\/457\/revisions\/471"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamf\/wp-json\/wp\/v2\/media?parent=457"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}