{"id":9,"date":"2021-11-08T08:57:30","date_gmt":"2021-11-08T08:57:30","guid":{"rendered":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamh\/?page_id=9"},"modified":"2022-12-13T06:16:33","modified_gmt":"2022-12-13T06:16:33","slug":"implementation","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2022teamh\/implementation\/","title":{"rendered":"IMPLEMENTATION"},"content":{"rendered":"\n

FULL SYSTEM DEVELOPMENT<\/h2>\n\n\n\n
\"\"<\/figure>
\n

This is the overall system of TouRI. The platform is a Stretch RE1 designed and built  by Hello Robot. It consists of a mobile base for navigation and a 2 DOF manipulator with a 1 DOF  end effector. It consists of various sensors such as the Intel RealSense d435i camera for performing perception tasks. The RP-Lidar A1which we are using for navigation and obstacle avoidance. A  9DOF IMU which we are fusing with the Lidar and wheel encoders to get a better estimate of the  robot\u2019s position during localization. It has wheel encoders in both the base and the arm to get the  robot pose. There is also a Pan Tilt display that we are designing and building ourselves which  will serve as a video call interface for the remote user during a virtual tour. We are developing this  interface so that only the display will have to rotate about, instead of the entire robot when the  remote user wishes to interact with something for which he is required to turn in another direction.<\/p>\n<\/div><\/div>\n\n\n\n

<\/p>\n\n\n\n

\n\n\n\n

INTERFACE <\/h2>\n\n\n\n

FVD Update: All user screens for user interface have been developed. Integration of the interface with all the subsystems is completed. <\/p>\n\n\n\n

\n
\n
\"\"
User screen<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"\"
Physical interaction<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n
\"\"<\/figure>
\n

PROGRESS<\/strong><\/p>\n\n\n\n

\u2705  Tele-op navigation controls<\/p>\n\n\n\n

\u2705 Tele-op manipulation controls<\/p>\n\n\n\n

\u2705 Database-link<\/p>\n\n\n\n

\u2705 Bot-side architecture<\/p>\n\n\n\n

\u2705   Tele-op gimbal controls<\/p>\n\n\n\n

\u2705 Autonomous commands<\/p>\n\n\n\n

\u2705 Audio-video link [Completed in Fall]<\/p>\n\n\n\n

\u2705 Asynchronous operation of the pan-tilt display [Completed in Fall]<\/p>\n\n\n\n

\u2705 System integration [Completed in Fall]<\/p>\n\n\n\n

Key<\/strong><\/p>\n\n\n\n

\u2705 Finished  \u2611\ufe0f On-Going   \u23f3TODO<\/p>\n<\/div><\/div>\n\n\n\n

\n\n\n\n

PERCEPTION<\/h2>\n\n\n\n

The 2D detection pipeline and 3D centroid estimation pipeline for souvenirs and shipping box have been implemented. Following images describe the pipeline’s high level detail:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n\n\n\n

All the development planned and integration of the perception system with manipulation and interface has been completed. <\/p>\n\n\n\n

\u2705 Test 2D object detection pipeline and 3D preprocessing pipeline for perception sub-system <\/p>\n\n\n\n

\u2705 Collect training data and train the model for shipping box detection.<\/p>\n\n\n\n

\u2705 Collect training data and train the model for souvenir objects.<\/p>\n\n\n\n

\u2705 Implement dropping pipeline<\/p>\n\n\n\n

\u2705 Refine grasping pipeline for multiple objects<\/p>\n\n\n\n

\u2705 Integration of manipulation and perception subsystems<\/p>\n\n\n\n

\u2705 Integration of picking and placing pipeline<\/p>\n\n\n\n

\u2705 Test the integrated system as a standalone module.<\/p>\n\n\n\n

\u2705 Improve inference speed of 2D pipeline ( socket communication to GPU)<\/p>\n\n\n\n

\u2705 Speed up 3D perception pipeline by porting over code to C++<\/p>\n\n\n\n

\u2705 Test the entire system.<\/p>\n\n\n\n

Key<\/strong><\/p>\n\n\n\n

\u2705 Finished  \u2611\ufe0f On-Going   \u23f3TODO<\/p>\n\n\n\n

 <\/p>\n\n\n\n

\n\n\n\n

NAVIGATION<\/h2>\n\n\n\n

FVD Update : Navigation subsystem is integrated with the user interface and now the user selects the location he wants to tour using the app and the robot navigates to the desired location.<\/p>\n\n\n\n

\n
\n
\"\"
User selects the location to tour<\/figcaption><\/figure>\n<\/div>\n\n\n\n
\n
\"This
Autonomous Navigation<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n

SVD Update : The autonomous navigation subsystem has been set up on the robot and tested. The robot is currently able to localize itself within a map, plan a path to a goal location, and avoid obstacles on the way.<\/p>\n\n\n\n