Design

Requirements

FR – Functional Requirement

PR – Performance Requirement

NF – Non-Functional Requirement

Functional Architecture

Input Handling: The robot autonomously parses the items in the list to generate an item plan. The input handling function uses an algorithm to select small and easy to grasp items and places them in the beginning of the list followed by larger and objects with no definite shape. This system keeps track of the objects modeled in the environment and ensures the system operates correctly and recovers from errors.

Perception: The perception function is responsible for scanning the shelf, scanning individual bins, determining item pose and providing the system with sufficient data to plan the manipulator trajectory to grasp the item from the shelf bin and place it in order bin.

Platform: The platform planner takes the item pose data as input and generates a valid collision-free motion plan to move the arm to a valid grasp position. Further, platform function also determines the reverse path to move the arm towards the order bin.

Grasping: the grasping function decides on the best grasp strategy and orients the end effector with respect to the object pose consisting of a suction system. Once the suction arm is close to the object, the grasping function switches on the suction mechanism and grasps the object

The functional architecture highlights the interaction between the four main functional areas. The user passes a Javascript Object Notation file which includes the item contents of all bins on the shelf and the desired order items. The robot generates an item plan to grasp the maximum number of objects in the desired timeframe. To do so, we plan to have an algorithm in place that assigns different weights such as ease of grasping, points associated etc. to each object. Finally input handling subsystem generates a work order that attempts to maximize the overall score.

Next, the perception system localizes the shelf and assigns it a frame in world coordinates. The details of the target item are passed to perception pipeline, where the Kinect2 RGBD sensor scans the shelf to determine item pose and passes the coordinates to the grasp planner. 

Once the perception pipeline returns the item pose, an ordered list of preferred grasps are generated from a precomputed set of valid grasp positions. The list is then sent to the arm planner and the arm planner iterates through the list to find a collision-free motion plan to the item. The suction system is activated and the motion plan is executed by the arm.

Once the motion plan is found, pressure sensor feedback provides the grasp status to the platform and once grasping is detected, the arm withdraws from the shelf. If the grasp was unsuccessful, the system will abort further attempts to grasp the current item and start on the next item in the list. Finally, the arm is moved over the order bin and the suction system is disengaged. The system iterates this loop of grasping and dropping until it has either picked up all items in the dictionary or the time limit has been reached. 

Physical Architecture

The physical architecture diagram shows the interaction of the robot with perception and suction subsystems and various components. As part of the global planner, we have a computer that runs on Ubuntu and ROS Indigo and takes care of the state controller including UR5 motion planning. The Gripper subsystem consists of the suction mechanism which is controlled by the ROS state controller through an Arduino microcontroller and relays. The pressure sensor also interacts with ROS state controller through the Arduino, using the ROSserial protocol and provides the grasp status feedback. On the perception subsystem, we are using the Kinect2 RGBD sensor which boasts higher resolution over the Kinect.

Software Architecture

The high-level software architecture diagram below explains the control and feedback mechanisms necessary in order to implment the functional architecture. The user input, in the form of a text file, is given to the master ROS controller, which begins the SMACH based state machine.

First, the master ROS controller passes in an item of interest which includes the specific bin number. The robot then aligns the Kinect2 and stitches several acquired point clouds together to improve the shelf scene resolution. The item recognition algorithms then determine the item position and orientation on the shelf. Specifically, using image segmentation techniques, bag of words classifiers, and search-based matching the approximate item location is determined. Using this data, the point cloud depth data is downsampled to the region of interest. Using ground truth 3D object data, the known geometry is fit to the scene. Algorithms available in OpenCV and the Point Cloud Library are used to simplify the vision task. This results in a position and orientation of the item of interest in world coordinates.

Finally, arm and item positions are sent to the grasp planner using trained methods of item acquisition, unique to each item. The arm planner uses the grasp list and finds a collision-free motion plan. A microcontroller is responsible for low level suction commands. Once the item is acquired, the ROS controller receives a grasp success signal from the grasp controller and then deposits the item into the order bin. This cycle repeats until all items from the input text file have been acquired.