{"id":265,"date":"2026-02-20T19:14:36","date_gmt":"2026-02-20T19:14:36","guid":{"rendered":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamg\/?page_id=265"},"modified":"2026-05-02T19:42:43","modified_gmt":"2026-05-02T19:42:43","slug":"system-implementation","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teamg\/system-implementation\/","title":{"rendered":"System Implementation"},"content":{"rendered":"\n

Labelling Subsystem<\/h2>\n\n\n\n

For our labelling subsystem, we utilize two Franka Emika Panda arms equipped with custom suction end-effectors. We have designated specific roles for each: the pick-and-place arm handles the introduction and removal of items from the workstation, while the labelling arm retrieves adhesive RFID tags from a dispenser and applies them to the items. <\/p>\n\n\n\n

Suction System<\/h3>\n\n\n\n

We developed a high-flow suction system powered by a Nessxa air compressor and a vacuum generator. This high-flow configuration is particularly effective for handling RFID labels, as their thin and flexible nature can make achieving a perfect air-tight seal difficult. The system compensates for minor leaks and more easily manipulates the lightweight labels. Initial testing in manual guide mode confirmed the pneumatic system’s ability to retrieve labels from the dispenser and utilize positive pressure for placement, and subsequent autonomous tests have verified the end-effector’s robustness.<\/p>\n\n\n\n

\n
\n
\"\"<\/figure>\n<\/div>\n\n\n\n
\n
\"\"<\/figure>\n<\/div>\n<\/div>\n\n\n\n

Labelling End-Effector: CAD model (left), 3D print (right)<\/p>\n\n\n\n

WWe also integrated a similar end-effector for the pick-and-place arm by leveraging a design from SNAAK, an MRSD team in the 2026 cohort. While this design is generally effective across various shapes and weights, we found that successful grasps on heavier objects require precise alignment with the center of mass. We address these requirements through our automated grasp pose generation logic.<\/p>\n\n\n\n

\n
<\/div>\n\n\n\n
\n
\"\"<\/figure>\n<\/div>\n\n\n\n
\n
\"\"<\/figure>\n<\/div>\n\n\n\n
<\/div>\n<\/div>\n\n\n\n

Pick-and-place End-Effector<\/p>\n\n\n\n

Motion Planning<\/h3>\n\n\n\n

The motion planning subsystem is responsible for generating collision-free paths through the labelling workspace for both the pick and place and labelling arms, enabling them to efficiently transport medical inventory and apply labels onto items. <\/p>\n\n\n\n

Since the primary goal of this subsystem is to ensure both arms can operate simultaneously within the workstation in a fast, repeatable, and collision-free manner, a waypoint-based planner was adopted for path generation.<\/p>\n\n\n\n

For low-level control, inverse kinematics (IK) is applied to each waypoint to obtain the required joint angles, which are then commanded to the onboard Franka controller to execute straight-line motion in configuration space. The joint configurations obtained from IK are additionally verified to be collision-free with respect to static obstacles, with dynamic inter-arm collisions handled separately by the state machine. If a configuration is found to be in collision, it is recomputed until a valid solution is obtained. All possible arm motions were validated to be collision-free, allowing the final system to transport medical inventory and apply labels at speeds significantly faster than would be achievable with more complex, sampling-based planners such as PRMs or RRTs.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Grasp Pose Generation – Input Zone<\/h3>\n\n\n\n

When an item enters the input zone, the system segments it using a combination of GroundingDINO and SAM. By merging these masks with RGB-D data from an ASUS Xtion PRO Live camera, we generate a point cloud of the top surface to calculate a best-fit plane. The system then computes a grasp pose at the segmentation mask’s centroid to ensure reliable object manipulation.<\/p>\n\n\n\n

\n
\n
\"\"<\/figure>\n\n\n\n

Segmented Object Masks<\/p>\n<\/div>\n\n\n\n

\n
\"\"<\/figure>\n\n\n\n

Depth Image from Xtion Camera<\/p>\n<\/div>\n<\/div>\n\n\n\n

\n
\n
\"\"<\/figure>\n\n\n\n

Object Point Clouds<\/p>\n<\/div>\n\n\n\n

\n
\"\"<\/figure>\n\n\n\n

Computed Grasp Pose<\/p>\n<\/div>\n<\/div>\n\n\n\n

Optimal Label Location – Labelling Zone<\/h3>\n\n\n\n

To ensure critical information remains visible to the end-user, our system must avoid occluding existing text or labels when applying new RFID tags to item packaging. We address this through a perception pipeline that processes RGB-D data from a top-down ZED Mini camera positioned above the labeling zone. After generating segmentation masks using GroundingDINO and SAM, the system computes a pixel-wise gradient magnitude to identify visual edges on the object’s surface, which will capture text and existing labels. A Euclidean Distance Transform (EDT) is then applied to a gradient image to determine the distance between each non-edge pixel and its nearest edge neighbor.<\/p>\n\n\n\n

By identifying local peaks in the EDT and filtering them via Adaptive Non-Maximal Suppression (ANMS), we derive candidate center locations that are intuitively the furthest from any visual clutter. Finally, we project a 40\u00d7120 pixel RFID label footprint onto these candidates to verify if a valid orientation exists where the label can be placed without overlapping any detected edges. The system then greedily selects the first valid center point for the final placement.<\/p>\n\n\n\n

\n
\n
\n
<\/div>\n\n\n\n
\n
\"\"<\/figure>\n\n\n\n

Original RGB Image<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n

\n
\n
\n
\"\"<\/figure>\n\n\n\n

Object Gradient Magnitudes<\/p>\n<\/div>\n\n\n\n

<\/div>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\n
\n
\"\"<\/figure>\n\n\n\n

Local Peaks of EDT<\/p>\n<\/div>\n\n\n\n

\n
\"\"<\/figure>\n\n\n\n

ANMS-Filtered Candidates <\/p>\n<\/div>\n\n\n\n

\n
\"\"<\/figure>\n\n\n\n

Final Label Location<\/p>\n<\/div>\n<\/div>\n\n\n\n

While utilizing gradient magnitude effectively highlights text and pre-existing labels, it also captures high-contrast color transitions and decorative graphics on the packaging. This sensitivity occasionally results in the algorithm being overly cautious or restrictive when identifying placement zones. We are continuing to iterate on this design to improve the system’s robustness and better distinguish critical information from purely aesthetic packaging elements.<\/p>\n\n\n\n

State Machine<\/h3>\n\n\n\n

The system utilizes the ROS SMACH (State Machine and Actionlib) library to manage the complex,concurrent workflows of the logistics and labeling arms. The architecture is designed as a hierarchical state machine, where a top-level concurrence container synchronizes two distinct sub-state machines:<\/p>\n\n\n\n