Functional Architecture

Cyberphysical Architecture

Full System Design Description

Labelling subsystem

The labelling subsystem is responsible for detecting incoming medical items, identifying their attributes and applying RFID labels using robotic manipulation. This subsystem ensures that every item entering the supply chain is uniquely identifiable and trackable. This system enables fully automated operation while maintaining high accuracy and generalizability across diverse item shapes and materials. Manual labelling introduces human error and increases labor costs, while conveyor systems lack flexibility for irregular, deformable or randomly oriented items.

Perception Module

Detects and isolates individual items from clutter. It takes multi-camera RGB-D images and uses them for item segmentation, grasp pose estimation and finding optimal label location. Multi-camera setups allow us to deal with occlusions easily, making the algorithm far more robust and improving grasp success rates. We selected RGB-D cameras instead of RGB-only cameras because depth information significantly improves segmentation accuracy and grasp planning reliability.

Manipulation Module

Computes safe trajectories and grasp configurations and executes pick-and-place and labelling motions. We select algorithms for the Franka arm such that the accuracy and repeatability of the arm is high, ensuring consistent and reliable label placement. We selected the Franka robotic arm due to its high precision, repeatability and safe human-robot interaction capabilities. A single-arm system reduces throughput and limits dexterity, while gantry systems lack the flexibility needed to handle diverse object geometries and orientations.

Tracking subsystem

The tracking subsystem monitors labelled inventory in the OR supply room using fixed RFID infrastructure and maintains real-time inventory awareness. We selected fixed RFID tracking because it enables continuous, automatic detection without requiring human intervention or line-of-sight. Compared to computer vision tracking, RFID is more reliable in cluttered environments and does not require direct visibility. Mobile RFID readers were also considered but discarded due to decreased performance under occlusion, latency and obstruction risks in hospital environments.

RFID Localization Module

Detects presence of tagged items and computes item spatial position. It uses the signal strength and phase angle from multiple antennas to estimate item location. This allows the system to determine which shelf or region an item occupies. We selected passive RFID tags because they are far more cost-effective per unit while also being reasonably accurate for localization. Alternative tracking methods such as barcodes were rejected because they require line-of-sight scanning and manual intervention.

Inventory Management Module

Updates inventory counts, location, and notifies staff of shortages or expiry. The RFID information obtained from the localization module is processed and stored in a central database, which provides real-time inventory tracking. This information is presented through a customer-friendly UI, allowing hospital staff to monitor inventory status and receive alerts.

Hardware

We use a bimanual Franka setup with multiple RGB-D cameras, custom grippers, and pneumatic systems. The dual-arm setup allows one arm to manipulate objects while the other applies labels, improving efficiency and flexibility. The custom grippers and pneumatic systems were designed to handle deformable, translucent and irregular medical items without damaging them. We also use passive RFID labels, including specialized tags for metal and liquid items, and a label dispenser for fast and reliable labelling. Fixed RFID readers and antennas are installed in the supply room to enable continuous tracking.