Project Description

Rapid growth in the worldwide market for warehouse automation and control systems is being driven by the global boom in e-commerce. Amazon is able to quickly package and ship millions of items to customers from a network of fulfillment centers all over the globe. Amazon sells 306 items per second and has 96 fulfillment centers across the United States. This would not be possible without leveraging cutting-edge advances in technology.

Amazon’s cutting edge automated Kiva shelves remove much of the walking and searching for items in the warehouses. However, commercially viable automated item selection and packaging in unstructured environments still remains a difficult challenge. In order to spur the advancement of these fundamental technologies, Amazon organized the first Amazon Picking Challenge (APC) at ICRA 2015. This competition aims to stimulate academic and industry interest in more generalized pick-and-place robotic systems with the ultimate of goal of automating the item retrieval task in Amazon’s order fulfilment process.

We are developing the Human Assistive Robotic Picker (HARP) as an entry to the competition 2016 APC. The goal of HARP is to enhance warehouse automation. Such a system would ensure faster processing of packaging and delivering items. HARP will require highly-sophisticated features, such as item identification and manipulation strategy in order to operate in dynamic environment and perform the core functions of item retrieval and item stowage. We will utilize intelligent perception, robust autonomous decision-making, a capable manipulation platform, and an innovative suction system. Team HARP is developing algorithms around the UR5 robotic arm to achieve this pick-and-place warehouse task. Universal Robots has generously made a UR5 available to Carnegie Mellon University for this competition.

Use Case

John’s final project demonstration for the MRSD fall validation experiment is due tomorrow. While running his final tests, the primary drive motor burnt out. With limited amount of time and no spares left, John thought he was out of luck. As a last resort, he logged onto to check how fact he could receive spare parts. Fortunately for John, Amazon recently implemented the Human Assistive Robotic Picker in its fully autonomous warehouses. John places his order and it is dispatched to a collection of robots in the warehouse. First, Kiva shelves autonomously drive from storage to their place in the order queue. This is where HARP comes into play.

HARP performs the task of grabbing items off shelves and boxing them for John. First, HARP parses John’s order and determines the item of interest. Next, sensors perceive the position and orientation of the drive motor on the shelf. Then a robotic arm strategically grabs the item off the shelf. Finally, HARP places the motor into the customer’s box.

In less than thirty minutes, Johns motor is out for delivery. Hours later, the box arrives on John’s doorstep, just in time to impress the professors before the demo.


John makes a purchase on Amazon. HARP boxes his item. It is out for delivery in under 30 minutes