System Summary

Autonomous Multi-Robot Material Handling for Manufacturing Futures Insititute’s Lego Testbed

GitHub: https://github.com/DockDockGo/

Problem Description

CMU’s Manufacturing Futures Institute (MFI) is investigating how to best apply automation to the under-served High-mix / Low Volume manufacturing. This requires finding cost effective ways to manufacture products that have a large number of skews, but only low volume production runs, with the overall mission of making automation more accessible to small to medium size businesses. For financial feaasbility, one core requirement is flexibility, more specifically: the ability to quickly reconfigure an assembly line to manufacture different skews.

Requirement: Rapid Reconfiguribility

MFI has decided to invest resources into building out a tech showcase and research platform towards realizing this vision of automation in HMLV assembly at its new Mill19 facility in Hazelwood, Pittsburgh. The goal is to generate datasets to support manufacturing research themes in AI-based robot planning and manipulation, production digital twins, and orchestration of human labor.

The first part of this vision is the Lego testbed, which comprises of multiple work cells and kitting stations. Let’s look at what we mean by a testbed work cell.

A work cell performs either autonomous assembly or disassembly for the purpose of research data collection for MFI.

Another part of this vision are what we refer to as kitting stations. A kitting station is able to autonomously compile requested lego parts into a pallet, which we define as the structural foundation of a unit payload. It is a transport structure, which supports goods in a stable fashion while being moved. A kitting station is also responsible for sorting a pallet with an assortment of Lego bricks and putting these back into storage.

In order to realize its requirement of rapid reconfigurability of the testbed factory floor, our primary stakeholder: MFI decided to get rid of conventional conveyor belts entirely, and instead use automated mobile robots, or AMRs to move materials between the testbed work cells and kitting stations spread over the testbed factory floor.

This sets the scope of our MRSD project. Our project aims to build an Autonomous Material Handling System for MFI’s Lego Testbed, where we define material handling as the manipulation and movement of material pallets from a source location to a destination location within the testbed environment.

For further clarity, the tasks of kitting, assembly and disassembly are the responsibility of the testbed, NOT our system. The testbed plays the role of a customer, or user to our system.

Sample Use Case

In this section, we detail a typical use case of our system once the automated testbed (our system’s user and customer) is fully operational. 

  1. A testbed end user uploads a CAD file depicting a desired Lego model to be assembled.
  2. The testbed deconstructs the model into the types of Lego bricks required for the assembly and delegates the procurement of these Lego bricks to the testbed stock room.
  3. A robotic arm at the testbed stock room collects the multiple bins containing the required type of parts into a pallet (with a unique ID, say pallet#1), and places this pallet onto one of the multiple payload docks located in the stock room, let’s call this dock#1.
  4. At this point, the testbed sends an API request to our system to our system to transport pallet#1 located at dock#1 in the stockroom to dock#2, located beside the kitting station.
  5. Our system selects an AMR from the fleet that can service this task fastest based on where the AMR is located and whether it is idle or servicing a backlog of tasks.
  6. This AMR navigates to dock#1 (payload source), picks up the pallet#1 payload using on-board actuation capabilities, then navigates to dock#2 (payload destination), where it uses on-board actuation to drop off pallet#1, and communicates the successful completion of the task to the testbed.
  7. Next the testbed kitting station executes the task of kitting i.e. picking the required quantities of Lego bricks from each of the bulk storage bins transported from the stock room into a ‘tray’ that is configured with separators to prevent bricks of different types from mixing. This tray is placed into pallet#2, which is then placed at dock#3, ready for transport.
  8. At this point, the testbeds send an API request to our system to transport pallet#2 located at dock#3 to dock#4 located at a testbed assembly work cell.
  9. Similar to step (5), our system selects an AMR from the fleet that can service this task fastest. This AMR navigates to dock#3 (payload source), picks up the pallet#2 payload using on-board actuation capabilities, then navigates to dock#4 (payload destination), where it uses on-board actuation to drop off pallet#2, and communicates the successful completion of the task to the testbed.
  10. Additionally, the testbed also sends another task request to transport pallet#1 containing stockroom bins that have already been used by the kitting station, from dock#2 (at the kitting station) back to dock#1(in the stockroom). This task is also serviced by an appropriately assigned AMR in parallel.
  11. Next, the testbed assembly station uses the kit supplied in the tray transported in pallet#2 to assemble the requested lego model. This model can also be transported to different parts of the testbed factory floor, as the testbed sees fit, using the transportation infrastructure provided by our system, as demonstrated above.

Note: The use case text in gray above is part of the testbed project scope, while the text in black is within the scope of our system

Use case modifications for the Fall Validation Demonstration:

Given that the testbed project is under development in parallel with our system, we chose to decouple our validation demonstrations from the progress / timeline of the testbed project. In order to achieve this, we made the design decision to manipulate lego bricks using human operators instead of robotic arms like that completed testbed project will. 

Figure 2.1: ​​For our system validation demos, we will be instructing human operators to manipulate lego bricks at various testbed work cells via Human-Robot interaction terminals

This required us to assume additional responsibility for communicating manipulation instructions to human operators via human-robot interaction (HRI) terminals located at the testbed work cells. We retain the same API test request schemas such that our system requires no modifications when swapping out human operator serviced work cells with the fully automated testbed upon the testbed project completion, which we further test in a simulation environment.

As an example, this would result in step (3) from the use case above translating to the following steps in the Fall Validation Demonstration:

  • Assuming that the parts are available in the required quantities, the server directs the HRI system to instruct the human worker in the stockroom to place bins containing the requested parts in bulk quantities inside pallet#1.
  • Once the human has placed this pallet on dock#1 (also included in the HRI instructions), he/she confirms the task completion via manual input to the HRI.