Simulation

For the project, our team has been developing simulation environments for the team to test other components. Our team decided to use Nvidia’s Isaac Sim considering the recent traction it has gained in the industry and our willingness to learn about the simulator.

As our team plans to use AI Maker Space for the spring validation demo, we have replicated the AI Maker Space in our simulation setup and modified it to replicate an Operating Room environment. Unlike Gazebo, interfacing ROS 2 with the simulator is non-straightforward in Isaac Sim. We need to create Action Graphs that take the values from the simulation and publish/subscribe to ROS-2 messages.

Interfacing with Fetch

1. ROS1 and ROS2 Bridge Setup

This process involved configuring Docker containers to run both ROS versions simultaneously, which required meticulous attention to networking and compatibility issues to ensure seamless data exchange between the two ROS environments.

2. Fetch & Jetson Xavier Integration with ROS2

The integration of Fetch with the NVIDIA Xavier computing platform is a pivotal component of our project, aiming to enhance the robot’s processing capabilities and enable more complex computational tasks.

Vending Machine Setup

The team decided to use a vending-shelf module (as shown in the figure below), an enhancement to the manipulation subsystem of the ORBot project which serves as the augmented shelve and eases the process of collecting the required item from the shelves. The initiative involved a comprehensive design and construction process, starting with the creation of a CAD model tailored to the system’s requirements. Following the design phase, the team procured the necessary components from the MRSD inventory, including 8020 aluminum profiles for the structural framework and suitable motors for actuation.

In conjunction with the hardware development, the team programmed an ESP32 board to facilitate wireless communication with ROS2 nodes. This task was critical to integrate the hardware with the software stack and involved establishing a reliable WiFi connection for data transfer. The programming also included writing scripts to control a DC motor via a 1298n motor driver, ensuring precise operation of the vending shelf.