2025 Team B

Software Instructions

Drone Autonomy System with Integrated Gimbal

Project Information

  • Class: Robotic Systems Development
  • Team: Team B, MRSD 2024
  • Team Members: Yufan Liu, Gweneth Ge, Joshua Pen, Jet Situ, Yi Wu
  • Description: Autonomous drone system with integrated gimbal control for target tracking and monitoring tasks

Introduction

This software system provides a complete solution for drone autonomy with integrated gimbal control. The system leverages ROS2 for communication between components, computer vision for object detection, and a multi-threaded architecture for efficient processing. The system includes components for UI visualization (Foxglove), gimbal control, camera stream processing, and object detection.

For detailed explanation of the software architecture, please refer to our GitHub repository.

Repository Structure

  • AirStack/: Air stack components for drone control
  • AirStack_GCS/: Ground control station components
  • humanflow/: ROS2-based gimbal control system
    • ros2_ghadron_gimbal/: Main gimbal control workspace
    • docker/: Docker configurations for the workspace
    • mcap_data/: Recorded data in MCAP format

Prerequisites and Installation

System Requirements

  • Docker and Docker Compose
  • ROS2 Humble
  • Ubuntu Linux (recommended)

Installing Docker and Docker Compose

# Install Docker
sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
sudo usermod -aG docker $USER

# Install Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/download/v2.18.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Cloning the Repository

git clone https://github.com/LifGorg/TeamB_mrsd_24.git
cd TeamB_mrsd_24

Network Configuration

Configure WIFI with the following IPv4 settings:

  • Manual address: 10.3.1.201
  • Netmask: 255.255.255.0
  • Gateway: 10.3.1.100

Set ROS domain ID:

export ROS_DOMAIN_ID=70

Verify connectivity:

ping 10.3.1.32
ros2 topic list

Docker Environment Setup

Robot Component

cd AirStack
docker compose --profile hitl build
docker compose --profile hitl up -d

Ground Control Station Component

cd AirStack_GCS
docker compose --profile hitl build
docker compose --profile hitl up -d

Gimbal Component

cd humanflow/ros2_ghadron_gimbal/dockers
docker compose -f docker-compose.yaml down
docker compose -f docker-compose.yaml up -d

Permission Settings

sudo chown -R $USER:$USER /home/dtc/humanflow/ros2_ghadron_gimbal
sudo chmod -R 755 /home/dtc/humanflow/ros2_ghadron_gimbal/install
sudo chmod -R 755 /home/dtc/humanflow/ros2_ghadron_gimbal/install/detect_and_track/lib/python3.10/site-packages

System Usage

Foxglove UI Setup

Source ROS2 environment:

source /opt/ros/humble/setup.bash

Run Foxglove bridge:

ros2 run foxglove_bridge foxglove_bridge

Development Workflow

  1. SSH into the remote machine:
ssh dtc@10.3.1.32
  1. Check Docker containers:
docker ps

Ensure that [robot_l4t] and [gimbal] containers are active.

  1. Set up local editing environment:
sshfs dtc@10.3.1.32:/home/dtc/humanflow [path to your desired working directory]/mount
  1. Connect to the Gimbal:
cd ~/mount/humanflow/ros2_ghadron_gimbal/src/inted_gimbal/
bash e_gim<Tab>  # to enter Docker
ros2 launch gim<tab>b<tab> g<tab>  # launches gimbal connection
  1. Build after editing:
# Remove existing build, install, log
./r_gimbal.sh
# Rebuild the selected packages
./b_gimbal.sh

Visualizing the Camera Stream

export ROS_DOMAIN_ID=70
cd Desktop/DroneSystem/
source install/setup.bash
python3 ros2_image.py

Gimbal Control Commands

Basic Control

# Send control commands (pitch=0, roll=0, yaw=90)
ros2 topic pub /gimbal_angles geometry_msgs/msg/Vector3 "{x: 0.0, y: 0.0, z: 90.0}"

Mode Control

# EO mode (Electro-Optical mode)
./eo_mode.sh
# IR mode (Infrared mode)
./ir_mode.sh
# Lock the gimbal
./lock.sh
# Point the gimbal downward
./pointdown.sh
# Return to home position
./homing.sh

System Architecture

Integrated-Gimbal Node Thread Layout

Stream Thread:

  • Uses GStreamer to receive frames through RTSP + TCP protocol at ~15 Hz
  • Provides frames for detection threads

Detection Thread:

  • Retrieves frames from the stream at 10 fps (reducing compute resource requirements)
  • Runs YOLO detection model
  • Estimates and publishes GPS coordinates
  • Publishes compressed images

Performance Optimizations

  • Frame Acquisition: Only the most recent decoded frame is kept in memory; earlier frames are discarded to prevent queue build-up
  • Parallel, Non-blocking Inference: Detection thread polls the latest frame, runs detection, then hands control back
  • Sensor Synchronization: Uses bounded search (max_sync_time_diff ≈ 0.3s) to attach fresh GPS/IMU samples to each frame
  • Publishing Strategy: Results are published with BEST_EFFORT reliability and queue depth 1
  • Profiling: Lightweight profiler records per-function timing, CPU usage, memory usage, and thread count

Troubleshooting

If you encounter issues with the gimbal connection, verify that:

  • The radio ethernet is correctly configured
  • All containers are running (docker ps)
  • The ROS domain ID is set correctly (export ROS_DOMAIN_ID=70)

Additional Notes

  • The pitch range of the gimbal is -90 (pointing down) to 90 (pointing up)
  • The yaw range of the gimbal is -120 (left) to 120 (right)
  • It’s recommended to change directions incrementally (e.g., by 10 degrees each time) to avoid gimbal lock
  • Main gimbal control path: /home/dtc/humanflow/ros2_ghadron_gimbal/src/inted_gimbal/
  • The robot_l4t container uses NVIDIA runtime for GPU acceleration
  • Both the robot_l4t and ground control station containers use host network mode for direct communication