Pure Pursuit Controller (Local Planning)

We implemented a pure-pursuit local planning module for F1TENTH autonomy stack, which takes the planned path and current vehicle state as input, computes a steering command, and publishes it to the vehicle control module. This enables closed-loop path tracking for local navigation.

Test 1: Standalone Visualization

Acceptance Criteria:

• A dummy grid map and dummy planned path are visualized
• A dummy vehicle successfully tracks the path using the pure-pursuit controller
• Vehicle motion qualitatively follows the path with stable steering behavior

Test 2: ROS 2 Node Integration

Acceptance Criteria:

• Node successfully subscribes to:
  1. Dummy path publication topic
  2. Dummy current vehicle state publication topic
• Node publishes steering commands to a steering topic
• Steering output updates correctly in response to changing path or state inputs

Here’s the video: https://www.youtube.com/watch?v=aePewyiDYb8

Waypoint Operation

The video here shows the operation results.

Route Generation

This function focuses on developing a backend route planning module that generates and visualizes a global route based on user-defined waypoints. Unlike the Autoware planner, which only computes a trajectory toward the current goal point, this module aims to provide a full route preview by connecting all user-selected waypoints using the lanelet map.

The backend will:

• Receive waypoints from the frontend
• Compute a continuous route along the lanelet graph
• Return the route for visualization in the UI
• Provide additional route-level information such as distance and estimated time of arrival (ETA)

This module is designed as a lightweight planner for visualization purposes only, without handling dynamic obstacle avoidance or execution-level control.

The video here should routes that generate with different waypoints.

Skeleton Detection

This subsystem adds real-time skeleton-based perception to the PatrolKnight anomaly detection pipeline. It uses OpenPifPaf to detect human skeletons from either offline MP4 videos or a live Intel RealSense D435i camera stream, then applies IOU-based tracking to maintain consistent person IDs across frames. The system publishes structured skeleton metadata and annotated image outputs for downstream anomaly detection, while also supporting MP4 and JSON artifact export for testing, debugging, and performance evaluation.

Test 1: Pre-Trained Model Pipeline Execution

Acceptance Criteria:

• The command-line output reports valid inference timing information
• The pre-trained model runs successfully with zero runtime errors.

Test 2: Single Local Image Skeleton Detection

Acceptance Criteria:

• Successfully detect > 95% of non-occluded human skeletons (joints & connections).

Test 3: Local Video Skeleton Detection

Acceptance Criteria:

• The system successfully processes multiple video frames from a local MP4 input without crashing.
• The output shows consistent skeleton detection across frames, including human joints, skeleton connections, and tracked human subjects.
• Re-identify and maintain tracking of a human target even when occluded < 50 frames.

Here’s the result video.

Test 4: Live RealSense Camera Skeleton Detection

Acceptance Criteria:

• The skeleton detection node successfully receives and processes live image frames from the RealSense D435i camera.
• The system publishes live skeleton detection outputs with visible joints, skeleton connections, and structured metadata for downstream anomaly detection.

Here’s the result video.

Anomaly Detection

This subsystem aims to detect anomaly actions, with anomaly score and actions labeled. Certain delay with the detection is acceptable since we are not having real-time anomaly detection system. We adopted a two-stage pipeline to balance detection reliability with action classification interpretability. In Stage 1, we selected the Graph-Jigsaw Conditioned Diffusion Model (GJCD) as our anomaly scoring backbone. Stage 2 is conditionally triggered only when the Stage 1 score exceeds a configurable threshold, activating a zero-shot natural language classification pipeline. The final anomaly score is a weighted fusion of the Stage 1 reconstruction score and Stage 2 classification confidence, ensuring the expensive classification step is invoked selectively while preserving interpretability through human-readable action labels.

Test 1: Pre-Trained Model Pipeline Execution

• The command-line output reports valid inference timing information
• The pre-trained model runs successfully with zero runtime errors.

Test 2: Anomaly Action Classification

• Label the correct action on the screen with anomaly score

-> This test does not reach the acceptance criteria. Will use different approach

Current result can be viewed at: https://drive.google.com/file/d/1oNhZA2zpQJ_BmL-QVpjfwpnWBm2JlEtK/view?usp=drive_link