Investigate application for AI/ML in our project

During our design brainstorming process, one of the AI/ML features we explored was robot-based suspect tracking and following. The motivation behind this idea was straightforward: if the robot detects a potentially dangerous individual but is not equipped to physically intervene, one possible way to improve community safety is for the robot to continue monitoring that individual’s movements while simultaneously notifying the operations center and law enforcement.

Our original autonomy pipeline was structured as:

UI → Global Planner → Local Planner → Pure Pursuit Controller

To support this new functionality, we proposed introducing a high-level behavior controller that could switch the robot between two operating modes:

• Normal behavior pipeline for routine patrol
• Emergency behavior pipeline for suspect tracking and following

In the proposed emergency pipeline, once a potential criminal is detected by the anomaly detection model, the system would use camera–LiDAR fusion to estimate the target’s global position. This would require transforming the detected person’s position from the camera frame to the vehicle baselink frame, and then into the global map frame or GNSS frame. Based on this estimated target position, control would switch from the standard path-following controller to a visual servoing controller.

This visual servoing module would use a PID-based control strategy to keep the target centered in the camera frame while maintaining an appropriate following distance. To preserve modularity in the overall software architecture, the controller would output the same type of low-level commands as the existing pure pursuit controller, namely steering angle and speed, so that it could interface cleanly with the existing chassis control system.

For future development, we also discussed extending this idea with reinforcement learning or imitation learning to handle more difficult scenarios, such as when the person being followed temporarily disappears from view after turning a corner or becoming occluded by walls or other obstacles.

Although the concept was promising, the company ultimately decided not to pursue it for this capstone project. Their main concern was that the feature would require major modifications to their already developed ACM autonomy control module. They also raised practical use-case concerns: for example, if a suspect crosses an obstacle or geofence that the vehicle cannot traverse, such as a flower bed or gated area, the robot would immediately lose the ability to continue following.

To address this limitation, we proposed a more advanced concept involving a multi-robot centralized control system, where another robot could take over the tracking task after the suspect moved beyond the first robot’s patrol zone. Under this approach, a central management system would reassign the follow mission to a second robot positioned in a more suitable area. However, this idea was also rejected because it significantly expanded the system scope beyond what was feasible for a capstone project.