This semester, our team aimed to achieve all system-level requirements, allowing for some flexibility. Our goal was to achieve all of the desired requirements with a lenient view on some of the parameters. For example our desired FPS was 30hz but we were able to achieve around 20-22 FPS. A detailed list of our targeted requirements is given in table 5.
The comprehensive system encompasses our sensor suite, consisting of off-board infrastructure cameras and on-board proximity sensors, alongside the G29 controller. Our software stack comprises several subsystems operating collaboratively. The data processing and Bird’s Eye View (BEV) generation pipeline is tasked with delivering a precise and responsive BEV. Simultaneously, the controls pipeline captures user inputs and channels them through a safety mechanism before implementation. The UI/UX module serves as the integration hub, providing users with an accurate visualization enriched with advanced visual overlays for teleoperation. Figure 1 depicts the overall flow of the system.
Figure 1: System OverviewSVD
FVD System Performance
Mechanical/Environmental Subsystem Testing
Test
Success Criteria
Outcome/Adjustment
Track Assembly Dimensional Check
Lane widths, parking spots within ±5% of design specs to ensure scaled realism.
Achieved ±3% accuracy. Sufficient for both teleop and autonomous runs.
Camera Coverage Verification
All cameras cover the entire track with overlapping fields of view, no blind spots.
Full coverage confirmed. Key areas (lanes, parking spots) always visible.
Marker Visibility Test
ArUco markers visible in all camera feeds for consistent calibration in both semesters.
All markers clearly detectable. Calibration routine succeeded as planned.
Ensuring flatness with bubble level & shims
Measured height variance & added shims to reduce unevenness
Reduced vertical variance to <2 mm, aiding stable motion
RC Car Platform Testing
Test
Success Criteria
Outcome/Adjustment
Servo Steering Response
Smooth proportional steering response to teleop inputs in Spring and planner commands in Fall.
Servo angles matched commanded inputs precisely in both modes. Stable control.
ESC Speed Control
ESC modulates motor speed linearly, maintaining ~10 cm/s average speed if requested.
Achieved stable speed control. Operator and planner targets met consistently.
Wireless Link Stability Check
Round-trip latency <200 ms for quick operator feedback (Spring) and stable autonomous updates (Fall).
Latency ~100–150 ms. Sufficient for real-time teleop and responsive autonomous corrections.
Proximity Sensor Trigger Test
Detect obstacles at ~15 cm and halt motions in both teleop and autonomous modes.
Car halted as soon as the obstacle detected. Overrides worked identically in both modes.
Sensing and Communication Testing
Test
Success Criteria
Outcome/Adjustment
BEV Generation Using Classical Perspective
BEV should accurately depict track from top-down with minimal distortion and correct alignment of ≥90% features.
BEV captured environment details within camera FOV. Alignment achieved; key lane features visible and consistent.
BEV Stitching Using Feature Matching
Stitched BEV from multiple cameras must present a seamless panorama without visible stitching artifacts.
Stitched BEV showed smooth transitions between camera frames. No major artifacts or misalignments observed.
Predict Trajectories from Odometry Data
Depth information should enhance obstacle shape perception, improving recognition of obstacles.
When depth sensor was tested, point cloud matched expected object shapes, improving obstacle cues.
Adding Depth Data
Measured height variance & added shims to reduce unevenness
Reduced vertical variance to <2 mm, aiding stable motion
Safety Checking with Visual Overlays
Visual warnings and overlays must appear when obstacles or lane departures threaten the car’s planned path.
Warnings appeared at ~15 cm proximity. Operator reported improved situational awareness.
Mechanical/Environmental Subsystem Testing
Test
Success Criteria
Outcome/Adjustment
Teleoperation Input Response Test
Car responds smoothly to operator’s steering/throttle (Spring scenario).
Operators noted prompt and stable response, no jerky motions.
Autonomous Waypoint Following
Car follows planner-generated waypoints within ±10 cm final position error (Fall scenario).
Achieved ~7 cm final position error. Planner corrections applied smoothly.
Safety Override on Obstacle
Car halts if object <15 cm away in both modes.
Immediate halt observed. Safety overrides worked seamlessly in teleop and autonomous runs.
Demonstrated Performance Requirements for Spring Semester
Procedure
Success Criteria
Requirements Satisfied
Demo 0: Operator selects one car in autonomous mode and issues a desired waypoint.
The car moves towards the selected waypoint in accordance with the generated path within the preset tolerance.
PR4, PR5
Demo 1: Operator selects the other car in autonomous mode and issues a desired waypoint. The second car is then teleoperated to a different parking lot.
The first car moves towards the selected waypoint in accordance with the generated path within the preset tolerance. The second car responds to teleoperation commands and moves towards its goal as per operator’s discretion in a smooth manner.
PR1, PR2, PR3, PR4, PR5
Demo 2:Operator selects both cars in autonomous mode and issues desired waypoints for both cars.
Both cars move towards the selected waypoint in accordance with the generated path within the preset tolerance.
PR2, PR3, PR4, PR5
Demo 3: Operator selects desired waypoints for cars when such that there are static obstacles present in its path.
Both cars move towards the selected waypoint in accordance with the generated path within the preset tolerance while avoiding obstacles.
PR2, PR3, PR5,
Demo 4:Operator selects desired waypoint for one car in autonomous mode and teleoperates the other car as a dynamic obstacle in the environment.
The first car moves towards the selected waypoint in accordance with the generated path within the preset tolerance while avoiding the dynamic obstacle behavior.
PR5, PR6
Strengths and Weaknesses:
Strengths:
Precise Teleoperation: The system excels in providing precise control over vehicles, enabling accurate navigation and parking maneuvers. This precision ensures reliability and operator confidence during teleoperation.
Consistent Car Detection and Tracking: The system reliably detects and tracks vehicles in real-time, which is critical for ensuring smooth teleoperation and autonomous navigation. The accuracy of the tracking has significantly contributed to overall system performance.
Minimal Latency in Teleoperation: Operators experience minimal delay between input commands and vehicle responses, enhancing real-time control and maneuverability. This low latency ensures that the teleoperation system performs efficiently, even in demanding scenarios.
Informative Visualizations for Operators: The system provides clear and meaningful visualizations to operators, aiding in decision-making and situational awareness. These visualizations ensure that operators can monitor vehicle performance and surroundings effectively.
Time-Synchronized Systems: All system components are time-synchronized, enabling seamless coordination across different subsystems. This synchronization minimizes discrepancies and ensures smooth overall operation, particularly when transitioning between different modes of control.
Weaknesses:
Noticeable Latency in Autonomous Control: During autonomous control, the system experiences latency ranging between 400 to 600 milliseconds, which impacts localization accuracy. This delay can lead to errors in vehicle positioning, ultimately affecting performance during autonomous navigation.
Swerving Motion: The localization system has a lag in updates and is affected by noise, which results in a swerving motion during vehicle control. This issue can compromise the stability and accuracy of vehicle movement, especially during high-speed maneuvers.
Unreliable Collision Avoidance: The current approach to collision avoidance is not fully integrated across teleoperation and autonomous control modes. In autonomous mode, obstacles are added to the costmap for avoidance, but this method allows the car to move very close to obstacles (“brushing motion”). In teleoperation mode, collision avoidance relies solely on ultrasonic sensors, which are prone to noise and may fail to detect obstacles with irregular or non-flat surfaces. This issue can be mitigated by using higher-quality proximity sensors and integrating collision avoidance strategies across both control modes.
User Experience: While the system is functionally robust, there is room for improvement in the user interface and overall user experience. Enhancements could include a more intuitive interface design, better visualization for multiple vehicles, and improved ease of use to optimize operator satisfaction. Additionally, the interface requires updates to support operations involving more than two vehicles, ensuring scalability and ease of management in future implementations.
