{"id":160,"date":"2024-05-03T20:30:19","date_gmt":"2024-05-03T20:30:19","guid":{"rendered":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teamc\/?page_id=160"},"modified":"2024-12-13T05:16:15","modified_gmt":"2024-12-13T05:16:15","slug":"system-performance","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teamc\/system-performance\/","title":{"rendered":"SVD System Performance"},"content":{"rendered":"\n

This semester, our team aimed to achieve all system-level requirements, allowing for some flexibility. Our goal was to achieve all of the desired requirements with a lenient view on some of the parameters. For example our desired FPS was 30hz but we were able to achieve around 20-22 FPS. A detailed list of our targeted requirements is given in table 5.<\/p>\n\n\n\n

Targeted Performance Requirements<\/strong><\/td><\/tr>
PR ID<\/td>Requirement<\/td>Subsystem<\/td>Status<\/td><\/tr>
PR1<\/td>Update Frequency of 10 hz<\/td>Perception<\/td>Passed <\/td><\/tr>
PR2<\/td>Successful parking is detected: 80% <\/td>Perception<\/td>Passed<\/td><\/tr>
PR3<\/td>Stationary tolerance: Y: \u58eb5 cm; X: \u58eb10 cm; Rotational \u58eb20; Degrees<\/td>Controls<\/td>Passed<\/td><\/tr>
PR4<\/td>Vehicle speed should be around 10 cm\/s<\/td>Controls<\/td>Passed<\/td><\/tr>
PR5<\/td>Safety Tolerance: 10 cm<\/td>Controls<\/td>Passed<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Overall System Description<\/h3>\n\n\n\n

The comprehensive system encompasses our sensor suite, consisting of off-board infrastructure cameras and on-board proximity sensors, alongside the G29 controller. Our software stack comprises several subsystems operating collaboratively. The data processing and Bird’s Eye View (BEV) generation pipeline is tasked with delivering a precise and responsive BEV. Simultaneously, the controls pipeline captures user inputs and channels them through a safety mechanism before implementation. The UI\/UX module serves as the integration hub, providing users with an accurate visualization enriched with advanced visual overlays for teleoperation. Figure 1 depicts the overall flow of the system.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Figure 1: System Overview<\/strong>SVD<\/strong><\/p>\n\n\n\n

FVD System Performance<\/h1>\n\n\n\n
Mechanical\/Environmental Subsystem Testing<\/strong><\/td><\/tr>
Test <\/td>Success Criteria <\/td>Outcome\/Adjustment <\/td><\/tr>
Track Assembly Dimensional Check<\/td>Lane widths, parking spots within \u00b15% of design specs to ensure scaled realism.<\/td>Achieved \u00b13% accuracy. Sufficient for both teleop and autonomous runs.<\/td><\/tr>
Camera Coverage Verification<\/td>All cameras cover the entire track with overlapping fields of view, no blind spots.<\/td>Full coverage confirmed. Key areas (lanes, parking spots) always visible.<\/td><\/tr>
Marker Visibility Test<\/td>ArUco markers visible in all camera feeds for consistent calibration in both semesters.<\/td>All markers clearly detectable. Calibration routine succeeded as planned.<\/td><\/tr>
Ensuring flatness with bubble level & shims<\/td>Measured height variance & added shims to reduce unevenness<\/td>Reduced vertical variance to <2 mm, aiding stable motion<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
RC Car Platform Testing<\/strong><\/td><\/tr>
Test <\/td>Success Criteria <\/td>Outcome\/Adjustment <\/td><\/tr>
Servo Steering Response<\/td>Smooth proportional steering response to teleop inputs in Spring and planner commands in Fall.<\/td>Servo angles matched commanded inputs precisely in both modes. Stable control.<\/td><\/tr>
ESC Speed Control<\/td>ESC modulates motor speed linearly, maintaining ~10 cm\/s average speed if requested.<\/td>Achieved stable speed control. Operator and planner targets met consistently.<\/td><\/tr>
Wireless Link Stability Check<\/td>Round-trip latency <200 ms for quick operator feedback (Spring) and stable autonomous updates (Fall).<\/td>Latency ~100\u2013150 ms. Sufficient for real-time teleop and responsive autonomous corrections.<\/td><\/tr>
Proximity Sensor Trigger Test<\/td>Detect obstacles at ~15 cm and halt motions in both teleop and autonomous modes.<\/td>Car halted as soon as the obstacle detected. Overrides worked identically in both modes.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
<\/strong>Sensing and Communication Testing<\/strong><\/td><\/tr>
Test <\/td>Success Criteria <\/td>Outcome\/Adjustment <\/td><\/tr>
BEV Generation Using Classical Perspective<\/td>BEV should accurately depict track from top-down with minimal distortion and correct alignment of \u226590% features.<\/td>BEV captured environment details within camera FOV. Alignment achieved; key lane features visible and consistent.<\/td><\/tr>
BEV Stitching Using Feature Matching<\/td>Stitched BEV from multiple cameras must present a seamless panorama without visible stitching artifacts.<\/td>Stitched BEV showed smooth transitions between camera frames. No major artifacts or misalignments observed.<\/td><\/tr>
Predict Trajectories from Odometry Data<\/td>Depth information should enhance obstacle shape perception, improving recognition of obstacles.<\/td>When depth sensor was tested, point cloud matched expected object shapes, improving obstacle cues.<\/td><\/tr>
Adding Depth Data<\/td>Measured height variance & added shims to reduce unevenness<\/td>Reduced vertical variance to <2 mm, aiding stable motion<\/td><\/tr>
Safety Checking with Visual Overlays<\/td>Visual warnings and overlays must appear when obstacles or lane departures threaten the car\u2019s planned path.<\/td>Warnings appeared at ~15 cm proximity. Operator reported improved situational awareness.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
Mechanical\/Environmental Subsystem Testing<\/strong><\/td><\/tr>
Test <\/td>Success Criteria <\/td>Outcome\/Adjustment <\/td><\/tr>
Teleoperation Input Response Test<\/td>Car responds smoothly to operator\u2019s steering\/throttle (Spring scenario).<\/td>Operators noted prompt and stable response, no jerky motions.<\/td><\/tr>
Autonomous Waypoint Following<\/td>Car follows planner-generated waypoints within \u00b110 cm final position error (Fall scenario).<\/td>Achieved ~7 cm final position error. Planner corrections applied smoothly.<\/td><\/tr>
Safety Override on Obstacle<\/td>Car halts if object <15 cm away in both modes.<\/td>Immediate halt observed. Safety overrides worked seamlessly in teleop and autonomous runs.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
Demonstrated Performance Requirements for Spring Semester<\/strong><\/td><\/tr>
Procedure <\/td>Success Criteria <\/td>Requirements Satisfied <\/td><\/tr>
Demo 0: Operator selects one car in autonomous mode and issues a desired waypoint.<\/td>The car moves towards the selected waypoint in  accordance with the generated path within the preset tolerance. <\/td>PR4, PR5<\/td><\/tr>
Demo 1: Operator selects the other  car in autonomous mode and issues a desired waypoint. The second car is then teleoperated to a different parking lot.<\/td>The first car moves towards the selected waypoint in  accordance with the generated path within the preset tolerance. The second car responds to teleoperation commands and moves towards its goal as per operator’s discretion in a smooth manner.<\/td>PR1, PR2, PR3, PR4, PR5<\/td><\/tr>
Demo 2:Operator selects both cars in autonomous mode and issues desired waypoints for both cars.<\/td>Both cars move towards the selected waypoint in  accordance with the generated path within the preset tolerance. <\/td>PR2, PR3, PR4, PR5<\/td><\/tr>
Demo 3: Operator selects desired waypoints for cars when such that there are static obstacles present in its path.<\/td>Both cars move towards the selected waypoint in  accordance with the generated path within the preset tolerance while avoiding obstacles.<\/td>PR2, PR3, PR5, <\/td><\/tr>
Demo 4:Operator selects desired waypoint for one car in autonomous mode and teleoperates the other car as a dynamic obstacle in the environment.<\/td>The first car moves towards the selected waypoint in  accordance with the generated path within the preset tolerance while avoiding the dynamic obstacle behavior.<\/td>PR5, PR6<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Strengths and Weaknesses:<\/h2>\n\n\n\n

Strengths:<\/strong><\/p>\n\n\n\n

    \n
  • Precise Teleoperation: The system excels in providing precise control over vehicles, enabling accurate navigation and parking maneuvers. This precision ensures reliability and operator confidence during teleoperation.<\/li>\n\n\n\n
  • Consistent Car Detection and Tracking: The system reliably detects and tracks vehicles in real-time, which is critical for ensuring smooth teleoperation and autonomous navigation. The accuracy of the tracking has significantly contributed to overall system performance.<\/li>\n\n\n\n
  • Minimal Latency in Teleoperation: Operators experience minimal delay between input commands and vehicle responses, enhancing real-time control and maneuverability. This low latency ensures that the teleoperation system performs efficiently, even in demanding scenarios.<\/li>\n\n\n\n
  • Informative Visualizations for Operators: The system provides clear and meaningful visualizations to operators, aiding in decision-making and situational awareness. These visualizations ensure that operators can monitor vehicle performance and surroundings effectively.<\/li>\n\n\n\n
  • Time-Synchronized Systems: All system components are time-synchronized, enabling seamless coordination across different subsystems. This synchronization minimizes discrepancies and ensures smooth overall operation, particularly when transitioning between different modes of control.<\/li>\n<\/ul>\n\n\n\n

    Weaknesses:<\/strong><\/p>\n\n\n\n

      \n
    • Noticeable Latency in Autonomous Control: During autonomous control, the system experiences latency ranging between 400 to 600 milliseconds, which impacts localization accuracy. This delay can lead to errors in vehicle positioning, ultimately affecting performance during autonomous navigation.<\/li>\n\n\n\n
    • Swerving Motion: The localization system has a lag in updates and is affected by noise, which results in a swerving motion during vehicle control. This issue can compromise the stability and accuracy of vehicle movement, especially during high-speed maneuvers.<\/li>\n\n\n\n
    • Unreliable Collision Avoidance: The current approach to collision avoidance is not fully integrated across teleoperation and autonomous control modes. In autonomous mode, obstacles are added to the costmap for avoidance, but this method allows the car to move very close to obstacles (\u201cbrushing motion\u201d). In teleoperation mode, collision avoidance relies solely on ultrasonic sensors, which are prone to noise and may fail to detect obstacles with irregular or non-flat surfaces. This issue can be mitigated by using higher-quality proximity sensors and integrating collision avoidance strategies across both control modes.<\/li>\n\n\n\n
    • User Experience: While the system is functionally robust, there is room for improvement in the user interface and overall user experience. Enhancements could include a more intuitive interface design, better visualization for multiple vehicles, and improved ease of use to optimize operator satisfaction. Additionally, the interface requires updates to support operations involving more than two vehicles, ensuring scalability and ease of management in future implementations.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"

      This semester, our team aimed to achieve all system-level requirements, allowing for some flexibility. Our goal was to achieve all of the desired requirements with a lenient view on some of the parameters. For example our desired FPS was 30hz but we were able to achieve around 20-22 FPS. A detailed list of our targeted requirements is given in table […]<\/p>\n","protected":false},"author":352,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-160","page","type-page","status-publish","hentry","clearfix"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teamc\/wp-json\/wp\/v2\/pages\/160","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teamc\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teamc\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teamc\/wp-json\/wp\/v2\/users\/352"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teamc\/wp-json\/wp\/v2\/comments?post=160"}],"version-history":[{"count":4,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teamc\/wp-json\/wp\/v2\/pages\/160\/revisions"}],"predecessor-version":[{"id":203,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teamc\/wp-json\/wp\/v2\/pages\/160\/revisions\/203"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teamc\/wp-json\/wp\/v2\/media?parent=160"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}