Detailed Design of Systems:
Fall 2026: Full-System Integration
In the fall the team moved from individually validated skills to chaining them into a complete elevator mission. Semantic mapping was removed from scope after the team lost one member.
Simulation Testbed:
The RIC elevator was reconstructed from a phone video into a MuJoCo scene with a moving cabin, cabin controls and front/rear 120° FOV cameras. It is used to test rule-based and agentic skill chaining and recovery behavior before running on hardware.
- Single-agent framework (Oct. 1 snapshot): 163 tests passed with 0 failures; the offline baseline completed the elevator trip in 55.9 s of simulated time with 11 decisions. Only 4 of 20 VLM cases were answered because of API rate limits, and in one case the VLM proposed entering the elevator with the door only 69% open, which is unsafe.
- Rule-based baseline: Approach & Call → Enter → Select floor → Ride → Check arrival → Exit.
- Agentic chaining: skills aligned with the simulator (object tracking, navigation, arm control) plus new skills for state checks and recovery behavior. The first end-to-end rollout in simulation is complete; it has not yet been tested on the real robot.
Real-Robot Skill Chaining:
A hand-written baseline chain runs on the G1: go to the button panel → press the button → enter the elevator and go to the in-cabin button panel.
Fall Validation Demonstration Plan:
The FVD mission at RIC is fully automatic and long-horizon with basic failure recovery: navigate to the call panel → press the call button → enter the elevator → press the floor button → exit at the target floor.
| FVD measurement | Target |
|---|---|
| Tracking rate | ≥15 FPS sustained |
| VLM response | ≤2 s per query |
| Button pose | ≤1 cm at 2 of 3 positions |
| Button presses | ≥80% outside / ≥70% inside |
| Door state | ≥95% of cycles |
| Moving floor detection | ≥90% of transitions |
| End-to-end mission | ≥1 of 3 trials |
Next steps: fix the navigation local planner, improve the button-pressing success rate, test skill chaining and collect failure modes, and add failure-recovery logic to the behavior tree.