Fall 2026 Updates

Repeatable Button Pressing:

Button pressing was reworked for repeatability: the arm motion was tuned (path smoothing and speed tuning), onboard tracking with position holding keeps the target fixed while the arm moves, and the final approach was slowed with a one-second pause at the pre-grasp pose. The robot has been tested pressing real elevator buttons.

Arm planning was also optimized (offline measurements from five desktop runs):

Metric	Before	After
Total planning time	2194–2583 ms	682–838 ms
IK calls per plan	22	11
Goal residual	186 µm	39 µm

Open issue: the robot still occasionally misses the button. Root-cause analysis is ongoing; the next step is to collect success rates while separating perception error from control error. An apparent wall motion of 11.95 mm peak-to-peak with a 3.01° tilt was measured during pressing; timing, robot-model parameters and motor encoders remain hypotheses for its cause.

Spring 2026

Arm Visual Servoing for Button Pressing:

We utilize Inverse-Dynamics to control the upper body (the arms). The system ensures a good precision and accuracy.

Fig: Robot Pressing a Button with Language Prompt.

Fig. Visual servo arm pipeline.

Door Opening Task:

We regard door opening as a whole-body loco-manipulation task and handle it using a trained joint-level control policy with Reinforcement Learning (RL). The simulation environment is built in IsaacSim.

The policy is optimized to open slightly damped doors with a crash bar. It takes in proprioceptions (to capture humanoid’s state) and processed perceptions data (to capture door’s state) and outputs the desired joint positions of 29 joints. A PD controller is then implemente to eventually transfer the joint position to torques.

To alleviate the sim2real gap, we further create the door model in simulator to match the real door.

Fig : Simulation Environment: The humanoid initializing in front of the modeled door.