{"id":163,"date":"2026-04-03T20:35:28","date_gmt":"2026-04-04T01:35:28","guid":{"rendered":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/?page_id=163"},"modified":"2026-04-24T14:01:34","modified_gmt":"2026-04-24T19:01:34","slug":"control-system","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/control-system\/","title":{"rendered":"Control Sub-system"},"content":{"rendered":"\n<p><strong>Arm Visual Servoing for Button Pressing:<\/strong><\/p>\n\n\n\n<p>We utilize Inverse-Dynamics to control the upper body (the arms). The system ensures a good precision and accuracy.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1369\" height=\"913\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/04\/image-11.png\" alt=\"\" class=\"wp-image-216\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/04\/image-11.png 1369w, https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/04\/image-11-300x200.png 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/04\/image-11-768x512.png 768w\" sizes=\"auto, (max-width: 1369px) 100vw, 1369px\" \/><\/figure>\n\n\n\n<p><strong>Fig: Robot Pressing a Button with Language Prompt.<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1708\" height=\"2048\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/04\/image-10.png\" alt=\"\" class=\"wp-image-214\" style=\"aspect-ratio:0.8339914705417786;width:286px;height:auto\" title=\"Untitled Diagram.drawio (2).png\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/04\/image-10.png 1708w, https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/04\/image-10-250x300.png 250w, https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/04\/image-10-768x921.png 768w, https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/04\/image-10-1281x1536.png 1281w\" sizes=\"auto, (max-width: 1708px) 100vw, 1708px\" \/><\/figure>\n\n\n\n<p><strong>Fig. Visual servo arm pipeline.<\/strong><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><strong>Door Opening Task: <\/strong><\/p>\n\n\n\n<p>We regard door opening as a whole-body loco-manipulation task and handle it using a trained joint-level control policy with Reinforcement Learning (RL). The simulation environment is built in IsaacSim. <\/p>\n\n\n\n<p>The policy is optimized to open slightly damped doors with a crash bar. It takes in proprioceptions (to capture humanoid&#8217;s state) and processed perceptions data (to capture door&#8217;s state) and outputs the desired joint positions of 29 joints. A PD controller is then implemente to eventually transfer the joint position to torques.<\/p>\n\n\n\n<p>To alleviate the sim2real gap, we further create the door model in simulator to match the real door.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"720\" data-id=\"122\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/02\/rl-video-step-0.mp4_20260220_232605.078.jpg\" alt=\"\" class=\"wp-image-122\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/02\/rl-video-step-0.mp4_20260220_232605.078.jpg 1280w, https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/02\/rl-video-step-0.mp4_20260220_232605.078-300x169.jpg 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/02\/rl-video-step-0.mp4_20260220_232605.078-768x432.jpg 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n<\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"720\" src=\"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/02\/rl-video-step-0.mp4_20260220_232610.096.jpg\" alt=\"\" class=\"wp-image-123\" srcset=\"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/02\/rl-video-step-0.mp4_20260220_232610.096.jpg 1280w, https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/02\/rl-video-step-0.mp4_20260220_232610.096-300x169.jpg 300w, https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-content\/uploads\/sites\/92\/2026\/02\/rl-video-step-0.mp4_20260220_232610.096-768x432.jpg 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p><strong>Fig :<\/strong> <strong>Simulation Environment:<\/strong> The humanoid initializing in front of the modeled door.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Arm Visual Servoing for Button Pressing: We utilize Inverse-Dynamics to control the upper body (the arms). The system ensures a good precision and accuracy. Fig: Robot Pressing a Button with Language Prompt. Fig. Visual servo arm pipeline. Door Opening Task: We regard door opening as a whole-body loco-manipulation task and handle it using a trained [&hellip;]<\/p>\n","protected":false},"author":434,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-163","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-json\/wp\/v2\/pages\/163","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-json\/wp\/v2\/users\/434"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-json\/wp\/v2\/comments?post=163"}],"version-history":[{"count":7,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-json\/wp\/v2\/pages\/163\/revisions"}],"predecessor-version":[{"id":219,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-json\/wp\/v2\/pages\/163\/revisions\/219"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2026teama\/wp-json\/wp\/v2\/media?parent=163"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}