{"id":210,"date":"2024-04-28T20:04:54","date_gmt":"2024-04-28T20:04:54","guid":{"rendered":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teami\/?page_id=210"},"modified":"2025-01-14T07:17:52","modified_gmt":"2025-01-14T07:17:52","slug":"system-performance","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teami\/system-performance\/","title":{"rendered":"System Performance"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Spring Validation Demo System Performance<\/h2>\n\n\n\n<p>Our overall system performance for Spring Validation Demo and Encore is as shown below:<\/p>\n\n\n\n<figure class=\"wp-block-table aligncenter\"><table><tbody><tr><td><\/td><td class=\"has-text-align-left\" data-align=\"left\"><strong>Performance Requirement<\/strong><\/td><td><strong>Performance Evaluation<\/strong><\/td><\/tr><tr><td>PR1<\/td><td class=\"has-text-align-left\" data-align=\"left\">The system shall receive and respond to the user&nbsp;within 5 seconds<\/td><td>achieved: &lt;=&nbsp;<strong>2 seconds<\/strong>\u200b<\/td><\/tr><tr><td>PR2<\/td><td class=\"has-text-align-left\" data-align=\"left\">The system shall localize in the pre-mapped environment&nbsp;within 5 cm<\/td><td>achieved:&nbsp;<strong>within 10 cm<\/strong>\u200b<\/td><\/tr><tr><td>PR3<\/td><td class=\"has-text-align-left\" data-align=\"left\">The system shall collect medical supplies&nbsp;with a success rate of over 80%<\/td><td>achieved:&nbsp;<strong>84%<\/strong>\u200b<\/td><\/tr><tr><td>PR4<\/td><td class=\"has-text-align-left\" data-align=\"left\">The system shall plan its path&nbsp;in less than 5 seconds&nbsp;and navigate to its destination&nbsp;with a maximum&nbsp;speed of 0.3 m\/s<\/td><td>achieved:<strong> 0.6 m\/s<\/strong>\u200b<\/td><\/tr><tr><td>PR5<\/td><td class=\"has-text-align-left\" data-align=\"left\">The system shall deliver medical supplies to the operating room&nbsp;within 5 minutes<\/td><td>achieved:<strong> 2&nbsp;minutes<\/strong><\/td><\/tr><tr><td>PR6<\/td><td class=\"has-text-align-left\" data-align=\"left\">The system shall inspect&nbsp;10 unique&nbsp;items&nbsp;with&nbsp;an accuracy of 2 item counts for the LOW&nbsp;segment<\/td><td>achieved:&nbsp;<strong>100% accuracy<\/strong>&nbsp;via weight-based inspection<\/td><\/tr><tr><td>PR7<\/td><td class=\"has-text-align-left\" data-align=\"left\">The&nbsp;system shall update the global inventory knowledge base&nbsp;at most every 12 hours<\/td><td>achieved:&nbsp;<strong>real-time inventory knowledge\u200b<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Fall Validation Demo System Performance<\/h2>\n\n\n\n<p>Overall, the system demonstrated consistent and reliable performance during testing, meeting the expectations set for its operation. While there were areas that could benefit from further optimization, the results generally validate the robustness and efficiency of the system in handling the tasks it was designed for.<\/p>\n\n\n\n<p>The table below summarizes our system performance for Fall Validation Demo and Encore:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>Test Case<\/strong><\/td><td><strong>Required Success Rate<\/strong><\/td><td><strong>Required Time<\/strong><\/td><td><strong>Total tests Conducted<\/strong><\/td><td><strong>Successful Tests<\/strong><\/td><td><strong>Success Rate<\/strong><\/td><td><strong>Average Time<\/strong><\/td><\/tr><tr><td><strong>Delivery\/Retrieval<\/strong><\/td><td>80%<\/td><td>5 min<\/td><td>20<\/td><td>16<\/td><td><strong>80%<\/strong><\/td><td><strong>3 min <\/strong><\/td><\/tr><tr><td><strong>Restocking<\/strong><\/td><td>80%<\/td><td>3 min<\/td><td>50<\/td><td>48<\/td><td><strong>96%<\/strong><\/td><td><strong>2 min<\/strong><\/td><\/tr><tr><td><strong>Combined <br>(4 restocking + 1 delivery)<\/strong><\/td><td>80%<\/td><td>23 min<\/td><td>12<\/td><td>11<\/td><td><strong>92%<\/strong><\/td><td><strong>12 min<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Overall Subsystem Performance<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>Subsystem<\/strong><\/td><td><strong>Description<\/strong><\/td><td><strong>Success<\/strong> <strong>rate<\/strong><\/td><\/tr><tr><td><strong>Suction Pose Generation<\/strong><\/td><td>Success rate of generating feasible suction pose<\/td><td><strong>88%<\/strong><\/td><\/tr><tr><td><strong>Motion Planning<\/strong><\/td><td>Success rate of planning a path to a given pose without collision<\/td><td><strong>90%<\/strong><\/td><\/tr><tr><td><strong>Scene Understanding<\/strong><\/td><td>Success rate of detecting all items in a scene<\/td><td><strong>80%<\/strong><\/td><\/tr><tr><td><strong>Navigation<\/strong><\/td><td>Success rate of reaching a goal pose within an error bound of 5cm<\/td><td><strong>95%<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p>We have tested our subsystem-level pipeline rigorously, all achieving <strong>80%+<\/strong> success rates, as depicted above.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Spring Validation Demo System Performance Our overall system performance for Spring Validation Demo and Encore is as shown below: Performance Requirement Performance Evaluation PR1 The system shall receive and respond to the user&nbsp;within 5 seconds achieved: &lt;=&nbsp;2 seconds\u200b PR2 The system shall localize in the pre-mapped environment&nbsp;within 5 cm achieved:&nbsp;within 10 cm\u200b PR3 The system &#8230; <a href=\"https:\/\/mrsdprojects.ri.cmu.edu\/2024teami\/system-performance\/\" class=\"more-link text-uppercase small\"><strong>Continue Reading<\/strong> <i class=\"fa fa-angle-double-right\" aria-hidden=\"true\"><\/i><\/a><\/p>\n","protected":false},"author":384,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-210","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teami\/wp-json\/wp\/v2\/pages\/210","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teami\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teami\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teami\/wp-json\/wp\/v2\/users\/384"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teami\/wp-json\/wp\/v2\/comments?post=210"}],"version-history":[{"count":5,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teami\/wp-json\/wp\/v2\/pages\/210\/revisions"}],"predecessor-version":[{"id":311,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teami\/wp-json\/wp\/v2\/pages\/210\/revisions\/311"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2024teami\/wp-json\/wp\/v2\/media?parent=210"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}