{"id":83,"date":"2017-10-19T21:53:58","date_gmt":"2017-10-20T01:53:58","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/?page_id=83"},"modified":"2018-05-11T20:52:38","modified_gmt":"2018-05-12T00:52:38","slug":"system-design","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/system-design\/","title":{"rendered":"Design"},"content":{"rendered":"<h2>Functional Architecture<\/h2>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-202 size-full aligncenter img-responsive\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-content\/uploads\/sites\/25\/2017\/10\/Functional-Architecture-1.png\" alt=\"\" width=\"1101\" height=\"561\" \/><\/p>\n<p>&nbsp;<\/p>\n<h2>Cyber-Physical Architecture<\/h2>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-203 size-full aligncenter img-responsive\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-content\/uploads\/sites\/25\/2017\/10\/Cyberphysical-Architecture-2.png\" alt=\"\" width=\"1099\" height=\"578\" \/><\/p>\n<p><b>FUNCTIONAL REQUIREMENTS<\/b><\/p>\n<ul>\n<li>Shall detect emotions from tri-modal data<\/li>\n<li>Shall output emotion chart<\/li>\n<li>Shall track user<\/li>\n<\/ul>\n<p><b>NON-FUNCTIONAL REQUIREMENTS<\/b><\/p>\n<ul>\n<li>Rests on tabletop<\/li>\n<li>On\/Off switch (Easy to use)<\/li>\n<li>Under $5000 (Affordable)<\/li>\n<li>Smaller than a microwave<\/li>\n<li>Less than 5 kg (Lightweight)<\/li>\n<\/ul>\n<p><b>PERFORMANCE REQUIREMENTS<\/b><\/p>\n<ul>\n<li>Will detect emotions from tri-modal data: 60% accuracy<\/li>\n<li>Will output emotion chart: 1 frame\/second<\/li>\n<li>Will track user: Real-time<\/li>\n<\/ul>\n<p><strong>SUB-SYSTEM DESIGNS AND DESCRIPTIONS:<\/strong><\/p>\n<h3 style=\"text-align: center\">Visual Modality<\/h3>\n<p>This modality detects facial features relevant to emotion detection. Batches of images are passed onto Resnet-18 and a feature vector is generated.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-235 aligncenter img-responsive\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-content\/uploads\/sites\/25\/2017\/10\/Visual_Modality.png\" alt=\"\" width=\"1050\" height=\"471\" \/><\/p>\n<h3 style=\"text-align: center\">Vocal Modality<\/h3>\n<p>Vocal modality combines the audio intonations of a user for emotion detection. It splits the waveform, normalises and standardises the sample rate to generate a feature vector.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-236 aligncenter img-responsive\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-content\/uploads\/sites\/25\/2017\/10\/Vocal_Modality.png\" alt=\"\" width=\"1059\" height=\"479\" \/><\/p>\n<h3 style=\"text-align: center\">Verbal Modality<\/h3>\n<p>The verbal modality processes the text data of speech by splitting the speech, generating word vectors and passing them through a Bidirectional LSTM to get context vectors.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-237 aligncenter img-responsive\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-content\/uploads\/sites\/25\/2017\/10\/Verbal_Modality.png\" alt=\"\" width=\"934\" height=\"408\" \/><\/p>\n<h3 style=\"text-align: center\">Attention Model<\/h3>\n<p>Attention model learns similarities and alignment between different modes of data and learns cooccurring features. This helps the network converge faster.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-239 aligncenter img-responsive\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-content\/uploads\/sites\/25\/2017\/10\/Attention-1.png\" alt=\"\" width=\"943\" height=\"531\" \/><\/p>\n<h3 style=\"text-align: center\">Shared Decoder<\/h3>\n<p>The shared decoder predicts emotions. A combined feature vector from the three modes and attention is passed to the LSTM network which learns to predict valence and activation from these features.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-240 aligncenter img-responsive\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-content\/uploads\/sites\/25\/2017\/10\/Decoder.png\" alt=\"\" width=\"951\" height=\"452\" \/><\/p>\n<h3 style=\"text-align: center\">Face Tracking<\/h3>\n<p>The video feed is obtained from a 2DOF mounted camera that tracks faces in the vertical and horizontal mimicking the movements of an attentive human. It recognises and tracks the face of the\u00a0user using two RC servos for dexterity. The 3D design of the set up is shown below.\u00a0<img loading=\"lazy\" decoding=\"async\" class=\" wp-image-287 aligncenter img-responsive\" src=\"http:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-content\/uploads\/sites\/25\/2018\/02\/lukabot2DOF.png\" alt=\"\" width=\"338\" height=\"512\" \/><\/p>\n<h3 style=\"text-align: center\">State Machine<\/h3>\n<p>The state machine is an internal module that responds to the detected emotion. It does not generate natural language like humans but selects from pre-prepared responses one that is appropriate for the situation.<\/p>\n<h3 style=\"text-align: center\">UI Subsystem -FVE<\/h3>\n<p>This module is the UI interface. It displays emotions of the user and the output from the state machine.<\/p>\n<p><iframe loading=\"lazy\" width=\"980\" height=\"551\" src=\"https:\/\/www.youtube.com\/embed\/Vxg4i80vxK0?feature=oembed\" frameborder=\"0\" allow=\"autoplay; encrypted-media\" allowfullscreen><\/iframe><\/p>\n<h3 style=\"text-align: center\">UI Subsystem -SVE<\/h3>\n<p><iframe loading=\"lazy\" width=\"980\" height=\"551\" src=\"https:\/\/www.youtube.com\/embed\/IG6LPBGgOv8?feature=oembed\" frameborder=\"0\" allow=\"autoplay; encrypted-media\" allowfullscreen><\/iframe><\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Functional Architecture &nbsp; &nbsp; Cyber-Physical Architecture &nbsp; FUNCTIONAL REQUIREMENTS Shall detect emotions from tri-modal data Shall output emotion chart Shall track user NON-FUNCTIONAL REQUIREMENTS Rests on tabletop On\/Off switch (Easy to use) Under $5000 (Affordable) Smaller than a microwave Less than 5 kg (Lightweight) PERFORMANCE REQUIREMENTS Will detect emotions from tri-modal data: 60% accuracy Will &#8230; <a href=\"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/system-design\/\" class=\"more-link text-uppercase small\"><strong>Continue Reading<\/strong> <i class=\"fa fa-angle-double-right\" aria-hidden=\"true\"><\/i><\/a><\/p>\n","protected":false},"author":111,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-83","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-json\/wp\/v2\/pages\/83","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-json\/wp\/v2\/users\/111"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-json\/wp\/v2\/comments?post=83"}],"version-history":[{"count":19,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-json\/wp\/v2\/pages\/83\/revisions"}],"predecessor-version":[{"id":325,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-json\/wp\/v2\/pages\/83\/revisions\/325"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2017teamd\/wp-json\/wp\/v2\/media?parent=83"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}