{"id":137,"date":"2019-01-22T23:21:31","date_gmt":"2019-01-22T23:21:31","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/?page_id=137"},"modified":"2019-12-13T23:09:55","modified_gmt":"2019-12-13T23:09:55","slug":"performance","status":"publish","type":"page","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/performance\/","title":{"rendered":"Performance"},"content":{"rendered":"

SVD Performance Evaluation<\/strong><\/h3>\n

\"\"<\/p>\n

Figure 1: The Sensing System for SVD
\n<\/span>(IMU\u2019s roll and pitch shown on the screen)<\/span><\/p>\n

 <\/p>\n

The Spring Validation Demonstration (SVD) allowed us to compare our system against our requirements. \u00a0To carry out the demonstration we set up a sensing system by attaching IMUs to the exoskeleton thigh brace on person\u2019s distal femurs, and attaching FSRs to both the toe and heel of each foot, as shown in Figure 1. Table 1 shows the validation demonstrations conducted and the results obtained.<\/span><\/p>\n

Table 1: Performance Evaluation against SVD<\/span><\/p>\n\n\n\n\n\n\n\n
Subsystem<\/b><\/td>\nRequirement Validated<\/b><\/td>\nPerformance<\/b><\/td>\n<\/tr>\n
Sensing Subsystem<\/span><\/td>\nMR2. Measure user\u2019s pelvic orientation within \u00b110\u00b0;<\/span><\/td>\nWe inclined the exoskeleton to specified angles of 0, 10, 20 degree using a protractor, and read the orientation output from IMU.<\/span><\/p>\n

For all trials, the IMU measurements were within 2 degrees of actual orientations. <\/span><\/td>\n<\/tr>\n

MR3. Determine user\u2019s hip joint angle within \u00b110\u00b0;<\/span><\/td>\nWe Rotated the joint by hand to specific angles of 0, 10, 20 degrees for both the HAA and HFE using a protractor, and read the orientation output from IMU.<\/span><\/p>\n

For all trials, the IMU measurements for both HAA and HFE were within 2 degrees of actual orientations.<\/span><\/td>\n<\/tr>\n

Processing Subsystem<\/span><\/td>\nMR1. Determine user\u2019s pelvic velocity within \u00a0\u00b1 0.3 m\/s;<\/span><\/td>\nWe had one teammate walk on the treadmill for 30 seconds and recorded the predicted forward center of mass velocity and compare the mean velocity with the treadmill velocity.<\/span><\/p>\n

While the treadmill velocity was 0.8 m\/s, the calculated mean forward and sideward velocities are 0.7 m\/s and -0.09 m\/s, respectively. The plots for calculated CoM velocity are shown in Figure 21.<\/span><\/td>\n<\/tr>\n

MR4a. Classify leg in swing with a recall of 70%<\/span><\/p>\n

MR4b. Classify leg in swing with a precision of 80%<\/span><\/td>\n

We had one teammate walk on the treadmill for another 30 seconds and recorded the predicted and ground-truth values of swing leg detections.<\/span><\/p>\n

For the left leg, we obtained 98% precision and 93% recall; for the right, we obtained 98% precision and 99% recall. The detected swing leg is plotted together with the ground truth in Figure 22.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

\"\" \"\"<\/p>\n

Figure 2: Performance Evaluation of CoM Velocity Calculation<\/span><\/p>\n

\"\"<\/p>\n

Figure 3: Performance Evaluation of Swing Leg Detection<\/span><\/p>\n

 <\/p>\n

From Table 1 and Figures 2 and 3, the performance of our system in SVD met our scenario and metrics. Other than those demo shown in SVD, we also had one teammate walk on treadmill with our one-side exoskeleton. While the passive walking, it fitted the person securely as shown in Figure 4.<\/span><\/p>\n

\"\"<\/p>\n

Figure 4: \u00a0Secure Fit Test<\/span><\/p>\n

 <\/p>\n

Table 2: Performance Validation against Test Plans<\/span><\/p>\n

\n\n\n\n\t\n\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t
Subsystem<\/th>Requirement Validated<\/th>Performance<\/th>\n<\/tr>\n<\/thead>\n
Electrical Subsystem - Motor Reverse Current Dissipation Circuit Test<\/td>MN3<\/td>Performance Reqs:
\nDigital output line goes high When the motors are spun by hand (signifies an over voltage is detected)
\n
\nThe MOSFET gate is opened.
\n
\nHow it performed:
\nBoth of these metrics were validated when this test was conducted. The RoboteQ GUI was used to ensure the digital output pin went high and a multimer was used to determine continuity between the gate and the drain of the MOSFET, therefore ensuring the gate had opened.
\n<\/td>\n<\/tr>\n
Electrical Subsystem - Power\/Emergency Stop test<\/td>MN3, MR1-9<\/td>Performance Reqs:
\nJetson, both motor controllers, USB hub, and IMUs are powered by the 12V supply (Jetson is on and power LEDs are illuminated)
\n
\nMotor Controller power switches power the motor controllers on and off
\n
\n48V supply provides power to the motors (they respond to actuation)
\n
\nThe emergency switch kills power to the motors (they do not respond to actuation), and all other components
\nremain powered.
\n
\nHow it performed:
\nVerified all electrical systems were powered on and the motors could be actuated. When the Emergency stop was activated, we used a multimeter to ensure 48V was no longer connected to the motors - that passed. The motors could no longer be actuated as well. But all other electrical components remained powered as required.
\n<\/td>\n<\/tr>\n
Mechanical Subsystem - Exoskeleton frame and its weight, balance, and user-fit requirements<\/td>MN1, MN2, MN3, MN4
\n<\/td>		Performance Reqs:
\nExoskeleton is built and complete
\nCalculated CoM is no more than 3cm from the user\u2019s CoM
\nWeight of exoskeleton is less than 10kg
\nThe exoskeleton does not slip and moves smoothly with user motions
\nUser does not feel pain or discomfort while wearing the exoskeleton
\n
\nHow it performed:
\nAll metrics were met during PR test. COM calculations were conducted using the CAD model in Fusion 360. The weight of the exoskeleton was measured using a bathroom scale to be 9.6 kg. The exoskeleton fit securely and was moving smoothly with user movements.
\n<\/td>\n<\/tr>\n
Electrical & Mechanical Subsystem - Successful integration of mechanical and electrical subsystems. <\/td>MR2, MR3, MR7, MR8<\/td>Performance Reqs:
\nFinal weight of exoskeleton is below 10kg.
\nExoskeleton is able to power on.
\nLeg supports of the exoskeleton are able to be actuated in both flexion\/extension and adduction\/abduction.
\nIMUs are powered, sampling, and outputting data to the Jetson.
\n
\nHow it performed:
\nThe final exoskeleton assembly weighed 9.6kg. Exoskeleton was able to successfully power on and actuate the leg linkages as well as read in sensor data from all IMUs.
\n<\/td>\n<\/tr>\n
Processing Subsystem- State Estimation (Sensing)<\/td>MR1, MR2,MR3,MR4(a),MR4(b)<\/td>Performance Reqs:
\nTo ensure that the output time taken by final individual algorithm blocks, namely the
\nswing leg detection and COM velocity, is below 100 ms
\n\u25cf To ensure that the swing leg prediction algorithm has a precision and recall of 80% and
\n70% respectively
\n\u25cf To ensure that the pelvic velocity is within \u00b1 0.3 m\/s
\n
\nHow the system performed:
\nThis Test was validated in depth during SVD. The system met all the requirements for the accuracy of the model as well as the. time criterion. The system also computed COM Velocity from the sensed IMU data at the knee accurately<\/td>\n<\/tr>\n
Processing Subsystem- Control Algorithms<\/td>MR5-9<\/td>Performance Reqs:
\nThe durations of capture point and trajectory planning algorithms must be less than 100 ms and
\n120 ms, respectively.
\n
\nHow the system performed:
\nThere was a delay in the system's desired motor command output vs Angela's pitch due to COM velocity and trajectory planner computation but the system satisfied the requirements stated with the entire system running in approx 0.1 seconds<\/td>\n<\/tr>\n

\n Actuation Subsystem - ROS
\n<\/td>
\nMR7, MR8, MR9
\n<\/td>		After launching the motor controller ROS node, and feeding desired position commands through ROS rqt topic publisher, the motor is actuated to the position, where the encoder reading is the same as the desired command.<\/td>\n<\/tr>\n
Actuation Subsystem - Mechanical Movement <\/td>MR7, MR8, MR9<\/td>We attached a pull force sensor to the end of the leg link, sent command through the GUI to actuate the motor to desired angle, and pulled the force sensor to get the maximum force.
\nAfter projecting the force into torque, it is more than 25 Nm, while the motor is still keeping the desired position. So The actuators are able to lift the leg linkage to the desired angles
\n<\/td>\n<\/tr>\n
Whole System Integration - Part 2: On Person
\n<\/td>		MR1-9<\/td>Performance Reqs:
\nWhile the person is walking with the exoskeleton, the exoskeleton can actuate the
\nperson\u2019s leg safely and reasonably.
\n
\nThe durations of capture point and trajectory planning algorithms are less than 100
\nms and 120 ms, respectively.
\n
\nThe landing angle measured by the IMU achieved at the end of swing (IMU
\nmeasurement) is within \u00b110\u00b0 of the calculated landing angle at the beginning of swing.
\n
\nThe target hip angle is reached before the time that the heel touches the ground.
\n<\/td>\n<\/tr>\n
Whole System Integration - Part 1: Off Person
\n<\/td>		MR1-9<\/td>Performance Reqs:
\nExoskeleton leg linkages move in accordance with the user\u2019s walking pattern
\n
\n
\nHow it performed:
\nValidated by plotting pitch of the user alongside pitch of the IMUs that were on the exoskeleton leg linkages. Both curves reached the same peak and showed similar rising and falling trends.
\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/p>\n

 <\/p>\n

Performance Validation against FVD Demonstration<\/span><\/h3>\n

Test 1: On person testing with swing leg ground truth.<\/strong><\/p>\n

During this test, the user wore our exoskeleton while walking. We used the ground truth from<\/p>\n

the IR sensors under the shoes to tell whether the leg is in swing and stance. The user felt good<\/p>\n

overall. At the start of the user\u2019s walking, the user felt little resistance from the exoskeleton.<\/p>\n

This was due to the small initialized center of mass velocity. Our system will generate desired<\/p>\n

motor commands corresponding capture points which are calculated from the center of mass<\/p>\n

velocity, so when there is a lowe COM velocity, the motor commands will be small. After the<\/p>\n

initial steps the user doesn\u2019t feel any resistance from the exoskeleton because the actuated link<\/p>\n

angles fitted the user\u2019s walking gait. From Figure 4 , the user\u2019s pitch (blue line) follows the<\/p>\n

desired motor command (red line) well for the angles. The motor command is ahead of the user\u2019s<\/p>\n

pitch, which leaves time for the actual actuations.<\/p>\n

 <\/p>\n

\"\"<\/p>\n

 <\/p>\n

Figure 4: User’s pitch vs motor command<\/p>\n

 <\/p>\n

Test 2: On person testing with swing leg prediction<\/strong><\/p>\n

For Test 2 we repeated Test 1 except that we used a machine learning model to classify<\/p>\n

whether the leg is in swing or stance instead of using ground truth data. Figure 31 compares our<\/p>\n

prediction with the ground truth. We achieved 0.88 for the recall and 0.91 for the precision<\/p>\n

which met our requirements. Other performance are almost the same as Test1.<\/p>\n

\"\"<\/p>\n

Figure 5: Swing leg Prediction<\/p>\n

 <\/p>\n

Test 3: Off person testing with swing leg prediction<\/strong><\/p>\n

For this test, the user walked wearing only the sensors, while the exoskeleton was fixed onto<\/p>\n

a cart and actuated off person. We used the machine learning model for swing leg classification.<\/p>\n

We compared the pitch angles measured by the IMUs on the exoskeleton\u2019s leg link and the<\/p>\n

user\u2019s pitch angles measured by the IMUs attached to the user\u2019s leg. From Figure 32, the user\u2019s<\/p>\n

pitch matched the exoskeleton\u2019s pitch very well. The error of the peak value is less than 5<\/p>\n

degrees on average. This test showed that the trajectory of the motion of the exoskeleton<\/p>\n

matched the gait of the user.<\/p>\n

\"\"<\/p>\n

\u00a0Figure 6: User’s pitch exoskeleton’s pitch<\/span><\/p>\n

Test4: Off person testing with perturbation<\/strong><\/p>\n

In this test, we demoed our exoskeleton can react to the perturbation from the side by<\/p>\n

actuating our adduction\/abduction motor. The perturbation is generated by pulling a rope<\/p>\n

attached to the user from the user\u2019s side while the user is walking. Figure 33 shows the desired<\/p>\n

motor command of the adduction\/abduction motor versus the roll angle of the user\u2019s leg. When<\/p>\n

the perturbation happened, there was a sudden increase of the user\u2019s roll angle, and our motor<\/p>\n

command match that roll angle well for both the time and the scale. This demo showed that our<\/p>\n

system reacts to perturbation correctly.<\/p>\n

\"\"<\/p>\n","protected":false},"excerpt":{"rendered":"

SVD Performance Evaluation Figure 1: The Sensing System for SVD (IMU\u2019s roll and pitch shown on the screen)   The Spring Validation Demonstration (SVD) allowed us to compare our system against our requirements. \u00a0To carry out the demonstration we set up a sensing system by attaching IMUs to the exoskeleton thigh brace on person\u2019s distal Read more about Performance<\/span>[…]<\/a><\/p>\n","protected":false},"author":143,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"template-fullwidth.php","meta":{"footnotes":""},"class_list":["post-137","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/pages\/137","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/users\/143"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/comments?post=137"}],"version-history":[{"count":6,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/pages\/137\/revisions"}],"predecessor-version":[{"id":630,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/pages\/137\/revisions\/630"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/media?parent=137"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}