{"id":395,"date":"2019-04-06T02:48:18","date_gmt":"2019-04-06T02:48:18","guid":{"rendered":"http:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/?p=395"},"modified":"2019-12-13T16:57:48","modified_gmt":"2019-12-13T16:57:48","slug":"software-and-processing-subsystem","status":"publish","type":"post","link":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/2019\/04\/06\/software-and-processing-subsystem\/","title":{"rendered":"Software and Processing Subsystem"},"content":{"rendered":"

The processing subsystem includes all algorithms within the system, including: signal filtering
\nand conditioning algorithms, prediction and calculation algorithms, and the control algorithm. It
\nwill take raw sensor data as an input into the subsystem and output torque values to the actuation
\nsubsystem.<\/p>\n

20th<\/sup>January 2019<\/strong><\/p>\n

An Initial Git Repository was set up in order to facilitate version control and code updates.<\/p>\n

 <\/p>\n

7st<\/sup>February 2019<\/strong><\/p>\n

Research Completed on on-board computers for the project. NVIDIA jetson was chosen.\u00a0Sparkfun Razor IMU 9 Dof was picked out, to be placed at knee to collect walking data.<\/p>\n

 <\/p>\n

16th<\/sup>February 2019<\/strong><\/p>\n

Sparkfun IMU was interfaced with Arduino Uno (I2C protocol). FSR\u2019s were connected to analog port of Arduino. The IMU\u2019s firmware was modified to give roll, pitch, yaw and quaternion values along with accelerometer, gyroscope and magnetometer values. The FSR\u2019s data is intended to provide ground truth for the prediction model.<\/p>\n

\"\"<\/p>\n

Fig 1: FSR and IMU setup with Arduino<\/u><\/p>\n

 <\/p>\n

20th<\/sup>February 2019<\/strong><\/p>\n

Preliminary data collection was done with the setup for one of the team members. The values were read from the Serial monitor of the Arduino\u00a0 IDE through a python library. The data collected was then converted to a pandas Dataframe to do exploratory analysis.<\/p>\n

\"\"<\/p>\n

Fig 2: Accelerometer Values from IMU<\/u><\/p>\n

 <\/p>\n

\"\"<\/p>\n

Fig 3: Gyroscope Values from IMU<\/u><\/p>\n

 <\/p>\n

7st<\/sup>March 2019<\/strong><\/p>\n

Additional data was collected with the IMU on the brace fitted at the front and properly calibrated. The analog values from FSR were binned \u00a0and binarized them (as shown in the figure below), by noting the values from the start of a heel strike till a toe-off. A threshold of 400 was set and binarized the values at the heel and toe.<\/p>\n

\"\"<\/p>\n

Fig 4: Binned FSR values<\/u><\/p>\n

 <\/p>\n

In the above diagram, for the purpose of clarity, the values at stance have been assigned 500 and values at swing have been assigned 0. For the purpose of training a model, we assigned 1 to the values in swing and 0 to the values in stance. This is added as the last two values of the data frame. Then we implemented a vanilla logistic regression on the model and performed a 10-fold cross-validation.<\/p>\n

The cross-validation accuracy was about 95%<\/p>\n

Then we prepared a different dataset altogether (completely exclusive from the training data) and fit the trained model to that. We were able to obtain a accuracy of 81%.<\/p>\n

 <\/p>\n

\"\"<\/p>\n

Fig 5: Test data vs Predicted Data<\/span><\/p>\n

\"\"<\/p>\n

Fig 6: ROC Curve for Logistic Regression<\/u><\/p>\n

 <\/p>\n

Understandably so, the model provides the prediction at that specific data point or instant independent of previous values.<\/p>\n

13th<\/sup>March 2019<\/strong><\/p>\n

Foot Placement:\u00a0For the Foot -placement section of the Processing Subsystem, we researched on the various models and chose the Linear Inverted Pendulum Model to calculate capture point.<\/p>\n

22nd<\/sup>March 2019<\/strong><\/p>\n

A Preliminary \u00a0Software Architecture pipeline was drawn up with our initial understanding and flow of all the algorithms.<\/p>\n

\"\"<\/p>\n

Fig7: Software Architecture<\/span><\/p>\n

1st<\/sup>April<\/strong><\/p>\n

 <\/p>\n

An initial ROS Environment was setup including the required packages for the YOST IMU and Razor IMU on Git.<\/p>\n

The Firmware on the Razor IMU was modified to read the FSR values. This is to remove the Arduino from the Pipeline and directly interface FSR to the IMU (SAMD21 Breakout Board) to the Jetson.<\/p>\n

The FSR and resistors (for the voltage divider circuit) were soldered to the perfboard, in order to collect data.<\/p>\n

5th April<\/strong><\/p>\n

ROS Environment:<\/span><\/p>\n

The Figure below shows the ROS architecture currently implemented for the exoskeleton system. \u00a0The Razor IMUs ROS package allows for the linear acceleration, angular velocity, and roll, pitch, and yaw angles to be published. \u00a0For testing and validation purposes we also set them to publish the raw FSR values to determine ground truth of whether the leg is in stance or swing (The FSRs are directly connected to the IMU breakout boards). \u00a0This data is published to the \/<\/span>imuR <\/span><\/i>topic for the right leg and the \/<\/span>imuL<\/span><\/i> topic for the left leg. \u00a0The <\/span>classification_model_node<\/span><\/i>, <\/span>COM_vel_cal_node<\/span><\/i> and <\/span>hip_exo<\/span><\/i> node all subscribe to this topic. \u00a0The <\/span>classification_model_node<\/span><\/i> determines if each leg is in swing or stance and publishes this information to the \/<\/span>swingLegRight<\/span><\/i> or \/<\/span>swingLegLeft<\/span><\/i> topic for the right and left legs respectively. \u00a0The <\/span>COM_vel_cal_node<\/span><\/i> and <\/span>hip_exo<\/span><\/i> node subscribe to these topics. The <\/span>COM_vel_cal_node<\/span><\/i> calculates the CoM velocity and publishes it to the \/<\/span>COM_vel<\/span><\/i> topic. The <\/span>hip_exo<\/span><\/i> node subscribes to this topic. Within the <\/span>hip_exo<\/span><\/i> node, the capture point is calculated and then transformed into a desired hip angle for the leg in swing. The HFE \u00a0and HAA desired angles are published to the \/<\/span>desiredAngleR<\/span><\/i> or \/<\/span>desiredAngleL<\/span><\/i> topic, depending on which leg is in swing. \u00a0The motor controller node will subscribe to these topics so that it can actuate the motors at the hip joints appropriately. \u00a0<\/span><\/p>\n

\"\"<\/p>\n

Fig 8: ROS Architecture<\/span><\/p>\n

7th April<\/strong><\/p>\n

Logistic Regression and the Random Forest Classifiers are the two machine learning models were considered for swing leg detection. \u00a0The nine features from the IMU (linear acceleration in x,y,z, angular velocity in x,y,z and roll pitch and yaw) are normalized before being used as inputs to the models.<\/span><\/p>\n

The data was split into training and testing sets. The two models were trained on the normalized dataset of IMU values and the target class. The cost of false positives is quite high for the exoskeleton use case. A false positive is when the leg is in-stance but detected to be in-swing. This would lead to the stance leg being actuated further in the software pipeline, which is potentially dangerous to the user. <\/span><\/p>\n

The confusion matrices for both logistic regression (Table 1) and random forest (Table 2) are as following. We achieved 94% overall accuracy for logistic regression and 97% for random forests.<\/span><\/p>\n

Table 1: Logistic Regression Confusion Matrix<\/span><\/p>\n\n\n\n\n\n
Logistic Regression<\/b><\/td>\nPredicted Leg in Stance<\/span><\/td>\nPredicted Leg in Swing<\/span><\/td>\n<\/tr>\n
Actual Leg in Stance<\/span><\/td>\n822 (TP)<\/b><\/td>\n53 (FP)<\/b><\/td>\n<\/tr>\n
Actual Leg in Swing<\/span><\/td>\n58 (FN)<\/b><\/td>\n943 (TN)<\/b><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Table 2: Random Forests Confusion Matrix<\/span><\/p>\n\n\n\n\n\n
Random Forests<\/b><\/p>\n

(50 estimators)<\/span><\/td>\n

Predicted Leg in Stance<\/span><\/td>\nPredicted Leg in Swing<\/span><\/td>\n<\/tr>\n
Actual Leg in Stance<\/span><\/td>\n870 (TP)<\/b><\/td>\n5 (FP)<\/b><\/td>\n<\/tr>\n
Actual Leg in Swing<\/span><\/td>\n57 (FN)<\/b><\/td>\n944 (TN)<\/b><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

It was concluded that the Random Forest Classifier with tuned hyperparameters provided the least number of false positives. Hence this classifier was used for the final system. \u00a0Precision and recall within the range of 93-99% were obtained for the final spring validation demonstration.<\/p>\n

\u00a0\u00a0\u00a0The subsystem should be improved by the decreasing the algorithm processing time to 100ms. \u00a0It would also be beneficial to smooth the predictions by taking into account the temporal nature of the data and noting the change or transitions in IMU values from swing to stance.<\/span><\/p>\n

10th April<\/strong><\/p>\n

CoM Velocity Calculation:<\/span><\/p>\n

\u00a0\u00a0\u00a0The CoM velocity calculation subsystem takes the roll and pitch angles from the thigh IMUs as well as predicted swing leg as inputs, and calculates the human\u2019s center of mass velocity which is used for the prediction of the capture points. We use the stance leg as an inverted pendulum model as shown in Figure 18<\/span>. Using a series of inputs of IMU angles and time stamps, we calculate the angular velocity. With a fixed length of human\u2019s leg, we can calculate the linear velocity of the end the pendulum, which can be regarded as human\u2019s CoM velocity.<\/span><\/p>\n

\"\"<\/p>\n

Figure 18: Inverted Pendulum Model for CoM Velocity Calculation<\/span><\/p>\n

Since the CoM Velocity calculation depends heavily on the correct detections of the Swing Leg Classification Subsystem, we first tested with the ground truth swing and stance leg values obtained from the FSRs under the feet. We started with sampling the IMU angles continuously and used the differences of angles divided by the time intervals to get the angular velocity. However, since the time intervals for continuous sampling is too short, a small difference caused by noise will result in large calculated velocity. To mitigate the influence of noise, we improved the sampling intervals. Also, we use the average of the calculated velocities at 10 different samples as our output velocity to further mitigate abnormal values.<\/span><\/p>\n

September 12 – Oct 1st ,2019<\/strong><\/p>\n

Time profiling of the software pipeline was done as shown in the. Figure below. The entire software pipeline was also revamped and three major packages were created: soteria sensing, Soteria controls and soteria motor control.<\/p>\n

\"\"<\/p>\n

Figure 19: time profile of pipeline<\/p>\n

\"\"<\/p>\n

Figure 20: software pipeline revamp<\/p>\n

September 28, 2019\u00a0<\/b><\/p>\n

 <\/p>\n

Came up with the winter dataset based trajectory planner concept. Sourced an excel version of the winter dataset and extracted the required columns for the trajectory planner.<\/span><\/p>\n

 <\/p>\n

October 22, 2019<\/b><\/p>\n

Began implementation of the winter trajectory planner in MATLAB . Wrote code in MATLAB to read data from the excel (.csv) file and import it into MATLAB as array objects. Thresholded the ground reaction force column and plotted it alongside the HFE curve to identify the swing phase and stance phase. Figure 1 shows one complete gait cycle along with a thresholded GRF plot.<\/span><\/p>\n

\"\"
\nFigure 1: One complete HFE trajectory cycle with the thresholder GRF plot denoting swing and stance<\/span><\/p>\n

October 23, 2019<\/b><\/p>\n

Extracted the section of the trajectory over swing, and divided the datapoints by the last swing-phase datapoint so as to normalize the trajectory to end at unity. The normalized swing portion of the winter trajectory can be seen in figure 2 below:<\/span><\/p>\n

\"\"
\nFigure 2: Normalized swing portion of the winter trajectory<\/span><\/p>\n

 <\/p>\n

November 5, 2019<\/b><\/p>\n

Ported the normalized winter trajectory from MATLAB to the soteria_controls package in C++. Wrote code to obtain the desired landing angle and swing times from the helper functions. Wrote code to scale the trajectory along the y-axis to match the desired angles, and imported and used the cubic spline interpolation library to interpolate the trajectory in order to scale it by the estimated swing time along the x-axis.<\/span><\/p>\n

 <\/p>\n

\u00a0<\/span>November 20, 2019<\/b><\/p>\n

Wrote alternative trajectory planner which only follows the rising portion of the curve to the desired hip angle instead of the complete rise and fall. Tested it on the user and determined it to be ill-suited to the problem. Implemented a phase shifter for the trajectory so that the trajectory can act earlier and actuate the leg linkage faster to minimize response time.<\/span><\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

The processing subsystem includes all algorithms within the system, including: signal filtering and conditioning algorithms, prediction and calculation algorithms, and the control algorithm. It will take raw sensor data as an input into the subsystem and output torque values to the actuation subsystem. 20thJanuary 2019 An Initial Git Repository was set up in order to Read more about Software and Processing Subsystem<\/span>[…]<\/a><\/p>\n","protected":false},"author":141,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17],"tags":[],"class_list":["post-395","post","type-post","status-publish","format-standard","hentry","category-processing-subsystem"],"_links":{"self":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/posts\/395","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/users\/141"}],"replies":[{"embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/comments?post=395"}],"version-history":[{"count":22,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/posts\/395\/revisions"}],"predecessor-version":[{"id":585,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/posts\/395\/revisions\/585"}],"wp:attachment":[{"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/media?parent=395"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/categories?post=395"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mrsdprojects.ri.cmu.edu\/2018teamb\/wp-json\/wp\/v2\/tags?post=395"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}