Work Plan
Work Schedule
Test Schedule
Parts List and Budget
Our updated, refined parts list can be found here.
Our total budget is $5000.00. We have spent 45% of our total budget, as shown in figure 8.4. Currently our major expenses are the servo motors and hard drive.
Risks
A categorisation of our risks can be found here.
Issues Log
11/7/2017: The SEMAINE dataset does not have values for the intensity label for ~ 20% of the total dataset. We decided to not use the intensity value for our emotion prediction and use the other 4 instead. This
decision was taken based on the fact that most information gain is contained within valence and arousal.
11/12/2017: The dataset does not contain sentence-level time transcripts for some parts within the dataset. It does, however, contain word-level time transcripts for the entire dataset. The sentence level transcript was generated using the word level transcript.
11/15/2017: The loss function is not converging on the entire dataset. After a lot of thinking, our reasoning is that the output from resnet does not show variance within the same batch. Since we have frozen gradient backpropagation on the resnet we are not changing its weights but only changing weights on the vocal net. We deduced that the vocal-net is not enough to further reduce the loss and since res-net was trained on image classification task it is giving nearly similar feature vectors for every type of face. We decided to unfreeze the resnet and let the gradient flow through it as well.
01/05/2018: Jitteriness of the stepper motors is heavily restricted the capability of our system. Fluid tracking becomes more important when dealing with 2 degrees of freedom because lag time and jitteriness combined together can make you lose the human in the scene quickly. We plan on solving this by using Servo motors instead of these steppers.
02/22/2018: Integrating and sequence in text modality has been a bit of a challenge. Difficult due to lack of experience in the team with the relevant subject matter.
03/22/2018: Dealing with jittery motion in the face tracking has been a challenge. This is being resolved by stepping the motion of the servo motors and editing the image to make sure face detection is more accurate
04/05/2018: Integrating the many subsystems led to many issues. For example, text input interfered with the display of the GUI. So we resolved this by multiprocessing for each separate task.
05/05/2018: The text subsystem confuses a lot of values with the ‘sadness’ class. This subsystem will have to be retrained.
05/07/2018: The state machine was found to interrupt the actor during tests. This was removed as it was not an SVE requirement.
05/08/2018: Lukabot loses tracking when the subject looks down and the face isn’t visible. This was fixed by smaller movements and a small degree of searching until face is in focus.