Finding Elevated Fear Through User Mouse Input

Over the last couple months, I have been contemplating ways to predict heightened levels of fear in a computer user based solely on their mouse inputs. I am classifying a heightened level of fear as an increase in heart rate and heart rate variability.

Originally, I planned on gathering a large data set that would consist of the user's mouse movements, heart rate, and heart rate variability, during a period of calming tasks as well as tasks designed to frighten the user. I was thinking I would then attempt to mathematically analyze the data set to find a correlation between heightened fear and specific patterns in mouse movements. Most prominently, I was hopeful to make us of the Broad Institute's MINE to assist me in this endeavor.

While I still plan to obtain a large data-set and analyze it with MINE, my priorities have shifted towards something more promising. In my research, I discovered that researchers have had a great deal of success in modeling the dynamics of the human arm, wrist, and hand by using a second-order model of a mass-spring-damper system. The damper in this context refers to muscle tension. Furthermore, not only has muscle tension been showed to have a direct correlation to elevated levels of stress, researchers at UC Berkeley have successfully been able to predict elevated levels of stress using only mouse movements. By retrieving the parameters of the mass-spring-damper system using the system's output (the mouse movements), they were able to predict elevated stress based off the damping frequency and coefficient.

The researcher's at UC Berkeley used an audio-processing technique called Linear Predictive Coding, a method of predicting future values of a signal from its previous values. LPC uses a feature of linear algebra called the Linear Prediction Model. By treating the mouse movements as a signal, and applying the Linear Model to it, I can retrieve a series of coefficients that describe the characteristic polynomial of the MSD system. The complex roots of this polynomial describe the various damping properties of the MSD.

Below can be seen the equation that describes the prediction model;

where x_hat(n) is the predicted value of the signal, x(n - i) is the previously observed value, and a_i is a p length series of coefficients. It is the coefficients a_i that describe the characteristic polynomial of the MSD.

To retrieve the values of a_i, I need to compute the least-squares solution to the equation Xa = b, which is described by the following matrices:

where m is the length of x. This can be reduced to a Yule-Walker equation as:

where r is the autocorrelation matrix of x.

By inverting the r matrix and multiplying it by the -r matrix, I will retrieve the transposed form of the a matrix, and henceforth will have determined the coefficients of the MSD's characteristic polynomial.

By following this process, I will have a foundation to start to predict elevated fear. However, while the UC Berkeley team achieved ~70-80% accuracy, it is highly likely that I will achieve noticeably less accuracy. This is due to the fact that their research used very task-specific models; tasks that require complex movement such as gaming very well may prove to be more troublesome. To remedy this, I will be using a more advanced form of the MSD system, as well as making use of the data set I will have at my disposal to further enhance the accuracy of this algorithm.

Currently, I am working on implementing this using SciPy