Hao Liu, one of the PhD students I supervised, defended his thesis “Biosignal Controlled Recommendation in Entertainment Systems”, on Monday, December 13, 2010.
[DOWNLOAD | PDF, 2M | Biosignal Controlled Recommendation in Entertainment Systems ]
With the explosive growth of entertainment content and the ubiquitous access to them via fixed or mobile computing devices, recommendation systems become essential tools to help the user to find the right entertainment at a given time and location. I envision that integration of the biosignal input into the recommendation process will not only help users to find interesting content, but will also be instrumental in increasing their comfort level.
The goal of this project was to develop a biosignal controlled entertainment recommendation system that increases the user’s comfort level by reducing his or her level of stress.
This project aims to contribute to the field of recommendation systems on two points. The first is the mechanism to embed the biosignal non-intrusively into the recommendation process. The second is the strategy behind the biosignal controlled recommendation to reduce stress. Heart rate controlled in-flight music recommendation is chosen as the application domain. The hypothesis is that the passenger’s heart rate deviates from the normal due to the unusual cabin environment during the long haul flight. By properly designing a music recommendation system to recommend heart rate controlled personalized music playlists to the passenger, the passengers’ heart rate can be uplifted, down-lifted back to normal or kept within the normal range, thus reducing his or her stress. Four research questions have been formulated based on this hypothesis.
After the literature study, the project went mainly through three phases: framework design, system implementation and user evaluation to answer these research questions.
During the framework design phase, the heart rate was firstly modeled as the states of bradycardia, normal and tachycardia. The objective of the framework is that, if the user’s heart rate is higher or lower than the normal heart rate, the system recommends a personalized music playlist accordingly to transfer the user’s heart rate back to normal, otherwise to keep it at normal. The adaptive framework integrates the concepts of context adaptive systems, user profiling, and the methods of using music to adjust the heart rate in a feedback control system. In the feedback loop, the playlists were composed. Yet, the framework allows the user to reject the recommendations and to manually select the favorite music items. During this process, the system logs the interactions between the user and the system for later learning the user’s latest music preferences.
The designed framework was then implemented with platform independent software architecture. The architecture has five abstraction levels. The lowest resource level contains the music source, the heart rate sensors and the user profile information. The second layer is for resource management. The manager components in this layer manage the resources of the first layer and modulate the access from upper layers to these resources. The third layer is the database, acting as a data repository. The fourth layer is for the adaptive control, which includes the user feedback log, the inference engine and the preference learning component. The top layer is the user interface. In this architecture, the layers and the components in the layers are loosely coupled, which ensures flexibility.
The implemented system was used in the user experiments to validate the hypothesis. The experiments simulated the long haul flights from Amsterdam to Shanghai with the same time schedule as the KLM flights. Twelve subjects were invited to participate in the experiments. Six were allocated to the control group and the others were allocated to the treatment group. In addition to a normal entertainment system for the control group, the treatment group was also provided with the heart rate controlled music recommendation system.
The experimental results validated the hypothesis and answered the research questions. The passenger’s heart rate deviates from normal values. In our user experiments, the passenger’s heart rate was in the bradycardia state 24.6% of time and was in the tachycardia state 7.3% of time. The recommended uplifting music reduces the average bradycardia state duration from 14.78 seconds in the control group to 6.86 seconds in the treatment group. The recommended keeping music increases the average normal state duration from 24.66 seconds in the control group to 29.79 seconds in the treatment group. The recommended down-lifting music reduces the average tachycardia state duration from 13.89 seconds in the control group to 6.53 seconds in the treatment group. Compared to the control group, the stress of the treatment group has been reduced significantly.