An Authoring Framework for Emotion-Aware User Interface of Mobile Applications

Abstract

Since affective computing has been introduced in 90s, affect recognition technology has achieved substantial progress recently. However, the application of user emotion recognition into software user interface is in its early stages. In this paper, we describe a new approach for developing mobile user interface which could react differently depending on user emotion states. First, an emotion reaction model is presented which determines user interface reactions for each emotional state. We introduce a pair of mappings from user states to different user interface versions. The reacting versions are implemented by a set of variations for a view. Further, we present an authoring framework to help developers/designers to create emotion-aware reactions based on the proposed emotion reaction model. The authoring framework is necessary to alleviate the burden of creating and handling multi versions for views at the development process. A prototype implementation is presented as an extension of the existing authoring tool DAT4UX. Moreover, a proof-of-concept application featuring an emotion-aware interface is developed using the tool.