J. Hu, “Move, But Right on Time,” in 1st European workshop on design and semantics of form and movement (DeSForM), Newcastle upon Tyne, 2005, pp. 130-131.

Move, but right on time

Jun Hu

Department of Industrial Design

Eindhoven University of Technology



We have entered a new media era: passive television programs become interactive with the red button on your remote control (Bennett, 2004). Video games come with many different controlling interfaces such as dancing mats, EyeToyimg cameras, driving wheels and boxing GametraksTM (In2Games, 2005). The D-BOXimg OdysseeTM motion simulation system even introduces realistic motion experiences, which were originally designed for theme parks, into our living rooms (D-BOX, 2005). In the vision of Ambient Intelligence (Aarts and Marzano, 2003), the next generation of people’s interactive media experience will not unfold only on a computer or television, or in a head set, but in the whole physical environment. The environments involve multiple devices that enable natural interactions and adapt to the users and their needs.

However involving multiple devices might also have a negative effect. It might increase the complexity of interaction. The environment together may become difficult to understand and to control. To ease the situation, embodied characters, such as eMuu (Bartneck, 2002) or Tony (Bartneck and Hu, 2004), may be used to give such an environment a concrete face. These characters have a physical embodiment and may present content through their behavior and interact with the user through speech and body language. They can even be used as input devices.

Timing and Meaning

This robot, Tony, is not more than just a toy. Built from Lego Mindstorms bricks, it does have a face and a body. If certain body parts are touched, it can make a smiling face, or draw a long face. It can also turn its body left and right. Great, with these mechanic movements, this toy is already a nice toy, a toy that moves.

But more can emerge from these mechanic movements. With an infrared link, it is connected to an interactive movie. The toy suddenly becomes an actor: It laughs at another character in the movie; it looses its attention because nothing exciting is happening; and it looks sad when the buddy in the movie fails to get a job – it almost becomes an actor for the movie, but embedded in a physical body, with physical movements.

With this demo, we show how emotions, attentions and even narratives can come out of the mechanic movements, by simply making these movements happen at right times. Timing gives the movements meanings.



The robot is a standalone application running on a Java Virtual Machine for Lego RCX, integrated with touch, rotation and infrared sensors, and movement, speech and lighting actuators. It receives the timely emotional events from the movie and reacts on it. The actual behaviors reacting on the events also depend on the mood of its one.

The movie, scripted in an XML based language (Interactive Play Markup Language, IPML for short), is played back by a central scheduler (called director in IPML). The director assigns actions for the distributed actors (in our demo, they are a full screen movie player, a robot, and a remote control) that are connected through a local network. Actions can be start or stop playing a video clip, showing a particular emotion or detecting an infrared signal. Actors are described in IPML scripts as requirements at an abstract level so that it can be presented in diverse environments. The director has to map these abstract requirements to available actors in a particular environment at run time.



No product is isolated from a context or an environment. Movements of products can be used as a new media to convey actions, emotions and even narratives in connection with other products, especially in an ambient intelligent environment. The movements can be designed for the product itself for its own sake, however, when in connecting with other products, the cooperation, especially in time, is important.


E. Aarts and S. Marzano. The New Everyday View on Ambient Intelligence. Uitgeverij 010 Publishers, 2003.

 C. Bartneck. eMuu - An Embodied Emotional Character for the Ambient Intelligent Home. Phd thesis, Eindhoven University of Technology, 2002.

C. Bartneck and J. Hu. Rapid prototyping for interactive robots. In F. Groen, N. Amato, A. Bonarini, E. Yoshida, and B. Kröse, editors, The 8th Conference on Intelligent Autonomous Systems (IAS-8), pages 136-145, Amsterdam, 2004. IOS press.

 J. Bennett. Red Button Revolution - Power to the People. 2004. URL http://www.bbc.co.uk/pressoffice/speeches/stories/bennett_mip.shtml.

D-BOX. D-BOX Motion Simulators. 2005. D-BOX Technologies Inc. website: http://www.d-box.com.

In2Games. Gametrak :: Come in and Play. 2005. In2Games Corporate and Press Site: http://www.in2games.uk.com.

Valid XHTML 1.1 Valid CSS!