Category: Articles

Jun Hu

The concept of Ambient Intelligence (AmI) is introduced by Philips Research as a new paradigm in how people interact with technology. It envisions digital environments to be sensitive, adaptive, and responsive to the presence of people. This PhD project believes that AmI environments will change the way people use multimedia services. The environments, which include many devices, will play interactive multimedia to engage people in a more immersive experience than just watching television shows or listening to radio programs. People will interact not only with the environment itself, but also with the interactive multimedia through the environment.

For many years, the research and development of multimedia technologies have increasingly focused on models for distributed applications, but the focus was mainly on the distribution of the media sources. Within the context of AmI, not only are the media sources distributed, the presentation of and the interaction with the media will also be distributed across interface devices. This PhD project focuses on the design of the structure of multimedia content and the distributed interfaces, believing that the user experience of multimedia in a distributed environment can be enriched by structuring both the media content at the production side and the playback system architecture at the user side in a proper way. The structure should enable both the media presentation and the user interaction to be distributed and synchronized over the networked devices in the environment. The presentation and interaction should be adaptive to the profiles and preferences of the users, and the dynamic configurations of the environment.

The design process went through three design iterations, following a spiral model. The first iteration was needed to get some first-hand experience and the preliminary requirements. It was concluded with the structure of StoryML and a demonstrator TOONS, which put the requirements for the second iteration on a stable foundation. More user requirements and technical challenges emerged in the second iteration during the development of the demonstrator DeepSea. DeepSea was used for the user evaluation of the concept of interactive multimedia in distributed environments. The design and development were brought forward based on the technical requirements and the experience gained from the second iteration. The design was completed with a full implementation of the proposed architecture, based on open standards and technologies. Three more demonstrators (TOONS in IPML, TheInterview, Mov’in) were built to validate the design. TheInterview was used to test the effect of the user’s cultural background on their perception of presence in interacting with these multimedia presentations in a distributed setting. During the project, the proposed architecture was used in implementing applications for various projects, ranging from a big EU project together with professional developers, to a small educational project by a team of four second year university students. Each resulted in a working demonstrator or prototype.

During the design process, several formal methods were used, including Object-Z, Broy’s component based framework, and the Petri net based OCPN and ASE. The formal approaches were helpful: They are abstract enough to make mechanisms explicit in a clear way and at the same time they are concrete enough to see the structure of the implementation. However it was not possible to formalize every aspect of the system. The system is far too complex for that. Several formalisms and several types of “syntactic sugar” were used to keep things manageable. Yet the most important design concepts were covered by formalizing them in a pragmatic way.

As a direct result of this design, a generic architecture has been implemented on top of existing network technologies and platform architectures, to enable playing IPML scripts in a networked environment with user preference and dynamic configurations taken into account. Aside from the architecture, this project also contributed to the body of scientific and engineering results. Examples are the method of rapid robotic prototyping, the extension of Petri nets as action synchronization engine, and new architectural patterns such as Timed Action, Synchronizable Object, Real-time Channel and Streaming Channel.

The results of user evaluations offer preliminary insight into how distribution, level of control and number of users influence user’s fun and presence experience in interacting with an AmI movie. Notably the level of control, and the number of the users, have a positive influence on the user experience. The influence of distribution on the user experiences is also observed and there can be an effect depending on the type of the distributed content and how the distribution is arranged. An effect of cultural background upon the perception of presence in a distributed setting is also observed. As far as known, it is the first time that the long-standing conjecture on this effect is confirmed experimentally.