PhD thesis by Siti Aisyah Binti Anas
Nowadays, objects are embedded with various sensors, making the objects knowledgeable and smart, gradually reducing users’ need to intervene. As a result, these intelligent objects that work quietly in the background are perceived as passive and reactive objects when interacting with users. The lack of communication between the objects and the users impedes the objects from being smarter and understanding the users’ requirements. Recently, many researchers actively and continuously research to improve the users’ interaction and engagement with the objects. The research objectives are to increase and improve the users’ awareness when interacting and engaging with the objects. Hence, it is essential to design useful feedback or feed-forward methods to indicate the objects’ internal operation state and facilitating communication with engaging users. Another different method, direct manipulation of the objects that exploits the human skills, also enhanced the interaction and engagement between the objects and the users. Looking at that, yet most human-object communication adopts one-directional communication, where the human always acts as the initiator when interacting and engaging with the objects. Thus, it is questionable and remains arguable to understand the objects’ smartness that could initiate and continuously communicate with the users.
The perceptual crossing paradigm is a paradigm that can create smartness in an object that could initiate and continuously communicate with the users when it is interacting and engaging with the users. The perceptual crossing paradigm also provides the necessary conditions that allow a person to recognize the other interacting entity as an intentional or reactive entity. Furthermore, the perceptual crossing paradigms emphasize the bidirectional and proactive communication between the person and the interacting entity. This paradigm also allows the interacting person to experience the differences during an interaction with an entity capable of initiating the communication or reacting towards the persons’ presence. Looking at the addressed limitations in interaction design and the perceptual crossing paradigm’s advantage, designers had adopted the perceptual crossing paradigm, which improves the richness and empathy of the human-object interaction. However, the conducted work does not emphasize the bidirectional and proactive interaction between the object and the interacting person. Hence, this thesis aims to design an object capable of expressing its intention to interact in a bidirectional interaction to improve the human-object interaction. Therefore, this work implemented the perceptual crossing paradigm to investigate the object’s bidirectional and proactive behavior towards the interacting and engaging person.
Three studies were conducted to investigate the bidirectional and proactive interaction design based on the perceptual crossing paradigm. Visual attention is chosen as an interaction method to allow the object to differentiate an intentional user from a potential user. For instance, a user looking at the object can be interpreted as an intentional user that wants to interact. In contrast,
a user presence nearby the object can be interpreted as a potential user. In the first study, a proactive object capable of expressing its intention to communicate bi-directionally with a user is designed and developed using a simple abstract motion. An eye tracker is used to detect the user’s visual attention and is integrated with the proactive object, allowing the object to initiate an interaction when a user’s visual attention is detected. An exploratory user study involving 30 participants is conducted to confirm the developed proactive prototype’s viability. The results show that the users did not achieve bidirectional interaction and unable to realize the prototype’s proactive behavior.
Based on the perceptual crossing paradigm, even if the communication channel is reduced to a bare minimum (i.e., touch perception as the perceptual quality), two intentional entities can still recognize each other as long as they experienced the same perceptual environment. These perceptual activity outcomes suggest that people can interact with each other by depending on a single interaction method if they are involved in the same perceptual environment. Therefore, with visual attention as the only perceptual quality used to interact with the object (i.e., eyes), the object should also show the same perceptual quality to enable the interaction to occur in the same perceptual environment. Therefore, in the first study, the absence of visible expressive perceptual quality makes the users unable to experience bidirectional interaction and understand the object’s proactive behavior.
In the second study, an improvised prototype mounted with a visible expressive perceptual quality similar to that of human eye contact (i.e., artificial eyes) is introduced. A camera module is used to detect the user’s visual attention and is mounted together with artificial eyes. Hence, the proposed artificial eyes allow the prototype to create back-and-forth eye-to-eye contact interaction with the intentional user, and therefore, bidirectional interaction can be achieved. An exploratory user study experiment is conducted to validate the human-object bidirectional interaction. The results show that the artificial eyes staring with blinking expression help the user experience bidirectional interaction and engagement with the prototype. However, to maintain a continuous bidirectional interaction, the artificial eyes need to express proactive behavior besides staring and blinking. Therefore, the artificial eyes are proposed with proactive expressions such as winking and pupil dilation to allow the user to maintain continuous bidirectional interaction with the prototype.
In the third study, a conceptual model called Session Initiation for Proactive Object (SIPO) based on the perceptual crossing paradigm is proposed. The SIPO conceptual is referred to achieve bidirectional and proactive interaction between the object and the user. To pre-evaluate the SIPO conceptual model in single- and multi-user scenarios, two pilot studies which are, 1) visibility of the perceptual quality and 2) expression of intentions, are conducted. The SIPO conceptual model is implemented into a prototype mounted with artificial eyes and a camera module on top. Abstract motion is integrated to allow the prototype to orient its body towards the users. The pilot studies in single- and multi-user scenarios are conducted to validate the SIPO conceptual model. The pre-evaluation results show that the prototype mounted with artificial eyes and abstract motion successfully achieved the bidirectional interaction and engagement with the users in single- and multi-user scenarios. The achieved results only show bidirectional aspects instead of bidirectional and proactive interaction and engagement. The study was then expanded, involving 28 participants using a real-environment user study. The real-environment user study introduces a primitive physical object mounted with artificial eyes that express staring, winking, blinking, and turning
viii
behaviors. These four different expressive behaviors are analyzed to validate the object’s proactive behavior towards the users in single-and multi-user scenarios. The results were taken into account for implementing a virtual object mounted with artificial eyes. A crowd-sourced video-based user study involving 240 participants with the virtual object expresses staring, winking, blinking, and turning behaviors is conducted. The results show and validate that the winking expression successfully revealed the object’s proactive behavior towards the interacting and engaging user and encourages reciprocal input. Therefore, in reflections of the achieved results, this research successfully implemented, evaluated, and validated the bidirectional and proactive interaction and engagement between the object and the user based on the perceptual crossing paradigm.
In conclusion, this thesis presents a perceptual crossing design with artificial eyes based on the perceptual crossing paradigm to improve human-object interaction and engagement. An initiative-taking was adopted. An object was augmented with a visible perceptual quality (i.e., artificial eyes) which provides the ability to express its desire to interact and engage with the person of interest. The design and development of the object are further improved by introducing the SIPO conceptual model, which enhances the object’s initiative-taking and the object’s proactive communication to maintain human-object interaction and engagement. The results show that the artificial eyes winking expression is proactive towards the interacting and engaging person. This thesis’s presented work could be a starting point for designers to develop a practical yet straightforward bidirectional and proactive interaction design based on the perceptual crossing paradigm to improve the human-object interaction and engagement.