RESEARCH QUESTION:
“How do users interpret different physical behavior, based on human body language, in a Smart Home Assistant in a multi-user scenario?”.

Overview

I approached this project during the Erasmus period at the Tu/e in Eindhoven in the Social Interactions with Shared Systems squad. Shared systems are used by multiple people, where the interaction of one person affects other people as well. The system can be used at the same time or with turn-taking. In your actions in shared systems, you have to think about other people. This is why the designers of these shared systems have to take into account aspects like awareness, accountability and translucency.

PERSONAL IMPROVEMENTS

It was very stimulating to compare myself with students and professors from another university and culture, relating to a new Design approach and improving my relationship skills. Furthermore, it was the first project in which I participated in the realization of a scientific paper. It was challenging to design and then carry out the study, it allowed me to face new situations but at the same time, it stimulated me. I definitely deepened my knowledge about design research and the methods used to bring insights out of it. Besides, with my group we worked very hard in the prototyping phase, improving different practical and IT skills.

Topics

Design research
Smart home assistant
Shared system
Research study
Paper writing
Prototyping
Coding

Design Research

Design Research Course - Group project

Apr - Jul 2018

The goal of this research was to make a framework for designers about the implementation of physical behavior in Smart Home Assistants for a multi-user scenario.

AUTHOR

Bart Bolluijt; Joline Frens; Riccardo Gualzetti; Lars de Langen

ABSTRACT

In this paper exploring research is done for the implementation of physical behavior in Smart Home Assistants for a multi-user scenario. Repertory grid was used to understand the perception of different prototypes that had added physical behavior. We tested 8 groups of 3 people, resulting in 24 participants, who were all students. The participants used the prototypes within their group, then covered the repertory grid individually and finally discussed the prototypes together. The findings and discussion can be used as a framework for future designers of Smart Home Assistants that will be used in a multi-user scenario. An important finding, is that as soon as a physical behavior becomes too vague or unclear, this will have a big influence on the interpretation of a Smart Home Assistant. This influences the device negatively for multi-user interaction.

Author Keywords: Smart home assistant; human behavior; multi user scenario; prototypes; body language;

PAPER

Download

PIE

The Pie prototype is based on the movement of leaning towards someone with whom you are having a conversation. The prototype looks like a flat cylinder cut into 6 equal parts. When someone talks, a part in their direction will move out.

POINTER

The prototype is based on turning your body towards the person and using hand to express attention. It was a high cylinder that could turn around its centre, with an arm that would move out when it was listening and talking to someone, pointing the person it was interacting with.

BALL

The Ball prototype was based on making eye contact and turning your head when you talk. It is more subtle than the Pointer prototype, but still based on a circular movement. It looked like a low cylinder with a circular engraving and a metal ball that would roll towards the person that is talking.

BEADS

The Beads prototype is a short cylinder with six sticks sticking out in a circle. This is based on the human behaviour where someone will raise their eyebrows when listening to someone. The sticks of the prototype move up towards the side where someone calls the Smart Home Assistant.

4. STUDY AND RESULT

For our study, we decided to use the research method repertory grid. In repertory grid, the participants are asked to rank the four prototypes on different scales. The study would start with 8 groups of 3 the participants each one, experiencing the prototypes. After this, the groups were split up and the participants were asked to rank the prototypes on eight different given scales individually. Finally, the participants would come back again and discuss their answers and why they give different answers.

Using the repertory grid gave us a lot of different possibilities for analyzing the data we got, and looking for correlations and interesting findings. We gathered our quantitative data from the repertory grid scales in one big spread sheet to try and get an overview of our findings. Using this data we calculated the average, standard deviation, median and mode for all the prototypes on the different scales. We then used this information to visualize our results in box-plots and provide ourselves with a clear visual overview of the data we got.

To extract our findings and conclusion we used the quantitative data to find interesting connections and phenomena between the scales, but also looked at specific experiences participants had with our prototypes. By looking at the qualitative data we tried to find reasons for connections between scales or divergent values in the repertory grid. We used these quotes to support our framework and put it in perspective so it is useful for future research.

FINDINGS

We have found an interesting correlation between the Clarity of a device and the ability to be Involved with the users. If the added physical behavior in a Smart Home Assistant is made very clear, this will most probably result in the user feeling that the device is involved in the conversation. Also, improved Privacy has a negative effect on the perceived Translucency in the device. When you add physical behavior which creates privacy for the user, the information for other people present will be limited and thus decrease the social translucency of the device.

Most other scale comparisons do not make up for interesting results at first glance. If you look at some scales compared with the scale Inviting for a multi-user scenario, the Pointer is a clear outlier. The comparison between Inviting for a multi-user scenario and Clarity including the Pointer prototype’s value, shows that they are not directly proportional. If we exclude the Pointer value, the relation does become proportional. This comparison and the comparison between Involved and Clarity make a directly proportional relation clear among the three scales: Clarity, Inviting for multi-user scenario and Involved. This means that if the interaction resulting from the added physical behavior on the device is more clear, it is more inviting for a multi-user scenario and also more involved.

2. DESIGN PROCESS

To find good designs which are useful to compare within our study, we went through an elaborate design process. We have made around 40 concepts and prototypes from scrap material to get a feel on what the Smart Home Assistant of the future may look like and explore the possibilities. We did this with a focus on the implementation of physical behavior in such a Smart Home Assistant. From this exploration, we discovered certain aspects in which we could divide all the concepts, and certain human behavior they mimicked.

After all this exploration, we extracted the most viable and distinguishable concepts. We mapped out these concepts looking at the openness of the concept and the possibility of expression. To make our research as valuable as possible, we decided to pick the 4 designs which we believed to be at the ends of both spectrum, from now on called Pointer, Ball, Pie and Beads. Comparing the final results of the research with these designs would provide more specific results in our design framework.

We realized we could not make four voice controlled prototypes. That is why we decided early on to use the Wizard of Oz method for the interaction with the prototype and control the prototypes ourselves. For this we made Graphical User Interfaces for all the prototypes in Processing.

3. PROTOTYPING

The goal was to create prototypes that were realistic enough to make the simulation effective. The prototypes were made of wood and were controlled by several servomotors connected to Arduino boards inserted inside them. The boards received the commands directly from the respective GUI, which were controlled by us based on the situation that was taking place in the simulation.

Knowing the importance of the reliability of the prototypes during the simulation, we worked hard to create four prototypes that gave the sensation of moving independently. In addition to improving our prototyping skills and mechanical knowledge, we learned how to program both for Arduino boards and GUI. The result was at the level of expectations and during the simulations, the students claimed that they were not influenced by the ‘Wizard of Oz’ method used to simulate the movement.

RESEARCH EXECUTION

1. IDEATION

At the beginning of our project, we decided to do a group brainstorm, bringing at least 20 ideas per person that were relevant to the squad topic. In this way, we have obtained more than 80 ideas that we have examined and evaluated together to find an interesting topic to carry out. Thanks to these 80 ideas we found three topics/scenarios that seemed the most interesting to us to explore: Personal Sound System, Shared Menu in a restaurant, and Decision Making in shared systems.

We decided to analyse these possible topics in more detail through four questions related to the concept of awareness in shared systems:

1. What do people need to be aware of?

2. What information is needed for this awareness?

3. How in the information gathered and presented?

4. How can people act based on this awareness?

During this process, discussing about an AI system useful to solve conflicts in a social shared situation, we started talking about voice assistants. The topic immediately seemed interesting and relevant to the context of our squad, since these devices still have obvious limitations in shared use. In this lack we have seen the possibility of a research that investigates how these devices can be developed to better adapt to a situation of sharing.

The next step was to learn more about the state of the art of these devices and their role in human-computer interaction. This research was useful to focus more precisely on the path to follow and to define a possible research question: “How does a Smart Home Assistant (SHA) influence the behavior of the human?”.

Subsequently, continuing the literature research and discussing with our coaches, we have redefined the question several times up to the final one.

CONCLUSION

With these findings, we tried to find the connection between the perception of human behaviour that was expressed by the device and its effect on a multi-user conversation. In that way, we tried to create a framework that allows designers to have a clear idea of which features they should put in a SHA to reach a particular interaction with users.

To design a SHA for a multi-user scenario, a few characteristics (which are caused by the implementation of certain human behaviour) were found to be more relevant than others. Our findings claim that Involving and Clarity are strictly connected to Inviting in a multi-user scenario. These characteristics can be reached through a direct connection to the user. Directly pointing is a great example where Clarity and Inviting for a multi-user scenario are positively influenced. Having feedback for the user whether they can speak increases clarity of the device. The possibility to point to more than one user at the same time has a great positive effect on the perception of the SHA stimulating multi-user interaction. Another benefit is when the movement is smooth, giving the possibility for other users to see what is happening. In addition, other characteristics like Privacy, Human-likeness and Natural resulted to not be so relevant in terms of Inviting in a multi-user scenario.