Symbiosis -
Living with Intelligent Systems



10-week Project

UID was invited to participate in the Microsoft Design Challenge 2016. The brief and general goals for the challenge alligned well with this course's learning objectives.

Design Challenge  

Achieving Symbiosis and the Conversational User Interface (CUI)

Design a product, service or solution that demonstrates the value and differentiation of the CUI. Your creation should demonstrate the best qualities of a symbiotic human-computer experience which features an interface designed to interpret human language and intent. Of course, language takes many forms - from speech, to text, to gesture, body language, and even thought. Your creation should clearly demonstrate foundational elements the CUI calls upon in order to delight people. It should meet a clear need and be extensible to wider applications. It may be near-term practical or blue sky, but the idea must be innovative, technically feasible, and have a realistic chance of adoption if instantiated. Of course, to deliver an optimal experience, much is implied - from data and identity permissions to cross-app agent and/or bot cooperation and coordination (first and third party); your design should minimally show awareness of these barriers or explore solutions to them.

UID interpretation
Symbiosis - Living with Intelligent Systems

At UID we took this brief as an opportunity to emphasize interaction design as defining the relationship between people and intelligent systems and the role of such systems in our lives. Complexity of real world contexts and people centered approach was initiated by connecting this course to parts of the work done in the preceding course on Communication Design for Co-creation, in which students engaged and immersed in real life contexts by means of ethnographic fieldwork.

16 students participated in 4 groups.
The resulting projects (summarised below):
*Nume - personalised nutrition
Koala - a conversational bus-service assistant
Luna - weaving bedtime stories
Epa - sustainable living assistant

*this project was selected to represent UID and the course at the 2016 Microsoft Research Design Expo.

Tutors from Microsoft
Don Coyner
Annika Ushio
Soramist Chintanamanus

Course responsible
Stoffel Kuenen

Instructor from UID
Niklas Andersson
Brendon Clark



Nume - personalised nutrition
by  Andre Kennedy, Joanna Pruchnicka, Yuanrui Li and Lene Rydningen

Food. It's essential to our wellbeing and way of life. However, our society has become disconnected in regards to how it affects us both physically and psychologically. But why is this, and how can we address it?

Our research demonstrated to us that there was a major gap in the communication of how the food we eat actually affects us. One banana for example contains B6 and a good source of manganese, vitamin C and so on. But what does this actually mean? Further the feedback of how this food will influence us is very delayed. These factors have been a major barrier for people making informed decisions about their diet. When approaching this we wanted to examine the following key questions: how can we empower people to understanding how their food affects things like sleep, energy and fitness. How can we teach people to change their cooking rituals according to what they like and need?

Nume is our solution to the issues addressed above. Nume is dynamic. She learns about the users; their eating habits, daily routine and so forth. Further she is informed. Using a transdermic patch she is able to see the nutritional blood values of each user. Armed with this knowledge she is able achieve a number of things. Firstly she can interpret what nutrients we are lacking and indicate how this may be affecting us. Secondly she is able to suggest meals based on what out body needs and our personal taste preferences. Thirdly she is able to provide real time feedback in laymen terms as to whether our new eating habits are improving the nutritional values identified as being low. Finally she is able to give goals to help motivate and encourage users to continue following these new eating habits. In combining these things she is able to create a feedback loop, where by she can educate users as to what their body needs and why it needs it. She is then able to motivate and push them through feedback and goals as a means to encourage them to change. In doing she creates a continuous learning cycle whereby users are finally able to understand what nutrition means to them.

Media: presentation video


Koala - a conversational bus-service assistant
by Melissa Hellmund, Christop Zobl, Shigeo Katsura-Gordon, Eduardo Pereira

Koala is a conversational service assistant for the public transportation system of Umeå. The project demonstrates the value and differentiation of CUIs (conversation user interface) within a real and familiar context, highlighting how language can and should be used in combination with additional sensory information, to make such interfaces usable and approachable. By taking the concept into a public service, Koala challenges the norm of simply applying a CUI as an input method for the common personal assistant bot on your phone, and helps to re-interpret the symbiotic relationship we have with technology. During the ethnographic research stage of this project, a number of social and cultural norms were observed, which are challenged with the help of Koala and the set of design principles it is based on.

To summarise, Koala creates a sense of presence through strategic use of lights responding to breach of personal space of the machine, as well as indicating when and who is to speak, defined as turn-taking. The language and tone Koala uses makes it clear that there is no human processing in the background, as this changes the way users interact. And finally, through building a conversation map and developing responses, Koala is able to gracefully divert from the its system's limits and guide conversation resulting in smooth and efficient interactions.

Media: presentation video


Luna - weaving bedtime stories
by Akansha Aggarwal, Eduardo Ferreira, Hector Mejia, Ben Lee

Telling bedtime stories to children has shown significantly higher brain development among them. Listening to bedtime stories not only leads to mental growth, but also inculcates moral, cultural values and enhances imagination among kids. Through our ethnographic research, we investigated into the current lifestyle of the parents; We found out that the culture of telling bedtime stories to kids is fading. Further, Some parents are good at telling stories, some of them are dreaded by the idea of it and some fail to keep the daily routine of narrating stories to them.

Luna is a system, which assists parents while telling stories. It consists of two elements: wearable token and the bedside device. The wearable token is the part of kids' daily life. The small token hears and understands the kid's surroundings by semantic listening. It doesn't act as a sneaky spy, but more as a friend of the kid. It process filters the conversations and selects the highlights in kid's daily activities. The second element of the system, bedside device gets activated when the token is embedded in it.

The child's day is processed and contextual stories are created in the system's story bank. Luna responds to voice commands and request of the kid and the parent. Both kids and the parents could pick and choose the story created by Luna.
In various situations, the system adapts to the scenario. In certain situation, it dominates the situation where it narrates the story. It creates dramatic pitch to engage and make story listening process interesting for the family. In assistance mode, Parent leads the scenario, where the system gives cues to the parent making storytelling process effortless for parents. It creates environmental sounds to make stories more fun and produces low fidelity visuals to give a gentle and a mellow end to the kid's day.

Luna is an attempt to formulate the conversational user interface for the kids. The system requires training and grows with the kid by learning his or her daily life. It stores and narrates stories that are tailored to kid's everyday life. It's a framework where telling a story is convenient and less painstaking for the parent. Luna is an effort to strengthen the bond between the parent and the kids in today's busy life. 

Media: presentation video


Epa - sustainable living assistant
EPA V1 1
by Yi-Ting Chien, Siddharth Hirwani, Piotr Kuklo, Yue Yuan

Epa is a conversational bot that lives in the future smart home.  It is a bot that would help change your consumption habits to live a sustainable lifestyle.  Epa is ubiquitous in your home and monitors your consumption to create a baseline to start from and then using tasks and challenges it helps you change your behaviour towards consumption of resources like water and electricity and also helps you sort waste correctly.  It is like a companion keeping you motivated through the the challenges, awarding you when you perform well and positively encouraging you even when you don't.  Short term and long term benefits ensure you stay motivated through the challenges and keep you going.  

The personality of the bot is related to the personality of each user.  The bot expresses itself not just in a conversational manner but also graphically, creating a physical presence.  It immerses itself into user's daily activities and home, understanding the appropriate time to appear and have a conversation rather that interrupt the user and his activities. Since the user may not always want to converse, Epa communicates in different ways like for example using music as guidance for the amount of water consumed while showering, switching from the users favourite songs to disliked ones.

Symbiosis comes from the partnership to help the user to change consumption habits with a playful experience.  With the help of a conversational user interface (CUI) it helps raise awareness for the environment.  If this can encourage each person to turn off the water tap or the switch off the lights or recycle, together as a community we can be more sustainable.

Media: presentation video