3 October - 29 November 2009
For the ninth in a series of ten artistic and scientific experiments at Fabrica, Tina Gonsalves and her team created Chameleon. For this installation, the gallery became an emotional theatre in which visitors and digital video portraits could interact with each other. Gonsalves and her team developed software that would seek out visitor’s emotions and then trigger an ’emotionally intelligent and learning’ computer system to playback video portraits in response:
“In Chameleon, individuals become intimately connected and implicated into varying emotionally provocative and reflexive social interactions leaving them with a better understanding of how their own non-verbal communications have an impact in social groups.”
In this exhibition of ever changing portraits, the emotional expressions of the visitors had a direct effect on the tone and emotional state of the gallery space itself. Gonsalves worked to highlight how often we attempt to make sense of each other and how we search for meaning and significance in expressions, dialogue and body language. By collaborating with a team of leading neuroscientists, affective computer scientists, emotional designers and curators, Gonsalves was able to further explore the intersections of art, technology and science.
The Chameleon Project
The Chameleon project has developed the understanding of emotional contagion from a variety of viewpoints during the course of its multiple iterations. It has involved a dynamic team of artists, scientists, designers and engineers in new research and engaged a broad range of participants in forming the video content and evaluating its effectiveness.
Chameleon is typical of Tina Gonsalves’ research-based approach to her practice, locating it in the intersection of art, everyday life, technology and science and integrating the development of her work within the residential programmes of leading artistic and academic institutions around the world.
Tina Gonsalves is currently working with world-leaders in psychology, neuroscience and emotion computing in order to research and produce moving image artworks mobile and wearable technology works that respond to emotional signatures of the body. Tina Gonsalves is artist in resident at the Wellcome Department of Neuroimaging London, UK, MIT Media Lab, Cambridge, USA, Nokia Research Labs Tampere, Finland and Brighton and Sussex Medical School Brighton, UK.
Hugo Critchley’s neuroscientific interests focus on brain mechanisms by which human social and motivational behaviour is controlled, both in healthy individuals and people with physical or psychological disorders. He is funded by the Wellcome Trust via a senior fellowship in clinical science for a programme of research entitled: Psychophysiological mechanisms underlying psychological and physical morbidity. He trained in Physiology and Medicine at Liverpool before undertaking a DPhil degree in Experimental Psychology at Oxford. Critchley’s specialist training in psychiatry at the Institutes of Psychiatry and Neurology combined clinical work with neuroscience research. He was appointed Foundation Chair in Psychiatry at Brighton and Sussex Medical School in October 2006. Critchley’s research programme focuses on understanding how interactions between brain and body influence emotions and physical and psychological wellbeing. This work continues with the support of the Wellcome Trust.
Chris Frith studied natural sciences at the Universaity of Cambridge. He subsequently trained in clinical psychology at the University of London’s Institute of Psychiatry. Since completing his Ph.D. (with H J Eysenck) on leaning and individual differences in 1969, he has worked as a research scientist funded initially by the Medical Research Council and subsequently by the Wellcome Trust. With the MRC he worked in Tim Crow’s unit at Northwick Park Hospital on the biological basis of schizophrenia exploring the neuropsychological basis of the symptoms of schizophrenia. Here he was involved in some of the very earliest studies of structural brain changes in schizophrenia. He then moved to the MRC cyclotron unit, where he carried out some of the earliest functional brain imaging studies of consciousness, volition and theory of mind. He had a key role in setting up the Functional Imaging Laboratory at the Institute of Neurology funded by the Wellcome Trust. He is currently Professor in Neuropsychology at UCL and Deputy Director of the Functional Imaging Laboratory and works on the neural basis of social interactions.
Rana El Kaliouby is currently a postdoctoral associate at MIT’s Media Laboratory, inventing novel technologies and experiences that enhance “mind-reading”, or social-emotional and empathic abilities of people and machines. She is passionate about creating new ways for people to capture, learn from and share their experiences and memories, drawing on and exploring the important role of affect in learning and memory. El Kaliouby is the 2006 recipient of the Global Women and Inventors Network, Higher Education & Learning Institutes (Gold Award). She has written several books chapters and refereed articles on the topic of Mind-reading Machines. She also exhibits her work regularly to engage the public in the research and to encourage more under-represented individuals to pursue a career in technology innovation – she has exhibited her work at the Royal Society Summer 2006 Science Exhibition in London and Scotland, where 3000 people interacted with the mindreading system in real time, exploring their expressions of emotion. Her work has been featured in the NewScientist, Reuters, CNET, Wired, the Boston Globe, New York Times, Slashdot and BoingBoing.
Rosalind W. Picard is founder and director of the Affective Computing Research Group at the Massachusetts Institute of Technology (MIT) Media Laboratory and co-director of the Things That Think Consortium, the largest industrial sponsorship organization at the lab. She holds a Bachelors in Electrical Engineering with highest honors from the Georgia Institute of Technology, and Masters and Doctorate degrees, both in Electrical Engineering and Computer Science, from the Massachusetts Institute of Technology (MIT). She has been a member of the faculty at the MIT Media Laboratory since 1991, with tenure since 1998. Prior to completing her doctorate at MIT, she was a Member of the Technical Staff at AT&T Bell Laboratories where she designed VLSI chips for digital signal processing and developed new methods of image compression and analysis. She was honored as a Fellow of the IEEE in 2005.
Helen Sloan has worked as a curator, researcher, writer, editor and producer in media arts and culture since late 1980s. Since 2003, she has been Director of SCAN, a networked organisation and creative development agency for media arts in the South of England working on media arts projects and strategic initiatives in arts organisations, academic institutions and further aspects of the public realm. Helen has worked both freelance and as a curator at organisations such as Camerawork, FACT, ICA and Site Gallery as well as directing festivals such as Across Two Cultures in Newcastle 1996 (an early conference on the overlapping practice of creative thinking in arts and science) and Metapod, Birmingham 2001 – 2. Current areas of interest and curatorial work include the points of intersection of science and culture, immersive environments, wearable technology, high speed networks, and media art and the creative economy.
Dr Nadia Berthouze is a Human-Computer interaction scientist and lecturer at the UCL Interaction Centre, University College London. Her research explores how affect, emotion, and subjective experience can be factored into the design of interactive technology. She was awarded a 2 years International Marie Curie Reintegration Grant in 2006 to investigate this issue in the clinical context and in the gaming industry.
Jeff Mann, Christian Topfner and Evan Raskob
Human Computer Interface Evaluation Researchers
Matt Iacobini and Kim Byers
It is self evident that emotions reflect particular mental and physiological states and give rise to a wide variety of feelings, thoughts and behaviour but in scientific terms, emotion is difficult to define. Emotions are varied and extremely complex phenomena that combine subjective experience, expressive behaviour, and neuro-chemical activity. Psychologically, it is generally agreed that emotions involve awareness of one’s environment or situation, bodily reactions, and approach or withdrawal behaviour.
“When people are in a certain mood, whether elation or depression, that mood is often communicated to others. When we are talking to someone who is depressed it may make us feel depressed, whereas if we talk to someone who is feeling self-confident and buoyant we are likely to feel good about ourselves. This phenomenon is known as emotional contagion.”
Elaine Hatfield, Studies in Emotion and Social Interaction, Cambridge University Press, 1994.
“Our mental life is about interacting with others. In daily encounters, people automatically and continuously synchronize with the facial expressions, voices, postures, movements of others. Some of these encounters happen in milli-seconds. Through unconscious mimicry, we forge a bond with each other through our gestures and movements – long before we utter a word. In essence, we are carriers, dancing with each other in harmonized body language, infecting each other with our emotions. Through these behavioural patterns, hierarchical and social power structures emerge.”
Affective computing is an area of research into human-computer interaction and concerns the design of computer systems that can recognise and respond to human emotions. A computer can gather clues to reading emotions from a variety of sources; facial expressions, posture, gestures, speech, physical activity and temperature changes can all indicate changes in emotional states. These can be detected using cameras, voice and sound recognition and other data receivers and then processed to provide a response for a variety of applications or changing situations.
Reading Faces and the Biological Basis of Emotion
“Human interaction is shaped by information from the face. Emotional expressions, in particular, enable us to judge the arousal state, predict the behaviour and infer the social motivations of another person. Visual decoding of emotional expressions is highly skilled, refined over the course of evolution and our own maturational development. Whether or not communication is actively intended, emotional expressions communicate value and meaning to an observer. Emotional information from the face is also communicated internally: Our own facial expressions change activity within emotional centres in our brain and can influence our own feelings and behaviour, including how we judge the emotions and behaviour of other people.”
Hugo Critchley Neuroscientist, Brighton and Sussex Medical School and collaborator on the Chameleon Project.
Responding to Emotion
“In many case we mirror the emotion we see. This is certainly the case for happiness, fear and disgust. But mirroring is not always the most appropriate response. Responding to anger with anger can lead to an escalation of emotion and a situation that has got out of hand. By responding to anger with sadness or compassion we can diffuse a difficult situation. Our study was a first step investigating more precisely the nature of emotional responses to different facial expressions.”
Emeritus Professor Chris Frith FRS Neuroscientist, Wellcome Trust Centre for Neuroimaging,
UCL and collaborator on the Chameleon Project.
Revealing the Chameleon Project
“The collaborative team have worked on Chameleon for nearly two years. This has been a research project drawing together methodologies from art and science. The connections and developments that have been made cross disciplines and to some individual research have given a richness and uniqueness to the Chameleon project that enables it to nimbly cross arts and science exhibition spaces and research environments. Embedding and linking both disciplines from the outset has ensured that the project has its own artistic signature while also enabling development of each collaborator’s particular research area.
"The project has undergone a number of phases or iterations in 10 stages. This exhibition at Fabrica marks stage 9 of Chameleon. In curating it is rare to have the chance to work on the exhibition of an artist’s process dedicated to one specific subject area. In Chameleon, Brighton has been particularly resonant in that respect with the collaboration between Fabrica, Lighthouse and InQbate allowing the various stages and qualities of the project to be displayed to play to the strength of those venues.”
Helen Sloan, Director of SCAN and consultant curator to Chameleon
“As a Human-Computer Interaction (HCI) expert, my role is to work with the artist and the audience to investigate how the audience interacts with and experiences Chameleon. I am interested in how people interpret or appropriate Chameleon and how to improve its design to facilitate a rich emotional experience that is not only defined by the artist but also by the meaning the audience itself attributes to the interaction unfolding.I am particularly interested in how non-verbal language can be used to modulate emotional experience through the use of affect-aware technology. What I want to learn is how people read and react to others’ emotions when the other is not a physical person, and what the factors are that can facilitate or inhibit emotional interaction.What I like in Chameleon is that it does not purely mimic the emotional expression of its audience. This would constrain the type of interpretation and experience possible. The loose emotional transfer mechanism integrated in Chameleon keeps a certain level of ambiguity that leaves space for personal interpretation and reflection. From an HCI perspective, it becomes an interesting platform to explore emotional communication in all its facets.”
Dr Nadia Berthouze Human-Computer Interaction scientist and lecturer at the UCL Interaction Centre,
University College London.