Liz Magic Laser using Visage Technologies
Introduction
Artist Liz Magic Laser offers guided exercises and findings from her experimental research study. She brings together leading experts from divergent fields, EMDR therapist Dr. Ameet Aggarwal working in functional medicine and market research expert Dan Hill Ph.D. who specialises in face tracking. Hill’s most recent book, First Blush, 2019, uses eye movement tracking to understand how we engage with famous works of art in museums from the Sphinx to Tracy Emin’s My Bed.
In this research Laser explores how eye movement can be manipulated to affect our emotional states, questioning the psychological impact of eye tracking technology such as Apple’s new VR headset (Apple Vision Pro). It seems the choreographing of eye movement will be the next frontier in the user interfaces that will reshape our perceptual experience and possibly rewire our brains in the era to come. Advertisers use eye movement tracking to grab our attention, thrusting our brains into high alert mode and elevating stress. EMDR therapy uses eye choreography to combat the effects of trauma and reduce our stress levels. Laser has devised the following guided exercises to offer an insight into how face tracking market research tools work and how EMDR therapy feels.
I want people to have an embodied experience of eye movement manipulation that’s been developed for polar opposite aims: for therapeutic versus profit driven purposes. Can EMDR tools help us navigate the triggers we encounter in cyberspace and art spaces?
Liz Magic Laser
1. View the tracker hacked
Liz Magic Laser and her research participants play with a face tracking algorithm to influence how their emotional states are perceived.
Laser conducted an experimental research study with participants who consented to Visage facial expression analysis while receiving EMDR therapy and informational briefings on eye tracking technology. The following animation shows Laser guiding participants to explore the facial expressions associated with the seven emotional states.
Hello. Let's begin. Let's find anger. So I've got a grimace in person. Squint my eyes.
Ok, disgust. Ick. The jaw. When I draw upper left lift up and lower right down.
Ahhhh...ahhhh
Ewwwwww
Ewwwwww
Ok, now fear. How do I get my fear back? Squish my forehead. Squish it. I'm squishing my forehead and I'm getting the fear all away.
Okay, now happy. Yeah, just smile. Just smile. Hmmmm. Yeah.
I'm puckering my lips out. Mouth wide open and eyes wide open.
Ahhhhhhhhhhhh
[Exhales]
Lips tight, or just loosely. Closed. Deep breath in and slow breath out for neutral.
2. Test drive
Your turn for expression exploration.
The facial expression and eye movement tracking algorithm used in Laser’s research was developed for market research by a company called Visage Technologies. It uses motion tracking to estimate gender, age and emotional state. Please take a few minutes to familiarise yourself with its options: play with your expressions or toggle the different option boxes on and off. How does the tracker say you are feeling? Do you feel it is accurate? Does it resonate? Can you alter how the software analyses you?
Consider each emotional state defined by the software. Try to achieve the facial expression the tracker associates with each emotion.
Start with anger. Where in your face does the tracker locate anger? In your forehead? Try moving your eyes. Does squinting affect your anger level? What happens if you cast a sideways glance or look down toward the bottom of the screen? Now work your way through the list of emotions...
Liz Magic Laser using Visage Technologies. Courtesy of the artist and Visage Technologies AB.
3. Try EMDR
Experience Eye Movement Desensitization Reprogramming therapy with Dr. Ameet Aggarwal.
EMDR (Eye Movement Desensitisation Reprogramming) therapy uses simple eye movement choreography to help people process trauma. The logic is that bilateral stimulation of the two sides of the body helps you integrate the two hemispheres of the brain.
After trying EMDR with the guided video below, take some time to reflect. Did you feel eye movement can rewire your brain?
If eye movement can reduce our stress levels, is the opposite also possible? Can a fixed gaze at one point on a screen or erratic eye movement patterns instigated by pop ups have harmful effects on our mental state?
Our eye movement patterns are already being choreographed by advertisers. If emerging technology like Apple’s VR headset (Apple Vision Pro) uses eye movement tracking for navigation then it is already dictating eye movement patterns as well? User interfaces that rely on touch will likely be replaced by AI algorithms that predict our eye movement to determine where we move in virtual space and even for simple internet surfing. No longer will we rely on haptics: touch screens, mouse or joystick clicking. Are we moving further and further towards disembodiment?
Morning, evening, afternoon everyone.
I'm sitting in Kenya at the moment, I just invite you to take your time. Take your time before listening to any of my words and notice how that feels the invitation to just take your time. Beautiful, and some of you will feel a settling in your nervous system. Often we're in a rush, we're pushed and we're on an agenda and when we're given the permission to take our time something settles in our nervous system, we go into parasympathetic mode. And in a similar way EMDR helps us get into that parasympathetic mode and come out of the stress response.
I'll invite you first to look around your room and just find something that gives you a sense of calmness, groundedness. Could be an object, it could be a picture, it could be your dog. When you take your time and then when your eyes land on that object, spend a moment there and notice what grounding feels like to you.
What changes in your breathing, in your jaw, in your throat. Nice. Notice anything else changing in your belly, in your arms and you can always come back to this object or to this scene, or to this feeling during this demonstration. It's your place of safety it's your place of disconnecting from the exercise, it's a place of just returning to your own environment where you have more and full control. Nice, and you can do that in other areas of your life whenever you're feeling stressed. You can always give yourself permission to come back to a place or scene which feels more grounding to you.
And now we're going to go into a bit of an EMDR demonstration. How EMDR works is I will get you to think of a mildly stressful memory. Today we're going to use a memory choose a memory that involves using the computer. Maybe opening up an email, um communicating with someone, shopping on Amazon, whatever it may be, when the sale is ending. Don't choose anything that's overly triggering or over stressful. And as you're thinking about this memory I'll get you to notice different sensations, right a physical feeling, an emotion and negative thoughts as well and as you do that then I'm going to get you to move your eyes from side. When you move your eyes from side to side try not to move your head just move your eyes left and right and you can also use your fingers and follow your fingers left to right. So the idea is to do bilateral stimulation, you're looking left or right or stimulating left and right nerve bundles. Today we're going to use eye movement because of this eye tracking software. So before we begin or well as we begin, the first part of this exercise is to think of a pleasant memory first.
A place of safety, a place of love, a place of connection, a place of freedom. For some of you it might be a place where you felt acknowledged or seen by someone, a loved one, or somebody important in your life. And just notice how that feels in your nervous system and really take your time as you connect to this memory I invite you to scan, become aware of your breathing your body sensations and then slowly begin moving your eyes from left to right.
Left, right, left, right, left, right.
Just allow this movement to ground the feeling of safety and comfort, more into your nervous system, more into your biology. And this is also a place you can come back to anytime you need to or feel like it during this exercise. And notice how that feels in your body the permission to come back to this place anytime you feel like it. And take a deep breath in and out let go of the eye movement and just relax. Good.
Now I invite you to think about a mildly stressful memory, ideally when you were using the computer, shopping, emailing, chatting, negotiating and as you do that go to the most stressful moment in time of that encounter. What does that mean? So the encounter might look like a movie, yeah a story of like 5 or 10 seconds, or a minute. Within that time frame choose a snapshot moment that was the most stimulating or stressful for you.
And as you connect to that moment I invite you to think about the negative thought you had about yourself. What does that mean? So instead of one thought could be “oh the sale is going to end” but the negative thought about yourself could be “I'm going to miss out” or “I'm going to, I'm not loved or, I'm going to fail I'm not going to do well, I'm not going to do well enough”. So think about the negative belief or thought you had about yourself.
So as we go back to this stressful memory, this moment, this negative belief you had about yourself, I invite you now to think about the emotion you had during that time with that memory and that thought. Is it frustration? Is it fear? Anxiety? Sadness? Really take your time and scan the sensations that you get and as you're connecting to these emotions, I invite you to also pinpoint what kind of physical sensation, where do you feel this in your body? Some of you might not be able to and that's okay, don't stress. And some of you can you might feel something in your jaw, in your heart, in your tummy. So you're connected to four sensations, or four senses right now the memory of the event, the negative belief you had about yourself at that time, the emotion connected to that memory and the negative belief and the physical sensation that you're having in your body. As you connected to those four different items or stimuli I invite you now to begin to move your eyes from left to right, left, right, or you can follow your fingers from left to right.
Left, right, left, right.
Do this for a few more seconds really take your time. Right now take a deep breath in and out and take a break from that and now notice how you're feeling. What is the same or different about that memory? Does it feel as strong? Or does it feel more distant? Has your perspective changed? Are you as overwhelmed or less overwhelmed? And as you connect now to the remainder of the memory and the sensations that are still left over and the negative belief you have about yourself. Notice how strong that negative belief still feels after the eye movement. So in the beginning if you thought "I'm going to miss" out felt like an 8 out of 10 Now when you look at the memory and the negative belief How true does it feel? When you say "I'm going to miss out" has it increased or decreased? So now I invite you to reconnect with the memory again with the remainder of the emotions, sensations and the intensity of the negative belief And begin to move your eyes left and right again.
Left, right, left, right.
Take your time. Really, take your time. Good. And then take a deep breath in and out and let it go and now I invite you to think about the memory again. What is left over with this memory? Has it changed? Has it changed in intensity? And then if you think about the negative belief you had about yourself I'm not loved, I'm going to miss out, I'm in danger. How true does it feel now compared to when you first started? And just take a moment to also scan your body. How intense do those sensations feel in your body? And the emotion with that negative belief, has something changed? And it's okay if nothing has changed, or maybe something has changed and something hasn't changed and that's okay as well and just notice what hasn't changed. Nice. And really take your time with that acknowledgement for yourself, nothing's wrong with that. It's just an observation, an invitation to bring love to those parts that still need a bit of attention and care. Beautiful.
And now let's go back to this first scene you started with where there was comfort, there was love, there was acknowledgement from someone important, your safe scene. And as you go back to that memory I invite you to begin to move your eyes from left to right again. As you allow your nervous system to indulge and absorb the positive thoughts thoughts and feelings of that safe memory and that safe place in your life. And really take your time here. In fact you can even move your eyes at a slower pace than you were before. Beautiful. Knowing that you can always come back here I invite you and if you need more time you can spend longer there. And for those of you who feel complete I invite you to take a deep breath in and out and gently let the exercise go. And then you can look around your room where you're sitting and maybe go back to that object that you were looking at in the beginning that gave you a sense of grounding familiarity and then just notice how it feels to look at this object in your room. The exercise is done.
You have full permission to disengage and just reconnect with your surroundings your environment give permission to your breathing to relax or do what it needs to do and just notice how your jaw feels, your belly feels, your eyes feel. Notice how it feels to give yourself permission to go on a journey of mildly stimulating your senses and coming back to grounding. Knowing that you can always invite this experience back for your nervous system, whenever you need to. Beautiful. Slowly take your time and when you feel ready, you're welcome to come back. Thank you.
4. Learn from an expert
Dan Hill gives the inside scoop on how eye and face tracking is being deployed by corporations.
In his 2007 book Emotionomics Dan Hill explains how the traditional focus group interview has a fundamental flaw: a person’s testimony is unreliable because they usually don't know how they actually feel. He advocated for augmenting market research surveys with facial expression analysis because your involuntary eye movements offer more insight into your true emotions than your words.
As Hill explains it, “facial coding helps us explore the SAY / FEEL gap: the discrepancy between what people say vs. what they actually feel.”
The way that eye tracking typically works is you're sitting in front of a screen, probably about two to three feet away and it will capture and calibrate where your gaze activity is going. Very specifically how it works is that infrared lights are used to triangulate where the pupil's attention is centred.
That does bring up one of the limitations of eye tracking. It does not capture peripheral vision because it's working off what is centred and triangulated. It also does not capture how you feel which is why facial coding is important, but eye tracking is great in terms of going after what is attracting our attention. Where do we pay attention? Really important here is a term called fixation which means that the eye needs you know 15 milliseconds to basically catch up the mind to the eye as to where you're going what you're taking in. That gets you to a fixation count, it's also important to know that with eye tracking you can capture what order typically are subjects moving in because we go around imagining that the tip of the iceberg is the iceberg and most of it is below the surface, it's below our level of awareness. So I have the US pattern for the linking of eye tracking and facial coding and I'm going to particularly emphasise here the eye tracking but I will bring in the facial coding. Facial coding will get you actually to seven emotions. So you're seeing here from the Visage, they're leaving out contempt which is a really important emotion
For instance I would mention that contempt is the most reliable indicator, unfortunately that a marriage will end that you don't trust you don't respect the other party. The two emotions that I've seen most common over my now quarter century of using facial coding is anger and happiness. With happiness you are approaching someone to hug them, with anger you are approaching to hit them. Down below in the right corner is sadness, which I could basically describe as longing to be hugged. So you're not happy but you are hoping to be happy.
You have the two kind of rejecting, scorning emotions of contempt and disgust where you're backing away as opposed to approaching. And then you have surprise and fear which are considered submissive and avoidance because you feel as if you're overwhelmed by the circumstances you find yourself in. Really important with facial coding is that you have 23 different muscle activities according to Dr Ekman's system and that those emotions typically happen very quickly, they come on and off the face. Emotion and motivation have the same root word in Latin to move to make something happen.
Your emotions are triggered by something meaningful in your environment. Faces, hugely important. We always talk about FaceTime, wanting FaceTime with someone if it's an important conversation. My research would indicate that about two-thirds of all your emotional responses and about two-thirds of those eye tracking fixations will be caused, elicited, generated due to taking in faces.
There are things that people won't say because they're being polite and there are things that people can't say because they're not even aware of how they feel or for that matter where their eyes are moving. The concretive estimation in the scientific community is that at least 95% of our mental activity is not fully conscious.
5. See the results
Our findings on the accuracy of facial expression and eye movement tracking tools.
Participants in Laser’s study were intermittently polled about the accuracy of Visage’s emotional tracking software. They reported approximately 50% accuracy, but who is the more reliable judge of emotions, the person or the algorithm?
Visualisation of the data obtained from Laser's research study
Visualisation of the data obtained from Laser's research study
Visualisation of the data obtained from Laser's research study
Visualisation of the data obtained from Laser's research study
Visualisation of the data obtained from Laser's research study
Backstory
Visage’s face tracking technology is derived from older 18th and 19th century classification systems for identifying emotions: the passions defined by René Descartes and Charles Darwin's Expression of Emotion in Man and Animal, 1872 . The more recent 20th Century studies were conducted by Dr. Paul Ekman at the University of California where he developed a program called FACS: Facial Action Coding System. Visage’s emotional expression classifications are largely based on Ekman’s work.
