Paul Ekman Interview: Conversations with History; Institute of International Studies, UC Berkeley

Face to Face: The Science of Reading Faces: Conversation with Paul Ekman, Professor of Psychology, University of California Medical School, San Francisco, January 14, 2004, by Harry Kreisler
Photo by Jane Scherr

Page 2 of 5

Developing the Science

Let's go back a minute, because you've talked about becoming a psychologist. But you've also said that in the army, you saw that it was important to become a scientist. The two aren't necessarily the same. They can be.

That's right.

And you came out here to do advanced studies.

Right, because I had been trained very well as a psychotherapist. I applied to graduate schools. I didn't know (nobody advised me) that you have to lie on your application. It's still true today. If you apply to the clinical psychology program here at Berkeley, and you say you want to be a psychotherapist, you go right in the waste paper basket. Eighty percent of their graduates end up doing psychotherapy, but they had been told, "Never say that. Say you want to do research." Because all the faculty researchers want researchers. Well, I wrote, "I want to do private practice psychotherapy." I got turned down by twenty-three out of twenty-four schools. The one school that took me wanted to train practitioners. I got that training very well, but my training research was rather meager. So I did a three-year post-doc at Langley Porter [at UCSF], and learned some research methods, and got started.

Now, at a certain point, you became interested in the face. I gather in the readings that I've done about you that you have two intellectual mentors, and maybe more, but two stand out: Darwin, on the one hand, and Silvan Tomkins on the other. They seem to combine these two perspectives. Tomkins was a psychologist?

No, he was actually a philosopher. His appointments were in psychology departments.

But he was a man of natural insight and hunch, in my reading of him, whereas, of course, Darwin was a great scientist. Talk a little about that, because those two strands come together in your work.

Actually, there's a third, Duchenne de Boulogne, a French neurologist who published in 1862. I'm a modern work; I am a descendant of all three.

I stayed away from the face. Lady Bird JohnsonIt was just too complicated, and I didn't know how to deal with it. No one did. There was a paper written a few years ago about all the very well-known psychologists who did one study of the face and stopped. I knew eight or nine of the most famous psychologists in the last century, and that was the case. Very few, very few stayed with it up until recently.

But Tomkins... Well, there was a series of accidents. He and I both submitted a paper to the same journal -- mine on body movement; his on the face -- at the same time, and the journal editor wrote the both of us and said, "You guys should meet each other." So I went to see him. He was such an exciting person. He was full of ideas and full of insights, and totally ignored by the field of psychology, because he was a theorist. At that time, psychology didn't like theoretical people. But he could also show you a lot of interesting things, and he did some things that were absolutely spectacular. By a strange quirk, which is its own story, I was given a grant (given, because I didn't apply for it) by ARPA -- the Advance Research Project Agency ...

DARPA. A Defense Department agency.

Well, DARPA is the current name.

Oh, yeah, it used to be ARPA.

Yeah, it used to be ARPA.

I was given a grant by them in 1966 to do basic research to find out what was universal and what was culture-specific about expression and gesture, because at that time, there was just a lot of argument and no definitive data. Initially, when they said, "Why don't you do this?" I said, "No, I'm not an anthropologist. I'm not trained." But they, for their own reasons, wrote the proposal and they gave me more money than I could spend in five years.

One of the things I did was to contact a neurologist at NIH who had been studying a disease called kuru in an isolated area of New Guinea, Carleton Gajdusek. He later got the Nobel Prize. Kuru is a disease [caused by] a slow virus, and it was his discovery that these viruses incubated. Kuru incubated for ten years. At the time I contacted him, he had not made that discovery, but he was working on it. But at the same time, he was working in the Stone Age cultures and had taken movie film.

These were the glory days of NIH, so he had a full-time cinematographer taking over 100,000 feet of film of two Stone Age cultures. I thought, "This is what I need. I can really settle the issues. New Guinea BoyThese people have not been contaminated by the media or by contact with the outside world. If Margaret Mead is right ...." I had talked with Mead, who thought [facial expression] was clearly culture specific. Gregory Bateson had become my friend; he thought exactly the same.

Tomkins was a modern-day Darwinian. At that point, I hadn't read Darwin, but I knew about him from Tomkins, and Tomkins believed this was all universal and innate. So it was a nice argument. I thought, "Great, here's an argument, and I've got the money. Can I settle it?" The Stone Age culture was crucial, because of their isolation. If Mead was right, when I looked at these films -- and Gajdusek kindly enough gave me a copy of all the footage; he had never looked at it. I mean, to look at 200,000 feet of movie film takes a long time. It took me a year. But I never saw an expression I hadn't seen before. There was nothing new. And whenever I could look [in the films] at what happened next [after the facial expression], my interpretation of the expression fit the social context.

So it was clear to me -- that's when I then read Darwin -- that Darwin and Tomkins were right. Now the question was, how to get the evidence? One of the things I did was to pull out from these films just the face. I brought Tomkins out to my lab and I showed him just the face, and he was able to identify what emotion it was just from the face. Moreover, he was able to tell me about the nature of the culture, because these two cultures that I had film on, one was very pacific and the other was very violent; he [knew] immediately just from the face. I said, "Where are you getting it from?" And he went up to the screen and he showed me where. I thought, "I've got to spend time ... not only do this research to settle this issue and get the evidence -- " because I knew these Stone Age cultures are going to disappear, and the ones that I studied were gone in two years after I was there; but I also felt I had to develop a scientific tool so that anybody could measure and get the kind of information that Tomkins did.

The virtue of these particular peoples and this particular footage was they were totally isolated from the world, so you couldn't say, "Oh, they have the same expressions, because they watch our movies or look at our magazines."

They had never seen a photograph. In one of these cultures, the one I did my work in, the nonviolent one (the violent one was too violent, and I got too scared to go in there), they had no still water. There were, of course, no mirrors, so they had never seen their own faces. I would light a match, and it was magic. A flashlight, unbelievable. So, it was sort of ... once they realized that all I wanted to do was show them these funny pictures ... Certainly, at that time, anthropologists had written that people had to learn how to look at a photograph. No. Especially these people: photographs? They knew exactly what was going on.

Were these photographs from the footage that you had taken? Were they special photographs ... ?

These were photographs that Tomkins and I selected on the basis of his and Darwin's theory about what configuration would signal each emotion. I took some from the footage of these people, because I was worried that if I showed them Caucasians, they wouldn't be able to do it. The first year I went, the first thing I found out [was that] it didn't matter who I showed them. A face is a face from their point of view. They couldn't tell whether you were a male or female if you were Caucasian, but they sure could tell the emotion.

So I asked them to make up stories. "Tell me what happened before. What's happening now? What's going to happen next?" That was pulling teeth. These were not a storytelling people. I took some film of them telling these stories, and the sweat is pouring off their faces. So I realized this was not the technique. But I used the stories to then go back the following year, and I would say to them, "friends have come," which is the one I got most often for what in the West we would consider happy expressions, smiling. "Here are three photographs. Point to the one where friends have come." Now, they didn't have to make the story, and they didn't have to say a word, just point. It worked spectacularly well. I also said, "Show me what your face would look like if friends had come." I said this in Pidgin, and then a boy who had been to a mission school and had learned Pidgin said it in their language. So it was still a multi-step. But those were the two techniques I used -- having them pose these stories, and having them pick the photograph that fits.

I then took the films of them posing and I showed them to students at San Francisco State. Well, if Mead was right, they would not know what these expressions meant. But, of course, you got the same result that we got when we showed them Western people. So it was really very strong evidence.

Now, on the other side, gestures -- things like "A-OK," those are totally a cultural product. They truly are a body language and they differ from one language and cultural setting to another. So the gestures are culture specific. The configurations on the face for emotion, for seven emotions, are quite universal.

Later in your research and in your new book you make a distinction between universal expressions that are probably innate, that all mankind and womankind demonstrate in some way, and I believe they're called themes. And then there are variations of the themes where there's more culturally determined variations on these. Talk a little about that.

Let me back up a bit. Just on expression itself, I spent a lot of time trying to find a culture-specific facial expression. I couldn't. There are culture-specific gestures like a wink, which actually isn't used very much anymore in our culture. When I was growing up, a wink was a kind of collusive agreement. I was, "I didn't really mean this." And it could also be used as a flirtation. Nobody flirts by winking anymore; they would think you were out of your mind!

The face isn't dedicated just to emotional expression; there are [facial] gestures. The French go "phoo," and they still do that. You know: "It doesn't matter." You can do it with your hands, but you can do some of them with your face.

But [as for] the emotional expressions, I have not been able to find an emotional expression that isn't universal. However, there are cultural differences in two things. In what we learn in the course of growing up about managing our expressions; I called that the "display rules." I did a study in Japan to show that in private we've got the same expressions in the same situations, but when there is an authority figure present, the Japanese engage in a lot of masking, which Berkeley students did not, which was my comparison group.

There are also differences in the triggers, which is what you were referring to. What brings forth the emotion? There you have to distinguish between the theme and the variations. There are universal themes, like if this nice chair I'm sitting in were to collapse, you would get a really great fear expression from me. You would get a great fear of expression from anyone [from] a sudden loss of gravity, or something that's moving very fast into your visual field as if it's going to hit you. You can't avoid moving back, and you'll see that lip stretch horizontally. Those are the themes.

But the variations are enormous even within a culture. My wife is afraid of mice. I'm not afraid of mice. I'm afraid of heights. She's not afraid of heights. So there are big differences within cultures and between cultures, but they're all variations on these common, biological themes. Some of them can be pretty remote. When I get angry by reading a story in the newspaper, let's say, a story about ... what can I get angry about? There was [a story] recently about a school official in San Francisco who had been embezzling money. That gets me angry. Well, I don't know that person. I haven't seen that person. I'm reading about it. I'm going through a symbolic process. I'm applying a set of moral [values.] There are cultures where nobody would get angry about that. You're a fool if you don't embezzle money. If you've got a government job, that's normative. Not here. So you can get pretty remote from the universal theme, and have it be a pretty strong trigger. The strongest triggers, the ones that are most unavoidable, are the ones that get established during childhood.

One of the things I write about in my new book is that it does appear (although I have to say the evidence isn't conclusive, but it's strongly suggestive) that if you learn a trigger early in life and you learn it with a lot of emotional charge, then it's with you for life. You can't erase it. I have two chapters in this book about how we can change what we become emotional about. To a large extent, we can't. We can weaken the trigger, but we can't remove it totally, and it will reappear under the right circumstance -- when we haven't enough sleep, if we're under stress, then the things that we thought no longer get us still do.

As part of your research, and this goes back a little ways, at a certain point you essentially saw yourself as an explorer and set out to map the face. Tell us a little about that, and what the FACS is, the Facial Action Coding System, which you developed.

It started by seeing what Tomkins could do and thinking I had to build a tool. If I had known it would take me eight years to do it (and that's all I did for eight years, myself and my research team; principally Wally Friesen, but a number of assistants and students) -- eight years is a long time to work on something. If I had failed, if it was unreliable, that would have been the end of my research career. Everybody thought I was nuts to begin with to want to get such precision.

The problem is that if you want to establish, for example, what are the signals for emotion, you can't do your descriptions in terms that mix the emotion with the description, you have to separate it. You have to describe things as precisely as you can, so you can find out [about] synonyms, for example: are there different ways of showing anger that have exactly the same significance? The only way you would know that is that is if you can measure everything the face can do. And that's what I set out to do.

So you're examining the muscles of the face?

I'm examining how the muscles change appearance ...

I see.

... so that if I do what I just did now, that's the inner part of the frontalis muscle that's doing that. That's the outer part of the same muscle. Now, before my work, the anatomists only did a dead anatomy, that is, on cadavers; they would take off the skin, and they would describe this as one muscle. But clearly, there is separate neural supply. If I can do this separately from that, then that means there's neural supply for this and for that. From a functional or a neurological point of view, it's two muscles. So this is Number 1 in our system, and that's Number 2 in our system. I learned how to do this by building on the work of the French neurologist Duchenne de Boulogne, who published in 1862. Darwin quoted him. His book at that time had been out of print for more than a hundred years, but we had a copy, one of eight existing copies, in our rare books collection at the medical school. One of my post-docs, Harriet Oster, was fluent in French. face encoded according to the system, by Paul EkmanSo I was able to take what Duchenne did and build on it. He studied primarily the combination of two muscles, and I went up to six muscles. So I examined 10,000 different facial combinations.

The system, which was published in 1978, just revised in 2002, is called the Facial Action Coding System. There are about 500 people who have learned this system. It's self-instructional. You don't learn it from me; you learn it from the materials. It is being used in research on everything from looking at mothers and infants, to evaluating advertising, to the impact of drugs on emotional life. Anytime you want to measure what the face does. Sign language in the deaf. It's not restricted to emotion. The face does all kinds of things unrelated to emotion.

It's a very slow and tedious system. There are now two major computer teams funded by the government, one at the University of Pittsburgh headed by Jeff Cohn, and one at UC San Diego - Salk, headed by Terry Sinowski and Marni Bartlett, that are trying to automate our system. Within five years, this will be an automatic system where as you're talking to me, a camera will be looking at you, and I would have a read-out on momentary changes in your emotional state, moment-by-moment.

Next page: Implications of Research

© Copyright 2004, Regents of the University of California