Emotional Design (21 page)

Trust is an essential ingredient in cooperative, human interaction. Alas, this also makes it a vulnerability, ready for exploitation by what is called
“social engineering,” the crooks, thieves, and terrorists who exploit and manipulate our trust and good nature for their gain. As more and more of our everyday objects are manufactured with computer chips inside, with intelligence and flexibility, and with communication channels to the other devices in our environment and to the worldwide network of information and services, it is critical to worry about those who would do harm, whether by accident, for the sake of mischief, for fun, or with malicious intent to defraud or harm. Crooks, thieves, criminals, and terrorists are experts at exploiting the willingness of people to help one another, both to figure out how to use onerous technology and when someone appears to be in urgent need of assistance.

A common approach to improved safety and security is to tighten up on procedures and to require redundant checking. But as more people are involved in checking a task, safety can decrease. This is called “bystander apathy,” a term that came from studies of the 1964 murder of Kitty Genovese on the streets of New York City. Although numerous people witnessed that incident, no one helped. At first the lack of response was simply blamed on the callousness of New York City residents, but social psychologists Bibb Latané and John Darley were able to repeat the bystander behavior, both in their laboratory and in field studies. They concluded that the more people watching, the less likely anyone would help. Why?

Think about your own reaction. If you were by yourself, walking along the streets of a large city and encountered what looked like a crime, you might be frightened and, therefore, reluctant to intervene. Still, you probably would try to call for help. But suppose a crowd of people were watching the incident? What would you do then? You probably would assume that you weren't witnessing anything serious, because if it were, people in the crowd would be doing something. The fact that nobody is doing anything must mean that nothing bad is happening. After all, in a large city, anything might happen: maybe it's actors making a movie.

Bystander apathy works in security as well. Suppose that you are working as a technician at a power plant. Among your jobs, you are
supposed to check the meter readings with one of your colleagues, another technician at the plant, a person you know and trust. Moreover, when you have finished, your supervisor will also do a check. The result is that you don't have to exert extra care on the task. After all, how could a mistake get through with so many people? The problem is that everyone feels this way. As a result, the more people that check on something, the less carefully each person performs the task. As more people are responsible, security may diminish: trust gets in the way.

The commercial aviation community has done an excellent job of fighting this tendency with its program of “Crew Resource Management.” All modern commercial aircraft have two pilots. One, the more senior, is the captain, who sits in the left-hand seat, while the other is the first-officer, who sits in the right-hand seat. Both are qualified pilots, however, and it is common for them to take turns piloting the aircraft. As a result, they are referred to by the terms “pilot flying” and “pilot not flying.” A major component of crew resource management is that the pilot who is not flying be an active critic, continually checking and questioning the actions taken by the pilot who is flying. The pilot flying is supposed to thank the other for the questions, even when they are unnecessary, or even wrong. Obviously, getting this process in place was difficult, for it involved major changes in the culture, especially when one pilot was junior. After all, when one person questions another's behavior, it implies a lack of trust; and when two people are supposed to work together, especially when one is superior to the other, trust is essential. It took a while before the aviation community learned to take the questioning as a mark of respect, rather than a lack of trust, and for senior pilots to insist that junior ones question all of their actions. The result has been increased safety.

Criminals and terrorists take advantage of misplaced trust. One strategy to break into a well-guarded place is to trigger the alarms repeatedly over the course of a few days, and then hide so that the security personnel cannot find any cause for the trigger. Eventually, in
frustration over the repeated false alarms, the security people will no longer trust them. It is then the criminals break in.

Not everyone is untrustworthy, just a few—but those few can be so severely disruptive that we have little choice but to relinquish trust and be suspicious of everyone, everything. There is a terrible tradeoff here: the very things that make security tighter are often those that make our lives more difficult or, in some cases, impossible. We need more realistic security that is cognizant of human behavior.

Security is more of a social or human problem than a technological one. Sure, put in all the technology you like. Those who wish to steal, corrupt, or disrupt will find a way to take advantage of human nature and bypass the security. Indeed, excessive technology gets in the way of security, because, by making the task of conscientious, everyday workers more difficult, it makes the job of bypassing the security measures even easier. When the security codes or procedures become too complex, people can't remember them, so they will write them down and post them on their computer terminals, under their keyboards or phones, or in their desk drawer (on top, though, where they are easy to get to).

As I was writing this book, I served on a committee of the United States National Research Council investigating information technology and counterterrorism. For my section of the report, I studied the social engineering practices used by terrorists, criminals, and other troublemakers. Actually, it's not difficult to find this information. The basic principles have been around for centuries and there are many books by ex-criminals, law-enforcement officers, and even guides to writing crime novels that provide relevant information. The internet makes the research easy.

Want to break into a secure facility? Walk up to the door carrying an armload of computers, parts, and dangling cords. Ask someone to hold open the door, and thank them. Carry the junk over to an empty cubicle, look for the password and login name, which will be posted somewhere, and log in (figure 5.2). If you can't log in, ask someone for help. Just ask. As one handbook that I found on the internet puts
it: Just shout, “Does anyone remember the password for this terminal?” You would be surprised how many people will tell you.

FIGURE 5.2a and b
How not to safeguard a password.

Figure a shows a note posted on the side of the computer display; figure b is an enlargement of the note. This is the sort of behavior that social engineers count on. But it is bad password policies that make us have to resort to this. Even if the password wasn't attached to the computer, a good social engineer could have guessed it: this computer is at the corporate headquarters of a major manufacturer of office furniture. “Chair”? Who would ever guess?
(Photograph by author.)

In the end, security is a systems problem, where the human is the most important component. When security procedures get in the way of well-meaning, dedicated workers, they will find work-arounds to avoid disruption, thus defeating the whole point of the procedure. The very attributes that make us effective, cooperative, creative workers, able to adapt to the unexpected and to provide assistance to others, make us vulnerable to those who would take advantage of us.

In my consulting work, I am often called upon to predict the next “killer application,” to discover the next product that will be so popular that everyone will have to own it. Unfortunately, if I have learned anything, it is that precise predictions of this sort are simply not possible. The field is littered with the bodies of those who have tried. Moreover, it is possible to be correct about a prediction, but very far off as to its time frame. I predict that automobiles will drive themselves. When? I have no idea: it might be twenty years, it might be one hundred. I predict that video telephones will become so popular that they will be everywhere, and we will simply take them for granted. In fact, people might complain if there weren't any video. When? Forecasters have been predicting widespread adoption of video phones “in just a few years” for the last fifty years. Even successful products can take decades before they catch on.

But even if exact prediction of successful products is not possible, we can be certain of one category that almost always guarantees success: social interaction. Throughout the last one hundred years, as technologies have changed, the importance of communication has remained high on the list of essentials. For individual communication, this has meant mail, the telephone, email, cell phones, and instant messaging and text messaging on computers and cell phones. For organizations, add the telegraph, the corporate memo and newsletter, the fax machine, and the intranet, that specialization of the internet for intracompany communication and interaction. And for societal groups, add the town crier, the daily newspaper, radio, and television.

Up to a few years ago, the increasing ease and lowered cost of travel had the unfortunate side effect of weakening the bonds that hold people together. Yes, through letters and telephone people could still be somewhat in touch, but this touch was limited. Two thousand years ago the Roman philosopher Seneca complained that travel led to many acquaintances but few friends, and up to recently this complaint still held true. Distance used to matter. Move away from family and friends, and the contact waned. Sure, one could use mail and telephone, but these were sparse communications amidst the busy activities of the day. People who separated physically would often separate socially and emotionally as well.

No more: today we can be in continual contact with friends and relatives no matter where we are, no matter the time of day. Today's technology makes it possible to stay in touch with friends and family on a continual basis. Email, instant messaging, text messages, and voice mail have no barriers in time or distance. Travel is relatively easy by auto, train, or airplane. The mail system reliably traverses the earth. The telephone is readily available and, with the cellular phone, always with us, always on. Email is ubiquitous. Billions of short messages are sent daily among the cell phones of the world. The isolation once imposed by distance and separation is no longer true. Today we can easily keep in touch with one another to an amount undreamed of earlier. Moreover, the communication revolution has barely begun: if it is so pervasive now, at the start of the twenty-first century, what will it be in one hundred years?

Most of the short text messages appear to be content-free. Among teenagers, they are apt to say: “What are you doing?”—or, in the highly abbreviated form they often take, “watrudoin”; “Where are you? (wru)”; “See you later (cul8r).” Among business people during the business day, they differ slightly: “Boring meeting”; “What are you doing?”; “Want a drink after work?” Occasionally, of course, they have real substance, as in business negotiations or in arranging meeting times or the details of a contract. But, on the whole, the point
of the frequent messages is not information sharing; it is emotional connecting. They are ways of saying to one another, “I'm here,” “you are there,” “we still like each other.” People need to communicate continually, for comfort, for reassurance.