Read Liars and Outliers Online

Authors: Bruce Schneier

Liars and Outliers (12 page)

BOOK: Liars and Outliers
11.37Mb size Format: txt, pdf, ePub
ads

We are also a species capable of profound immorality. And while moral pressure works, it also regularly fails. When it does, it fails for several specific reasons:

People vary in their individual behavior
. Sure, most people will cooperate most of the time, but some people will defect some of the time, and almost everyone will defect once in a while.

Morals often conflict.
We'll talk about this more in Chapter 11. Sometimes defectors are people whose morals lead to different imperatives than those reached by the cooperators. These people will be largely unaffected by societal moral pressure. Society will have an easier time convincing a potential thief that stealing is wrong than it will have convincing an abolitionist that slavery is good.

Morals often overreach
. It's relatively easy to use morals to enforce basic prosocial behaviors, because those are aligned with what's already in our brains. Enforcing arbitrary moral codes is much harder. If the group norm goes against any of Haidt's five fundamental moral systems, more people will have conflicting morals, and more will defect.

Throughout history, totalitarian regimes have attempted to impose moral codes on their citizens, suppressing some heretofore acceptable behaviors and inventing new obligations. Perhaps the most well-known modern example of an authoritarian attempt to reorient popular moral sensibilities was the Soviet Union's unsuccessful prohibition on the
practice of religion
, which threatened to undermine materialist Communist ideology. This kind of thing isn't rare. When I visited Myanmar in 1991, I saw large billboards everywhere, courtesy of the government's “People's Desire Campaign,” exhorting the populace to believe and act in all sorts of pro-government ways. These campaigns often just drive the behaviors underground.

Morals can be manipulated
.
Confidence tricksters
, in particular, manipulate the very same traits that make us cooperate: kindness, altruism, fairness.

Morals scale badly
. They fail as societies become larger and the moral ties that bind their members weaken.

Remember Baron-Cohen's theory of empathy. As the group gets larger and more anonymous, there's less empathy. Joseph Stalin said, “the death of one man is a tragedy, the death of millions is a statistic”; similarly,
we have trouble
thinking about large groups in the same moral way we think of the people closest to us.

All of these reasons make morals the weakest of the societal pressures. Morals are the societal pressure that works “when no one is watching.” They determine whether we keep a wallet that no one saw us find and pick up, whether we litter on a deserted street, whether we conserve energy or crank up the air conditioning, and whether we help ourselves to a bagel on a tray in an empty break room.

When nothing other than moral pressure influences a societal dilemma, the number of defectors will be at its largest. However, opportunities for individuals to make moral choices when they are unobserved represent only a small portion of societal dilemmas. We humans are a social species, and more often than not someone
is
watching. And that makes an enormous difference.

Chapter 8

Reputational Pressures

From the perspective of trust, societal dilemmas involve a Red Queen Effect. On one hand, defectors should evolve to be better able to fool cooperators. And on the other, cooperators should evolve to better recognize defectors. It's a race between the
ability to deceive
and the ability to detect deception.

There's a lot of research on detecting deception, and humans seem not to be very good at it. There are exceptions, and people can learn to be better at it—but in general, we can't tell liars from truth-tellers. Like the Lake Wobegon children who are all above average, most of us think we're much better at detecting deception than we actually are. We're better, but still not great, at predicting cooperators and defectors.
1

This is surprising. The Red Queen Effect means both sides improve in order to stay in place, yet in this case, defectors have the upper hand. A possible reason is that we have developed another method for figuring out who to trust and who not to. We're a social species, and in our evolutionary past we interacted with the same people over and over again. We don't have to be that good at predicting bad behavior, because we're really good at detecting it after the fact—and using reputation to punish it.
2

In fact, our brains have specially evolved to deal with cheating after the fact. Perhaps the most vivid demonstration of this can be seen with the
Wason Selection Task
. The test compares people's ability to solve a generic logical reasoning problem with their ability to solve the same problem presented in a framework of detecting cheaters: for example, “if Alice went to Boston, she took the train” versus “if Alice is served alcohol, she is over 21.” People are generally much
better at solving
the latter. Additionally, fMRI
scans of the brains
performing this task show that we have specific brain circuitry for cheater detection.

Think back to how contrived and artificial the sealed bag exchange from Chapter 5 seemed. That's because there's more going on than short-term decision making. Commerce isn't a one-time event. It happens again and again, day after day, often between the same people. We know the individuals and companies with whom we interact, maybe personally, maybe casually, maybe by their brand. Everyone has a reputation, and it's important. While morals are part of the reason we cooperate with each other, the preponderance of the evidence—both observational and experimental—supports the hypothesis that
we cooperate primarily
because we crave reward (engagement) and fear punishment (exclusion) from other members of our group.
3

Bob depends on his reputation as an honest merchant. If he cheats Alice, she won't do business with him again. Even worse, she'll tell her friends.
4
Bob couldn't survive as a merchant if he had a reputation as a cheater. If we assume that the cost to Bob's reputation if he defects is greater than the value of the item being purchased, he has no dilemma. He is better off cooperating, regardless of what Alice does. Reputation is such a major factor for Bob that he almost certainly allows Alice to reverse the transaction after the fact, a process commonly known as returning the purchase.
5
This is the fundamental threat of damage to your reputation. A business works to make its customers happy, because it knows its reputation will be damaged if it doesn't deliver. Customers, knowing this is true, are more willing to trust the business.
6

Societal Dilemma: Cheating customers.
Society: Group of merchants/society as a whole.
Group interest: Merchants are trusted.
Competing interest: Maximize short-term profits.
Group norm: Don't cheat customers.
Corresponding defection: Cheat customers when possible.
To encourage people to act in the group interest, society implements these societal pressures:

Moral: Guilt, shame, sense of fairness, kindness, etc.

Reputational: Merchants want to be seen as trustworthy. Customers share their experiences with merchants, making merchants less likely to cheat customers so as to retain their good reputation.

Customer reputation used to be a bigger deal than it is today. When commerce was transacted entirely in local markets with local merchants, in situations where buyers and sellers knew each other and knew they would need to do business with each other many times in the future, reputation mattered just as much to the customer as it did to the merchant. In today's world of global commerce, where potential customers may be located a half a world away, customer reputation matters much less than merchant reputation. Means for ascertaining the integrity of potential customers are coming back, though. Online reputation systems, like eBay's feedback mechanism, gave both merchants and customers
reputation information
about each other. (In 2008,
eBay changed this
, and no longer allows merchants to give feedback on customers, citing abuse of the process by merchants.)

We take our reputations very seriously, and spend a lot of time and effort maintaining them, sometimes defending them to the point of death.
7
We go to these extremes because we recognize that if we want others to trust us and cooperate with us, we need a good reputation. So we keep our reputation clean, cover up blemishes, or fake our reputation completely.

Tellingly, psychological and brain research both show that we
remember negative
information about people more vividly, with more detail, and for a longer time than positive information. It seems that knowing who will defect is more important than knowing whom to trust.

We're also good at keeping up with the reputation of others. There's a
theory that gossip
originated as a mechanism for learning about the reputations of others and helping us know whom to trust. Of course, gossip requires language. Humans are unique on the planet for our ability to gossip,
8
and humans everywhere on the planet are enthusiastic about it. It tells us who is likely to be cooperative and who is not, so we know
whom to interact
with. It helps
establish group interests
and group norms. It works as a societal pressure system, too; both observational studies and experiments show that
gossip helps keep
people in line. Social networking sites are the most modern manifestations of these ancient needs.

Reputation is a common mechanism for raising the costs of defecting and increasing the benefits of cooperating. Buskers generally don't disrupt each other's acts because they don't want a bad reputation among their peers.
Diamond merchants
generally pay their debts promptly, don't pocket other people's diamonds, and don't substitute worthless stones for valuable ones, because they don't want to jeopardize their reputation within the community. And prisoners sometimes won't testify against each other because they don't want to be known as stool pigeons by the other criminals in town.

Here's how finely tuned we are to others watching our actions. The coffee room at the Division of Psychology at the University of Newcastle in Australia works on the honor system, just like Feldman's bagel business. Researchers found that if they put a sign above the pay box with a picture of
a pair of eyes
—not an entire face, just a pair of eyes—people put almost three times as much money in the box as they did when the sign had an image of flowers. Similarly, children who were told to take only one piece of Halloween candy but were left alone with a full bowl defected less when the bowl was placed in front of a mirror. And they defected even less when they were asked their
names and addresses
before being given the same opportunity. Along the same lines, religion often provides a universal observer. God is omniscient and the arbiter of one's final reputation, and a calculating believer behaves accordingly.
9

Not only do we guard our reputation against blemishes, we also take pains to advertise our good reputation. This can be as grandiose as a company touting its customer satisfaction ratings or product quality awards, or as mundane as those small “I Voted” stickers that many polling places in the United States give to voters to wear for the rest of Election Day. The effect is both reputational and moral; voters can publicly demonstrate that they behaved in the group interest and voted, and simultaneously remind others of their civic responsibility. There's even a German expression,
“Tu Gutes und rede darüber”
: “Do good and talk about it.”
10

We need several more pieces to make a reputational pressure system work. We don't always have perfect information about what other people are doing. Maybe they cooperated when we thought they defected, or vice versa. Or they might have done the wrong thing accidentally or because they weren't thinking clearly. Our reputational systems have to work despite the occasional mistake. This requires two things: contrition and forgiveness . If you defect by accident, apologize, make amends, and then return to cooperating. And if someone does that to you, forgive and return to cooperating.

I'm glossing over a lot of subtleties here. Forgiveness is a complicated emotion, and there's a fine line between being forgiving and being a sucker, and between being contrite and being a doormat.
There's a great word
from the Tshiluba language, spoken in southeastern Democratic Republic of Congo, that regularly appears on impossible-to-translate word lists.
Ilunga
means someone who forgives any abuse the first time it occurs, tolerates it the second time, and neither forgives nor tolerates it the third time. The English saying is snappier: “Fool me once, shame on you; fool me twice, shame on me.”

Throughout most of history, commerce was a local phenomenon. Reputation made it work, and reputation was local. The emergence of long-distance commerce in the Western world was aided in great part by the involvement of European Quakers, who earned a reputation for dealing honorably with their business partners. Prior to the mid-17th century, European traders ran a significant risk that trading partners from other countries would act in their own self-interest and renege on promises they had made; overseas contracts were often unenforceable, so the potential for profit often outweighed the likelihood of punishment. However, the Quakers' religious commitment to integrity and simple living, and their belief in the essential worth of every individual, informed all of their business dealings. Being upright with God was more important to them than making a fast buck. The moral benefit they experienced from acting in accord with their consciences, and the ensuing reputational benefits within both their religious and business communities, outweighed any short-term financial gains that might have come from shady dealing. A Quaker found to have dealt with others dishonestly ran the risk not only of losing business opportunities, but of being expelled from his religious community. As a result,
Quakers would cooperate
even if it went against their self-interest, and—as they consolidated their positions in industry—there was a gradual increase of trust in them among overseas traders.

The Quakers were an exception. The problem with reputation is that it doesn't naturally scale well. Recall Dunbar's numbers. We can recognize 1,500 faces, but the number of people we know enough about to know their reputation is much lower—maybe 500 or even 150. Once our societies get larger than that, we need other mechanisms by which to infer reputation than direct knowledge of the other person. And, as you'd expect, we have developed several of these.

One of the ways to scale reputation is to generalize based on group membership. So we might believe that people with a particular skin color, or who speak a particular language, or who worship a particular God, are untrustworthy. We might believe that a Quaker is trustworthy.

During the mid-17th century, being a Quaker meant something to the general community. In different periods of history, so did being a Freemason, or a member of the Medici family. In the 12th century, you could take a Templar letter of credit issued in England all the way to Jerusalem. In the 11th century, the
Maghribi traders
of the medieval Mediterranean had a reputation similar to the Quakers. A thousand years earlier, Roman letters of introduction were similarly trusted throughout the empire.

Political scientist
Robert Putnam
has argued that mistrust increases in a community as ethnic diversity increases. Evidence of this effect comes from sources as diverse as studies of carpooling, Peruvian micro-credit cooperatives, and Civil War deserters. Even worse, this inherent mistrust of those in other ethnic groups isn't offset by an increase in trust of those in one's own ethnic group; trust across the board weakens in more ethnically diverse communities.
11

So it should come as no surprise that we have an enormous number of membership markers that we use to determine who is like us: language, dress, ethnicity, gang tags, haircuts, tattoos, jewelry, T-shirt slogans, food choices, gestures, secret handshakes, turns of phrase in speech, formal membership credentials, and so on. We generalize based on profession, city of residence, political affiliation, religious affiliation, sexual orientation, interests, and pretty much any other category you can think of. The theory is that all of these are
vestigial remnants
of prehistoric kin recognition mechanisms. But while these might have worked better in our evolutionary past than they do today, our brains are still stuck on them.

BOOK: Liars and Outliers
11.37Mb size Format: txt, pdf, ePub
ads

Other books

A Simple Mistake by Andrea Grigg
If She Should Die by Carlene Thompson
Sempre (Forever) by Darhower, JM
The Promise by Freda Lightfoot
Surrogate by Ellison James
The Meaning of Liff by Douglas Adams, John Lloyd
Fallen Eden by Williams, Nicole
Solea by Jean-Claude Izzo, Howard Curtis