How can an organization maximize the extent to which its managers take an outside view in their decisions? Pick a specific type of organizational decision (e.g., hiring, firing, picking a vendor, acquiring a company, etc.) and be specific in your prescriptions. Be aware of potential resistance to these changes. HINT: Make sure to reference both the HBR article AND the Cognitive Repairs article. Do NOT simply repeat the steps from the HBR reading or describe what a well-known company currently does. 

Product 4279

www.hbr.org

A R T I C L E

Delusions of
Success

How Optimism Undermines
Executives’ Decisions

by Dan Lovallo and Daniel Kahneman

Included with this full-text

Harvard Business Review

article

:

1

Article Summary

The Idea in Brief—

the core idea

The Idea in Practice—

putting the idea to work

2

Delusions of Success

10

Further Read

ing

A list of related material, with annotations to guide further
exploration of the article’s ideas and applications

http://harvardbusinessonline.hbsp.harvard.edu/relay.jhtml?name=itemdetail&referral=4320&id=4279

http://hbrarchives.harvardbusinessonline.org

page 1 of 10

Delusions of Success

How Optimism Undermines Executives’ Decisions

©
2

0
0
3
B

Y
H

A
R

V
A

R
D

B
U

S
IN

E
S

S
S

C
H

O
O

L
P

U
B

L
IS

H
IN

G
C

O
R

P
O

R
A

T
IO

N
. A

L
L

R
IG

H
T

S
R

E
S

E
R

V
E

D

.

The Idea in Brief

Three-quarters of business initiatives floun-

der—new manufacturing plants close pre-

maturely, mergers and acquisitions don’t

pay off, start-ups fail to gain market share.

Why?

Delusional optimism

: We overem-

phasize projects’ potential benefits and un-

derestimate likely costs, spinning success

scenarios while ignoring the possibility of

mistakes.

The culprits? Cognitive biases and orga-

nizational pressures to accentuate the posi-

tive. We can’t eradicate either, but we

can

take a more objective view of an initiative’s

likely outcome. How?

Reference forecast-

ing

: comparing a project’s potential out-

comes with those of similar, past projects—

to produce more accurate predictions.

The Idea in Practice

R O S E – C O L O R E D G L A S S E S

We’re subject to numerous

cognitive biases

:

Anchoring.

Competing for limited funding, we create

project proposals accentuating the positive.

These initial forecasts skew subsequent analy-

ses of market and financial information toward

overoptimism: We don’t adjust our original esti-

mates enough to account for inevitable prob-

lems.

Competitor neglect.

We ignore competitors’ capabilities and plans.

Rushing to secure a new market, for example,

we forget that rivals will follow suit. As compet-

itors ramp up production and marketing, sup-

ply outstrips demand—rendering the market

unprofitable.

Exaggerating our abilities and control.

We take credit for positive outcomes while at-

tributing negative outcomes to external factors

and deny the role of chance in our plans’ out-

comes. Result? We assume we can avoid or over-

come all project problems.

We also fall victim to

organizational pressures

:

We approve proposals with the highest
probability of failure.

Since only the most promising proposals attract

investment dollars, we make overoptimistic

forecasts.

Highly

overoptimistic proposals are

approved.

We reward optimism and interpret pessi-
mism as disloyalty.

Reinforcing one another’s unrealistic views of

the future, we undermine our company’s criti-

cal thinking.

T H E O U T S I D E V I E W

How to counteract cognitive biases and organi-

zational pressures? Awareness

and

a more ob-

jective forecasting method—especially with

never-before-attempted initiatives. These steps

can give us an “outside view” to augment our

intuitive “inside view”:

Select a set of past projects to serve as
your reference class.

A studio executive forecasting sales of a new

film selects recent films in the same genre, fea-

turing similar actors and comparable budgets.

Assess the distribution of outcomes.

Identify the average and extremes in the refer-

ence-class projects’ outcomes. The studio execu-

tive’s reference-class movies sold $40 million in

tickets on average. But 10% sold less than $2 mil-

lion and 5% sold more than $120 million.

Predict your project’s position in the dis-
tribution.

Intuitively estimate where your project would

fall along the reference class’s distribution. The

studio executive predicted $95 million as his

new film’s sales.

Assess your prediction’s reliability.

Counteract your biased prediction from Step 3.

Based on how well your past predictions

matched actual outcomes, estimate the correla-

tion between your

intuitive

prediction and the

actual

outcome. Express your estimate as a coef-

ficient between 0 and 1 (0 = no correlation; 1 =

complete correlation). The studio executive ex-

pressed his correlation coefficient as 0.6.

Correct your intuitive estimate.

Adjust your intuitive prediction based on your

predictability analysis. The studio executive’s

corrected

estimate was $62 million: $95M + [0.6

($40M – $95M)].

harvard business review • july 2003 page 2 of 10

©
2

0
0
3
B
Y
H
A
R
V
A
R
D
B
U
S
IN
E
S
S
S
C
H
O
O
L
P
U
B
L
IS
H
IN
G
C
O
R
P
O
R
A
T
IO
N
. A
L
L
R
IG
H
T
S
R
E
S
E
R
V
E
D
.

In planning major initiatives, executives routinely exaggerate the

bene�ts and discount the costs, setting themselves up for failure. Here’s

how to inject more reality into forecasting.

Delusions of
Success

How Optimism Undermines
Executives’ Decisions

by Dan Lovallo and Daniel Kahneman

In 1992, Oxford Health Plans started to build a
complex new computer system for processing
claims and payments. From the start, the
project was hampered by unforeseen prob-
lems and delays. As the company fell further
behind schedule and budget, it struggled,
vainly, to stem an ever rising flood of paper-
work. When, on October 27, 1997, Oxford dis-
closed that its system and its accounts were in
disarray, the company’s stock price dropped
63%, destroying more than $3 billion in share-
holder value in a single day.

Early in the 1980s, the United Kingdom,
Germany, Italy, and Spain announced that
they would work together to build the Euro-
fighter, an advanced military jet. The project
was expected to cost $20 billion, and the jet
was slated to go into service in 1997. Today,
after nearly two decades of technical glitches
and unexpected expenses, the aircraft has yet
to be deployed, and projected costs have more
than doubled, to approximately $45 billion.

In 1996, the Union Pacific railroad bought
its competitor Southern Pacific for $3.9 bil-

lion, creating the largest rail carrier in North
America. Almost immediately, the two compa-
nies began to have serious difficulties merging
their operations, leading to snarled traffic, lost
cargo, and massive delays. As the situation got
worse, and the company’s stock price tum-
bled, customers and shareholders sued the
railroad, and it had to cut its dividend and
raise new capital to address the problems.

Debacles like these are all too common in
business. Most large capital investment
projects come in late and over budget, never
living up to expectations. More than 70% of
new manufacturing plants in North America,
for example, close within their first decade of
operation. Approximately three-quarters of
mergers and acquisitions never pay off—the
acquiring firm’s shareholders lose more than
the acquired firm’s shareholders gain. And ef-
forts to enter new markets fare no better; the
vast majority end up being abandoned within
a few years.

According to standard economic theory, the
high failure rates are simple to explain: The

Delusions of Success

harvard business review • july 2003 page 3 of 10

frequency of poor outcomes is an unavoidable
result of companies taking rational risks in un-
certain situations. Entrepreneurs and manag-
ers know and accept the odds because the re-
wards of success are sufficiently enticing. In
the long run, the gains from a few successes
will outweigh the losses from many failures.

This is, to be sure, an attractive argument
from the perspective of executives. It effec-
tively relieves them of blame for failed
projects—after all, they were just taking rea-
sonable risks. But having examined this phe-
nomenon from two very different points of
view—a business scholar’s and a psycholo-
gist’s—we have come to a different conclu-
sion. We don’t believe that the high number of
business failures is best explained as the result
of rational choices gone wrong. Rather, we see
it as a consequence of flawed decision making.
When forecasting the outcomes of risky
projects, executives all too easily fall victim to
what psychologists call the planning fallacy. In
its grip, managers make decisions based on de-
lusional optimism rather than on a rational
weighting of gains, losses, and probabilities.
They overestimate benefits and underestimate
costs. They spin scenarios of success while
overlooking the potential for mistakes and
miscalculations. As a result, managers pursue
initiatives that are unlikely to come in on bud-
get or on time—or to ever deliver the expected
returns.

Executives’ overoptimism can be traced
both to cognitive biases—to errors in the way
the mind processes information—and to orga-
nizational pressures. These biases and pres-
sures are ubiquitous, but their effects can be
tempered. By supplementing traditional fore-
casting processes, which tend to focus on a
company’s own capabilities, experiences, and
expectations, with a simple statistical analysis
of analogous efforts completed earlier, execu-
tives can gain a much more accurate under-
standing of a project’s likely outcome. Such an

outside view

, as we call it, provides a reality
check on the more intuitive

inside view

, reduc-
ing the odds that a company will rush blindly
into a disastrous investment of money and
time.

Rose-Colored Glasses

Most people are highly optimistic most of the
time. Research into human cognition has
traced this overoptimism to many sources.

One of the most powerful is the tendency of
individuals to exaggerate their own talents—
to believe they are above average in their en-
dowment of positive traits and abilities. Con-
sider a survey of 1 million students conducted
by the College Board in the 1970s. When asked
to rate themselves in comparison to their
peers, 70% of the students said they were
above average in leadership ability, while only
2% rated themselves below average. For ath-
letic prowess, 60% saw themselves above the
median, 6% below. When assessing their abil-
ity to get along with others, 60% of the stu-
dents judged themselves to be in the top
decile, and fully 25% considered themselves to
be in the top 1%.

The inclination to exaggerate our talents is
amplified by our tendency to misperceive the
causes of certain events. The typical pattern of
such attribution errors, as psychologists call
them, is for people to take credit for positive
outcomes and to attribute negative outcomes
to external factors, no matter what their true
cause. One study of letters to shareholders in
annual reports, for example, found that execu-
tives tend to attribute favorable outcomes to
factors under their control, such as their cor-
porate strategy or their R&D programs. Unfa-
vorable outcomes, by contrast, were more
likely to be attributed to uncontrollable exter-
nal factors such as weather or inflation. Simi-
lar self-serving attributions have been found
in other studies of annual reports and execu-
tive speeches.

We also tend to exaggerate the degree of
control we have over events, discounting the
role of luck. In one series of studies, partici-
pants were asked to press a button that could
illuminate a red light. The people were told
that whether the light flashed was determined
by a combination of their action and random
chance. Afterward, they were asked to assess
what they experienced. Most people grossly
overstated the influence of their action in de-
termining whether the light flashed.

Executives and entrepreneurs seem to be
highly susceptible to these biases. Studies that
compare the actual outcomes of capital invest-
ment projects, mergers and acquisitions, and
market entries with managers’ original expec-
tations for those ventures show a strong ten-
dency toward overoptimism. An analysis of
start-up ventures in a wide range of industries
found, for example, that more than 80% failed

No matter how detailed, the

business scenarios used in

planning are generally

inadequate.

Dan Lovallo

is a senior lecturer at the
Australian Graduate School of Manage-
ment at the University of New South
Wales and a former strategy specialist
at McKinsey & Company.

Daniel Kahne-
man

is the Eugene Higgins Professor of
Psychology at Princeton University in
New Jersey and a professor of public af-
fairs at Princeton’s Woodrow Wilson
School; he received the Nobel Prize in
economic sciences in 2002.

Delusions of Success

harvard business review • july 2003 page 4 of 10

to achieve their market-share target. The stud-
ies are backed up by observations of execu-
tives. Like other people, business leaders rou-
tinely exaggerate their personal abilities,
particularly for ambiguous, hard-to-measure
traits like managerial skill. Their self-confi-
dence can lead them to assume that they’ll be
able to avoid or easily overcome potential
problems in executing a project. This misap-
prehension is further exaggerated by manag-
ers’ tendency to take personal credit for lucky
breaks. Think of mergers and acquisitions, for
instance. Mergers tend to come in waves, dur-
ing periods of economic expansion. At such
times, executives can overattribute their com-
pany’s strong performance to their own ac-
tions and abilities rather than to the buoyant
economy. This can, in turn, lead them to an
inflated belief in their own talents. Conse-
quently, many M&A decisions may be the re-
sult of hubris, as the executives evaluating an
acquisition candidate come to believe that,
with proper planning and superior manage-
ment skills, they could make it more valuable.
Research on postmerger performance sug-
gests that, on average, they are mistaken.

Managers are also prone to the illusion that
they are in control. Sometimes, in fact, they
will explicitly deny the role of chance in the
outcome of their plans. They see risk as a chal-
lenge to be met by the exercise of skill, and
they believe results are determined purely by
their own actions and those of their organiza-
tions. In their idealized self-image, these exec-
utives are not gamblers but prudent and deter-
mined agents, who are in control of both
people and events. When it comes to making
forecasts, therefore, they tend to ignore or
downplay the possibility of random or uncon-
trollable occurrences that may impede their
progress toward a goal.

The cognitive biases that produce overopti-
mism are compounded by the limits of human
imagination. No matter how detailed, the
business scenarios used in planning are gener-
ally inadequate. The reason is simple: Any
complex project is subject to myriad prob-
lems—from technology failures to shifts in ex-
change rates to bad weather—and it is beyond
the reach of the human imagination to foresee
all of them at the outset. As a result, scenario
planning can seriously understate the proba-
bility of things going awry. Often, for instance,
managers will establish a “most likely” sce-

nario and then assume that its outcome is in
fact the most likely outcome. But that assump-
tion can be wrong. Because the managers have
not fully considered all the possible sequences
of events that might delay or otherwise dis-
rupt the project, they are likely to understate
the overall probability of unfavorable out-
comes. Even though any one of those out-
comes may have only a small chance of occur-
ring, in combination they may actually be far
more likely to happen than the so-called most
likely scenario.

Accentuating the Positive

In business situations, people’s native opti-
mism is further magnified by two other kinds
of cognitive bias—anchoring and competitor
neglect—as well as political pressures to em-
phasize the positive and downplay the nega-
tive. Let’s look briefly at each of these three
phenomena.

Anchoring.

When executives and their sub-
ordinates make forecasts about a project, they
typically have, as a starting point, a prelimi-
nary plan drawn up by the person or team pro-
posing the initiative. They adjust this original
plan based on market research, financial anal-
ysis, or their own professional judgment be-
fore arriving at decisions about whether and
how to proceed. This intuitive and seemingly
unobjectionable process has serious pitfalls,
however. Because the initial plan will tend to
accentuate the positive—as a proposal, it’s de-
signed to make the case for the project—it will
skew the subsequent analysis toward overopti-
mism. This phenomenon is the result of an-
choring, one of the strongest and most preva-
lent of cognitive biases.

In one experiment that revealed the power
of anchoring, people were asked for the last
four digits of their Social Security number.
They were then asked whether the number of
physicians in Manhattan is larger or smaller
than the number formed by those four digits.
Finally, they were asked to estimate what the
number of Manhattan physicians actually is.
The correlation between the Social Security
number and the estimate was significantly
positive. The subjects started from a random
series of digits and then insufficiently adjusted
their estimate away from it.

Anchoring can be especially pernicious
when it comes to forecasting the cost of major
capital projects. When executives set budgets

When pessimistic opinions

are suppressed, while

optimistic ones are

rewarded, an

organization’s ability to

think critically is

undermined.

Delusions of Success

harvard business review • july 2003 page 5 of 10

for such initiatives, they build in contingency
funds to cover overruns. Often, however, they
fail to put in enough. That’s because they’re
anchored to their original cost estimates and
don’t adjust them sufficiently to account for
the likelihood of problems and delays, not to
mention expansions in the scope of the
projects. One Rand Corporation study of 44
chemical-processing plants owned by major
companies like 3M, DuPont, and Texaco found
that, on average, the factories’ actual construc-
tion costs were more than double the initial
estimates. Furthermore, even a year after
start-up, about half the plants produced at less
than 75% of their design capacity, with a quar-
ter producing at less than 50%. Many of the
plants had their performance expectations
permanently lowered, and the owners never
realized a return on their investments.

Competitor Neglect.

One of the key factors
influencing the outcome of a business initia-
tive is competitors’ behavior. In making fore-
casts, however, executives tend to focus on
their own company’s capabilities and plans
and are thus prone to neglect the potential
abilities and actions of rivals. Here, again, the
result is an underestimation of the potential
for negative events—in this case, price wars,
overcapacity, and the like. Joe Roth, the
former chairman of Walt Disney Studios, ex-
pressed the problem well in a 1996 interview
with the

Los Angeles Times

: “If you only think
about your own business, you think, ‘I’ve got a
good story department, I’ve got a good mar-
keting department, we’re going to go out and
do this.’ And you don’t think that everybody
else is thinking the same way.”

Neglecting competitors can be particularly
destructive in efforts to enter new markets.
When a company identifies a rapidly growing
market well suited to its products and capabil-
ities, it will often rush to gain a beachhead in
it, investing heavily in production capacity
and marketing. The effort is often justified by
the creation of attractive pro forma forecasts
of financial results. But such forecasts rarely
account for the fact that many other competi-
tors will also target the market, convinced that
they, too, have what it takes to succeed. As all
these companies invest, supply outstrips de-
mand, quickly rendering the new market un-
profitable. Even savvy venture capitalists fell
into this trap during the recent ill-fated Inter-
net boom.

Organizational Pressure.

Every company
has only a limited amount of money and time
to devote to new projects. Competition for this
time and money is intense, as individuals and
units jockey to present their own proposals as
being the most attractive for investment. Be-
cause forecasts are critical weapons in these
battles, individuals and units have big incen-
tives to accentuate the positive in laying out
prospective outcomes. This has two ill effects.
First, it ensures that the forecasts used for
planning are overoptimistic, which, as we de-
scribed in our discussion of anchoring, distorts
all further analysis. Second, it raises the odds
that the projects chosen for investment will be
those with the most overoptimistic forecasts—
and hence the highest probability of disap-
pointment.

Other organizational practices also encour-
age optimism. Senior executives tend, for in-
stance, to stress the importance of stretch
goals for their business units. This can have
the salutary effect of increasing motivation,
but it can also lead unit managers to further
skew their forecasts toward unrealistically rosy
outcomes. (And when these forecasts become
the basis for compensation targets, the prac-
tice can push employees to behave in danger-
ously risky ways.) Organizations also actively
discourage pessimism, which is often inter-
preted as disloyalty. The bearers of bad news
tend to become pariahs, shunned and ignored
by other employees. When pessimistic opin-
ions are suppressed, while optimistic ones are
rewarded, an organization’s ability to think
critically is undermined. The optimistic biases
of individual employees become mutually re-
inforcing, and unrealistic views of the future
are validated by the group.

The Outside View

For most of us, the tendency toward optimism
is unavoidable. And it’s unlikely that compa-
nies can, or would even want to, remove the
organizational pressures that promote opti-
mism. Still, optimism can, and should, be tem-
pered. Simply understanding the sources of
overoptimism can help planners challenge as-
sumptions, bring in alternative perspectives,
and in general take a balanced view of the fu-
ture.

But there’s also a more formal way to im-
prove the reliability of forecasts. Companies
can introduce into their planning processes an

Delusions of Success

harvard business review • july 2003 page 6 of 10

objective forecasting method that counteracts
the personal and organizational sources of op-
timism. We’ll begin our exploration of this ap-
proach with an anecdote that illustrates both
the traditional mode of forecasting and the
suggested alternative.

In 1976, one of us was involved in a project
to develop a curriculum for a new subject area
for high schools in Israel. The project was con-
ducted by a small team of academics and
teachers. When the team had been operating
for about a year and had some significant
achievements under its belt, its discussions
turned to the question of how long the project
would take. Everyone on the team was asked
to write on a slip of paper the number of
months that would be needed to finish the
project—defined as having a complete report
ready for submission to the Ministry of Educa-
tion. The estimates ranged from 18 to 30
months.

One of the team members—a distinguished
expert in curriculum development—was then
posed a challenge by another team member:
“Surely, we’re not the only team to have tried
to develop a curriculum where none existed
before. Try to recall as many such projects as
you can. Think of them as they were in a stage
comparable to ours at present. How long did it
take them at that point to reach completion?”
After a long silence, the curriculum expert
said, with some discomfort, “First, I should say
that not all the teams that I can think of, that
were at a comparable stage, ever did complete
their task. About 40% of them eventually gave
up. Of the remaining, I cannot think of any
that completed their task in less than seven
years, nor of any that took more than ten.” He
was then asked if he had reason to believe that
the present team was more skilled in curricu-
lum development than the earlier ones had
been. “No,” he replied, “I cannot think of any
relevant factor that distinguishes us favorably
from the teams I have been thinking about. In-
deed, my impression is that we are slightly
below average in terms of resources and po-
tential.” The wise decision at this point would
probably have been for the team to disband.
Instead, the members ignored the pessimistic
information and proceeded with the project.
They finally completed the initiative eight
years later, and their efforts went largely for
naught—the resulting curriculum was rarely
used.

In this example, the curriculum expert made
two forecasts for the same problem and arrived
at very different answers. We call these two dis-
tinct modes of forecasting the inside view and
the outside view. The inside view is the one that
the expert and all the other team members
spontaneously adopted. They made forecasts
by focusing tightly on the case at hand—consid-
ering its objective, the resources they brought
to it, and the obstacles to its completion; con-
structing in their minds scenarios of their com-
ing progress; and extrapolating current trends
into the future. Not surprisingly, the resulting
forecasts, even the most conservative ones,
were exceedingly optimistic.

The outside view, also known as reference-
class forecasting, is the one that the curricu-
lum expert was encouraged to adopt. It com-
pletely ignored the details of the project at
hand, and it involved no attempt at forecast-
ing the events that would influence the
project’s future course. Instead, it examined
the experiences of a class of similar projects,
laid out a rough distribution of outcomes for
this reference class, and then positioned the
current project in that distribution. The result-
ing forecast, as it turned out, was much more
accurate.

The contrast between inside and outside
views has been confirmed in systematic re-
search. Recent studies have shown that when
people are asked simple questions requiring
them to take an outside view, their forecasts
become significantly more objective and reli-
able. For example, a group of students enroll-
ing at a college were asked to rate their future
academic performance relative to their peers
in their major. On average, these students ex-
pected to perform better than 84% of their
peers, which is logically impossible. Another
group of incoming students from the same
major were asked about their entrance scores
and their peers’ scores before being asked
about their expected performance. This sim-
ple detour into pertinent outside-view infor-
mation, which both groups of subjects were
aware of, reduced the second group’s average
expected performance ratings by 20%. That’s
still overconfident, but it’s much more realistic
than the forecast made by the first group.

Most individuals and organizations are in-
clined to adopt the inside view in planning
major initiatives. It’s not only the traditional
approach; it’s also the intuitive one. The natu-

Delusions of Success

harvard business review • july 2003 page 7 of 10

ral way to think about a complex project is to
focus on the project itself—to bring to bear all
one knows about it, paying special attention to
its unique or unusual features. The thought of
going out and gathering statistics about related
cases seldom enters a planner’s mind. The cur-
riculum expert, for example, did not take the
outside view until prompted—even though he
already had all the information he needed.
Even when companies bring in independent
consultants to assist in forecasting, they often
remain stuck in the inside view. If the consult-
ants provide comparative data on other compa-
nies or projects, they can spur useful outside-
view thinking. But if they concentrate on the
project itself, their analysis will also tend to be
distorted by cognitive biases.

While understandable, managers’ prefer-
ence for the inside view over the outside view
is unfortunate. When both forecasting meth-
ods are applied with equal intelligence and
skill, the outside view is much more likely to
yield a realistic estimate. That’s because it by-
passes cognitive and organizational biases. In
the outside view, managers aren’t required to
weave scenarios, imagine events, or gauge
their own levels of ability and control—so they
can’t get all those things wrong. And it doesn’t
matter if managers aren’t good at assessing
competitors’ abilities and actions; the impact
of those abilities and actions is already re-
flected in the outcomes of the earlier projects
within the reference class. It’s true that the
outside view, being based on historical prece-
dent, may fail to predict extreme outcomes—
those that lie outside all historical precedents.
But for most projects, the outside view will
produce superior results.

The outside view’s advantage is most pro-
nounced for initiatives that companies have
never attempted before—like building a plant
with a new manufacturing technology or en-
tering an entirely new market. It is in the plan-
ning of such de novo efforts that the biases to-
ward optimism are likely to be great.
Ironically, however, such cases are precisely
where the organizational and personal pres-
sures to apply the inside view are most in-
tense. Managers feel that if they don’t fully ac-
count for the intricacies of the proposed
project, they would be derelict in their duties.
Indeed, the preference for the inside view over
the outside view can feel almost like a moral
imperative. The inside view is embraced as a

serious attempt to come to grips with the com-
plexities of a unique challenge, while the out-
side view is rejected as relying on a crude anal-
ogy to superficially similar instances. Yet the
fact remains: The outside view is more likely
to produce accurate forecasts and much less
likely to deliver highly

unrealistic ones.

Of course, choosing the right class of analo-
gous cases becomes more difficult when exec-
utives are forecasting initiatives for which pre-
cedents are not easily found. It’s not like in the
curriculum example, where many similar ef-
forts had already been undertaken. Imagine
that planners have to forecast the results of an
investment in a new and unfamiliar technol-
ogy. Should they look at their company’s ear-
lier investments in new technologies? Or
should they look at how other companies car-
ried out projects involving similar technolo-
gies? Neither is perfect, but each will provide
useful insights—so the planners should ana-
lyze both sets of analogous cases. We provide a
fuller explanation of how to identify and ana-
lyze a reference class in the sidebar “How to
Take the Outside View.”

Putting Optimism in Its Place

We are not suggesting that optimism is bad, or
that managers should try to root it out of
themselves or their organizations. Optimism
generates much more enthusiasm than does
realism (not to mention pessimism), and it en-
ables people to be resilient when confronting
difficult situations or challenging goals. Com-
panies have to promote optimism to keep em-
ployees motivated and focused. At the same
time, though, they have to generate realistic
forecasts, especially when large sums of
money are at stake. There needs to be a bal-
ance between optimism and realism—be-
tween goals and forecasts. Aggressive goals
can motivate the troops and improve the
chances of success, but outside-view forecasts
should be used to decide whether or not to
make a commitment in the first place.

The ideal is to draw a clear distinction be-
tween those functions and positions that in-
volve or support decision making and those
that promote or guide action. The former
should be imbued with a realistic outlook,
while the latter will often benefit from a sense
of optimism. An optimistic CFO, for example,
could mean disaster for a company, just as a
lack of optimism would undermine the vision-

The outside view is more

likely to produce accurate

forecasts and much less

likely to deliver highly

unrealistic ones.

Delusions of Success

harvard business review • july 2003 page 8 of 10

How to Take the Outside View
Making a forecast using the outside view requires planners to
identify a reference class of analogous past initiatives, deter-
mine the distribution of outcomes for those initiatives, and
place the project at hand at an appropriate point along that
distribution. This effort is best organized into five steps:1

1. Select a reference class. Identifying the right reference
class involves both art and science. You usually have to weigh sim-
ilarities and differences on many variables and determine which
are the most meaningful in judging how your own initiative will
play out. Sometimes that’s easy. If you’re a studio executive trying
to forecast sales of a new film, you’ll formulate a reference class
based on recent films in the same genre, starring similar actors,
with comparable budgets, and so on. In other cases, it’s much
trickier. If you’re a manager at a chemical company that is consid-
ering building an olefin plant incorporating a new processing
technology, you may instinctively think that your reference class
would include olefin plants now in operation. But you may actu-
ally get better results by looking at other chemical plants built
with new processing technologies. The plant’s outcome, in other
words, may be more influenced by the newness of its technology
than by what it produces. In forecasting an outcome in a compet-
itive situation, such as the market share for a new venture, you
need to consider industrial structure and market factors in de-
signing a reference class. The key is to choose a class that is broad
enough to be statistically meaningful but narrow enough to be
truly comparable to the project at hand.

2. Assess the distribution of outcomes. Once the refer-
ence class is chosen, you have to document the outcomes of
the prior projects and arrange them as a distribution, showing
the extremes, the median, and any clusters. Sometimes you
won’t be able to precisely document the outcomes of every
member of the class. But you can still arrive at a rough distri-
bution by calculating the average outcome as well as a mea-
sure of variability. In the film example, for instance, you may
find that the reference-class movies sold $40 million worth of
tickets on average, but that 10% sold less than $2 million worth
of tickets and 5% sold more than $120 million worth.

3. Make an intuitive prediction of your project’s position
in the distribution. Based on your own understanding of the
project at hand and how it compares with the projects in the
reference class, predict where it would fall along the distribu-
tion. Because your intuitive estimate will likely be biased, the
final two steps are intended to adjust the estimate in order to
arrive at a more accurate forecast.

4. Assess the reliability of your prediction. Some events
are easier to foresee than others. A meteorologist’s forecast

of temperatures two days from now, for example, will be
more reliable than a sportscaster’s prediction of the score of
next year’s Super Bowl. This step is intended to gauge the
reliability of the forecast you made in Step 3. The goal is to
estimate the correlation between the forecast and the actual
outcome, expressed as a coefficient between 0 and 1, where 0
indicates no correlation and 1 indicates complete correlation.
In the best case, information will be available on how well
your past predictions matched the actual outcomes. You can
then estimate the correlation based on historical precedent.
In the absence of such information, assessments of predict-
ability become more subjective. You may, for instance, be
able to arrive at an estimate of predictability based on how
the situation at hand compares with other forecasting situa-
tions. To return to the movie example, say that you are fairly
confident that your ability to predict the sales of films ex-
ceeds the ability of sportscasters to predict point spreads in
football games but is not as good as the ability of weather
forecasters to predict temperatures two days out. Through a
diligent statistical analysis, you could construct a rough scale
of predictability based on computed correlations between
predictions and outcomes for football scores and tempera-
tures. You can then estimate where your ability to predict
film scores lies on this scale. When the calculations are com-
plex, it may help to bring in a skilled statistician.

5. Correct the intuitive estimate. Due to bias, the intuitive
estimate made in Step 3 will likely be optimistic—deviating
too far from the average outcome of the reference class. In this
final step, you adjust the estimate toward the average based on
your analysis of predictability in Step 4. The less reliable the
prediction, the more the estimate needs to be regressed to-
ward the mean. Suppose that your intuitive prediction of a
film’s sales is $95 million and that, on average, films in the
reference class do $40 million worth of business. Suppose fur-
ther that you have estimated the correlation coefficient to be
0.6. The regressed estimate of ticket sales would be:

$95M + [0.6 ($40M–$95M)] = $62M
As you see, the adjustment for optimism will often be sub-

stantial, particularly in highly uncertain situations where pre-
dictions are unreliable.

1. This discussion builds on “Intuitive Predictions: Biases and Corrective
Procedures,” a 1979 article by Daniel Kahneman and Amos Tversky that ap-
peared in TIMS Studies in Management Science, volume 12 (Elsevier/North
Holland).

Delusions of Success

harvard business review • july 2003 page 9 of 10

ary qualities essential for superior R&D and
the esprit de corps central to a successful sales
force. Indeed, those charged with implement-
ing a plan should probably not even see the
outside-view forecasts, which might reduce
their incentive to perform at their best.

Of course, clean distinctions between deci-
sion making and action break down at the top.
CEOs, unit managers, and project champions
need to be optimistic and realistic at the same
time. If you happen to be in one of these posi-
tions, you should make sure that you and your
planners adopt an outside view in deciding
where to invest among competing initiatives.
More objective forecasts will help you choose

your goals wisely and your means prudently.
Once an organization is committed to a course
of action, however, constantly revising and re-
viewing the odds of success is unlikely to be
good for its morale or performance. Indeed, a
healthy dose of optimism will give you and
your subordinates an advantage in tackling
the challenges that are sure to lie ahead.

Reprint R0307D;

Harvard Business Review

OnPoint 4279

To order, see the next page

or go to www.hbr.org
or call 800-988-0886 or 617-783-7500

http://hbrarchives.harvardbusinessonline.org

http://harvardbusinessonline.hbsp.harvard.edu/relay.jhtml?name=itemdetail&referral=4320&id=4279

http://harvardbusinessonline.hbsp.harvard.edu/relay.jhtml?name=itemdetail&referral=4320&id=R0307D

page 10 of 10

To Order

For reprints,

Harvard Business Review

OnPoint orders, and subscriptions to

Harvard Business Review

:
Call 800-988-0886 or 617-783-7500.
Go to www.hbr.org

For customized and quantity orders of
reprints and

Harvard Business Review

OnPoint products:
Call Frank Tamoshunas at

or e-mail him at
ftamoshunas@hbsp.harvard.edu

Delusions of Success

How Optimism Undermines Executives’ Decisions

Further Reading

A R T I C L E S

The High Cost of Accurate Knowledge

by Kathleen M. Sutcliffe and Klaus Weber

Harvard Business Review

May 2003

Product no. R0305E

These authors agree that senior managers’ abil-

ity to interpret information is critical to making

better decisions. Today’s complex information,

they maintain, is rarely precise—and often am-

biguous and conflicting. Therefore, companies

should think carefully about whether to invest

heavily in systems for collecting and organizing

vast amounts of competitive data. Information’s

accuracy and abundance are less important for

strategy and organizational change than the

ways in which executives

interpret

such informa-

tion—and communicate their interpretations.

In other words, executives must manage

mean-

ing

more than they manage

information

.

Sutcliffe and Weber aren’t suggesting that accu-

rate information doesn’t matter at all. Corpo-

rate leaders must have clear knowledge of their

industries. But managers’ interpretive

outlooks

determine their companies’ competitive advan-

tage more than the information itself. And the

most successful leaders interpret information

through a curious blend of optimism and pessi-

mism that the authors call “humble optimism”:

They embrace opportunities—but they’re not

overly confident in their ability to control those

opportunities. They thus manage ambiguity—

while simultaneously mobilizing action.

What Do Managers Know, Anyway?

by John M. Mezias and William H. Starbuck

Harvard Business Review

May 2003

Product no. F0305A

Mezias and Starbuck also contend that accurate

competitive information may be less important

to a company’s success than previously as-

sumed. And they add another important piece

to the decision-making puzzle: managers’ will-

ingness to seek and make wise use of feedback.

Managers, the authors maintain, often have

badly distorted pictures of their businesses and

their competitive environments. And they have

great confidence in their own distorted percep-

tions. Why? They tend to focus on what’s hap-

pening right now, in their specific jobs, in their

specific business units, operating in their very

specific competitive worlds. Busy among the

trees, they lose sight of the forest. They also base

their analyses on sources of varied reliability—

such as corporate documents they often misun-

derstand, personal experiences, and rumors.

And they also surround themselves with

yea-sayers.

This may sound like a recipe for disastrously

large errors in judgment. But when managers

get prompt feedback on the impact of their de-

cisions, their misperceptions may cause only

small errors—if they respond appropriately.

The challenge lies in overcoming managers’

fear of sanctions if they’re wrong. Companies

must accept that distorted perception is a fact of

management—and design decision processes

that work despite inaccurate perceptions. The

implications? Encourage managers to admit

their errors and modify their approaches

accordingly.

617-783-7626,

http://harvardbusinessonline.hbsp.harvard.edu/relay.jhtml?name=itemdetail&referral=4320&id=F0305A

http://harvardbusinessonline.hbsp.harvard.edu/relay.jhtml?name=itemdetail&referral=4320&id=R0305E

mailto:ftamoshunas@hbsp.harvard.edu

http://hbrarchives.harvardbusinessonline.org

COGNITIVE REPAIRS:
HOW ORGANIZATIONAL PRACTICESCAN

COMPENSATEFOR
INDIVIDUAL SHORTCOMINGS

ChipHeath,RichardP. Larrick, andJoshuaKlayman

ABSTRACT

The literaturein cognitive psychology has described a varietyof shortcomings that
preventindividuals from learningeffectively. We review this literature and provide
examplesofa numberoforganizational practices that may effectivelyrepairthecog-
nitive shortcomingsof individuals. We call these practicescognitiverepairs. We
then discusssix tradeoffs that affect the success of cognitive repairs.We close by
consideringhow a cognitive perspective might benefitthosewho studyorganiza-
tional learning and those who manageit.

Research inOrganizational Behavior, Volume 20, pages

1

-37.
Copyright © 1998 hy JAI Press Inc.
All rights of reproduction in any form reserved.
ISBN: 0-7623-0366-2

1

CHIP HEATH, RICHARD P. LARRICK, andJOSHUA KLAYMAN CognitiveRepairs 3

INTRODUCTION

In afamous speech, Hamletdeclares,“What a pieceof work is man.How noble in
reason,how infinite in faculties” (Hamlet, II, 2). An observer whosummarizedthe
psychologyof the late twentiethcenturywould probably choose very different
phrases to describe the humancondition—perhaps,“What fools these mortals be”
(Midsummer Night’sDream, III, 2).

Are people“infinite in faculties” and “noble inreason”?HerbertSimon won a
Nobel prize for arguingthat social science mustunderstandthe waysthat human
facultiesare limited. Instead of being infinite in faculties,Simon’s humans could
be only “boundedlyrational” because theircognitive abilities—their ability to
perceive,remember,andprocessinformation—were restricted. Well, then, ifpeo-
ple arenot infinite in faculties,a.re they “noble inreason”?Cognitive psycholo-
gists havespent30 years examining the actualprocessesthat people usewhen
they collect information, combineit, anddraw inferences about their world(Nis-
bett & Ross, 1980; Kahneman,Slovic, & Tversky, 1982; Holland,Holyoak,Nis-
bett, & Thagard,1986).Insteadof depicting people as “noble” (or magnificent) in
reason, this researchhas arguedthat peoplereasonin ways that producesystem-
atic errors. A pessimistic modern Hamlet might combine the observations of these
two research streams and describe humans as equipped with primitive hardwnre
andbuggy software.

However, outsidershave not alwaysacceptedthe pessimistic description of
human faculties and reason that is found in the research literature.As one skeptic
put it, “If we are so stupid,how did we get to the moon?” (Nisbett & Ross,1980).

How should we resolve the apparent discrepancy betweenthe pessimisticliter-
atureon human shortcomings and the optimistic evidence of humanaccomplish-
ment? One way is todismiss the laboratory research. Someresearchershave
argued that the shortcomings that have beendocumentedin the lab areso minor
that they do not constitute mistakes ofany real consequence(Funder, 1987;
Cohen,1981).Others havearguedthat individuals areless likely to make errors in
natural environmentsthan in contrived laboratory experiments (Anderson,1991;
Gigerenzer,1996; Cheng& Holyoak, 1985;Hilton, 1995).

We proposeanotherway to resolvethe discrepancy.Unlike someresearchers,
we do not dismissthe examples of limitations, errors, and biases reported in thelit-
erature; we assume that individuals arelimited, their decision processes are
biased, andthat they often make costly mistakes onimportant decisions. We
resolve the apparent discrepancy between evidence of individualshortcomings
and the empirical factof moonwalksby observing thatindividualsdid not make it
to themoon, NASA did.

Organizationslike NASA may have discovered ways toavoid~orrepair theindi-
vidual shortcomingsthat have been documentedby cognitive researchers.Orga-
nizations may develop such repairs through deliberate analysis, learn them
throughtrial and error, or discoverthem through serendipitous accident.In some

cases, repairsmay derive from formal academicdisciplineslike economicsor sta-
tistics (e.g., Nisbett,1992;Nisbett, Krantz, Jepson, &Kunda, 1983; Larrick, Mor-
gan, & Nisbett, 1990), but in most casesthey will not: They will be ad hoc,
intuitive rulesthat emerge from day-to-daypractice. Our thesis,then, isthat indi-
viduals indeed face cognitive limitations and shortcomings, but thatorganizations
can provide individuals withnormsandproceduresthat mitigate theirlimitations
and reduce their shortcomings.

In this paper we describe a variety of potentiallyseriousshortcomingsthat have
beendocumentedin researchon humanjudgmentand reasoning. We focus inpar-
ticular on learning and hypothesis testing,that is, how people use information to
develop and revise theirmental model of the world.Foreach cognitiveshortcom-
ing we discuss, we provide examples of organizational practices thatmay repair
this shortcoming. We call these practicescognitive repairsto emphasize the fact
that they correct somecognitive processthat wasinitially flawed and in need of
repair.

We identify potential cognitive repairs to spurresearchersto considerhow such
repairs might look and function.Althoughat thispoint, we canmake only ananec-
dotal case forinterpretingcertain practicesas “repairs,” we hope that,by pointing
out someplausible examples of such repairs, we willpromptresearchersin both
psychologyand organizationsto considermore systematicallyhow such repairs
might function.

More generally, the concept oforganizationalcognitive repairsillustratesthat
researchers may find interesting relationships betweenindividual cognitionand
organizational practice.Theserelationships have not received the attention they
deserve.On the one side, research in cognitive psychology has largely treated
individual learnersas “rugged individualists”who face a difficult environment
alone, equippedonly with their own, flawed cognitive strategies. On the other
side, organizational research has largely ignored the literature on individualcog-
nition, focusing insteadon issues of motivation orincentives.By studyingorgani-
zational sources of cognitive repairs, we bringtogethertwo frequently disparate
literatures anddemonstratehow knowledge at onelevel of analysiscaninform the
other.

By reviewing individual shortcomingsand identifying potential cognitive
repairs, wealsohope tocontributeto the academic andpopular literatureonorga-
nizational learning (Epple,Argote, & Devadas, 1991; Deming, 1982; Senge,
1990; Cohen, 1991; Miner & Mezias, 1996).One important means tofacilitate
learningby organizations is to develop processes that overcome theshortcomings
of individuals within the organization.

Below, we start with a briefexampleof the kinds of repairs that weconsiderin
this paper. Then we introduce a framework that describes different stages in the
learning process, and we use it toreview the literature onindividual shortcomings
and to suggest potential repairs.As a preliminary reply toHamlet,we say that
even ifruggedindividualsare unlikely to be infinite in faculties and noble inrea-

4 CHIP HEATH, RICHARD P. LARRICK, and JOSHUA KLAYMAN CognitiveRepairs

son, individualswho have access to organizational and cultural repairs maysome-
times appearso.

An Exampleof OurApproach

Consider one studythat might be regardedas an ominous indication of ignoble
reasoningby individual experts. Hynes andVanmarcke(1976) asked seven“inter-
nationally known” civil engineers to predict the height of a structurethat would
cause a foundation tofail; they alsoasked the engineers toset a 50 percentconfi-
dence interval around their prediction sothat their confidenceinterval was wide
enough to have a 50 percent chance of enclosing the true failure height. Theresults
were quitesobering: not one engineer correctly predicted the truefailure height
within his or herconfidenceinterval.

Evidently, the civil engineers thoughtthey knew more thanthey did—if they
had been awareof the limitations of their analysis,they would haveset widercon-
fidenceintervals andwould have predicted the truefailure heightmorecorrectly.
In the psychologicalliterature this kind of finding has been labeled“overconfi-
dence,”and it is not anaberration.Similarresultshave been observed with anum-
ber of individual professionals(e.g., Russo & Schoemaker,1992).In summarizing
the evidence,Griffin and Tversky (1992) quipped that experts are “often wrong
but rarely in doubt.”

To illustrate why this study paintsan ominous picture of individual reasoning,
considerthat (unlessyou arereading this paper outside)you are sitting in abuild-
ing that was constructedby civil engineers who were substantiallyless accom-
plishedthanthe internationally known experts in thestudy. Your civil engineers
made numerous decisions to ensure the stability and safety of yourbuilding; they
decidedhow strong to makeits roofsupports andhow stableto makeits founda-
tton. If evenexpertengineersare overconfident,shouldyou be concerned about
your safety?

The answer, we believe, is no. Fortunately, the engineering profession has
developeda particular repair, called“safety factors,” that mitigatethe overconfi-
dent reasoningof individual engineers.In an actual assignment civil engineers
would preciselycalculatethe amount andstrengthof foundationmaterialsneces-
sary to hold a structure of a particular height,then they would multiply their pre-
cise answerby a safety factor(i.e., a number between three and eight), and use the
larger figure to build the foundation.Were the confidenceintervals of the engi-
neerstoo narrow?Yes. Werethey toonarrow by a factor of three? No.

Safetyfactors arean example of thekindof cognitive repair weconsiderin this
paper.An organization (e.g., an engineeringfirm or the engineering profession at
large) provides its members with a repair thathelps combat a systematic and
potentially serious bias inindividual judgment. As a result,the organization
shields individualswithin the organization from actingon their flawed decisions,

5

and it shields individuals inside andoutsidethe organizationfrom suffering the
consequences.

Whatis a CognitiveRepair?

Organizational repairscanroughly bedivided into two classes: (1) motivational
repairs increase theenergy and enthusiasm with which individuals pursue a task
and (2) cognitive repairs improve the mentalproceduresindividuals use todecide
which task to pursue andhow to pursueit. Organizational research onmotivation
and incentives can be regarded as the study of motivational repairs(Milgrom &
Roberts,1992; Eisenhardt,1989).Organizations may need to repair motivational
problems in order to encourage individuals tolearn(e.g., see Heath,Knez, &

Camerer,1993).For example, individualsmay not be willing toexperimentwith
newtasks becausethey have becomeendowedwith the benefits associated with
the old task.

Although previous work hasrecognizedtheimportanceof motivational repairs,
it hasneglected cognitive repairs. Even when individuals have the right incentives
and resources, theymay not learnfrom their experience ifthey use the wrong
mentalprocess togeneratehypotheses,collectinformation,anddrawconclusions.
The civil engineers who misestimated the stability of the clayembankment.were
adequatelymotivatedto getthe right answer. However,they did not on their.own
invoke the kind of correctives(e.g., safety factors)that might havemadetheir
guesses more appropriately cautious.

REVIEW OF INDIVIDUAL LEARNING AND
EXAMPLES OF COGNITIVE REPAIRS

In this section we organize the literatureon learning and decision making around
three different stages of the learning process. Effective learners must (1) generate
hypotheses that explain thecausalstructureof the world, (2)collectinformation to
distinguishamong theirhypotheses,(3) draw conclusions that areappropriateand
~autious. Theboundariesbetween these stages arefuzzy—theyare interrelated
~ndinterconnected(Klayman, 1995). However, we distinguish among them

ecause theyinvolve different psychological processes.
Our strategythroughoutthe review is to considerfirst the individual then the
‘ganization. For each stageof learning, we describe howan ideal individual
irner might reason, andreview psychological research showing how realmdi-
luals depart from this ideal. Then, we describe potential cognitive repairsby
ich organizations might correct the individual shortcoming in question.

6 CHIP HEATH, RICHARD P. LARRICK, and JOSHUAKLAYMAN CognitiveRepairs 7

Generating Hypotheses

In the first stage of the learning process individuals must generate hypotheses
about the relationships amongevents.Subjectto constraints of time andinforma-
tion, individuals should generate hypotheses that are deep(i.e., by considering
causesthat are more general or systemic) and broad(i.e., by considering a larger
numberof potential causes).However,a greatdeal of psychological researchsug-
geststhat individuals develop hypothesesthat are shallowand narrow.

Individuals GenerateHypotheses thatareShallowRatherthanDeep

Individuals Searchfor Explanationsthat MakeThemselvesLook Good

Individuals often conductshallow searches when theytry to explain success or
failure becausethey search in a self-serving way(i.e., in a way that allows them
to feel good about themselves).In a meta-analysisof 91 tests of this self-serving
bias, Mullen and Riordan (1988)show that individuals typically conclude that
their successesresulted from stable, internal factors (e.g., ability), butthat their
Jailures resultedfrom unstable,environmentalfactors (e.g., thedifficulty of the
environment, insufficient effort, or bad luck)(see alsoFiske & Taylor, 1991,
pp. 78-82).

How might organizations repair self-servingbiases?Some repairsmay be quite
simple: Traderson Wall Street are warned, “Don’t confuse brains and a bullmar-
ket” (Odean, 1996).This compactphrase prompts individual traders toconsider
the base rate of success inthe market, and it makes it moredifficult for them to
indulge in self-servingexplanations for their success.

At FloridaPower andLight employeesdevelopeda new way to fightself-serv-
ing biases afteran incident thatprominentlyfeatured aJapaneseinspector for the
Deming Prizewho laterbecamea folk hero withinthe company(Walton, 1990,
p. 61). To impressthe inspector,FP&L managerstook him to visit a new facility
that had been constructed faster and moreeconomicallythan any facility in the
history of theindustry.However, the Deming inspector did notsimply accept the
results at face value and congratulate themon their “quality” projectmanagement;
instead, he asked a number ofquestionsto determinewhythey were so successful.
Themanagers’answerswere so inadequate that it soon becameclearthatthey did
not understandenough about their success torecreateit in the future. The inspec-
tor dismissed their “success” in hisJapanese-accentedEnglish—”you were
rucky.” Later on his phrase, complete withaccent,became a common repairfor
self-serving interpretationsof success.

The Deminginspectordeflated a self-serving biasby consideringalternative
hypotheses for success (e.g., luck rather thanskill or knowledge). Traditionally at
FP&L, managers were not questioned as long as they achievedgood results. After
this incident managerswere much more likely to be asked to explain their sue-

cesses.If they could notdo so, the verdictwould be delivered: “you wererucky”
(Walton, 1990,p. 61). The strategic use of the accent was designed to remindman-
agers about theearlier incident whereluck produceddramaticresults that were
unlikely to be repeated.

Individuals Focuson PeopleRather thanSituations

Individuals alsogeneratea shallow set of hypotheses because social settings
tend to highlight people as causes.In Westernculture individuals typically choose
to explain events in terms of people’s actions and traits rather than situationalfac-
tors (Gilbert & Malone, 1995; Ross, 1977; Ross & Nisbett, 1991). In a recent
study observers heardanother studentgive a pro-life or pro-choice speechon
abortion. Afterward, observers assumedspeakersheld attitudes consistent with
their speeches even though the speeches were derived fromscripts written by the
experimentersand even though the observers themselves told the speakers which
position to speak for (Gilbert & Jones,1986).Similarly, Deming (1982) describes
a company that used a varietyof flammableproducts in their production process.
After analyzing the dataon fires, Deming found that the fires were a stable and
predictableoutcomeof the production process.However, according to Deming,
the companypresidentfocusedhis attentionselsewhere.He “sent a letter toevery
one ofthe 10,500employeesof the company to plead with themtoset fewer fires”
(p. 325).

People’s actions are frequently more obvious than their situations.Therefore,
when individuals generate hypotheses about whyan event occurred, their first
hypothesis is likely to bethat someperson caused it(e.g., Ross & Nisbett,1991).
This tendency to focuson people rather than situations has beendocumentedby-so
manyinvestigatorsin so many situations that it has beencalledthefundamental
attributionerror(Ross,1977; for recent reviews see Ross & Nisbett, 1991; Gilbert
& Malone, 1995).

Organizations might repair thefundamentalattributionerrorby remindingiridi-
viduals toconsidercauses other than people,especiallythe peoplewho are likely
to be closest to anyproblem: front-line workers.For example,an old military
adage says, “Thereare no such things as bad troops, only badofficers” (Cohen &
Gooch, 1990, p. 228). Parallel repairs are found in total quality management
(TQM). Ishikawa says, “whenever mistakes occur, two-thirdsto four-fifths of
responsibility rests with management” (Ishikawa,1985,p. ix). Such maximsmay
partially repair thefundamentalattribution errorbecausethey encourageindivid-
uals tolookbeyondthe front line. On the otherhand,they may simply focus the
erroron people at ahigherlevel. Thus, abetter repairmay be one from Deming,
who tells managers thatof the problems he has seen, “94% belong to the system”
(Deming, 1982, p. 315). Ishikawa and Deming both use vivid statistics toover-
comethe fundamentalattribution error even though it isunlikely that either has

8 CHIP HEATH, RICHARD P. LARRICK, and JOSHUAKLAYMAN CognitiveRepairs 9

conducteda preciseempirical analysis.Deming’s “94%” is particularlynotewor-
thy because ofits apparent precision.

Individuals StopSearchingasSoon as TheyGenerateOne Hypothesis

Self-serving biases andthe fundamentalattribution error are specialcasesof a
much broader tendency:Individuals tend to stopsearchingfor a causeas soonas
they locate a plausible candidate hypothesis(Gregory, Cialdini, & Carpenter,
1982;Hoch, 1984).

To counterthis generaltendency,organizations have developedsome repairs
that arewidely applicableacrossa numberof domains.In onetechniqueknown as
the “Five Whys,” workersat Toyota learned to ask “why?”five times beforethey
stoppedgenerating hypotheses. Whenthey did so, they were more likely tofind a
root cause ratherthan a superficialone.Imai (1986) illustrates thetechniquewith
the following example:

Question1:
Answer I:
Question2:
Answer2:
Question3:
Answer3:
Question4:
Answer4:
Question5:
AnswerS:

Why did the machine stop?
Because the fuseblew dueto an overload.
Why wastherean overload?
Becausethe bearinglubrication was inadequate.
Why wasthe lubricationinadequate?
Becausethe lubrication pumpwas not functioningright.
Why wasn’tthe lubricating pumpworking right’?
Becausethe pump axle woreout.
Why wasit worn out?
Because sludgegot in.

Imai arguesthat by asking “why” five times,workers identified “the real cause
and therefore the realsolution: attaching astrainerto the lubricating pump. If
workers had notgone through such repetitive questions,they might havesettled
with an intermediatecountermeasure,such asreplacingthe fuse” (Imai, 1986, p.
50). Another illustrationof the Five Whys deals directly with thefundamental
attribution error: “Problem: He doesn’t manage well. (1)Why? He’s not on the
floor. (2)Why?He’s in the equipment room. (3) Why? The newest equipment isn’t
working. (4) Why? Purchasing gave the supplier a short leadtime. (5) Why?Poor
planning system”(Forum, 1992, p. 54). In general, when individuals ask“why”
the first time, they are likely to developanswersthat invoke somesalient, recent,
or proximal event (e.g.,someperson’sactions). Subsequentwhys are likely to
cause individuals to think more broadly and situationally.

Although the Five Whys is an admirable cognitive repair becauseof its power
and simplicity,individuals may find it difficult to executeby themselves.When
individuals have one good hypothesis in mind,that hypothesisoften blocks their
ability to seealternatives(Gregory,Cialdini, & Carpenter,1982; Gnepp & Klay-
man,1992; Mynatt, Doherty,& Dragan, 1993).For example, Hoch (1984) found

that subjectswho generatedpro reasons for buying aproducthad more difficulty
generating conreasonsimmediately afterward.

If individuals find itdifficult to generate alternate hypotheseson their own,then
organizationsmay repairshallow searchby confronting individuals with others
who are expertin asking questionsthat reveal deep causes. AtMicrosoft, Bill
Gateshas by personal example, encouraged aculturethat relieson relentlessques-
tioning. Says oneWindows manager, “yougo into the meetings andyou come out
just sweating because, if there is anyflaw, he will land on it immediately and pick
itto bits” (Cusumano &Selby, 1995,p. 25). Employees “overuse” terms borrowed
from Gates,like “drill down” asa euphemism for “going into more detail” (“What
Bill GatesReally Wants,” 1995).

A similar cognitive repair isfound in theorganizationthat administersthe Dem-
ing quality prize. Here, official Deming inspectorsexaminemanagers using a
techniquecalled“single-caseborequestions.”They begin with broad exploratory
queries andthen relentlessly delve down intoweaknessesand omissions inthe
answersthey receive. Single-case borequestionssometimes identifycausesthat
arequite deep. For example,FloridaPower andLight often had to deal with power
outagesthat occurredwhen a treefell on a powerline and severedit. To improve
the reliability of its service,FP&L organized a unit to trim all the trees in sites
where damage had occurred, and thus prevent future outages. Managers at FP&L
congratulated themselves for creating aprocedurethat preventedfutureproblems.
However, the Deming inspectors were not satisfied with theproceduresince~itpre-
vented problems only in areas that had already experienced acrisis.Theysearched
for a solution at adeeperlevel, and askedmanagersa numberof questionsabout
what might beconsideredforestry! What kind of trees grow in theregion?Do
palms grow faster or slowerthan oaks? Managersat FP&L realized they did not
know the answers to these questions, and thatthey had not searcheddeeply
enough tosolve their problems. After their experience with single-caseboreques-
tions, FP&L managers consulted with foresters anddevelopeda regularmainte-
nanceprocedureto trim trees basedon their growth ratesand acrossthe entire
region, notjust in areas wheretreeshad previously severed lines Afterparticipat-
ing in sessionsof this kind with the Deminginspectors,managersat the firm
learned to ask single-case borequestionsin their own internal discussions, thus
institutionalizingthis cognitive repair(Walton, 1990,pp. 57-63).

Individuals GenerateHypothesesthat areNarrow RatherThanBroad

In an ideal world individual learnerswould not only generatedeeperhypothe-
ses;they would alsoconsidera broad ratherthan narrow set of potentialhypothe-
ses. However, even when individuals generate alternativehypotheses,their
“alternatives” often differonly slightly from one another,and all lie within the
same general frame.For example,participants in oneexperimentwere asked to
considerthe serious parking problem facedby their university, and they were

10 CHIP HEATH, RICHARD P. LARRICK, and JOSHUAKLAYMAN
Cognitive Repairs 11

given time togenerateas manysolutionsasthey could (Gettys etal., 1987).Com-
bined,participants generated about300 solutionsthat researcherswere later able
to classify into about seven majorcategories.One category, for example,sug-
gested ways to reduce demand for parking (e.g.,by increasing parking fees) and
anothersuggested waystouse parking moreefficiently (e.g.,by segregatingpark-
ing slots according tosize). The averageparticipantproposed about11 solutions
but these11 solutionsrepresented only about three of the seven possiblecatego-
ries. The authors askedan independentpanel ofexpertsto compile a completelist
of high-qualitysolutions, andthey used thiscompletelist to assesshow many
solutionswere missedby eachindividual. The typical participant missed from70
to 80 percent of the high-quality solutions.However, when asked, individuals
believedthey had missed only25 percent.

Evenexpertsfail toconsiderahrQadrangeof alternative hypotheses.Forexam-
ple, one group ofresearchersshowed professional automechanicsa “fault tree”
that listed a number of hypotheses aboutwhy a car might notstart (e.g., battery,
starting system,fuel system,ignition). Some mechanics were presented with a
“full tree” that containedsevenspecific hypotheses,otherswere givena ‘~pruned
tree” that omittedsome important hypotheses (e.g. the ignition system). The
resultsindicatedthat when hypotheses were prunedoff the tree, mechanics did not
adequatelyconsiderthem (Fischhoff,Slovic, & Lichtenstein,1978).

How might organizationsrepair narrow search by individuals?Individuals
might search more broadly ifthey arecuedto think about a problem from different
perspectives.At Sharp, employees are told to be “dragonfliesbut not flatfish.”
Dragonflieshavecompound eyes and see things from multiple perspectives at
once, butflatfish havelargeeyes that onlylookin one direction (Nonaka &Takeu-
chi, 1995).

The “dragonfly” repair exhorts individuals toconsiderdifferent perspectives,
but thismay be difficult for individuals to do by themselves. Organizations might
repairnarrow search more effectivelyby encouraging individualsto- recruit others
who havedifferent perspectives. A good example of this is providedby Bridge-
stone Tire,which conducts “kokai watches” togeneratealternative hypotheses for
improving workpractices.During akokai watch a group ofup to a dozenpeople,
from different areas of afactory, gather for a few hours to watch otherswork. In
one four-hour watch adozenpeople identified63 potential dangers with a new
machine(Walton, 1990,pp. 200-201).

The kokai watchhas a numberof features that ensure that watchersgeneratea
broadarray of hypotheses. First, it mandates alargenumber of watchers (up to
12). Second, it selects watchers from a variety of different areas—in one kokai
watchthat examineddie and material changes, watchers included a plantvisitor,
a memberof the humanresourcesstaff, a chemist, and a project manager. “The
idea was that people could observe a process, even thosewho were strangers toit,
with fresheyes,seeing thingsthat closelyinvolved workersmight not” (Walton,
1990, p. 200). Third, it ensures that watchers do notdiscardhypothesesprema-

turely. The watchersare instructed to “write down anything,‘Hey, looks like the
guy is walking toomuch,’ or ‘Looks like he’snot handlingthe knife right”’ (Wal-
ton, 1990,p. 201). Only after watchers generate hypothesesindependentlyarethe
results combined andfiltered.

Otherorganizational procedures also repairnarrow individual searchby ensur-
ing that individuals generate hypothesesindependently.For example, when
Motorola formscross-functionalteams toevaluate new products,they do not
allow employees who have participated in one product team to participate in
anotherteam with a similarproduct. This prohibition limitsthe pool of potential
team members in acostlyway; evaluation teamsinvolve six to nine people and
spendtwo to three months to develop a business plan for the newproduct. How-
ever, by consciously disregardingprevious experience,Motorola allows new
teams to develop recommendations independently from previous teams. At the
same time, Motorolaavoids losing the knowledgeof previous “veterans”—they
serve asa “review team” that evaluatesthe recommendationsof the newestteam.2

Other repairsensure that a broad range ofalternativesare consideredsimulta-
neously.Somecompaniesdivide aproductdevelopmentteam into competingsub-
groups which develop separate projectproposals,and only later recombine to
debatethe advantages and disadvantages of theindependent proposals.Again, this
strategy is costly because it is redundant.However, it may have advantages
becausethe built-in independenceensuresthat different subgroups will approach
a problemfrom different perspectives (Nonaka & Takeuchi,1995,p. 14).

CollectingInformation

In the second stage of the learning process ideal learnerscollectinformation to
test and revise their hypotheses. Therearetwo main sources of such information:
the information that individuals already have in their memory and theinformation
that they collect from theenvironment.Both kinds of information have potential
flaws, but individuals mightminimizethese flawsif they collectedinformationin
a rigorousway. However, learnersdo not always actas thoughthey are awareof
the potential flaws in their information—they frequentlycollect only a small,
biasedsample.

Individuals OftenCollectSmall Samplesof Information

Individuals often collect only a limited sample of informationbecausethey are
constrainedby time or attention.In a classicstudy, Payne (1976) asked hissub-
jectsto chooseone apartmentout of a number of alternatives, each of which was
describedon severaldifferent dimensions(e.g., rent, cleanliness, landlordquality,
noiselevel). Whensubjects chose among only two apartments, theytendedto con-
sider all of the information beforethey decided.However, individuals searcheda
smaller and smaller percentage of information as more informationbecameavail-

12 CHIP HEATH, RICHARD P. LARRICK, andJOSHUAKLAYMAN Cognitive Repairs 13

able. For example, one subject, whowas decidingamong 12 apartmentscharac-
terized on eight different dimensions,looked at only about 25 percent of the
information beforemakinga final choice.

It would be reasonable forindividual learners to collect only a smallsampleof

information ifthey performed acost/benefitanalysis and decidedthat collecting a
large sample was too costly. However, there is evidence that individuals collect
only a smallsampleof information becausethey systematically underestimate the
benefits of larger samples. Tversky andKahneman(1971) arguethat individuals
typically believe that small sampleswill be quite similar to the population from
which theyare drawn. They labeled thisbeliefthe “lawof small numbers” tohigh-
light that it contradictsthe statistical “law of large numbers,” whicharguesthat
samplescan yield an accurate picture of a population whenthey are sufficiently
large. When individuals believe in the law of small numbers,they assumethat any
samplewill be sufficient,no matterhow small.

At the extreme,individuals may not collect any information from the external
environment becausethey believe that they already have adequate information
stored in theirhead.Organizations may overcome thistendencyby encouraging or
requiring individuals tocollectlarger samples. This kind of repair ispervasivein
writings on TQM. “In promoting statistical qualitycontrol,we have used theslo-
gan, ‘Let ustalk with data”’ (Ishikawa,1985,p. 200). At manyTQM companies
one of the main principles of the quality effort is “Managementby Fact” (Walton,
1990,p. 37).

And TQM not only talks aboutdata, it provides individuals with toolsthat help
themcollectand analyzedata.For example, six of the “SevenTools” of TQM pro-
vide ways tocollectdata (e.g., checksheets) or to simplify and display largequan-
tities of data(e.g., histograms, scatter plots, Pareto diagrams, control charts)
(Deming, 1982;Imai, 1986;Ishikawa,1982, 1985;Juran,1992).

Individuals Collect BiasedSamplesof Information

Individual learners not only collect smallsamplesof information,they alsotend
to collect samplesthat are biased (i.e., that are unrepresentativeof the larger
world). Consider the commonclaims that “the otherline alwaysmovesfaster” or
“it only rains after I wash mycar.” Unless we want to believethat a malevolent

spirit is in charge of such harassment, these examplesdemonstratethat ourmem-
oriesdo not store a randomsampleof all waiting times orall rainstorms—weare
more likely torememberthe rainstorms thatspoil the finishon-ourfreshlywashed

car. Even whenindividualscollectinformation from theoutsidework1(rath~vtha~i
from memory),they do not always attend tothe most relevant andimportantinfor-
mation.Below, we discuss anumberof factorsthat might lead individual learners
to collect biased samples.

Individuals Only Consider AvailableInformation

As indicatedby the car wash example, individuals often collect biased samples
because they collect information that is easilyavailable in memory, for example,
because it isespeciallyvivid or recent. The problemis that individuals typically
assumethat the information that is availableis also most frequent, probable, and
causally important(Tversky & Kahneman, 1973). This assumption is often
wrong. Individuals dramaticallyoverestimatethe likelihood of vivid causes of
deathlike accidents or homicides, and they underestimate thelikelihood of less
vivid causeslike disease or strokes.Individuals estimate that accidents causedas
many deaths as diseases andthat homicides wereas commonas strokes. Infact,
diseasestake 16 times more livesthan accidents and strokes take11 times more
lives than homicides. Individuals also overweight recentinformation. They
assume that the most recent flood provides an upper boundon possibleflood dam-
age, and the purchase of earthquake insurance “increases sharply after aquakeand
then decreases steadilyas memories fade”(Slovic, Fischhoff, & Lichtenstein,
1982,p. 465).

Many organizationsrepair individuals’ tendencyto rely on biased,available
informationby instituting a process that collects information more systematically.
At a Motorola division that develops equipment forcellular phone systems, one
group realizedthat an availability bias was causing it to overlook certaincustom-
ers when it evaluated new products. The unit assigned account managers only to
large accounts, so when managers evaluated newproducts,they primarily consid-
ered the needs and requirements of only largecustomers.However, the unit also
serveda numberof smaller customersthat did not have theirown accountman-
ager. Together,these small customersaccountedfor a largepercentageof reve-
nues. Motorola overcame theavailability bias by developing a Feature
PrioritizationProcess; theysurveyedcustomersup to four times a year andthen
weightedall of the inputsbasedon customervolume andpriority.3

Hospitals also have a variety ofproceduresto force individuals tocollectinfor-
mation more systematically. Trauma physicians are often confrontedby vivid but
potentially misleading information. One doctor states that, contrary to what one
might expect, stabbings and bullet woundsare “relatively straightforward affairs”
becausethey leave“clear trackson the body.” Other injuries are more difficult to
treat becausethey leave no visible cues. “It would be all too human to focuson a
laceratedscalp—agory but basically insignificantinjury—and miss a fractured
thighbone that hadinvisibly severedamajorartery” (Rosenthal,1994,p. 48). The
medical profession has developed a series of strict protocolsfor traumasituations
that allow doctors toquickly collect all the relevant information, not just that
which is salient. For example, when a patient firstentersthe emergencyroom,
physiciansfollow the “ABCs”; they establishairway, then breathing, thencircu-
lation.4 For situationsthat are more critical, such ascardiacemergencies,proto-
cols are even more rigorous and specific.

14 CHIP HEATH, RICHARD P. LARRICK, and JOSHUA KLAYMAN 15
Cognitive Repairs

If individuals tend to focuson information that is highlyavailable,it is notter-
ribly surprising that they are frequently unaware of missing information. Even
when information is present, learnersdo not pay as much attention to whatdoesn’t
happenas what does (Agostinelli, Sherman,Fazio, & Hearst, 1986; Newman,
Wolff, & Hearst,1980).

Certain professions andorganizationshave learned to repair the tendency to
ignore missing information. Homicide detectives learn to notice the absenceof
itemsat murderscenes, sincemany murderers takebacksomething that belongs to
them after committingthe crime.“You look atwhat’s beentaken andyou find out
who it belongedto originally” (Fletcher,1990,p. 75).

A particularly importantform of missing information is the absence ofexperi-
ence with highly unusualevents.Bank examiners rarely see a bank fail,nuclear
techniciansrarely see a meltdown, airline personnel rarelywitnessa crash(March,
Sproull, & Tamuz, 1991; Perrow, 1984). Certain organizations institutionalize
proceduresthat encourage individuals to pay attention to suchinformation:desjtito
the factthat such eventsare unlikely to beavailablein their own experience. For
example,at the Federal Reserve Bank, which certifies the security of banks, senior
bank examinersdeliberately recountstoriesof failed banks to keepjunior exam-
iners aware thatthey should bevigilant.5 At one bank’s commerciallending
department, senior creditofficers would hold seminars and informal brown-bag
lunches todiscusspast lending mistakes, particularly in areas characterizedby
unusual or rare events(e.g., “problems with highly leveraged companies, real

6
estate, environmentalliability on contaminatedproperty”). By forcing individu-
als to rehearse suchinformation,organizations help individuals learn fromvicar-
ious experiencesthat are rare but highlyinformative. Furthermore,organizations
remind individuals of potentially painful information that self-serving biases
would make thempreferto ignore.

IndividualsCollect BiasedInformation Basedon Their PreexistingTheories

Researchsuggests that individuals tend to think of “facts, experiences, and
arguments thatsupporta current hypothesis more readilythan those that refuteit”
(Klayman, 1995; see also, Baron,1988; Kunda, 1990; Nisbett & Ross, 1980).
Thus, whenindividuals collect information from memory,they may focus on
information that supports their preexistingtheories.Individuals may also do this
whenthey collectinformation from the externalenvironment.For example, when
individuals collect information fromothers,they often askspecific, directive
questions thatare likely to elicit the answer they expect (Hodgins& Zuckerman,
1993; Zuckerman,Knee,Hodgins, & Miyake, 1995).

The Chicago Board of Trade has astaffof in-houseinvestigatorswhoscrutinize
tradesthat may violate exchange rules. Intheseinvestigations,which areobvi-
ously quite sensitive,it is veryimportantthat investigatorsdo not collectinforma-
tion that is biasedby their initial theories.To repair thistendency,the investigators

are trainedto avoid questionsthat canbe answered with a yes or no response.
“This forcesan investigator to askopen-endedquestionsand allows her to draw
out as much information aboutthe situation as possible.” Byasking open-ended
questions,investigatorsavoid the possibility of directing the interview in a way
thatelicits only information that is consistentwith their preexistingtheories.7

Someorganizationshave developed maxims that seem designed to encourage
individuals tocollectdataratherthan relyingon their (potentially biased) theories.
At BridgestoneTire employees use two Japaneseterms: genbutsu(actual product)
and genba(actual place)(Walton, 1990,p. 194). Theseterms remind employees
not to rely on their own theories, but to actuallygo out andinvestigatethe actual
product in the actualplace where the problemsarose. Another group (Forum,
1992)uses a similar cognitive repair they call thethreeactualrule: (I) Go to the
actual place; (2)Seethe actual problem;(3) Talk to the actual peopleinvolved.

Individuals ConsiderOnly Part of the RelevantInformation

Finally, individual learners maycollectbiased samples because they tendto col-
lect information fromonly one small corner of the universe of information. This
arises from basic cognitive processes. Memory isassociative—whenindividuals
retrieve onepiece of information, theytend to think of other information thatis
linked to it by strong associations,commonfeatures,or similar meaning. Even
when individualscollect information from the external environment,they are
likely to collect informationbasedon the samekind of associative process.
Researchin cognitive psychology hasshownthat individuals attend to andprocess
information more comprehensively when they have a mentalschemathat tells
them what information is needed in a given situation and where to findit (Ander-
son, 1995).

Accordingly,organizationscan repair biased informationcollectionby provid-
ing individuals with aschemathat reminds them of the full range of relevantinfor-
mation.Many schemas of this kindcanbe found in thefinancialservicesindustry,
where individuals must assess a wide varietyof information todeterminewhether
to buy, sell, or lend.At the FederalReserveBank of NewYork, the BankExami-
nationsgroup protects the FDIC insurance fund by ensuring that individual banks
are in soundfinancialcondition.Whenreviewing eachbank,examiners use arat-
ing system knownas CAMEL: they review Capital adequacy,Assetquality, Man-
agement, Earnings, andLiquidity.8 In another bank’s commercial loan
department,creditanalysts use the “Five Cs of Credit”:Collateral,Capacity,Cap-
ital, Conditions, andCharacter.9

Organizational schemaslike CAMEL and the Five Cs are likely to encourage
individuals tocollect a broaderrangeof information than they wouldnormally
collect.It wouldbe very easyfor individual learnersto collectinformationon only
on themost salientfactors(such as cashflow in a loan decision).Although cash
flow is certainlyimportant, it canalsobe misleadingor unreliable,particularly in

16 CHIPHEATH, RICHARD P. LARRICK, andJOSHUAKLAYMAN Cognitive Repairs 17

an environment where conditions are changing. By emphasizing the FiveCs, a
bank canrepair the tendencyof individual analysts to neglect informationabout
importantvariablesthat are less obvious orare harder to assess. For example, the
Five Cs reminds loanofficers to considercharacter—Whatare the management
skills of the owners? Dothey have good personalcredit records?Although the
answersto such questionsare quite important, individualanalystsmight forget to
ask them in a numbers-oriented environmentlike a bank, without acognitive
repairlike the FiveCs.

Individuals WhoCollect BiasedInformation Fail to Correctfor Bias

We have discussed a number of factorsthat might lead individual learners to
collect biased information.However, even if learnerscollectbiased information,
they might still be able to draw effective conclusion as long as theyrecognizedthe
bias and corrected forit. Forexample,supposean individual made the statement,
“the otherline alwaysmovesfaster,”but thenremindedherself that suchsituations
might be overlyavailablein her memory.This kind of correction improves the
conclusions drawn from even a biased sample.On the other hand, even ifindivid-
uals are aware thatthey have collected biased information,they may not know
how to correctfor biases after thefact. For example, after individuals ask biased
questions and therefore receive biased answers,they do not take into accounthow
much theanswerswere biasedby their initial questions (Zuckerman, Knee, Hod-
gins, & Miyake, 1995).

Because individualsdo not always correcttheir information forbiases,some
organizationsattempt to ensure that individualscollectunbiased samplesfrom-the
start. Microsoft requires itssoftware developersto use the sameprogramsand
machinesthat areusedby their customers.For example, programmers whowere
developingthe new Windows NT operating system ran the current day’sversion

of the programasthey programmed the next day’s version. AtMicrosoftthis pro-
cess isknown as “eating yourown dog food.” It ensuresthat developers collect a
large, unbiasedsample of information about the current state of the program.If
Windows NT crashed while a developer wasdesigninga new printerdriver, he
had tofix the problem with NT before he could return to his driver (Cusumano &
Selby, 1995,p. 331). Microsoft also requiresdevelopersto use the same machine

usedby customers,a requirement thathas been “controversial at times” because
developers like to have thefastest, coolest machineson their desks. However,
when developershavebettertechnology than theaveragecustomerthey collect
biased information abouthow well their software programsperform.One man-
ager said,“every time I’ve had a project where thedevelopershad hardware that
was a generation beyond what customers had, the [software] always hadperfor-
mance problems” (Cusumano &Selby, 1995,p. 347). Byrequiringdevelopersto
use the same machines as their customers,Microsoft forces them to collectan

unbiased sample of information about theoperatingspeed and memory demands
of the softwarethey are developing.

DrawingConclusions

After generating hypotheses and collectinginformation,ideal learners should
evaluatethe information they have collected and drawconclusionsthat areappro-
priate andcautious.Researchers have suggestedthreemain classesof problems
that real individuals facewhenthey interpretevidence. First, they often weigh
information in a way that is not statisticallyappropriate—forexample, they
emphasizethe importanceof extreme evidence butthey do not emphasize therel-
ative amount of extreme versus non-extreme evidence. A second, even more
insidious problem is that individuals use their initial theoriesto interpretthe evi-
dence. While individuals may readily accept information that is consistentwith
their initial hypothesis,they cast a critical eyeon information thatcontradictsit.
Third, as a resultof the two previous processes andothers,individuals frequently
drawconclusionsthat are overconfident and overly optimistic.

Individuals WeighVivid and Extreme EvidenceMore Heavily

Once individuals have collected information,how shouldthey combine it and
weigh it? An ideal learnerwould weigh informationbasedon the quality of the
information.However,actual learnersdo not alwaysassignappropriateweights to
all aspectsof the decision.For example, they tendto weigh more vivid, easily
imagined information more heavily(Keller & McGill, 1994).They also focuson
the extremity or strength of theavailableinformation (e.g., the warmth of arec-
ommendation letter) without adequately attending tothe amount or weightof the
evidence(e.g., the writer’s amountof contactwith the recommendee)(Griffin &
Tversky, 1992).

If individuals tend tooveremphasizevivid or extremeinformation, organiza-
tions might prevent thisby requiring individuals toconsciouslyclassifyinforma-
tion according to its appropriate weight. Manycompanieshave internal audit
groupsthat examine the records of company divisions to ensure that they areusing
proper accounting proceduresand spending moneyon legitimate expenses.An
audit usually uncoversa variety of major andminor “exceptions”(i.e., situations
where correctprocedureswere not followed). One auditor saysthat auditors must
be careful not to “place too much emphasison memorableerrors, e.g., an error in
the president’s expensereportor the misuseof the companycar.” One auditing
group repaired this temptationby first classifying eachexceptionas major or

in
minor then consciously ignoring theminor issues.

Consistent with the tendency tooverweightthe extremityof information and
ignore the amount, individuals frequentlyplace higher weighton one vivid case
than on a much larger sample of information. Joseph Stalin is reported to have

18 CHIP HEATH, RICHARD P. LARRICK, andJOSHUA KLAYMAN CognitiveRepairs 19

said, “The death of a single Russian soldier isa tragedy.A million deathsis a sta-
tistic” (Nisbett & Ross,1980,p. 43). In a studythat supports this observation,Bor-
gida andNisbett (1977) showedsome students a statistical summary ofhow
dozensof students hadratedvariouscoursesin the previous term. Other students
attended a panel discussion, during whichtwo or three upper-division students
rated eachcourseon a numerical scale and providedsomegeneric, uninformative
comments. Despite the factthat the statistical summary provided students with a
larger amountof information, individuals who heard the smallsample of vivid
information were more likely to change thecoursesthey selected.

Microsoft also discoveredthat individualsdiscountlargesamplesof statistical
information.At one point, Microsoft started surveying users to seehow many of
them found it easy tousea particularfeature. Softwaredevelopers often refused to
believe the statistics.“The usability groupwould tell the development group‘Six
out often couldn’t dothis.’ And thedeveloper’sreactionwould be, ‘Where’d you
find six dumbpeople?”’(Cusumano &Selby, 1995,p. 379).In order to repair this
tendencyto ignore base rateinformation,Microsoft made the information more
vivid. It built a “usability test lab” where developers can watch real usersstruggle
with new productsfrom behind a one-waymirror. Instead of presentingdevelop-
ers with pallid statistics, the test lab presents them with real people(albeita mxtch
smallersample).The lab managersaysthat when developers see auser, “twenty
ideasjust immediately come tomind. First of all, you immediately empathize with
the person. Theusual nonsense answer‘Well, they canjust look in the manual if
they don’t know how to useit,’ or ‘My idea is brilliant;you just found ten stupid
people’…that kind of stuffjust goes out thedoor…” (Cusumano &Selby, 1995,p.
379). This cognitive repair is interesting because it uses onekind of bias (over-
weighting of extreme, orvivid information) to fight another (underweighting of
statisticalinformation).

IndividualsUse Their PreexistingTheoriesto Interpret the Evidence

Individuals not only weigh information inappropriately,they also havediffi-
culty interpreting information independently of their preexistingtheories.Instead
of using the information to test theirtheories,they use their theories to test their
information. This often leads them todiscountinformation that disagrees with
their preexistingbeliefs.

In a classic demonstration of such discounting, Lord, Ross, and Lepper (1979)
selectedundergraduateswho strongly supported or opposed capital punishment
and presented them withtwo purportedacademic studiesthat evaluated capital
punishment’seffectivenessusing very different methods. A study using one
methodfound that capital punishmentwas effective and a study using the other
methodfound it was ineffective (the researcherscounterbalancedwhich method
was associated with whichresult). Participantsapplaudedthe positive aspects of
whichevermethodsupported theirown preexistingtheory, andthey critiqued the

“design flaws” in the other. In fact, after receivingmixed results fromthe two
studies, subjectsbecamemoreconvincedof the validity of their original position.
Seemingly, they regardedthe evidence as “one good studythat supports my
beliefs, and one lousy studythat draws the wrong conclusions.”Individual sub-
jects thus failed to evaluate the incoming information separately from theirpreex-
isting theories. Unfortunately, similar results have been notedwith professional
scientists(Mahoney,1976;Koehler, 1993).

One bank helped itsloan officers repair the waythey interpret evidenceby
encouragingthem toconsidera nonstandardtheory of lending.In mortgagelend-
ing, loan officers oftenlookfor reasonsto deny loans because loans are difficult to
make (they are subject to a mountain ofregulations)and potentially quite costly
(e.g.,foreclosureon a badloan maycostup to 20% of the property value). Thus,
the initial hypothesis in manyloan decisions isthat an applicant should be denied
a loan unless proven otherwise. One mortgageloan departmentgrew at an annual
rate of30 percentby forcing loan officers toconsideran alternative to the standard
hypothesis.Instead of asking whetheran applicant should bedenieda mortgage
loan, it asked whether theapplicantshould beapproved.This reversalled the
departmentto develop specialprogramsfor qualified applicantswho had low
incomes or other specialcircumstances.11

Individuals use their theoriestodevelopexpectationsabout what is normal, and
they frequently label unexpected events as “problems” or “failures.” These labels
may be misleading,however, particularly in research anddevelopmentwhere
unexpected events may point the way to importantbreakthroughs.One research
organization has developed a repair that discourages individuals from thinking
that unexpected events are failures(Sapolsky,1997).JacksonLaboratoriesbreeds
mice that exhibit physiologicalor behavioraltraits that are of interest to medical
researchers.For example,it sells mice that lack growthhormoneto researchers
who are interested inunderstandingthe biology of mammalian growth. It found
that the animal technicians (e.g.,the peoplewho cleanedthe cages) often noticed
unusual behaviorthat was scientifically important. The mice that lacked growth
hormone were discoveredby a technician who noticed aparticularmouse that
didn’t grow at a normal rate. Another technician noticed a mousethat didn’t
respond normally to the loudnoisesthatoccurredwhen the cages werecleaned–
its offspring were found to be susceptible tohereditarydeafness.After several
experienceslike this where unexpectedbehaviorproducedimportantdiscoveries,
the company started holding regular meetings with animaltechniciansto inquire
whetherthey have spottedanything unusual.Theseforums for highlighting the
importanceof unexpected eventsare called“deviant searches.”

CRSS,an architecturalfirm, developeda special position to repair the problem
of theory-based interpretationof evidence.“Most designerslove to draw,to make
‘thumbnail sketches’,”says one manager, but this rush todraw conclusionsis
often premature.CRSScreateda uniquejob description,the “programmer,” to
ensurethat somemembers of its design teams were not allowing theirown theo-

20 CHIP HEATH, RICHARD P. LARRICK, andJOSHUAKLAYMAN Cognitive Repairs 21

ries to dominate the waythey evaluated information from clients.Programmers
arenot in charge ofdesigningor problem solving,they are in chargeof “problem
seeking.” They are trained touse techniques that help them to resist premature
conclusions,and thus listen more carefully to clients. “The experienced, creative
I programmerj withholdsjudgment,resists pre-conceived solutions andthe pres-
sure to synthesize…he refuses to make sketches until heknows the client’sprob-
lem” (Peters,1992,p. 402).

Often, organizationsensure that individuals weigh information effectivelyby
forcing them to interact with others who might weighthe informationdifferently.
One researcher hasexploredwhether training as a scientist cures the problemsthat
other individuals have in evaluatingevidence(Dunbar,1995).The answer isno.
For example,scientists,especiallyyoung ones, often believethat a singleexperi-
mental result hasjust resolvedan important problem. However, when Dunbar
studied aset of microbiologylabsthat had been particularly successful, he found
that they placed more emphasison group lab meetings. At thesemeetingsanindi-
vidual scientist presented his or herresultsto a varietyof skeptical,uninvolved
peers.Whenthe individual scientist presented a striking new piece of evidence
(e.g., I have detectedEnzymeZ in a biological process where ithas never been
observedbefore), theindividual’s peers were typically quite willing to propose
alternate ways of interpreting the evidence (e.g.,the samplewas contaminated
with residual Enzyme Z from a prior procedure). In successful labs, even when
individual scientists failed to weigh a particular piece of evidence appropriately,
their peersdid so for them. Moreover, the most successful labs werethose that
includedmemberswith different training and backgrounds.Such “lab meetings”
are not limited to successfulmolecularbiology labs; similar meetings takeplaceat
venture capitalfirms wherefirms decide whether to allocate money tonew ven-
tures (Kaplan,1995).

IndividualsDraw Conclusions thatare Overconfidentand Overly Optimistic

Imagine that individuals havegenerateda set of hypotheses, collectedsome
new information, and interpreted the relevanceof the information for the initial
hypotheses. How much confidence should they place in the conclusionsthey have
drawn?If individual learners were adequately cautious, their conclusionswould
reflect the degree ofuncertaintyin the dataon which they are based.Over the
years, researchhas documentedthat individuals often express morecertaintyin
their conclusionsthan is warrantedby the factsavailable to them (or by their
actual performance). This kind of problem has beendocumentedextensively in
laboratorystudies,but also infield studies of individualjudgmentin a variety of
professions,like the civil engineers in theintroduction(Griffin & Tversky, 1992;
Lichtenstein,Fischhoff, &Phillips, 1982; Russo & Schoemaker,1992).

Individualsoften exhibit aparticularkind of overconfidencethat we mightlabel
a planningfallacy (Buehler, Griffin,& Ross,1994)or anoptimismbias. Thisopti-

mism bias ispervasivein work environments. Software developers at Microsoft
often experience burnout becausethey “grossly underestimate”how long it will
takethem to accomplish certaintasks (Cusumano &Selby, 1995, p. 94). Organi-
zationsdo not always successfully overcome this individual bias. A study ofpio-
neerprocessplantsrevealed that thetypical plant experienced actual construction
costs that were almostdouble the originalprojections;similarly, a study ofstart-
ups showed that morethan 80 percentfell short of their projected marketshare
(Davis, 1985). Theseexamples suggest that individualsdraw conclusionsthat
underestimate the amount of uncertainty and error in their predictions, butthey
tend todo it asymmetrically—theyrarely overestimate a project’s cost or time to
completion.

Of course, individuals maydisplayan optimismbiasbecausethey confrontmis-
alignedincentives.Perhaps if engineerscorrectlyestimated the true cost of a new
energy plant, decision makers might choose not to buildit. However, the real
causesof theoptimismbias seem to be cognitive, since individuals areoverconfi-
dent by the same magnitude even in labexperimentsthat rewardaccuracy.For
example,individuals typically assumethat their predictions are more precisethan
they are.Whenthey are askedtoset confidenceintervals around a quantity, so that
theirconfidenceinterval has a98 percentchanceof including the true answer,they
are typically surprisedby the true answer not2 percent of the time, but20 to 50
percent (Lichtenstein, Fischhoff, &Phillips, 1982; Russo & Schoemaker,1992).

How might organizationsrepairindividual tendencies towardoptimism bias
and overconfidence? One strategy is to allow individuals to makeoverconfident
predictions, thenadjustthem overtly. This was the strategypursuedby the engi-
neering profession withits safety factors. Microsoft uses asimilar strategy tocor-
rect the overly optimistic projections of individual softwaredevelopers:It has
rules about the amountof buffertime that should beaddedto projects.For reason-
ably well-understoodprogrammingchallenges,suchas applicationsprogramslike
Excel andWord, Microsoft typically adds buffer timethat constitutes30 percent
of the schedule.However,for operatingsystemslike Windows, where developers
mustcreatea system thathas to mesh effectively with numerouspiecesof hard-
ware and software, Microsoft may add buffertime that reaches50 percent
(Cusumano &Selby, 1995).Similar repairs have evolved inotherindustries. At
one Big Six accounting firm, where teams mustprepareformal plans for acon-
sulting engagement,projectleadersdevelop their best estimatesof time, expense,

t2
andcontingencycosts, thenincreasethe finalnumberby 15 percent. This repair
has evolved despite the fact that this environmentprovidessome incentives to
underestimatecosts—bidsthat are too highmay not be accepted.

WhenMicrosoft adds buffer time to a schedule, it corrects the predictions of
overconfident individualsby overriding them.However, it has also developed
proceduresthat help individual developers decrease their initiallevel of overcon-
fidence. For example, the company has improved itsschedulesby requiringdevel-
opers tocreatea detailed work planthat specifies which tasksthey will perform

22 CHIP HEATH, RICHARD P. LARRICK, andJOSHUAKLAYMAN Cognitive Repairs

during specific windowsof time. Says one manager, “The classicexampleis you
aska developerhow long it will take him to do something andhe’ll say a month,
becausea month equalsan infinite amount of time.And you say, ‘Okay,a month
has22 working days init. What arethe 22 thingsyou’re going to do during those
22 days?’ And the guy will say, 0h, well, maybe it will taketwo months.’ Even
by breakingit down into22 taskshe realizes,‘Oh, it’s a lot harder than I thought”’

(Cusumano &Selby, 1995, p. 254).
Some organizations repair overconfidenceby forcing individuals to interact

with others whoare trained to question their conclusions. For example,the Penta-
gon for manyyears had what they calledthe “murder board,” a group ofexperi-
enced officersthat reviewed theplans for importantmissions, withthe goal of
killing the mission. According to Pentagonlore, the failed Iranian hostage rescue
during theCarteryears wasnotvettedby this boardbecausehigh governmentoffi-
cials were too concerned about securityleaks.13

Other organizationshave developednorms of frank feedback to ensure that
individuals question others’ conclusionshonestlyand openly.In its featureanima-
tion unit, Disney regularly holds “Gong Shows” where personnel(including
departmentsecretaries)canpitch ideas to a group ofseniorexecutives. Gong
Shows mayattract40 people whopresenttheir idea to theexecutivesand other
presentersfor three to five minutes. The seniorexecutivesare careful to give
exceptionallyfrank feedback atthe end of the session,highlightingbothgood and
bad aspectsof the presentations. “Somebody may have agreatconcept, but the
story may not be very good. [Wecan’tsay] ‘Oh, that’s fabulous. Great pitchguys!’
and whenthey leave, mumble,‘That was awful!’….We don’t pull our punches.
[Eventually] people begin tounderstandthat no matterhow good,bad,or indiffer-
ent theidea, it can be expressed,accepted, and thought about”(McGowan, 1996).

GENERAL DISCUSSION

In this paper we have reviewed the literatureon individual learningusing a simple
frameworkthat considersthreebroad stagesof the learning process.We argued
that ideal learnerswould generate a broad and deepset of hypotheses,test them
with a large, unbiasedset of information,and draw conclusions in a cautious and
balancedway. Thepsychologicalliteratureindicates, however,that realindividu-
als are not ideal learners; they think and act in ways that reduce their ability to
learneffectively.

Fortunately, individual learners do not have togo it alone.We have argued that
organizationsfrequently repair theshortcomingsof individual learners through
the use of sayings, informal routines, and formal procedures.We believe the
examples we have offered illustratethe tremendouspromise of organizational
sources of cognitive repairs.

23

Nevertheless,we do not thinkthat cognitive repairs will overcomeevery indi-
vidual problem. Cognitive repairs are heuristics—like themental processesthey
repair,they arepragmaticand often efficient, but alsoapproximateand inexact.
For example,they maysolve 75 percent of individualshortcomingswhile incur-
ring only one-thirdof the costs of optimalprocedures(e.g., fromeconomicsor sta-
tistics). However,they are unlikely to beperfect.

Consider thefive whys. It undoubtedlyprompts individuals to think more
deeply about causes, but it is only a roughheuristic.Why five questionsand not
three orseven?And which questions? “Problem: He doesn’t manage well.” (1)
Why? He doesn’t manage conflict well. (2)Why? He grewup in a dysfunctional
family. (3) Why?His parentswere alcoholics…” In this example, the answers took
an unhelpful detouraway from potential solutionssometimearound answer 2.

Even when repairs are reasonablyeffective, they may still leave roomfor fur-
therrepair.Consider,for example, the military’s partial repairfor thefundamental
attribution error: “There are no bad troops,only bad officers.”This adage may
repair tendencies to attribute blame tothe people whoareclosest to a problem (the
troops who areon the battlefield);however,it merely focuses attentionon another
group of people. Thus, itmay prevent individuals from fixing systems orproce-
duresthat have basic flaws (Cohen &Gooch, 1990).A more effectiverepairmight
say, “There are no bad people, only bad systems.”

Otherrepairsmay be imperfect becausethey fix one problem well, butexacer-
bate others.For example, the Five Cs may help individual loanofficers collect
more kinds of information but theymay createsecondaryproblems.First, by
emphasizing character, the Five Cs may provoke thefundamentalattribution error.
Second, although theyexpandthe set of factors loan officers willconsiderin a
loan decision,they may also institutionalize any tendencythat they may have to
ignoreotherpotentially relevant factors. Third, they may help loan officerscollect
information, butthey do not necessarily help theminterpretit. They seem toindi-
cate that each C should beweightedequally, whereasan ideal statisticalmodel
would weigh someCs more heavilythan others.

As these caveats illustrate, cognitive repairs areunlikely to completelyrepair
the shortcomingsof individual learners.Thus, when we assesswhethera given
cognitive repair is successful, we mustconsiderthe costs and benefitsof the
repair. Below,we considersix dimensions that may affect the costs and benefitsof
repairs, and therefore their success.

Tradeoffs Associatedwith Successful Repairs

In order to be successful, a cognitive repair must be effective—it must mend

some individual shortcoming and improve learning relative to the status quo.To
be truly successful,however,a cognitive repair must also beacceptedin the orga-
nization and actively used. A repair that is not implemented is not arepair.

24 CHIP HEATH, RICHARD P. LARRICK, andJOSHUA KLAYMAN CognitiveRepairs

Cognitive repairsare a kind of innovation,and as such, theiruse will undoubt-
edly be affectedby many of the characteristics that havepreviously beenmen-
tioned in literatureson diffusion andadoption (Rogers,1995; Scott, 1995).We
will focus on innovation characteristicsthat are particularlyrelevantfor cognitive
repairs. Cognitiveshortcomingsnot only createthe need for a repair,they also
limit what repairsmay succeed.

Below, we consider six dimensions that affect whether a repair will be success-
ful: simple versuscomplex,domain-specific versus domain-general,familiar ver-
sus novel, corrective versuspreventative,social versusindividual, and top-down
versusbottom-up.(We will typically focus on the endpointsof these dimensions,
but they shouldbe regardedas continuousrather thandichotomous.)Most dimen-
sionsinvolve tradeoffs.For example, qualitiesthat make a repair more effective in
solving an individual shortcoming sometimes reduce the chancesthat it will be
acceptedand usedby individuals.In the absenceof perfectly effective andaccept-
able repairs, we mustrecognizeand understandthe tradeoffsthat make repairs
more or less successful.

Simple versus Complex

One obviousdimensionalong which cognitiverepairs vary is whetherthey are
relatively simple or complex.Many of the repairs we have discussed in this paper
are strikingly simple—they require an individual torememberand apply aproce-
durethat is only a fewstepslong (e.g., the five whys or the physiciansABCs). In
contrast,many of the proceduresthat are taught as formal repairs in academic
environmentsare quite complex, andinvolve many stagesof sorting, arranging,
and calculating(e.g., formal financial or statisticalanalysis).

Simple repairshave profound advantages over complex repairs. First,they are
more likely to be used because thecostsare small; individuals will find iteasierto
learnand implementshorterprocedures. By contrast, complex repairs typically
require extensive backgroundknowledgeand tax basic cognitive resourceslike
attention and memory (Bell,Raiffa, & Tversky, 1988; Nisbett & Ross, 1980).
Thus, when individuals encounter a complex repair, they are likely to perceive the
costs of learning it as large and immediate, and the benefits of using itas small,
uncertain, and delayed.

Second, simple repairs are easier torememberand reconstructthan complex
repairs, and this increases the probability that individuals will accurately apply
them and accuratelytransmitthemto others. Because complex repairs require
individuals to remember a number of stages,they are more likely to be distorted
whenthey are transmitted from individual toindividual. This problem will bepar-
ticularly pronounced in situationsthat requirelearningby observation andimita-
tion (DiMaggio & Powell, 1983). Individuals who learn a repair through
observationmay find it difficult to infer the completerules of behaviorfor com-

25

plex repairsbecauseinformation about the rules is incomplete, unavailable, ordis-
tributed across time in a waythat makeslearningdifficult.

Although simple repairshave profound advantages overcomplexrepairs, they
also havesome disadvantages. Fundamentally, thetradeoff between simple and
complex repairs is atradeoffbetween ease ofuse and accuracy.Complexproce-
dures are oftencomplicatedbecause they attempt to be precise. Simple repairs
gain easeof useby sacrificing accuracy.For example, a simple aphorism suchas
“don’t confuse brains and a bullmarket” suggests the correct direction toadjust
one’sjudgment,but providesno guidance aboutexactlyhow much one shoulddis-
credit the success of an individualtrader. To precisely estimate theamount of
credit due to brains versus themarket,an individualwould have to perform a more
complex procedure,suchascalculatingthe overall market performance andmea-
suring an individual’s performance relative to the dispersion and centraltendency
of the market.

Domain-Speczflcversus Domain-General

Cognitive repairs also vary in the rangeof contextsto which they canbe applied
(Nisbett, 1992), with some repairs being relatively more domain-specific and
somebeing more domain-general.Domain-specificrepairs are tailorednarrowly
for a specific context(e.g., the FeaturePrioritization Process at Motorola or the
Five Cs of Credit). Domain-generalrepairs aredescribedso generally, and
abstractlythat they apply across mostjudgmenttasks(e.g.,the Five Whys or most
economicor statistical principles).

Domain-specific rules haveat leasttwo advantages overdomain-generalrules.
First, individualsfind it easier torecognizethat a domainspecific rule is relevant
becausethe situation itself remindsthem to usethe rule (e.g., acreditanalyst who
has learned to think aboutthe Five Csof Credit will find it difficult to think about
lending decisionswithout consideringall five categories of information). Second,
individualsmay find it easierto apply domain-specific thandomain-generalrules.
Consider, forexamplea loan officer who istryingto apply a general rulelike “cal-
culate the netpresentvalue (NPV)” of makingthe loan.This domain-generalrule
appliesto many more financial decisions thanthe Five Cs; but it contains no hints
about how it should be applied to aloan decision. In contrast, the Five Cs pointout
specificaspects of theloan decisionthat might affect the loan’squality. Similarly,
securities traders mightfind it hardto benefitfrom a domain-generalwarning
against self-serving biases(e.g., “pay attention tosituational determinantsof suc-
cess, and don’t over-attribute achievement topersonalcharacteristics”).In con-
trast,they are unlikely to miss the point of a moredomain-specificwarning not to
confuse brains and a bullmarket.

Although domain-specificrules have advantages, they also have limits.Their
specificcontentwill make them more likely tospreadwithin their domain, but it
may also preventthem from spreadingacross domains.For example,engineers

26 CHIP HEATH, RICHARD P. LARRICK, andJOSHUA KLAYMAN Cognitive Repairs 27

have safety factors and softwaredevelopershave buffer time, butknowing about
safetyfactors does notautomaticallysuggest the need forbuffertime. And even a
single individualmay use a rule effectively in one domain butfail to-seeits impli-
cations foranother.Auditors areoften quitegood at ignoring theirpreexistingThe-
ones about aclient’s financial health whenthey investigateauditing problems.
However, they arelesslikely to do so whenthey confront similar problemsoutside

the auditing domain, even if the problem relates to their other professionalactivi-
ties (Smith & Kida, 1991).

A second limitationof domain-specific repairs is that theyaretightly tailored to
fit a particular task andenvironment.Because of this tightfit, they maybe less
successful thandomain-generalrepairs when the task environment is in flux. A
buffer factor designed during a specific period oftime—”multiply all time-to-
delivery estimatesby 1.5—maylose its effectiveness whentechnologicalor eco-
nomic conditions change. Considerthat Microsoft had to develop separate buffer
factors to repair overconfidencein applications andoperating-systents?.In general,
domain-specific rules will be helpful incompaniesor divisions where the tasks
and environmentsare stable over time, whiledomain-generalapproacheswill be
helpful in situations where tasks and environments change frequently(e.g., at
higher levels in an organizationwhere tasks and decisions areless routine).

A potential method of combining the advantages of domain-specific and
domain-generalrules maybe to give individuals a domain-specific repair and then
train them to generalize that repair to otherdomains(Fong & Nisbett, 1990;Lar-
rick, Morgan,& Nisbett, 1990).Individualstypically find it easier to generalizeby
analogy from onespecificinstance toanotherthan to map fromgeneraLprinciples
down to specifics (Bassok,1990).For example, peoplewho learn to ignore sunk
costs in financial investments correctlyrecognizethat this rule applies toinvest-
ments of time as well(Larrick, Morgan, & Nisbett,1990).Similarly, amanagerin
industry may find it easier to applythe specific military adage about their being
no such thingas bad troops” than to apply a more generallessonabout thefun-

damental attribution error.

CorrectiveversusPreventative

Cognitive repairs also differ in whethertheypreventor correcttheshortcomings
of individuals. Corrective repairs interveneduring or after aparticularcognitive
process (e.g., the accounting team that corrects their tendency tooverweightvivid
exceptions by forcing themselves to consciously classify each exceptionas major
or minor). At the extreme, a corrective repair might only intervene at thevery end
of a processto correctthe overall outcome(e.g., safety factors). Preventative
repairs intervene early in a cognitive process before shortcomings have had a
chance toact. Microsoft prevents developers from acquiringa biased sample
about the speed of their programsby forcing them to develop programson the
same machines usedby customers.

Someshortcomingsareeasierto correctthan others.Forexample, when ashort-
coming arises because individuals have the wrongrule, they maynot find it diffi-
cult to substitute a different rule(Wilson & Brekke, 1994).Trauma physicians
may learnto check airway before breathing, and accountantsmay learn to ignore
vivid but minorexceptions. In general, corrective repairs will be appropriate when
individuals accept the need for a repair andthey understandhow to execute the
correction.

However,when a shortcoming arises becauseof some basic cognitive process,
organizationsmay need to intervene more forcefullyby bypassingor eliminating
the faulty process (Arkes,1991;Wilson & Brekke, 1994).Forexample,individu-
als may find it difficult to generate abroadand independentset of hypotheses
because associative memory leads them toconsiderthe samealternativestheypre-
viously considered.Theoretically, Motorola could instruct individuals whoare
developing anew consumerproduct to “ignore what you’ve done in the past and
approach this problemcreatively.”However, individuals mightfind it difficult to
ignore their previoussolutions.Thus, Motorola prevents the problemby prohibit-
ing them from serving on morethan one productdevelopmentteam. Similarly, the
Chicago Board of Trade could warn itsinvestigatorsto discountthe answersthey
receive whenthey ask leading questions. Instead it prevents individualinvestiga-
tors from asking yesor no questions, and thusensuresthat they receivelessbiased
information in thefirst place.

Familiar versusNovel

Repairsmay also vary in the extent to which theyarenovel ratherthan familiar.
Novel repairs require individuals to change their assumptions or tolearna proce-
dure from scratch. For example the “programmers” at the CRSSarchitecturalfirm
had to learn to resist theirtendencyto sketch solutions before evaluatingall the
informationfrom a client. On the otherhand,familiar repairs buildon preexisting
knowledge (e.g., the CAMELschemafor bank examiners or the ABCs fortrauma
physicians).Theserepairs have familiarcontent-traumaphysicians knowthat they
should attend tobreathingand circulation, and bank examiners knowthey should
pay attentionto capital and earnings. They also have afamiliar form—they are
organizedby a simple acronymthat individuals already haveas a part of their
mental lexicon.

Familiarrepairsmay be at anadvantageovernovel repairsbecausethey areless
costlyto use and their benefitsmay be moreapparent.CAMEL and the ABCs
reduce costsby using a familiaracronymto remind individuals tocollecttypes of
information that they know they should be collecting. In contrast,the CRSSpro-
grammershad to work hard to overcome thebehaviorsthey had learnedas archi-
tects, andthey mayhave questioned the benefits of theelaboratenew procedures
they were being taught.Familiarrepairsare alsoless likely to provoke resistance

28 CHIPHEATH, RICHARD P. LARRICK, and JOSHUAKLAYMAN Cognitive Repairs 29

thannovel repairs. Anything that requires individualstothrow out old practices or
adopt new beliefsmay be technically and psychologically difficult.

However, familiar repairsmay sometimes be too familiar for theirown good.
First, they may be lesslikely to createenthusiasm.If individuals thinkthat a new
repairdiffers only trivially from current practice,they mayseeno advantage toit.
Because individuals often ignore thefamiliar, would-be change agents often strive
to createthe perception that their programs arenovel and unique(Abrahamson,
1996).Second, familiar repairsmay be subject todistortion.If a repair seemspar-
tially familiar, individuals may neglect itsunfamiliar aspects or force them to
mimic the more familiar aspects (aprocessthat psychologists call assimilation).
For example, the propertechniquefor brainstorming requires aspecificsequence
of steps:first, a creative, idea-generation stage which does notallow criticism,
then a stage where ideasare evaluatedand selected. Although organizationsfre-
quently report thatthey use brainstorming, careful examinationrevealsthat the
organizationsare actually engaged in a more familiar activity: a basic business
meeting (Zbaracki, in press). Thenovel aspectsof the brainstorming procedure,
suchas the separation of stages and the “no criticism”rule, are often lostas brain-
storming isassimilated to the more familiar practice of the standard business
meeting.In the end, only theattractivelabel remains. In situations whereassimt-
lation is a problem, repairsthat are novel may be less likely to suffer distortion
than repairsthat are more familiar becausenovel repairsdo not evokethe preex-
isting knowledgethat leadsto assimilation.

SocialversusIndividual

Many of the cognitiverepairs we have considered are social;they work because
individuals interact with others(e.g., single-caseborequestions in the Deming
Prize organization,or the murderhoard atthe Pentagon).Otherrepairsare indi-
vidual; individualsapply them to theirown learning processes without theinter-
vention of others(e.g., individuals learn to use the Five Whys, and individual
investigatorsat the Board ofTrade learn to avoid “yes or no” questions).

In general, we suspect that manysuccessfulrepairs will be social becauseindi-
vidualsmay not recognizethe need to repair themselves. The very cognitiveshort-
comings that organizations might hope to repairwill make it difficult for
individuals to see theirown flaws. As we havediscussed,individuals tendtoretain
current assumptionsin the face of conflicting data (Klayman,1995). Also, they
interpretevents in waysthat protect their self-image; theyavoid potentiallythreat-
ening feedback(Frey, 1986; Larrick, 1993)and attribute their poor outcomes to
luck or forces outside theircontrol. Although individualsmay find it hard torec-
ognizetheir own biases,they may findit easier torecognizethe biases ofothers.
Many of the repairs we document have the feelof (friendly) socialgamesmanship.
Forexample, learners at FP&L did notconsiderthe hypothesisthatthey had been
“rucky”—their colleagues considered it for them. Similarly,during weekly micro-

biology lab meetings,researchersdid not have to suggest alternative ways ofinter-
preting theirevidence,their peers didso. Social competitionamong individuals
aids the spreadof repairs even when individualsare overconfident and believe
they would have donejust aswell without therepair.

Social repairsdo have to overcomesomedisadvantages.For example,individ-
uals may not appreciate otherswho attempt to repair their biases, andthey may
dismissthe repairattemptsas theproductof picky or abrasive personalities. Thus,
social repairs may be moresuccessfulwhen an individual understandsthat his or
hertormentorsare playing an important, formal role. Individualsmay find it easier
to entertainan antagonist’scritiques when he or she islabeledas a“devil’s advo-
cate,” or whenthe individual is appearing before the “murder board.” Disneyclar-
ified the role of the evaluators and theoccasionby establishingits norm of frank
feedback andby labelingits tryouts as “The Gong Show.”

Eventually,social repairsmay be transformed into individual repairs asindivid-
ualslearnto imitate the patterns of analysis forcedon them by others. In order for
individuals to learn, they needvivid, immediate feedback.Social encountersare
likely to provide a keysource of such feedback.For example, when Deming
examiners asksingle-case-borequestions, or when lab colleaguestry to shoot
holes in a lab presentation, individual learnersmay eventually learn to engage in
preemptiveself-criticismin order tolookbetterduring social encounters(Tetlock,
1992).(Many academicpapers arebetterbecause authorslearnto mentallysimu-
late potentialreviewer’scomments.)Such repairsinvoke social forces at twodif-
ferentlevels: individuals who anticipatesocial interactionmay be more aware of
someof their own shortcomings, andthenactualsocial interactionmay overcome
additionalshortcomingsthat individualscannotrepair on their own.

Top-DownversusBottom-Up

Some cognitive repairs originate from“top-down” within an organization.Typ-
ically these repairsare deliberately designed and implemented by managers or
outsideexperts.Others arise frombottom-upthrough informal observationorser-
endipitousdiscovery fromthe people who aredoingthe work.

The source of the repair isimportantbecause it is likely to affect itsform. In
general, bottomup repairs, suchas organizational adages, willbe simpler and
more domain-specific than top-down repairsdesignedby technically sophisti-
catedengineers,statisticians, or management gurus.Thelocal originof bottom-up
repairs may also make them feel more familiar and acceptable than top-down
repairs. Thus, theorigin of a repair will be highlycorrelatedwith many of the
tradeoffs we have already discussed.

More importantly, the origin of the repair is also likely to affecthow potential
adopters perceiveit. Top-down repairsmay be perceivedwith suspicion orhostil-
ity precisely becausethey originateoutsidethe organizationor are imposedfrom
above. Front-line workers may doubt that outsiderslike consultantsunderstand

30 CHIP HEATH, RICHARD P. LARRICK, andJOSHUAKLAYMAN Cognitive Repairs 31

their situationwell enough to make wise recommendations. When managerssug-
gest a repair,they seem toimply that employees havebeenperformingpoorly or
cannot be trusted to perform theirjob correctly. If so, then individualsmay resist
adopting a repairbecauseof the same kind of self-serving biases we discussedear-
lier.

Top-downrepairsmay also be resisted becausethey will be perceivedasdriven
by politics or fashion and notby the demands of the task. Some top-downrepairs
may be resisted becausethey seem too political. Particularly when top-down
repairs rely on fixed procedures,they may provoke resistance because individuals
may think thatthey aredesigned tocentralizecontrol or removeindividual-discre-
tion. Other top-downrepairsmay be resisted becausethey seem tobe driven by
fashion. Institutional theorists contend that organizations adopt new practices for
reasonsother than greaterefficacy. Organizational membersmay share similar,
cynical intuitions (Adams,1996),and will resist repairsthat they seeasmerewin-
dow dressing oras this year’s fad. Whenindividuals have trouble recognizing
their shortcomings, thenthey may be particularly likely to attribute top-down
repairs to politics or fashion becausethey will not recognizethe repair’s true
value.

Bottom-up repairs will often benefit from their local, homegrown origins. Local
repairs have a meaningful history that makesthem memorableand appealing.
Even a repairthat is potentially threatening, suchas “you were rucky,” may be
more acceptable if organizational members see itas their own invention. Just as
lawyers are entitled to tell lawyerjokes,organizational members are entitled to
develop self-critical repairs and toconveytheir insider statusby using iliem. And
homegrown repairs evoke a stronger sense of ownership; at the same time that
they call attentionto a potential shortcoming,they also givethe n’~er creditfor fix-
ing it.

Conclusions: SuccessfulRepairs

We have considered six differentdimensionsalong which cognitive repairscan
be classified. For example, the physician’s ABCsare simple, domain-specific,
corrective,familiar, individual, and top-down. Although we have suggestedsome
advantages and disadvantagesofeach endpoint of each dimension, we believe-that
our discussion suggests at leastsomepreliminary conclusions about successful
repairs. Forexample,because individuals have limited faculties,organizations
who wish individuals to learncomplex,domain-generalrepairs will find them-
selvesdevoting a greatdeal of scarce time, money, and effortto ensurethat such
repairs are learned and used. Similarly, because individuals are overconfident
about theirown conclusions, they may not spontaneously execute individual
repairs tocorrecttheir own biases. Basedon the advantages and disadvantages we
have considered, we suspect that the most successful repairs will be simple,
domain-specific,socially administered, and evolved frombottom-uprather than

developed from top-down.We find this conclusion intriguingbecauseit describes
repairs that differ sharply from those that arerecommendedin academiclitera-
tureson decision analysis, statistics, and economics.

Implications for Research

Cognitive psychologistsoften think of people asruggedcognitive individual-
ists, constrainedby their own cognitive limitations and a poor environment in
which to learn. Cognitiveresearcherscontinueto argueoverhow well individuals
actually perform the cognitive tasksthey encounterin their lives (see Anderson,
1991, and accompanyingcommentaries,and the debate between Gigerenzer,
1996; and Kahneman &Tversky, 1996). However, it is importantto remember
that much of what people do, including muchof their cognition, takes place in the
contextof organizationsand other social structures.

Some recent approaches in psychology doexplorecultural andsocial contribu-
tions to individual learning. For example, work on “transactivememory” (Weg-
ner, Erber, & Raymond, 1991; Liang, Moreland, & Argote, 1995) showshow
individuals reduce their memory limitationsby distributing memory-heavytasks
across multiple people. Thus,therecan becollectivememory that does notreside
in any individual. Our concept of organizational repairs is in the samespirit, but it
deals with“higher order” cognition: Reasoning and decision making can alsobe
improved through socialstructureand culturalbootstrapping.

We also believethat organizationalpsychologists couldbetterunderstandorga-
nizational processes ifthey thoughtmore about the cognitiveprocessesof the indi-
viduals who makeup the organization.Researchtying individual psychology to
organizationalbehavior certainlyhas along and venerable pedigree (March &
Simon, 1958),but recently,some researchershave expressed concern that that
approachis still underutilized. For example, in the contextof institutional diffu-
sion processes,Zucker(1991) has warnedthat “without a solid cognitive, micro-
level foundation, we risk treatinginstitutionalizationas ablackbox at theorgani-
zational level, focusing oncontentat the exclusion of developing a systematic
explanatorytheory of process” (p.105). And Miner (1994) warns,“most evolu-
tionary discussions of organizational changediscussroutines asthoughthey exist
independentof individual human beings” and“evoke images ofdisembodiedenti-
ties removed from day-to-day human interaction” (p. 87). Knowledge ofindivid-
ual cognitioncan be crucial tounderstandingwhy thingshappenastheydoiri-an
organization.Forexample,why do engineeringfirms use a systemby which the
best engineers make their bestestimates,only to have the firm second-guess them
by adding a huge safety factor?Any explanationwould beincompletewithout an
understandingof individual overconfidence.Understandingthe abilities andcon-
straints of individualspermits a kind of cognitive archaeologyof organizational
practices thatmay allow organizationalresearchersto better understandwhy cer-
tain rules, norms, andproceduresdevelop in organizations, andwhy others fail.

32 CHIP HEATH, RICHARD P.LARRICK, andJOSHUAKLAYMAN Cognitive Repairs 33

Implicationsfor Practice

Managers who think explicitly about cognitive repairswill, we think, be in a
betterposition tofoster improvements in their organizations. Managers already
think about factors such as incentive systems andinformation-technoLQgyas tools
to fosterlearning andinnovation.We believe that cognitive repairswill be a useful
additionto the toolbox. Managerswho consciouslyconsiderindividual cognition
may be able torecognizea larger number of repairopportunitiesandmay target
top-down repairs moreeffectively. Furthermore,they may designmore effective
repairs ifthey take a cognitive approach andconsiderrepair dimensionslike the
six we discussedearlier.

Even whenrepairsare developedbottom-uprather than top-down, a manager
who is informedabout individualcognitionmight have a positive influence.As
has beenobservedby researchers who think about evolutionary approaches to
organizations, oneof the critical components of organizational learning is tostart
with a rich and varied pool of alternative practices(Levitt & March, 1988;Miner,
1994). Savvymanagers canenhancethe pool of alternatives that areavailableby
teachingorganizationmembers about the concept of repairs andby encouraging
them toidentify existing repairs and to seek new repair opportunities. Managers
canrecognizeand reward individuals whodiscovercognitiverepairs,and-they-can
disseminate effectiverepairs via demonstration, training,communication,and
rotation of personnel.

Managersmay also find it veryimportantto think about cognitive repairs when
they evaluate existing organizationalpractices. Consider, for example, a new
executivewho discovers that her development groupis split up into six separate
teams, eachtrying to solve the same problem without anycommunicationwith
each other untilwell into the process.Thismight seemlike a paradigmof bureau-
cratic inefficiency. Yet, this is the kind of systemthat Motorola has found to be
effective ingeneratinga broaderset of options. Without understanding thepoten-
tial value of this repair, the newexecutivemight besadly surprisedby the results
of streamlining theprocess.

Managers might also benefit from a cognitive approachbecausecognitive
repairs, like other innovations,may suffer from the law of unintendedconse-
quences. A repairthat is intended tofix one problem can well end up creating
another.Recall, for example,Microsoft’s laudableattempt to make thecustomer
moresalientto program developersby having them watch live-customers attempt
to use their products in the usabilitytestlab. Thetestlab repairedthe developer’s
tendency to beunmovedby cold statistics, but it probablyexacerbatedtheir ten-
dency to believethat small samples were reliably representative.In responseto
seeing onecustomerin the test lab,developersmight wastetimealtering a feature
that would have been okay for mostcustomers.Managers who take a cognitive
approach would,we hope, be more likely to avoid unwantedside effects, or at
least bein a betterposition to recognize and cope with them.

Final Words

In contrast to Hamlet’s enthusiasm, we argue thatthereis good reason to be
aware ofthe limitations of individual learners. People are not “infinite infacul-
ties” or “noble in reason.” As individuals, we make systematicmistakesthat com-
promiseour ability tolearnfrom our experiences and tounderstandthe world.On
the otherhand, wemortals are notall fools. We are able toform socialstructures
that havethe potential to magnifysomeof our abilities and to minimizesomeof
our failings. In thispaperwe have concentrated ondemonstratingthat effective
organizational repairs happen. Wedo not mean to imply thatorganizationscure
any and all individual shortcomings, nor eventhat organizationsalways make
thingsbetterrather thanworse.Nevertheless, we do believe that theorganizations
in which we work can provide us with norms, reminders, protocols, andproce-
duresthat help usmovebeyond our individual shortcomings.

ACKNOWLEDGMENTS

We thankthe Graduate Schoolof Business,University of Chicagoandthe FuquaSchoolof

Business,Duke University, for researchsupport. For helpful commentson this project, we

thank participantsat Rod Kramer’s 1996 conferenceon OrganizationalLearning; we also
thank Jim March, Cade Massey, Elizabeth McSweeny, Sim Sitkin, Dick Thaler, Elke

Weber, andJohnWright.

NOTES

1 We primarily want to distinguish problemsof incentives fromproblemsof mental process. In
this reviewwe will not distinguish between mental errors that arise from “motivatedreasoning”and
thosethat arise from “colder” processes.

2. Personalcommunication,Abhay Joshi.
3. Personalcommunication,RobertAlan Fisch.
4. Personalcommunication,Jan Elsbach.
5. Personalcommunication,Francisco Bayron.
6. Personalcommunication,DeanSiewert.
7. Personalcommunication,JustinScott Bradley.
8. Personalcommunication,Francisco Bayron.
9. Personalcommunication,DeanSiewert.

10. Personalcommunication,Ken Myszkowski.

II. Personalcommunication,Leo Tucker.
12. Personalcommunication,Raymond Stukel.
13. Personalcommunication,John Payne

REFERENCES

Abrahamson, E. (1996). Management fashion.Academy ofManagementReview, 21, 254-285.
Adams,S. (1996). The Dilbert principle. New York: Harper Business.

34 CHIP HEATH, RICHARD P. LARRICK, andJOSHUAKLAYMAN
Cognitive Repairs 35

Agostinelli,0, Sherman,S. J., Fazio, R. H.,& Hearst, E.5. (1986). Detecting and identifying change:
Addition versus deletion.Journal ofExperimental Psychology. Human Perception and Perfor-
mance, 12, 445-454.

Anderson,J.R. (1991). Is human cognitionadaptive? Behavioral and Brain Sciences,14,471-517.
Anderson,JR. (1995).Cognitivepsychologyand its implications.New York: W.H. Freeman& Co.
Arkes, H. R. (199 I). Costs and benefitsofjudgmenterrors: Implications for debiasing.Psychological

Bulletin, 110, 486-498.
Baron,J. (1988). Thinking and deciding. Cambridge, UK: Cambridge University Press.
Bassok,M. (1990). Transferof domain-specificproblem-solvingprocedures.Journal ofExperimental

Psychology: Learning, Memory, and Cognition, 16, 522-533.
Bell, D. E., Raiffa, H.,& Tversky, A. (1988).Decision making: Descriptive, normative, and prescrip-

tive interactions.Cambridge: Cambridge University Press.
Borgida, E.,& Nisbett, R. E. (1977). The differential impactof abstractvs. concrete information on

decisions.JournalofAppliedSocialPsychology7, 258-271.
Buehler,R., Griffin, 0., & Ross,M. (1994). Exploring the “planning fallacy”: Why peopleunderesti-

mate their task completiontimes. Journalof Personality and SocialPsychology67, 366-381.
Cheng,P. VV., & Holyoak, K. J.(1985). Pragmatic reasoningschemas.CognitivePsychology,17, 391-

416.
Cohen, L.1. (1981). Can human irrationality beexperimentally demonstrated?Behavioral and Brain

Sciences, 4, 317-331.
Cohen,M. D. (1991). Individual learning and oganizational routine: Emerging connections.Organi-

zation Science, 2, 135-139.
Cohen,E. A., & Gooch,J. (1990).Military misfortunes:The anatomyoffailurein war. New York: Free

Press.
Cusumano,M. A., & Selby, R. W. (1995).Microsoft secrets.New York: Free Press.
Davis, 0. (1985,March-April). New projects: Bewareof false economies.Har’.’ard BusinessReview,

95-11)1.
Deming, W. E. (1982).Out of the crisis. Boston: Massachusetts Instituteof Technology Center for

Advanced Engineering Study.

DiMaggio, P. J., & Powell Xv. W. (1983). The iron cage revisited: Institutionalisomorphismandcol-
lective rationality in organizational fields.American SociologicalReview,48, 1,47-160.

Dunbar,K. (1995). How scientists reallyreason:Scientific reasoning in real-world laboratories. In R.
J. Steinberg& J. Davidson (Eds.),The natureof insight.Cambridge,MA: MIT Press.

Eisenhardt,K. M. (1989). Agency theory: An assessment andreview. Academyof Management
Review, 14(1), 57-74.

Eppel, D.,Argote, L., & Devadas, R. (1991). Organizational learningcurves:A methodfor investigat-
ing intra-plant transferof knowledgeacquiredthroughReamingby doing. OrganizationalSci-
ence, 2, 58-70.

Fischhoff,B., Slavic,P.. & Lichtenstein, 5.(1978). Fault trees: Sensitivityof estimatedfailure proba-
bilities to problem representation.Journal of Experimental Psychology: HumanPerception
andPerformance,4, 330-344.

Fiske, S. T., & Taylor,S. E. (1991).Social cognition(2nd ed.). NewYork: McGraw-Hill.
Fletcher,C. (1990).What cops know.New York: Pocket Books.
Fong, G. T., & Nisbett, R. E. (1990). Immediate anddelayedtransferof training effectsin statistical

reasoning.Journalof Experimental Psychology:General, 120, 34-45.
Forum Corporation(1992).P.R.O.G.R.E.S.S.tool kit. Forum Corporation.
Frey, D. (1986). Recent research on selective exposure to information.Advancesin Experitnental

SocialPsychology19, 41-80.
Funder,D.C. (1987). Errors andmistakes:Evaluating the accuracyof social judgment.Psychological

Bulletin, 101, 75-90.

Gettys, C.F. et al. (1987). An evaluationofhuman act generation performance.OrganizationalBehav-
ior and Human Decision Processes,39, 23-51.

Gigerenzer, G. (1996). On narrownorms and vague heuristics: A replyto Kahneman and Tversky
(1996).PsychologicalReview, 103, 592-596.

Gilbert, D. T., & Jones, E. E.(1986). Perceiver-induced constraint: Interpretationsof self-generated
reality. Journalof Personality and SocialPsychology50, 269-280.

Gilbert, D. T.,& Malone,P.S.(1995). The correspondence bias.Psychological Bulletin,117, 21-38.
Gnepp,J., & Klayman,J. (1992). Recognitionof uncertaintyin emotional inferences: Reasoningabout

emotionallyequivocalsituations.DevelopmentalPsychology28, 145-158.
Gregory, Xv. L., Cialdini, R. B., & Carpenter, K.M. (1982). Self-relevant scenarios as mediatorsof

likelihoodestimatesand compliance: Does imagining makeit so?Journalof Personality and
Social Psychology43, 89-99.

Griffin, 0., & Tversky,A. (1992). The weighingof evidence andthedeterminantsof confidence.Cog-
nitive Psychology24,411-435.

Heath,C., Knez,M., & Camerer, C. (1993). The strategicmanagementofthe entitlement processin the
employmentrelationship.StrategicManagementJournal, 14, 75-93.

Hilton, D.J. (1995). The social contextof reasoning: Conversational inference and rational judgment.
Psychological Bulletin, 118, 248-271.

Hoch,S. J. (1984).Availability and interference in predictive judgment.JournalofExperimentalPsy-
chology:Learning, Memory, andCognition, 10, 649-662.

Hodgins,H. S.,& Zuckerman,M. (1993). Beyond selecting information: Biasesin spontaneousques-
tions and resultantconclusions.Journalof Experimental SocialPsychology29, 387-407.

Holland, J. H., Holyoak, K.J., Nisbett, R. E., & Thagard,P. T. (1986).Induction: Processesof infer-
ence,learning, and discovery.Cambridge,MA: Bradford Books/MIT Press.

Hynes,M., & Vanmareke, E. (1976).Reliability of embankmentperformancepredictions.Proceedings
of the ASCE Engineering Mechanics Division SpecialtyConference.Waterloo, Ontario,Can-
ada: University of Waterloo Press.

lmai, M. (1986).Kaizen: The key to Japan’scompetitivesuccess.New York: McGraw-Hill.
Ishikawa, K. (1982).Guide to quality control(2nd revised ed.). Hong Kong: Asia Productivity Orga-

nization.
Ishikawa, K. (1985).What is totalqualiry control? TheJapanese way.Englewood Cliffs,NJ: Prentice-

Hall.
Juran, J.M. (1992).Juranon quality by design:The new stepsfor planningquality into goods andser-

vices.New York: Free Press.
Kahneman, D., Slovic,P., & Tversky, A. (1982). Judgmentunderuncertainty: Heuristicsand biases.

Cambridge, England: Cambridge University Press.
Kahneman, D.,& Tversky, A. (1996). On the realityof cognitive illusions.PsychologicalReview,103,

582-591.
Kaplan, J. (1995).Start-up: A silicon valiry adventure.Boston: Houghton, Muffin Co.
Keller, P. A., & McGill, A. L. (1994). Differencesin the relative influenceof product attributes under

altemative processing conditions: Attributeimportanceversus attribute easeof imagability.
Journalof Consumer Psychology3, 29-49.

Koehler,I. J. (1993). The influenceof prior beliefson scientificjudgmentsofevidencequality. Orga-
nizational Behavior and Human Decision Processes,56, 28-55.

Klayman, J. (1995). Varietiesof confirmation bias.The Psychologyof Learning and Motivation,32,
385-418.

Kunda, Z. (1991)).The case formotivatedreasoning.Psychological Bulleti,t,108, 480-498.
Larrick, R. P. (1993). Motivational factorsin decision theories: The roleof self-protection.Psycholog-

ical Bulletin, 113, 440-450.
Larrick, R. P., Morgan, J. N., & Nisbett, R. E. (1990). Teaching the useof cost-benefitreasoningin

everyday life.PyschologicalScience,1, 362-370.

36 CHIPHEATH, RICHARD P. LARRICK, andJOSHUA KLAYMAN Cognitive Repairs 37

Levitt, B., & March, J. G. (1988). Organizational leaming.AnnualReviewofSociology14, 319-40.
Liang, D. Xv., Moreland,R., & Argote, L. (1995). Group versusindividual training and groupperfor-

mance:The mediatingfactorof transactivememory.Personality& SocialPsychologyBulletin,
21, 384-393.

Lichtenstein, S., Fischhoff,B., & Phillips, L. D. (1982). Calibrationofprobabilities:The stateofthe art
to 1980. In A. Tversky, D. Kahneman, &P. Slovic (Eds.),Judgment underuncertainty:Heu-
ristics and biases(pp. 306-334).New York: Cambridge.

Lord, C., Ross, L., & Lepper,M. R. (1979). Biasedassimilationand attitude polarization: The effects
ofprior theorieson subsequentlyconsidered evidence.Journalof PersonaliryandSocialPsy-
chology,37, 2098-2109.

Mahoney,M. J. (1976).Scientist as subject:Thepsychologicalimperative.Cambridge,MA: Ballinger.
Making quality more than afad. (1992, May 18). Fortune,pp. 12-13.
March,J. G., Sproull, L. S., &Tamuz,M. (1991).Learningfrom samplesofone orfewer. Organization

Science,2, 1-13.
March, J. G.,& Simon, H. A. (1958).Organizations.New York: John Wiley.
McGowan, J. (1996, April1). How Disney keeps ideascoming. Fortune,pp. 131-133.
Milgrom, P., & Roberts,1. (1992).Economics,organization, and management.EnglewoodCliffs, NJ:

Prentice Hall.
Miner, A. 5. (1994). Seekingadaptiveadvantage: Evolutionary theory and managerial action. In J. A.

C. Baum & J. V. Siogh (Eds.),Evolutionarydynamicsof organizations (pp. 76-89).New York:
Oxford University Press.

Miner, A. S. & Mezias, S. J. (1996).Ugly duckling no more: Pasts and futuresof organizationalleam-
ing research.OrganizationScience,7, 88-99.

Mullen, B., & Riordan, C. A. (1988). Self-serving attributions for performancein naturalisticsettings~
A meta-analyticreview. JournalofAppliedSocialPsychology18, 3-22.

Mynatt, C. R,Doherty, M. E., & Dragan,Xv. (1993). Information relevance, working memory, and the
considerationof altematives.QuaterlyJournalof ExperimentalPsychology46A,759-778.

Newman, J.,Wolff, Xv. T., & Hearst, E. (1980). The featurepositive effect in adult human subjects.
Journal ofExperimental Psychology: Human Learning andMemory, 6, 630-650.

Nisbett, R., & Ross, L.(1980). Human inference: Strategiesand shortcomingsof social judgment.
Englewood Cliffs,NJ: Prentice-Hall.

Nisbett, R. E. (1992). Reasoning, abstraction, and theprejudicesof 20th centurypsychology. In R. E.
Nisbett (Ed),Rulesfor reasoning(pp. 1-12).Hillsdale, NJ: Erlbaum.

Nisbett, R. E.,.Krantz,D. H.,Jepson,D., & Kunda, Z. (1983). The useof statisticalheuristicsin every-
day reasoning.PsychologicalReview,90, 3 39-363.

Nonaka, I.,& Takeuchi, H. (1995).The knowledge-creating company: How Japanese companiescre-
ate the dynamicsof innovation.New York: Oxford University Press.

Odean,T. (1996). Volume,volatility, price and profitwhen all traders areabove average.Working
paper.University of California, Berkeley.

Payne, J.Xv. (1976). Task complexity andcontingentprocessingin decisionmaking: An information
search and protocol analysis.Organizational Behavior and HumanPerformance,16, 366-387.

Perrow,C. (1984). Normal accidents:Living with high-risktechnologies.New York: Basic.
Peters,T. (1992). Liberation management:Necessarydisorganizationfrr the nanosecond nineties.

New York: Alfred A. Knopf.
Rogers, E.(1995). Thediffusion of innovations(4th ed.). NewYork: Free Press.
Rosenthal,E. (1994,May). Invisible injuries. Discover,pp. 48-51.
Ross, L., & Nisbett,R. E. (1991).Thepersonand the situation: Perspectivesofsocial psychology.New

York: McGraw-Hill.
Ross, L.(1977). The intuitive psychologist and his shortcomings. In L. Berkowitz (Ed.),Advancesin

experimentalsocialpsychology(Vol. 10). New York: Academic Press.

Russo,J. E.,& Schoemaker,P.J. H. (1992). Managing overconfidence.SloanManagementReview,33,
7-17.

Sapolsky,R. (1997, November). Ontheroleof upholsteryin cardiovascularphysiology.Discover,pp.
58-66.

Scott,Xv. R. (1995). Institutions andorganizations.ThousandOaks, CA: Sage.
Senge,P. M. (1990). Thefifth discipline. New York: Doubleday Currency.
Slovic,P., Fischhoff,B., & Lichtenstein,5. (1982). Facts versus fears: Understandingperceivedrisk.

In A. Tversky, D. Kahneman,& P. Slovic (Eds.),Judgmentunderuncertainty: Heuristicsand
biases(pp. 463-492).New York: Cambridge.

Smith, J. F., & Kida, T. (1991). Heuristics and biases: Expertise and taskrealismin auditing. Psycho-
logical Bulletin, 109,472-489.

Tetlock, P. E. (1992). Theimpactof accountabilityon judgmentandchoice: Toward a socialcontin-
gency model.Advancesin Experimental SocialPsychology25, 331-376.

Tversky, A., & Kahneman, D. (1971). Belief in the lawof small numbers.PsychologicalBulletin, 76,
105-110.

Tversky, A., & Kahneman, D. (1973).Availability: A heuristicforjudging frequency and probability.
CognitivePsychology5, 207-232.

Walton, M. (1990).Demingmanagementat work. New York: Putnam.
Wegner, D. M., Erber,R., & Raymond,P. (1991). Transactive memoryin close relationships.Journal

of Personality and SocialPsychology61, 923-929.
What Bill Gates really wants. (1995,January16). Fortune,p. 34.
Wilson, T. D., & Brekie, N. (1994). Mental contaminationand mentalcorrection:Unwantedinflu-

enceson judgmentsand evaluations.Psychological Bulletin,116, 117-142.
Zbaracki,M. (in press). The rhetoric and realityof total quality management.AdministrativeScience

Quarterly.
Zucker, L. G. (1991). Postscript:Microfoundationsof institutional thought. InXv. Xv. Powell & P. J.

DiMaggio(Eds.),The newinstitutionalismin organizationalanalysis(pp. 103-107).Chicago:
Universityof Chicago.

Zuckerman,M., Knee, C.R., Hodgins, H. S.,& Miyake, K. (1995). Hypothesis confirmation: Thejoint
effectof positivetest strategy and acquiescenceresponseset.JournalofPersonalityand Social
Psychology68, 52-60.