Why Success Always Starts With Failure

The ego is the enemy of innovation. Economist Tim Harford explains why a complex world demands that we accept our mistakes and adapt rapidly.

“Few of our own failures are fatal,” economist and Financial Times columnist Tim Harford writes in his new book, Adapt: Why Success Always Starts With Failure. This may be true, but we certainly don’t act like it. When our mistakes stare us in the face, we often find it so upsetting that we miss out on the primary benefit of failing (yes, benefit): the chance to get over our egos and come back with a stronger, smarter approach.

According to Adapt, “success comes through rapidly fixing our mistakes rather than getting things right first time.” To prove his point, Harford cites compelling examples innovation by trial-and-error from visionaries as varied as choreographer Twyla Tharp and US Forces Commander David Petraeus. I interviewed Harford over email to dig deeper into the counter-intuitive lessons of Adapt. What follows is a series of key takeaways on the psychology of failure and adaptation, combining insights from our conversation and the book itself.

The Wrong Way To React To Failure

When it comes to failing, our egos are our own worst enemies. As soon as things start going wrong, our defense mechanisms kick in, tempting us to do what we can to save face. Yet, these very normal reactions — denial, chasing your losses, and hedonic editing — wreak havoc on our ability to adapt.

Denial.

“It seems to be the hardest thing in the world to admit we’ve made a mistake and try to put it right. It requires you to challenge a status quo of your own making.”

Chasing your losses.

We’re so anxious not to “draw a line under a decision we regret” that we end up causing still more damage while trying to erase it. For example, poker players who’ve just lost some money are primed to make riskier bets than they’d normally take, in a hasty attempt to win the lost money back and “erase” the mistake.

Hedonic editing.

When we engage in “hedonic editing,” we try to convince ourselves that the mistake doesn’t matter, bundling our losses with our gains or finding some way to reinterpret our failures as successes.

We’re so anxious not to “draw a line under a decision we regret” that we end up causing still more damage while trying to erase it.

The Recipe for Successful Adaptation

At the crux of Adapt lies this conviction: In a complex world, we must use an adaptive, experimental approach to succeed. Harford argues, “the more complex and elusive our problems are, the more effective trial and error becomes.” We can’t begin to predict whether our “great idea” will actually sink or swim once it’s out there.Harford outlines three principles for failing productively: You have to cast a wide net, “practice failing” in a safe space, and be primed to let go of your idea if you’ve missed the mark.

Try new things.

“Expose yourself to lots of different ideas and try lots of different approaches, on the grounds that failure is common.”

Experiment where failure is survivable.

“Look for experimental approaches where there’s lots to learn – projects with small downsides but bigger upsides. Too often we take on projects where the cost of failure is prohibitive, and just hope for the best.”

Recognize when you haven’t succeeded.

“The third principle is the easiest to state and the hardest to stick to: know when you’ve failed.”

The more complex and elusive our problems are, the more effective trial and error becomes.

How To Recognize Failure

This is the hard part. We’ve been trained that “persistence pays off,” so it feels wrong to cut our losses and label an idea a failure. But if you’re truly self-aware and listening closely after a “release” of your idea, you can’t go wrong. Being able to recognize a failure just means that you’ll be able to re-cast it into something more likely to succeed.

Gather feedback.

“Above all, feedback is essential for determining which experiments have succeeded and which have failed. Get advice, not just from one person, but from several.” Some professions have build-in feedback: reviews if you’re in the arts, sales and analytics if you release a web product, comments if you’re a blogger. If the feedback is harsh, be objective, “take the venom out,” and dig out the real advice.

Remove emotions from the equation.

“It’s important to be dispassionate: forget whether you’re ahead or behind, and try to look at the likely costs and benefits of continuing from when you are.”

Don’t get too attached to your plan.

“There’s nothing wrong with a plan, but remember Von Moltke’s famous dictum that no plan survives first contact with the enemy. The danger is a plan that seduces us into thinking failure is impossible and adaptation is unnecessary – a kind of ‘Titanic’ plan, unsinkable (until it hits the iceberg).”

Being able to recognize a failure just means that you’ll be able to re-cast it into something more likely to succeed.

Creating Safe Spaces to Fail

Twyla Tharp says, “The best failures are the private ones you commit in the confines of your own room, with no strangers watching.” She rises as 5:30 AM and videotapes herself freestyling for 3 hours each morning, happy if she extracts just 30 seconds of usable material from the whole tape. This is a great example of a “safe space to fail.” But many of us don’t have this luxury of time or freedom. So how do we create this space?

Practice disciplined pluralism.

Markets work by this process, encouraging the exploration of many new ideas as well as the ruthless weeding out of the ones that fall short. “Pluralism works because life is not worth living without new experiences.” Try a lot of things, and commit only to what’s working.

Finding “a safe space to fail is a state of mind.”

Assuming that you don’t operate a nuclear power plant for a living, you can probably infuse a bit more freedom and flexibility into your workday. Give yourself permission to test out a few off-the-wall ideas mixed in with the by-the-book ideas.

Imitate the college experience.

“College is an amazing safe space to fail. We are experimenting with new friends, a new city, new hobbies and new ideas – and we’ll often mess up academically and socially as a result. But we know that as long as we don’t screw up too dramatically, we’ll finish college, graduate, and move on – that mix of risk and safety is intoxicating. Yet somehow as we grow older we lose it.” —

What’s Your Take?

Do you think that trial and error is the most effective approach for innovation? What are your tips for surviving failure?