What Every Founder Needs to Know About Survivorship Bias

Starting a company is hard. Really hard. The odds of success are stacked against you, with data showing that nearly 90% of startups end in failure. Yet we‘re constantly bombarded with stories in the media and our social feeds about the breakout successes – the Ubers, Instagrams, and Stripes of the world.

We put these companies and their founders up on a pedestal, breathlessly analyzing everything from their morning routines to their hiring practices, assuming there must be some "secret sauce" we can copy. But this type of thinking is a cognitive trap known as survivorship bias, and it can lead us as founders to make poor decisions based on incomplete information.

In this post, we‘ll take a deep dive into survivorship bias, examining what it is, why it‘s so pervasive in the world of startups, and how you can identify and avoid its pitfalls as you build your company.

What Is Survivorship Bias?

Survivorship bias is the logical error of focusing on the people or things that "survived" or succeeded some process while overlooking those that failed, typically because they are hidden from view. It‘s a type of selection bias that can lead us to overestimate the chances of success and draw false conclusions by only examining the winners, not the entire pool of participants.

The concept was first popularized by World War II mathematician Abraham Wald, who was tasked with figuring out how to minimize bomber losses to enemy fire. The military knew armoring the entire plane would make it too heavy, so they wanted to selectively reinforce the areas that took the most damage.

Wald examined the bombers that returned from their missions and found the bullet holes tended to be clustered on the wings, tail, and fuselage. So the military‘s first instinct was to add extra armor to those areas. But Wald realized they were only looking at the survivors. What about the planes that had been shot down?

As it turned out, the parts of the returning planes that had no bullet holes – like the engines – were actually the most vulnerable. The planes hit in those critical areas never made it back. By only looking at the survivors, you miss key information that could be even more important.

And that‘s the core of survivorship bias – drawing conclusions based only on the examples that make it past some selection process, while ignoring those that don‘t. Some other classic examples:

• A financial advisor cherry-picking their top-performing funds in an ad, conveniently failing to mention the many funds that tanked
• Basing a medical study only on patients that responded well to a treatment, not those that dropped out due to side effects or lack of results
• Assuming the habits of famous entrepreneurs are the key to success, while ignoring the many entrepreneurs with the same habits that failed

Survivorship Bias in Business & Entrepreneurship

Perhaps nowhere is survivorship bias more rampant than in the world of startups and business. We are inundated with survivor stories – the college dropouts who became billionaires, the companies that went from garage to IPO, the founders who beat the odds.

But the reality is, these are the exceptions, not the rule. For every Mark Zuckerberg, Sara Blakely or Elon Musk, there are thousands of entrepreneurs you‘ve never heard of who pursued similar paths and failed. The base rate of success in startups is low – really low.

A 2021 report analyzing 111 startup post-mortems found that the top reasons for failure were:

• No market need (42%)
• Ran out of cash (29%)
• Not the right team (23%)
• Got outcompeted (19%)

The vast majority of founders pour their hearts and souls into companies that never take off due to factors both within and outside their control. But we tend to put the spotlight on the rare successes, creating a distorted view of what it takes to build a business.

Some common examples of survivorship bias in startups and business:

Trying to Copy Successful Companies

How many pitches have you heard that opened with "We‘re the Uber for X"? Too many founders look at breakout successes like Warby Parker or Airbnb and try to copy their playbook verbatim, without fully analyzing the market conditions, competitive landscape, and unique circumstances that enabled their success.

Just because a direct-to-consumer model worked in eyeglasses or a marketplace model worked in hospitality doesn‘t mean the same tactics will automatically translate to domains like healthcare, education or construction. You have to do your own homework.

Anecdotal Advice From Outliers

Entire cottage industries and content verticals have sprung up around analyzing and copying the habits of top founders and business leaders. What do they eat for breakfast? What books do they read? What is their sleep schedule?

But again, just because Elon Musk or Marie Kondo have some quirky habits doesn‘t mean those habits are the key to success. For all we know, they‘re succeeding in spite of those habits, not because of them. Always consider the source and scope of advice.

Flawed Hiring & Evaluation Practices

A famous example of survivorship bias in hiring is the brainteaser questions that used to be popular in Silicon Valley job interviews. Companies would ask candidates to solve some elaborate logic puzzle, convinced that the ability to think through these types of problems was predictive of on-the-job success.

But Google famously banned brainteaser questions after analyzing their hiring data and finding no correlation between ability to solve brainteasers and actual job performance. They were focusing only on the people who got hired (the survivors), not the many outstanding candidates those questions may have filtered out.

Poor Pricing & Revenue Decisions

Imagine you sell a SaaS product and are considering raising prices. If you base your pricing analysis solely on feedback from existing customers and ignore prospects who never bought because your pricing was already too high, you‘re likely to draw the wrong conclusions. You must factor in the people that didn‘t "survive" your current pricing as well.

Why We Fall for Survivorship Bias

Survivorship bias is so pervasive because it plays into many of our existing cognitive tendencies and mental models:

• We like success stories and find failures depressing, so the media tends to give outsize coverage to the rare successes while ignoring the masses of failures
• Our brains are wired for pattern-matching and love anecdotes and stories over analyzing boring, aggregate data
• We tend to ascribe success to individual skill and effort, while chalking up failures to bad luck or external circumstances
• We are drawn to the highly visible and newsworthy, while the mundane and typical tend to fly under our radars
• Successful people and companies are more vocal and likely to share their stories and advice, while failed founders often fade into the background

All of these mental shortcuts and information asymmetries combine to create an environment where survivorship bias can really thrive. So how do we push back against it?

How to Identify and Avoid Survivorship Bias as a Founder

The first step to overcoming survivorship bias is simply being aware of it. When you catch yourself making generalizations based only on success stories, or focusing only on the examples that made it through some filter, pause and ask what information might be missing.

Some specific strategies you can employ:

1. Collect Full Data Sets

Make sure you‘re gathering data from the entire pool of customers, investors, employees etc – not just the ones that are the most visible or responsive. Understand why people didn‘t buy or didn‘t respond. Continuously seek to expand your data set rather than a making decisions based on a small subset.

2. Know the Base Rates

Don‘t just focus on the numerator (absolute number of successes), but also the denominator (total number of people or companies that pursued that path). If a VC firm brags that three of their portfolio companies went public, that‘s meaningless unless you know they‘ve invested in 300 companies. Always ask "out of how many?"

3. Intentionally Seek Disconfirming Evidence

It‘s easy to find information that confirms our existing ideas and assumptions. It‘s much harder to seek out evidence that contradicts or disproves our beliefs. Whenever you‘re testing a hypothesis, actively look for data that would prove it false, not just information that supports it.

4. Get Diverse Perspectives

We tend to surround ourselves with people who share our backgrounds and ways of thinking. Make an effort to talk to people with wildly different life experiences and perspectives from your own. You may be surprised by the angles and considerations you‘ve been missing.

5. Imagine Alternative Outcomes

When we‘re sitting on top of the mountain as the survivor in a situation, everything about our path can seem obvious and inevitable in hindsight. But what about all the people who pursued similar paths and didn‘t make it? How does the world look through their eyes? What would you be seeing and thinking if you were in their shoes?

Also remember, just because someone is successful doesn‘t mean everything they did was optimal or even correct. And just because someone failed doesn‘t mean everything they did was wrong. Judging a decision based only on the outcome is another dangerous logical fallacy.

The truth is, success in startups and business is often due to a complex mix of team quality, market conditions, competitive landscape, timing, specific decisions, and a heavy dose of luck. Trying to determine the "key factors" using only a handful of cherrypicked successes is a recipe for bad conclusions.

Conclusion

The siren song of survivorship bias is hard to resist, especially for founders. We look at the rare legends who made it big and desperately want to believe that if we can just crack the code of what they did right, we can follow the same trajectory.

But this "success leaves clues" mentality often leads us to focus on the wrong things while ignoring more mundane but critical factors. We put on blinders that constrict our thinking and cause us to make decisions based on incomplete and biased information.

The truth is, for every thriving survivior, there are exponentially more "failures" you don‘t see. And those failures often hold just as many valuable lessons as the successes, if not more.

By being aware of survivorship bias, collecting complete data, knowing your base rates, seeking disconfirming evidence, and imagining alternatives, you can begin to identify and overcome these blind spots. You can make decisions based on a more objective analysis of the entire landscape rather than a skewed subset.

And you can realize that while success absolutely does leave clues sometimes, applying blanket patterns without carefully considering your specific situation is more likely to lead you astray than to Promised Land.

Survivorship bias doesn‘t have to sink your company. By proactively identifying it and taking concrete steps to counteract it, you can keep it from clouding your judgment and decision making. So the next time you hear some tidbit of advice or supposed strategy for success, always remember to ask: what about the ones that didn‘t survive?