What it is
Survivorship bias occurs when analysis is conducted on the subset of cases that passed some selection filter - companies that are still operating, products that are still on the market, leaders who are still in post - while the cases that failed and therefore disappeared are excluded. The survivors appear to share characteristics that "explain" their success, but those same characteristics may have been equally common among the failures that are no longer visible. The bias produces lessons that are artefacts of the data sample rather than genuine causal factors.
Where it shows up
In competitive analysis and strategy decisions, survivorship bias appears when teams benchmark against successful competitors and conclude that their practices are the cause of success. Firms that pursued the same practices and failed are not in the comparison set. Similarly, when evaluating whether to adopt a SaaS product, teams often reference customer case studies published by the vendor - a population selected entirely from successes - rather than researching churn rates, failed implementations, or customers who reverted.
What Rubicon Probity does
When Rubicon Probity encounters a decision record at the Diagnose stage where the evidence base consists primarily of published case studies, analyst reports, or vendor references, it raises a NOTE flag and requests that the team document evidence sourced from non-survivors - churned customers, discontinued products, or failed comparable initiatives. Both populations must be represented before the analysis moves forward.
Detection questions
- Does your evidence base include information about organisations or projects that attempted this and failed - or only those that succeeded?
- If you looked at the full population of companies or projects that used this approach, what fraction achieved the outcome you are targeting?
- Are the case studies in this decision record selected by you, or by a vendor or third party with an interest in showing successes?