Amongst all the chaos, a quietly brilliant Hungarian Jewish mathematician, Abraham Wald (whose family escaped Nazi persecution by emigrating to the United States) noticed a peculiarity of Allied bombers. While at Columbia University, he was tasked with saving as many bombers and pilot lives as possible. Easier said than done. However, Wald came up with a simple, yet genius trick. He realised that the bombers that come back, while riddled with bullet holes, *come back*. It was those that didn’t return from their missions that needed to be studied - those were the planes where the bullets hit the fuel storage, a wing weak spot, or glass in the cockpit. 

The popular concept at the time was to reinforce Allied bomber planes in the areas where the most bullet holes were present. Having understood the bias of only looking at surviving planes and where the bullets hit them, he proposed that reinforcements be built on the areas with *fewer* bullet holes, since those were the areas where it was likely that the plane wouldn’t survive the hit. 

In a nutshell, and removing it from its military origin, survivorship bias is the idea that by only looking at the surviving population/record, we are omitting that part of the sample that ‘didn’t make it’ and coming to faulty conclusions by not observing the whole spectrum of evidence.

This way of thinking, coined the ‘survivorship bias’, was groundbreaking and contributed significantly to the area of operational research. We continue to see ripples of it echo all around us, from its application in the world of finance, to education, to policy-making. In this blog post, we’d like to show you how it affects the world of digital organising and campaigning.

The challenge to survivorship bias 

The critique of survivorship bias, also sometimes morbidly referred to as ‘corpse bias’, is that failures and weaker examples are focused on so profoundly that that in itself creates reverse bias. An example of ‘corpse bias’ (and where a dash of ‘survivorship bias’ might in fact prove useful), is when you focus on unsuccessful campaigns and try to see the best in them or disregard winning campaigns and take zero lessons from their victories because you disagree with what they stand for. Or, on a more granular level, focusing on unpicking word-for-word why your latest email got a lower open rate rather than focusing on your broader email strategy and how it could improve overall engagement.

‘Corpse bias’ is in fact something quite deep-seated in political campaigning, particularly with the polarisation of the political scene in recent years. 

Many progressives, for example, credit the victories of Trump and Brexit to populism, ‘being at the right place at the right time’, throwing money at the problem, or simply cheating. While some of these are no doubt strong supporting factors that contributed to their overall victories, it would be wrong to assume that these campaigns hadn’t been deliberately strategic. In fact, tactics such as Trump’s Twitter or Leave campaign’s decentralised local campaigning, ended up having the desired impact - winning an election.

So where does survivorship bias come into it? Well, as you think about how to develop a successful campaign, it’s important to look at both what’s working and not working not just for you, but your opposition, too. Many progressives strongly disagree with the positions and tactics of the extreme right. However, that doesn’t mean that the right doesn’t have clever campaigning tricks up their sleeves which progressives can use to their advantage. (Which is why we have a “Know Thy Enemy” section of our Tectonica Organising Network newsletter).  However, this doesn't mean going too far the other way. Just because Trump uses Twitter to deliver quick, biting cheap shots doesn’t mean you should too. Going for imitating quick, sexy wins, is often going to be equally as detrimental as not willing to learn from your rivals at all. Creating a holistic, deep, digital organising strategy that is brave enough both to learn from people you may disagree with, to not being tempted to simply trial one-shot tricks without thinking long-term is the key to campaigning and organising success.

How on earth does survivorship bias relate to day-to-day digital organising?

If you’re anything like us, mathematics and statistical models aren’t necessarily your go-to for quiet night-in reading. That being said, concepts such as ‘survivorship bias’, which are mainly used in statistical applications, can be used with incredible efficacy to give a fresh direction to a wide range of verticals, not just those revolving around statistics and number crunching like finance. Political campaigns and movements deploy plenty of statistical analyses to measure success and make themselves more efficient. They are also prime examples of areas where theoretical mathematical concepts like survivorship bias can be used successfully.

In digital organising, there are four main ways to use survivorship bias, and we provide a few success/fail case studies to demonstrate how to apply it in the real world of organising:

• Defining your message

Understanding the way your messages have failed in the past, rather than solely replicating the ones that succeeded, is an example of how survivorship bias can be applied in practice.

Hillary Clinton’s campaign and its changing of themes and slogans is a prime example of where survivorship bias sank its marketing ship. Dismayed at the lethal and effective ‘Crooked Hillary’ terminology of her opponent, her campaign (in many ways, understandably), decided to a/b(and c/d/e/f) test around half a dozen types of slogans and themes, drawing on what had worked in the past not just for her, but for other democratic candidates in other elections. Sadly, what resulted was a diffused message which, unlike Trump’s, could not be easily slapped onto a hat because it lacked clarity and consistency. Clinton’s campaign is an example of how overly-cautious, meticulous replication of various campaigning elements led to her lacking a unique voice, and instead created a mosaic from disjointed successes of other campaigns in different contexts. Understanding where Democrats had failed in their messaging instead would have been a far more valuable endeavour.

What does this look like day-to-day? Stop focusing on the message and getting bogged down in which word works and which word doesn’t. Think big. You can semantically test your message to smithereens, but if the overarching strategy isn’t there, you’ll have a hard time being successful. Don’t A/B test before you know the purpose of that email going out in the first place. It’s better to have no email at all, than a clinically researched email with no purpose.

• Defining the methods in which you connect with your supporters

It’s not just about the words in your campaign message - it’s also about how you engage with your community to develop it.

UK politics, in particular, have been a hotspot for this with the recent post-referendum turbulence, as have more adventurous campaigns such as that of Andrew Yang in the US. With the former example, UK politicians understood that traditional campaigning and door-knocking was not storing data in a readable and digestible way. Understanding their failure in reaching undecided voters, these parties moved to the NationBuilder platform en masse to be able to more closely connect with undecided voters and separate them from strong supporters/opposers. As such, these parties were able to reach undecided voters with nuanced email campaigns and specific asks to build meaningful relationships with them.

Now, imagine you’re working for the Andrew Yang campaign. You’re using NationBuilder, which has pretty much all the functionalities you could possibly want. From fundraising, to email, to the website, to a comprehensive database. Like a dog who’s been given too big of a bone and doesn’t quite know what to do with it, it would be so easy to get bogged down in small details and frantically try to replicate what worked on other campaigns (and something which Andrew Yang hasn’t done, which was one of the keys to his meteoric rise). ‘Obama used the snowflake model? Quick, let's create 200 filters to that effect!’ The thing is, again, you might be risking riding the bike while trying to ride it. Understanding *how* those filters will interact with your fundraising efforts, and the user experience of those people on the website, is a far more important first strategic step than ticking boxes you’ve seen work on previous campaigns. Even better, a good point of departure would be to see where other similar campaigns have gone wrong, and try and figure out how you can create a holistic strategy to prevent that from happening in your case. This is something Andrew Yang is doing brilliantly, and his focus on technology and the bigger picture, rather than just being a slightly different version of the same old, replicating the same old microstrategies, is proving hugely successful.

In these two cases, understanding failed organising methodologies was pivotal for helping campaigns tighten up their digital efforts and achieve better results. Here, casting survivorship bias aside is key to surviving.

• Defining campaign successes and failures

What does a successful digital organising campaign look like anyway?

Survivorship bias can help understand previous failures and successes, and how to improve them. Macron’s ‘En Marche campaign did well to wait until the last minute to launch both the campaign and the campaign’s KPIs by learning from the patterns of failure of Fillon, Le Pen, Melenchon, and other frontrunners. The campaign’s understanding of what constitutes media failures, for example, and how that affects messaging and popular opinion, was a large part of Macron winning the presidency. 

Similarly, Obama’s campaign learned from his first campaign by harnessing the power of A/B testing and understanding when his messaging was failing. This led to him being able to increase donation conversions by 49% and sign-up conversions by 161%.

Here, being able to understand and quantify your failures rather than successes can help define your overarching KPIs, which in turn can mean you reach your goals with added ease.

• Focus on where you are going, not just where you are.

It frequently happens that when we first start talking with clients and assessing their digital needs, the focus of their requests revolves around problems they’re having with their current systems. For example, we’ve had whole first strategy sessions where a laundry list of complaints surface about the problems of the current website: a) it’s hard to edit content, b) it has a confusing structure, and c) it doesn’t relay our message well. It may well be that all those things are frustrating and causes of the website not working. But just like looking at the bullet-riddled planes that you have in front of you, you may be missing the whole picture if you don’t step back and look at the bigger picture. It’s important not to miss key factors like if the website is all static communication and no engagement, or if there is a key tool of digital which is not even being considered. We encourage our clients to not just focus on the problem they are having now but to first define where they need to go in terms of their digital, and then fill in the plan with all the opportunities to get there.

Conclusion

Survivorship bias is a double-edged sword. On one hand, it can uncover fascinating lessons from successful organising tactics. On the other hand, by only focusing on your wins, you’re omitting all your failures - the slogans that never made it, the websites that nobody visited, the Tweets that fell flat. All of which can provide useful lessons in helping you iterate your digital organising campaign. Especially these days, politics is about winning. Social media is all about going viral. But, as throughout human evolution, the fails are often what teach us the most and iterating through error and being brave enough to accept where you went wrong is one of the most powerful things anyone can do for their campaign.