Survivor Bias Makes Fools of Us All

Survivorship bias or survival bias is the logical error of concentrating on the people or things that "survived" some process and inadvertently overlooking those that did not because of their lack of visibility.

We’ve all seen the Facebook memes:

“Share if you rode in the back of a truck as a kid and didn’t die!”

“Like if you played in a spent nuclear fuel pool as a kid and turned out fine!”

“Who else rode their bike without a helmet as a kid?”

(Okay, I may have made one of those up.  And yet it’s way safer than you probably think.)

The point is, these memes are classic examples of survivor bias, because anyone who DID die, or didn’t turn out fine, or had a massive head injury because of a bike accident (a) isn’t invited to respond; and (b) couldn’t actually respond anyways on account of being, y’know, dead.  The results from such a survey (which is unscientific in approximately 947 ways, anyway) are, of course, worthless.

But then, we all know that this isn’t really about a survey; it’s a passive-aggressive way to trying to show that a particular activity is just fine, and we’re all swotty little nancy boys for caring or trying to make people stop doing that.

This is a stupid attitude.

Seemingly random story time: I work with the FAA on a near-daily basis as part of my job at a rather large aerospace company formerly based in the Puget Sound region.  (You’ve probably never heard of it.)  The FAA, as with many governmental agencies, has a lot of regulations that our products are required to meet before we can sell them and our customers can fly them, and these are constantly being updated to be more stringent, or more effective (not always the same thing).  The FAA likes to say that amending a regulation doesn’t mean it wasn’t safe before, just that it’s more safe now.  And on the whole, this is true.

I’d like to draw your attention to pages 17 and 18 of this file. The accident rate on an airplane is ridiculously low; so low that the last several years have to be broken out into their own scale to see the year-to-year changes.  It would be easy to simply say that it’s good enough.  We’ve done what we can, the death rate from flying is really low, and let’s celebrate our success!  Except we don’t; we keep trying to find ways to make things even safer.  Sure, there may come a point of diminishing returns; we can’t make things so expensive to produce, in the name of making them safer, that people can’t afford to fly.  But we’re not there yet.

I mean, look at page 18.  It’s twice as likely that a crash will damage the airplane enough to write off the whole airplane than that the crash will kill someone.  Look at the incident at Heathrow a few years ago.  That was the first hull loss of a 777 (over 10 years after being introduced), and there still wasn’t a single fatality.  This is how ridiculously safe it is to fly now.  But it’s not perfectly safe, so we keep trying to improve.

Did plenty of people fly without dying before those regulations?  Of course.  But far more people died then than now, and we decided that it wasn’t acceptable.  We didn't say "I flew without all those regulations and I was just fine."  We said, "Hey, maybe we should fix things so that all those crashes that are happening don't, even though most people make it just fine."

Most people mock anti-vaxxers because “Share if you had measles and nothing bad happened” would be widely recognized as a terrible meme, and that's what most of their arguments boil down to.  (That’s obviously not the entire reason, but you get my point.)  We all know (or should) how bad measles can be, and we take steps to make sure it doesn’t happen to our kids or to other kids.

So what’s the difference?  Why do some activities get the survivor bias treatment (“I survived, so did other people I know, it’s clearly not that bad, why worry about it?”) and others don’t?  Well, like most things, it probably comes down to a risk analysis and exposure.  What feels normal?  What are we used to?  What do we find scary?

Sadly, most people are crap at risk analysis, in that we’ll worry far more about unlikely but drastic issues than we will the far more common dangers (flying vs. driving, for example).  I’m sure the second meme up there scared most of you far more than the others, even though, barring diving to the bottom of the pool down next to the fuel rods, it’s literally just as safe as swimming in a regular pool, which I’m sure doesn’t scare you at all.

So, yeah.  It’s a stupid attitude, which disregards everyone who was affected or could be affected in favor of the fact that you weren’t affected.

It’s not just physical activities that this applies to, though.  It’s also about the way we talk, and the way we treat people.

“Political correctness” gets a lot of play these days.  And while I suppose it’s possible to go overboard, most of the anti-PC complaining I see tends to boil down to, “I’m mad I can’t just say and do whatever I want any more without thinking about other people.”

And it’s true; you can’t.  Ethnic slurs?  No longer acceptable.  Calling something “gay” or “retarded” as an insult?  Acceptable only at the fringes of society; most people certainly aren’t going to accept it.  Casual misogyny?  No longer just written off by most people as "locker room talk" or "guys being guys".  God knows I certainly have a hard time coming up with names and insults in my Poor Driving posts (of which there will be more, don’t worry), because so many of them are really based on demeaning a particular group of people.

For example, "cocksucker" as an insult implies being gay is something to be ashamed of.  "Bitch" is a gendered insult.  "Pussy" or "twat" imply that being female is to be less than.  I know some people will think it's absurd to take that sort of thing into consideration, but that's my point.  Just because you're used to being able to say what you want without thought doesn't mean you should.  That's not "PC", that's just basic decent human respect.

Just because something used to be okay doesn’t mean it should be, or even should have been.  Just because someone said to you “You’re so gay” and you “turned out fine”, that doesn’t mean someone else did.

This is the sort of attitude that legitimizes bullying.  This is the sort of attitude that drives kids – KIDS – to suicide because they can’t deal with it.  This is the sort of attitude that leads to catcalling on the street and rape culture.  This is the sort of attitude that STILL stigmatizes mental illnesses as weakness, despite the mountains of evidence that (a) it often has a physiological component behind it, making it just as much a physical illness (which is far more socially acceptable), and (b) why did you even need a favorable comparison to physical illness because they’re both real things that can be wrong with you and one is no more “legitimate” than the other just because we understand it better.

Gender roles.  Sterotypes.  Being a “man”.  Being a “lady”.  Being "strong", not "weak", whatever you define those terms to mean.  (Why do you think I underlined “running away” in the last post?)  Just because people dealt with these things growing up doesn’t mean they’re right.  It doesn’t mean they turned out fine.  It doesn’t mean it’s right to push them onto your own kids, or other peoples’ kids.  Just because you survived something doesn’t mean everyone does.

This is a stupid, selfish attitude.  It's caring only about yourself and not others, and that's not how society works.  It's not!  Societies don't work if people don't take care of each other; that's the entire point of a society.  Don't believe me?  Even David Brooks gets it, in a column I'm still not quite sure I believe he really wrote.

There are SO MANY THINGS that we, as people, as societies, as human beings, now realize were REALLY STUPID WHY DID WE EVER THINK THEY WERE GOOD IDEAS.  And people at the time trotted out the survivor bias then, too.  And it was wrong then, too.

We learn.  We improve.  We do better.  Just because we did something in the past and lived to tell doesn’t mean we can or should continue to do it in the future.

Because it’s hard to hear from those who won’t, or can’t, tell how they didn’t.