Beware the Bimodal

Missouri incumbent Senator Claire McCaskill thinks Missourians should vote for her because she is in the middle, #50, in a ranking of Senators based on how liberal/conservative they are.

Be careful of the political spin. This ranking doesn’t mean that she’s a ‘centrist’, as she likes to claim. This is the group that does the rankings and here is how it is calculated.

They grade on a curve. Think of a test in high school. If her and fellow Democrat Senators all scored 85 or higher there’s a couple of ways to look at the scores.

First, we might say they all scored well and deserve As and Bs.

Second, we could rank them by their scores and we’d find Claire ranked #50 with a score of 85.

Which method do you think is a better indicator of her true performance? The #50 ranking or the score of 85? I’d argue that the score of 85 is better.

By no means would we say that she’s closer in ability to a person that scores 50 or lower. But that’s exactly what she wants you to believe with her touting of the #50 ranking.

In statistics, a test score distribution where you have a clump of people at 85 and above and another clump of people scoring 40 and below, with very few in between, is a bimodal distribution (or two modes).

In the Senate, you are pretty much either liberal or conservative. Saying you are the least liberal doesn’t mean much, except that you are comfortable misleading people.

While I don’t agree with everything about all conservatives, I don’t very often see a conservative try to mislead folks into believing they aren’t conservative. It should be distressing to liberals when their candidates try to leverage the general public’s weak understanding of statistics to pretend they are not liberal.

 

Advertisements

I’m skeptical of the chocolate study

While my own biases really, really want this one to be true, I am just as skeptical of the study that says: Eating Lots of Chocolate Helps People Stay Thin,” as I am about the red meat study.

I have many of the same concerns. The main one is that the folks who eat a lot of chocolate may simply lead healthier lives.

Other concerns I have about the study include:

  • Just like in the red meat study, is that the amount of chocolate eaten is determined by self-reporting, which is not reliable. For example, unhealthy people may simply have under reported their chocolate intake.
  • Small sample size. The study contained 1,000 people. That’s small.
  • The average age was 57 and we don’t know how long these folks have had their chocolate habit. In 57 years, there could be many more things that influence health than chocolate consumption.

So, it’s possible that this research shows that healthier people in this group of 1,000 said they eat more chocolate than the less healthy people, which really doesn’t mean much.

I will, however, give the researcher credit. She wasn’t nearly as pompous about her findings as the red meat researcher.  When asked if we should all eat more chocolate she responded:

Our findings – that more frequent chocolate intake is linked to lower BMI – are intriguing, however it is not a siren call to go out and eat 20 pounds of chocolate a day.

Though, I’ll point out that while linked is a better word than causes, I think the statistically-challenged will not appreciate that fine point and will likely treat the two words as synonyms.

Moneyball

I watched the movie Moneyball this week and enjoyed it. I’ve heard of the book and avoided reading it due to my own biases.

I’m a skeptic of statistical analysis, which often (even in the movie) gets confused with science.  My statistics-loving friends raved about how the book showed just how valid and effective the use of statistics is, even in a sport.

I see this confusion and misapplication of statistics as science in everything from how we run our schools, climatology, economics, fitness and diets and how we run our businesses and other organizations.  I’ve observed enough attempts at “scientific management” in my career to know that the use of it does not guarantee success — and sometimes can make things worse off (it certainly didn’t help in the housing crisis).

But, based on what I saw in the movie, this isn’t quite the case of baseball science that many people believe it is.  I realize movies simplify the story, but what I saw in the movie is more in line with what I have seen to be effective in real life:  focusing on meaningful facts over biases.

It wasn’t the use of statistics that improved the performance of the team.  Rather, it was the use of meaningful performance and output measures to overcome deep-seated biases in the coaching the recruiting staff.

This is demonstrated in one scene of the movie when the team’s talent scouts are discussing potential new players to add to the team.   This is a good-looking kid.  He has a nice swing.  Other teams like him. I like this kid.

I’ve seen this in real life all too often.  I’ve seen business programs die and good talent passed over for promotion because of similar biases.  Usually the bias is as simple as: That’s not what I’d do or That person doesn’t do the things the way I do them.  Very rarely is the true performance of the project or person even discussed.

The key insight of the statistician in Moneyball wasn’t the use of statistics, or sabermetrics, per se, but in using meaningful output measures to trump the biases. Baseball fans want wins. Wins come from scoring and scoring comes from getting on base. Good defense is a must, but not quite as important as scoring. So, instead of worrying about whether this was a good-looking kid, they’d worry more about whether he could get on base.  Facts trumped biases.

Myth Meet Fact

Don Boudreaux of Cafe Hayek does an excellent job in this post/letter to the editor of the Washington Times, A Hypothesis Easily Tested, Daily in exposing a long running myth about the male/female pay gap.

The long-running myth is that gender discrimination is the or a major cause of the actual gender pay gap — why women on average are paid less than men.

This myth has been around a long-time.  I first remember seeing it decades ago.  Unfortunately, I don’t remember such a common sense suggestion as Don’s.  It amazes me that it’s taken this long for someone to think of this approach and I’m disappointed that I didn’t!

Don suggests that if the discrimination hypothesis is real then businesses should exploit this valuable finding and gain a competitive cost advantage by employing only women at lower pay than equally productive men.

From Don’s letter:

But to those persons who believe that women are indeed consistently underpaid, boy do I have a deal for you!  Start your own firms and hire only women.  If it’s true that women are consistently underpaid, you’ll be able to hire outstanding employees by paying them more than the relative pittances they currently earn, while you still profit handsomely from employing them.

On Success

If I need a statistician to tell me whether something worked or not, it didn’t work.

That’s a sentence that I think I came up with, but I wouldn’t doubt that I lifted it off someone and can’t remember who. Until proven otherwise, I’ll take credit.

This sentence came from my years working with statisticians employed by managers of for-profit companies to find out if something they tried worked or not.

My sentence ruffles the feathers of statisticians.  Many statisticians understand the limitations of their trade, but many  don’t.  Many non-statisticians don’t either.  When managers use statisticians to determine the effectiveness of a project, beware.

To me, the project needs to offer what I call a clear advantage.  I should be able to look at the results and clearly see that what I tried was better.

Continue reading