Are Feminist Blog Stats Atypically Bad?

February 20th, 2014
statcheck, statistics
Occasionally Scott Alexander will get annoyed enough by a statistic that's going around Facebook to write up an in-depth piece on how it's completely missing the point. These are generally interesting and informative. The most recent one is no exception, but instead of simply talking about the flaws in the the statistics Scott goes on to say:

I have been debunking bad statistics for a long time. In medicine, in psychology, in politics. Click on the statistics tag of this blog if you don't believe me. Yet the feminist blogosphere is the only place where I consistently see things wrong by factors of several thousand get reblogged by thousands of usually very smart people without anyone ever bothering to think critically about them. Like, thirty five thousand feminists – including some who self-identify as rationalists! – saw an article that literally said a guy was more likely to get hit by a comet than get falsely accused of rape, and said "Yeah, sure, that sounds plausible."

So please permit me to keep griping just one moment longer. Do not trust anything that comes out of the feminist blogosphere. When you see something in the feminist blogosphere, your default assumption should be that it is approximately as honest as this Clymer article.

That's a strong claim, not that feminist blogs tend to present bad statistics but that they're much worse than everyone else. I've also looked into tricky statistics before and found problems (efficiency of meat, fair trade opposition) and these weren't feminist blogs. Without looking into it my guess would be that most groups making political claims present misleading statistics. Not because they're trying to mislead, but because not making mistakes is hard. Plus even if someone does notice a stat seems fishy they don't evaluate it because that seems like being a traitor to their cause, doing the work of the political adversaries.

But let's actually look into this, with as much of a survey as I can find time for. I've collected the first ten feminist blog posts I find that use statistics, and the first ten "men's rights" blogposts for the same, and then look into them to see how they compare for accuracy.

For finding feminist blog posts that use stats, Jezebel has a nice tag. I selected the first ten, excluding any that didn't present statistics. For men's rights posts went down r/MensRights and selected the first 10 self posts and links to blog posts that had positive upvote counts. The posts are:



I'll save the rest for later and just look at the first one here: Breaking News! Female Protagonists Are Still Rare in Gaming and Movies. They present a few stats:

Women make up 45% of the gaming community and 4% of the main characters of the 25 biggest games of the year.
This isn't so good. For the "gaming community" they're including people who play "casual" games online or on phones, while the "biggest games" is measured in top sellers at retail which excludes those categories. If we figure popularity by players instead of money we have the problem that many popular games don't even really have "main characters"; consider Candy Crush Saga.

(On the other hand, the primary bird in Angry Birds, Red, would have made sense as either gender but they went with male.)

Men make up 35% of the cinema audience and 84% of the protagonists of the 25 biggest movies of the year.
This stat seems reasonable; we're comparing the same categories. Better would be something like "male protagonists as a fraction of movie-views" but that seems like a lot harder to calculate and probably not that different.

A third (more or less) of the top ten and a single blockbuster comedy is by no means equal, but it is an improvement over the 2012 field, which featured only two top-ten, female driven movies (The Hunger Games and Brave). It's also light years better than the 2011 field, which featured no female-driven movies in the top ten, unless you count Breaking Dawn, which would be perhaps too optimistic. The progress may be slow, but the "bottlenecks" ... are slowly being broken up by the egalitarian forces of the internet
That 2013 (3 of 10) was better than 2012 (2 of 10), which was better than 2011 (0 of 10) is not really enough to detemine that the internet is breaking down barriers. Let's make a larger chart, looking at the last 20 years of best selling movies and counting the ones where a woman is credited first on imdb's autocomplete:

Things are definitely improving, but over a longer timescale than you'd think from 2011: 0, 2012: 2, 2013: 3. They also shifted from "top 25" counting to "top 10" counting for the quote above. What happens if we look at the whole top 25?

It looks like much of this change in the top 10 is movies breaking the top-10-barrier, not an overall increase in female-led movies. Maybe we can attribute this to the internet but it's not as much of a change it looked like at first. This sets us up for a different feminist argument: men are most of the main characters and it's not getting better. It's less positive, but still in line with Jezebel's politics.

(Data is at the end, footnote [1]. Note that this data isn't ideal; first billing can be more about popularity than role in the film. For example, Hairspray billed John Travolta (Edna) first even though Nikki Blonsky (Tracy) was clearly the main character. This probably happened several times in the data, but I don't know movies well enough to go by "main character".)

Summary: three claims, one was fine, one was conflating two different populations, one was extrapolating from too little data and overstated recent change. Even the ones that were off were far from being off by "factors of several thousand".

[1] Movie data:

year female-led movies in top 10
2013 Hunger Games: CF, Frozen, Gravity
2012 Hunger Games, Twilight: Breaking Dawn II, Brave
2011 Twilight: Breaking Dawn
2010 Alice in Wonderland, Twilight: Eclipse, Tangled
2009 Twilight: New Moon
2008 Twilight
2007 none
2006 none
2005 Narnia, King Kong
2004 none
2003 none
2002 My Big Fat Greek Wedding, Chicago
2001 none
2000 Scary Movie
1999 Runaway Bride, The Blair Witch Project
1998 none
1997 My Best Friend's Wedding
1996 Twister
1995 none
1994 none

year female-led movies in rank 11-25
2013 The Heat
2012 Snow White and the Huntsman, Prometheus
2011 The Help, Bridesmaids, Super 8
2010 Salt, Valentine's Day, Black Swan
2009 Monsters vs Aliens, The Proposal, Cloudy with a Chance of Meatballs
2008 Sex and the City, Mamma Mia, Wanted, Four Christmases
2007 Juno, Enchanted
2006 The Devil Wears Prada, The Break-Up, Dreamgirls, Scary Movie 4, Charlotte's Web
2005 Flightplan, Monster-in-Law
2004 The Village, The Grudge, Million Dollar Baby, The Princess Diaries II
2003 Freaky Friday, Scary Movie 3, How to Lose a Guy in 10 Days
2002 Lilo and Stitch, The Ring, Sweet Home Alabama, Panic Room
2001 Tomb Raider, Spy Kids, The Princess Diaries, The Others, Legally Blonde, America's Sweethearts, Save the Last Dance
2000 Erin Brockovich, Charlie's Angels, Miss Congeniality
1999 Double Jeopardy, Pokemon: The First Movie
1998 Mulan, The Rugrats Movie, Shakespeare in Love, Stepmom
1997 Scream 2, Contact, I Know What You Did Last Summer
1996 The First Wives Club, Scream, The Hunchback of Notre Dame
1995 Dangerous Minds, While You Were Sleeping, Congo
1994 The Client

Comment via: google plus, facebook

Recent posts on blogs I like:

Book Review: Outlive

Actually a book review and not a summary

via Thing of Things May 21, 2024

How bad is alcohol?

Unfortunately we landed on a pretty bad drug as a default. The post How bad is alcohol? appeared first on Otherwise.

via Otherwise May 6, 2024

Clarendon Postmortem

I posted a postmortem of a community I worked to help build, Clarendon, in Cambridge MA, over at Supernuclear.

via Home March 19, 2024

more     (via openring)