Bechdel vs. Science
The gist of the video is that the majority of best picture nominees for this year’s Oscars fail the Bechdel test. For those new to the idea, the Bechdel test is a way of determining whether or not a movie contains meaningful female characters. It was initially presented at a tongue-in-cheek shot at the gender bias in movies in the comic strip Dykes to Watch Out For. To pass the test, a movie has to have three things: (1)Two [named] female characters that (2) talk to each other (3)about something other than a man.
However, being mostly a joke rather than a carefully constructed, double-blinded study, it lacks a certain amount of critical credibility.
Ms. Sarkeesian seems to have explicitly missed an important failing of the test. Starting at 7:45 in her video she says:
“In response to the Bechdel test I’m often asked ‘Well, what about the reverse?’ Why isn’t there a test to determine whether two men talk to each other about something other than a woman? The answer to that is simple. The test is meant to indicate a problem and there isn’t a problem with a lack of men interacting with one another.”
That right there, that is bad science. And we at Mad Art Lab cannot abide bad art science. Why is it bad science? Let us break it down.
The Bechdel test only checks for female interaction. Failing the Bechdel test is meant to indicate a lack of female character presence. However, it does not necessarily indicate an institutional bias. Simply failing the Bechdel test does not indicate anything except that a movie has failed to meet that standard. However, buried in the discussion of the test is the unstated premise that most movies pass the male version of the test or the ‘Reverse Bechdel.’ This is not logically necessary. It is possible for a movie to pass both the male and female versions of the test, just as it is possible for a film to fail both. For example, March of the Penguins utterly fails both tests simply for there being no dialogue between any characters and Underworld:Awakening manages to pass on both counts. There are lots of ways for a movie to fail the test in either direction and passing the test or its reverse at all isn’t actually that easy.
Claiming that it is a problem that only two Best Picture nominees this year pass the test is equivalent to claiming that your shampoo will increase shininess by 27% or your mouthwash makes your breath twice as fresh. They’re meaningless without some basis for comparison. There needs to be some baseline or benchmark. Therefore, rather than being irrelevant, running the Reverse Bechdel test is necessary in order to have something meaningful against which we can compare. It is technically possible for there to be even fewer films that pass the Reverse Bechdel test. It is pointless to get angry about how few films pass the Bechdel test until a basis for comparison has been established.
Let’s do some science now. We’re going to do the same thing as Feminist Frequency tried to do, but with a bit more scientific rigor and a lot more academic pretention.
Hypothesis: There is a male character bias in films being nominated for Best Picture Academy Awards.
Test Method: Apply the Bechdel method to both male and female characters in Best Picture Nominees and compare the ratio of films that pass each test.
Sample: 2011 Academy Award Best Picture Nominees
Bechdel – Two Pass easily two ambiguously so. The Descendants and The Help both passed the test easily while Hugo and Midnight in Paris squeaked through with one or two sentence exchanges.
Reverse Bechdel - The only film that manages to clearly fail the Reverse Bechdel Test is The Help. The other eight have the requisite conversation between males (some were young boys) about something besides women.
Well that’s pretty definitive. Eight to two, pitchfork and torch time right? Not quite. Unfortunately we have only taken a very small sample. From these data, we can only really comment on the male-female role balance in this year’s Best Picture nominees. It could easily be an unusual year, an outlier. Before we have the mathematical grounding for making any conclusions regarding trends or patterns, we’d have to do this for several years. The only conclusion that I can confidently make at this point is that this was a pretty rough year for female characters in Oscar-bait films. However, the results of this small study do indicate that there is reason for further investigation and have not given us any cause to reject our hypothesis.
Now that I’ve ranted for this long, I think it’s worth mentioning that I believe that there is a terrible imbalance in Hollywood cinema along the gender lines. But I believe that from personal recollection and through whatever biases I carry. I do not know it to be true. We can only know by actually checking.
Bechdell Test and Minorities
The video above also discusses the Bechdel test as applied to minorities. This, too lacks a bit of mathematical insight.
The Bechdel test is meaningful when comparing male to female characters because it is easy to argue that there should be a pretty clear 50/50 split. Almost exactly half of the population is female so it follows that half of the characters should be as well and consequently we could expect roughly a quarter of non-romantic interactions to be between women. One can argue pretty easily that the same number of movies should pass the Bechdel as the Reverse Bechdel.
This, however, is not true for minority groups. As a group gets smaller, it becomes exponentially unlikely that they will pass their version of the bechdel test. Let’s use African-Americans as an example as that group was directly discussed in the video.
According to the 2010 Census of the United States, roughly 12.4% of Americans identify as black. That is convenient as it means about one in eight people in the US are black and that makes for easy math.
I would argue that to be represented totally fairly, without racial bias, that same proportion should show up in cinema. Thus for every eight movie characters, we can expect one to be black. Therefore in a film with a small cast it isn’t unlikely that there would be no black characters at all. Furthermore, having two named black characters wouldn’t be an expectation until a movie had a cast of around sixteen. In the rare case that a film hits a cast that large, what are the odds that any two characters will have a meaningful exchange within it?
So the Bechdel test is pretty useless for evaluating character presence of minority groups. The group can actually be well over-represented before they’ll start passing the test at all. So if you actually want to evaluate the representation of a particular group, you will need to come up with a different test. I would lean towards something that can be compared against actual presence in the population.
What is the proportion of homosexual protagonists in Oscar nominated films?
What is the proportion of black characters that survive to the end of horror movies?
What is there a trend in the proportion of inter-racial relationships in film in the past 20 years?
These are all things that can be tested and can give mathematically meaningful data. Unfortunately, what cannot be quantified is how fairly any group is being represented. Presence alone does not imply goodness. That discussion involves value judgements and interpretation which lie outside of the safety of basic statistics and therefore are well beyond the scope of this article.
A Final Note
I have shredded Ms. Sarkeesian’s arguments a fair bit here, but it is important to point out that I haven’t refuted her conclusions, just her methods. She may have missed a few mathematical nuances, but that doesn’t mean she’s wrong.
Special thanks to Amy, Smashley, Anne, Maria, Melissa, Donna and Cloe for helping me test the films.