Streetlight Psychology

Recent article on Vox: Do Violent Video Games Actually Make People More Violent? It’s a good and even-handed article which concludes:

The short answer [is] we don’t really know. Some studies have indeed found that, in lab settings, people can become more aggressive after playing violent games. Some have also found that people who play violent games are more likely to commit violent acts in real life. But it’d be just as easy to conclude that inherently violent people are simply drawn to violent games — and indeed, several studies have come to that conclusion. Moreover, not all observational studies have found any correlation between gaming and violence. Perhaps most importantly, there’s been no surge in violence among youth over the past few decades in gaming countries to accompany the rising popularity of violent games. On the whole, the evidence is decidedly mixed.

I would, however, like to make one important point I think they left out, which also happens to be a very broad criticism of entire subfields of psychology.

They say that “lab experiments have shown violent games can increase aggression”, citing a meta-analysis of ninety-eight studies but using as their specific example a recent Italian study. In this experiment, a bunch of high-schoolers were randomly assigned to play Grand Theft Auto or some less violent game. Then they were asked to participate in an experiment where they got to blast loud noises at a partner, which seems like a pretty violent thing to do. The kids who had played Grand Theft Auto were more likely to give their partner louder, more violent blasts. A nice, elegant result very typical of results in this field.

Vox’s analysis:

There are a few caveats to keep in mind when considering these studies. One is that they use aggression and self-control as proxies for real-world violence, because researchers can’t actually allow violence to occur in a lab. The idea is that, in increasing aggressiveness, these games make it more likely that someone considering violence will be pushed over the edge and actually commit it. But there’s a huge difference between blasting someone with a loud noise, or scoring higher on a questionnaire intended to measure aggressiveness, and actually resorting to violence in the real world.

I think this is insufficiently skeptical.

Suppose I have a different hypothesis. I hypothesize that sad video games cause depression. It seems prima facie plausible. We’re having a lot of trouble with child and adolescent depression nowadays. And do you remember that thing that happened in Final Fantasy VII, with that one character (no spoilers)? Really sad. Maybe there are lots of video games like that, where you see your favorite characters or people you really like die or suffer terrible travails. In the past, maybe only a couple of people would lose people close to them, but through the vicarious experience of video games we’ve all experienced seeing our friends “die” dozens of times by the time we graduate high school. Hence, adolescent depression. Right?

We propose an experiment to test this. Fifty Italian teenagers play the appropriate scene in Final Fantasy VII; a control group of another fifty play something neutral like Tetris or whatever. Afterwards, we measure sadness in some way. For example, maybe we make them watch a comedy program and record how many times they laugh; if they don’t laugh much, we assume they’re sad. Or maybe we just give them a self-rating questionnaire. Whatever.

I predict that this study will easily find that the intervention group is sadder than the control group. I also predict that the original hypothesis, that sad video games cause depression, remains stupid.

The difference seems to be that between a temporary state of sad mood, and a long-term alteration of personality that amplifies a sadness “trait” (or more specifically depression, which sometimes presents similar to sadness but may have totally different causes). While it’s easy to imagine that sad video games affect the first, it’s much harder to believe they can do the second, and act in an additive way to make you a very sad person ten years down the line.

But the same should apply to the violence studies! Yeah, okay, you’re priming violence, you’ve got people in a high-arousal, aggressive state, they’ll probably be more violent until the effect wears off. But there’s a long way between that and saying that their personality becomes more and more violent until they shoot up a school. Especially since, as far as I know, no mass murderer is known to have played video games immediately before their crime, the period in which these kinds of effects might be relevant.

There is an old story about a drunk who loses his keys:

A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost his keys and they both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost them here, and the drunk replies, no, that he lost them in the park. The policeman asks why he is searching here, and the drunk replies, “This is where the light is.”

This has since been dubbed the “streetlight effect”, and these studies seem to be an example.

The policeman asks the psychologists what they’re doing. The psychologists say “Studying propensity to violence in the few minutes after someone plays a video game.” The policeman asks “And this is going to reveal important principles about why people are violent in real life?” The psychologists say “No, but it’s really easy to study.”

I want to stop for a second and say that this is not an argument that the field of videogamology is hopelessly flawed or that videogames cannot cause violence. Most of the videogamologists I have read freely acknowledge this problem. That they continue to do these kinds of studies time after time doesn’t surprise me – I’m working on a study right now for work which I don’t think will be hugely contributory to the Sum Of Human Knowledge, because I’m supposed to do something and this was a moderately interesting and do-able thing that came up. A society that rewards researchers per study, like one that rewards factories per kilogram of machinery, will get what it pays for. The videogamologists have taken this in stride and along with these sorts of laboratory studies are doing their best to do longitudinal studies adjusted for various confounders some of which reveal the same pattern. I haven’t studied the field and I have no opinion on it beyond what’s in the Vox article. I just want to make sure people don’t pay too much attention to the streetlight studies.

Which, by the way, are a problem far beyond videogamology.

Here’s an article called Arabs as terrorists: Effects of stereotypes within violent contexts on attitudes, perceptions, and affect. Participants consume some media with depictions of terrorist Arabs in it. Then they are asked what they think of Arabs. They are more likely than the control group to say they think of Arabs as terrorists. Therefore, media with stereotypical portrayals causes people to stereotype groups in various ways.

Okay. How long did the researchers give between the media in question and the survey? Five minutes? Ten? And we’re supposed to believe that this gives us important information about whether, if one of these people throws a brick at a mosque in ten years, it’s going to be because of all of the anti-Arabic media he’s been gradually consuming up until that time?

In fact, let me make a stronger claim. The “success” of this experiment actually disproves its hypothesis. Assume that everyone in both the experimental and control groups have probably seen hundreds of depictions of Arabs as terrorists throughout their lives. What the experiment finds is that one recent depiction of terrorists is enough to overwhelm all of those hundreds of past depictions, such that a clear and obvious difference shows up between the control group (who have seen hundreds of media depictions of terrorist Arabs throughout their entire lives) and the experimental group (who have seen hundreds of media depictions of terrorist Arabs throughout their entire lives, plus one extra one five minutes ago). What we’re actually finding is that any media depiction effect decays really really rapidly, so much so that people who have been told their entire lives that Arabs are terrorists can be used as the control group in an Arabs/terrorism experiment without anyone batting an eye.

Of course, it’s possible that there are two different effects – a very weak long-term effect that builds up with many exposures over time and existed in all of the study participants, and a very strong short-term effect you get when you just saw a documentary on Osama five minutes before someone asked your opinion. But in that case, you shouldn’t go proving the existence of the second effect and then pretending you’ve learned anything at all about the first.

A big chunk of the science in the area of prejudice and racism is exactly like these Arab studies, unfortunately.

Let me bring up one more area in which I think this effect distorts research.

Judith Harris’ book The Nurture Assumption makes the bold claim that most differences in how parents bring up their children don’t really change the children’s long-term outcomes that much. This obviously contradicts a wide body of research, and she goes at length into why she thinks the wide body of research is wrong.

I don’t have the book here, so I’m working from memory and it’s going to be more a general form than a specific example, but – suppose that some scientists find mothers who yell at their kids, and mothers who don’t yell at their kids, and find that the children of the yellers act out more and cause more trouble. You can conclude – and a lot of psychologists did – that yelling changes kids’ personalities and makes them more aggressive or rebellious or have some other trait desire to act out.

But other studies show that when the kids are in school, or after they’re taken away by social services, or twenty years later when they’re grown up, there’s no difference between the two groups. Why? Harris suggests that what we’re actually seeing is children responding to a dynamic. The presence of yelly parents makes kids act out, in the same way that the presence of cold makes kids put on a sweater. Cold doesn’t permanently change children’s thermoregulation and cause them to be more likely to wear a sweater twenty years later, it just causes an immediate reaction. Likewise, yelling doesn’t change children’s traits and characteristics and cause them to be more rebellious twenty years later, it just causes an immediate reaction. Remove stimulus, remove response. And this is all my vague half-remembered analogy to something I think was in the book, but I think it’s an important distinction.

Situations can cause immediate responses to those situations, or long-term changes in the way people think. You can’t just demonstrate the former and assume you’ve proven the latter.