College And Critical Thinking
[epistemic status: My bias is against the current college system doing much good. I have tried not to be bogged down by this bias, but take it into account when reading my interpretations below.]
[EDIT: An earlier version of this post claimed that one paper had shown a u-shaped relationship between time spent in college and critical thinking. A commenter pointed out this was true only of a subset in two-year colleges, but not of four-year colleges or college in general – which shows the expected linear relationship. I am sorry for the error, and correcting it somewhat increases my confidence in college building critical thinking.]
Over Thanksgiving, I was discussing tulip subsidies with the pro-Bernie-Sanders faction of my family, and my uncle claimed that we needed college because “it teaches you how to think critically”.
The evidence sort of supports him, but with the usual caveats and uncertainties.
First of all, what the heck is critical thinking? Luckily, we have a very objective scientific answer: critical thinking is the ability to score highly on the Watson-Glaser Critical Thinking Appraisal. Now that we’ve answered that, we can move on to the question at hand.
Most studies on this issue are terrible because they lack control groups. That is, they measure students when they enter college, measure them again when they leave college, and find that their critical thinking ability has improved. But this could be for any number of reasons. Maybe older people generally have better critical thinking than younger people. Maybe life experience builds critical thinking. Maybe college had nothing to do with any of it. The best meta-analysis of such studies, MacMillan 1987, finds exactly this, and concludes:
Overall these studies suggest that seniors, in the main, are probably better at critical thinking than freshmen. However, since the most compelling data were gathered through weak pretest-posttest or longitudinal designs, it is difficult to separate out the effect of college from the maturational effects that occur despite college.
I like the phrase “maturation that occurs despite college”, although I don’t think they meant it in that way.
But in any case we need a better study design to conclude anything from this. There are two studies with moderately good designs, both by a guy named Pascarella. The first compares 30 college students to 17 matched non-college students and follows them up for one year. They find that while both students and non-students gain critical thinking skills over the course of the year, the college students gain 17% more, corresponding to an effect size of 0.44.
The second, larger study compares students doing college full-time to students doing college part-time, under the theory that if college is causing the effect, then a little college should cause a small effect, but lots of college should cause a big effect. They find this in the four-year college sample, and a garbled u-shaped mess in the two-year college sample. At least the four-year sample, which is what most people are interested in, looks good.
On the other hand, some other studies find less impressive effect sizes. Arum and Roska recently wrote a book on this kind of thing, Academically Adrift
College entrance to end of sophomore (ie half of college) improves critical thinking by 0.18 SD. This might have been greater in the past “Pascarella and Terenzini estimated that seniors had an 0.5 SD advantage over freshmen in the 1990s. In contrast, during the 1980s students developed their skills at twice the rate: seniors had an advantage over freshmen of one standard deviation.”
Note that we’re comparing unlike with unlike. Four years of college need not produce an effect twice as great as two years of college, any more than a space heater that increases the temperature of a room 10 degrees after being left on for one hour will increase the temperature 87600 degrees after being left on for a year. Indeed, some studies suggest that most of the gains happen in freshman year. But even so, there’s a very clear downward trend here. As usual, we have no good way of knowing if that’s caused by gradually-improving studies, gradually-improving college critical thinking training, or gradually-deteriorating colleges.
One more question: do we know the specific aspects of the college experience that cause critical-thinking gains?
Specific explicitly-advertised “critical thinking classes” don’t. Being a college that prides itself on a specific “critical thinking focus” doesn’t. This sort of thing seems very well replicated, although there are a few individual studies that don’t really interact with the rest of the literature that seem to have at least temporary positive results. Classes that you might think would teach critical thinking, like logic, or philosophy, or statistics, don’t seem to do especially well (here’s a specific in-depth discussion of philosophy). The only positive result anyone’s been able to find is that “liberal arts” (here viewed as a broad category including science and mathematics) seems to do better than occupational skills, and “education, social work, communications, and business” seem to do worse.
In terms of broader factors, – one study finds that quantifiable college experiences explain “between 7 and 17%” of the variability in first-year critical thinking gains (other sources say classes explain 2.5% and extracurriculars 2.9%). Studying a lot seems to help. So does reading unassigned books. Aside from that, the biggest finding is kind of concerning:
Students who characterized their relationships with other students as “competitive, uninvolved…alienated” were more likely to show gains in critical thinking than were students who portrayed their peer relations as “friendly, supportive, or a sense of belonging” The data in this study do not permit confident explanation of this relation, but one might speculate that a sense of participation in a friendly, supportive peer environment may require a partial suspension of one’s critical thinking skills.
…wow. I was going to say something like “students busy spending time with their friends have less time to learn stuff”, but your cynical awful explanation works too.
So what’s the big picture?
Well, we know that people will gain critical thinking skills during the four years from age 18 to age 22. We have an small study that finds college helps a little with this process and a larger study that shows dose-dependent effects of college. We have some hard-to-compare effect sizes ranging from 0.18/2-years to 0.44/year. That’s modest but appreciable, and it’s probably at least somewhat real.
But those of you who went to my talk last week hopefully know what my next question will be: how long does this last?
For example, we know that parents’ personalities have all sorts of interesting effects on their children while those children are living with their parents. We also know that as soon as children leave their parents, those effects go down to near zero. What people tried to interpret as some deep fact about development was actually just a reflection of the environment that those children were in. Likewise, preschool makes children do much better in kindergarten, but by third grade the preschool-educated kids are doing the same or worse as the others. It’s not fundamentally altering their developmental trajectory, it’s just creating an short-term effect.
Every one of the studies I’m citing here was done in freshman or sophomore year (one study cited another done in senior year, but I couldn’t find it directly). No one has ever looked at students who have been out of college a year – let alone out of college thirty years – to see if the effect continues. I would bet that it doesn’t.
Until somebody checks, enjoy your opportunity to tell people that the evidence backs college building critical thinking skills.