Are The Amish Unhappy? Super Happy? Just Meh?

I.

Recently on Marginal Revolution: Are the Amish unhappy?

The average levels of life satisfaction [among the Amish] was 4.4; just above the neutral point…the Amish fall lower than members of many other groups. In a study of more than 13 thousand college students from 31 nations, for example, only students from Kenya (whose average life satisfaction was 4.0) scored lower than the Amish (Diener & Diener, 1995).

Sounds like Amish people are quite unhappy. This came as a surprise to me, since I’d heard from Jonah Lehrer and Business Insider that the average Amish person is as happy as the average non-Amish billionaire, proving once and for all that community and old-fashioned values are more important than money:

As an illustration of the striking disconnect between money and happiness, the average life satisfaction of Forbes magazine’s 400 richest Americans was 5.8 on a 7-point scale. Yet the average life satisfaction of the Pennsylvania Amish is also 5.8, despite the fact that their average annual salary is several billion dollars lower.

I actually care about getting this one right. There’s a lot of discussion over whether modern society produces ennui, meaninglessness, atomization, etc – and whether our material wealth has really brought us happiness. The data tend to support a story where more modern and developed countries are happier, but not without some ambiguity and contradiction (a few Latin American countries seem to do better than much richer European ones). But there’s always a concern that the least-developed areas today – like sub-Saharan Africa – are places that have absorbed the worst parts of modernity – like totalitarianism, pollution, and slums – but not the good parts like not-dying-of-cholera. The Amish are about as close as we can get to surveying 1700s-Europe. If we can figure out how happy they are, maybe it would tell us something new about the good life.

Unfortunately, we can’t. This field is full of conflicting data, shifting methods, unreplicated surveys, and – and I didn’t even realize this was a problem it was possible for a field to have – is super-confusing because everyone involved is named Diener. I tried to get the above-cited Diener & Diener 1995, but carelessly bought Diener, Diener & Diener 1995 instead. Cowen’s source is a book by Robert Biswas-Diener, who is comparing a study by Diener & Diener to a study by Biswas-Diener, Vittersø, & Diener, the last of which was do-able only because:

It was both coincidental, and helpful, that my surname — Diener — is also a relatively common Amish surname. This curious point of contact allowed me to introduce myself and my project.

Sure. Whatever.

The Marginal Revolution excerpt comes from Biswas-Diener, Vittersø, & Diener (after this: BDVD). This study was part of the authors’ project to prove that most people are happy. In a previous paper, they had determined that most people in modern societies are happy; in this one, they look at three different traditional societies (Amish, Inughuit Eskimos, and Maasai) to see if they are mostly happy was well. On a 1 – 7 scale, they find:

They don’t formally compare these to the modern societies numbers in the paper. The Biswas-Diener book linked by MR does compare them, finding the Amish are lower than every modern society except Kenya, but there are two important caveats.

First, the Biswas-Diener book compares the survey of Amish (mean age 44) to the Diener & Diener survey of college students in modern society. If college students are happier than 44-year-olds (they are), that’s a potential confounder.

Second, the book notes that the Amish’s self-reported total happiness is lower than their self-reported happiness with any individual facet of their lives. Just look at the table above – their romantic life is a 6.1, their health is a 5.7, their attractiveness is a 5.1 (those beards, right?) – but the two totals, self-satisfaction and life-satisfaction – are 4.2 and 4.4 respectively. It seems like they’re averaging a bunch of numbers and getting an average lower than any of the individual inputs. This is especially bad since modern people tend to do the opposite; report an average happiness higher than their happiness with any individual part of their lives. Biswas-Diener guesses that modern people like to present themselves well (the “have a Facebook feed full of spectacular parties and meticulously-prepared plates of food even when your life is falling apart” effect), and traditional societies are more likely to value humility and treat pride as a sin. As far as I know, there are no studies that have ever measured a non-Amish society using this exact breakdown.

Third, as far as I can tell, the Diener & Diener paper doesn’t actually show Kenyans having the lowest life satisfaction, or Kenyans having a life satisfaction of 4.0:

This is the male table. There’s another one for women, but it’s very similar, and neither the male table, the female table, nor the average of the two tables matches the claim that Kenya is 4.0 or that nobody else is less than 4.4. My guess is all the Dieners share preliminary data with each other, and Biswas-Diener is going off some older or unpublished version of this, but I’m not sure and I might just be missing something.

The public version of Diener & Diener does generally backs up the claim that 4.4 is kind of on the lowish side. But on the other hand, the Maasai number of 5.4 would be the highest one on the whole chart (tied with Finland), which would suggest there’s no clear traditional vs. modern society dichotomy.

So the result of this comparison seems to be “The Amish are above neutral happiness, but less happy than almost any modern society. On the other hand, the Maasai are more happy than almost any modern society. However, a lot of this could be about self-presentation, and the data for modern societies are kind of unclear.”

II.

What about the “Amish are as happy as billionaires” claim?

The billionaire numbers come from Diener’s Happiness Of The Very Wealthy and seem to check out. It finds billionaires are happier, though not vastly happier, than everyone else. Billionaires have an average happiness of 5.8 – remember, the highest national sample above was Finland at 5.4.

This time, the Amish numbers come from Diener and Seligman’s Beyond Money: Toward An Economy Of Well-Being, which presents this graph:

It doesn’t explain where the table comes from, but the use of the same three traditional societies (Maasai, Inughuit, and Amish) suggest the BDVD paper above (which shares one of its Dieners with the Beyond Money paper). But its numbers for all three groups are very different, and its headline result – the one about Pennsylvania Amish as happy as billionaires – isn’t in the BDVD paper at all and has no citation.

The only clue to this discrepancy is that the Beyond Money paper cites the BDVD paper as “manuscript submitted for publication”. Perhaps the peer reviewers made comments which caused BDVD to drop their Pennsylvania Amish result and analyze some of the other results differently? In any case, since the Amish = billionaires data seems to have been quietly dropped by the authors, we probably shouldn’t put much stock in it.

Does this mean we should default to the “Amish less happy than anyone else except Kenyans” data? I say no. If the study authors got data consistent with “Amish are as happy as billionaires”, and later on it got changed to “Amish less happy than anyone”, plus they changed the Kenyan stuff around too, then I really don’t care about the “correction”, this research isn’t rigorous enough, or fixed enough, to convince me of anything.

All of this is from long before the replication crisis started improving methodologies, and I don’t trust it enough to consider it worth trying to smush everything together into a coherent whole. Sweep the billionaires/Kenyans issue under the rug, and there are still too many questions. Is there really a gulf between Pennsylvania Amish and Illinois Amish as vast as that between Swedes and Calcutta slum-dwellers? Are the Maasai really so much happier than modern societies, even as the Amish are so much less happy? Is there really an entire scientific field where everyone is named “Diener”? We just don’t know.