SSC Survey: Scattered Negative Results

Traffic to this blog is declining. I need to act decisively to draw people back. Write something so interesting it can’t help but go viral. I’m going to write about…negative results from the perception questions on last year’s survey.

The last SSC survey had a lot of optical illusions and visual riddles. I had hoped to expand on some of the work in Why Are Transgender People Immune To Optical Illusions and Can We Link Perception And Cognition? This post is a very brief summary of results and, basically, an admission of failure. While I was able to replicate the same suggestive results as in the last survey, I was unable to expand on them, strengthen them, or really turn them into any kind of interesting framework.

I was able to weakly replicate the headline result from Why Are Transgender People Immune To Optical Illusions: transgender status still correlated with all three mask illusions, and with the average of all three mask illusions, but very weakly: r = -0.04, p = 0.001. This was true even when I excluded everyone who took place in last year’s survey, providing an independent confirmation of the result. But with correlations this low, it’s hard to get too excited.

I was also able to weakly replicate the headline result from Can We Link Perception And Cognition?. I haphazardly gave people a “weirdness score” based on them having more mental illnesses, more unusual political opinions, and more minority sexual/gender identities (without looking at their illusion results). People with higher weirdness scores consistently had more ambiguity-tolerant results on illusions, with correlations around r = 0.05 for most tests. They also had notably higher average Tolerance of Uncertainty Test scores. But none of these results were very striking and there was minimal individual structure in them. If I was going to take this further I would have come up with a more principled definition of weirdness, but at this point it doesn’t seem worth it.

What do I mean by saying these results are weak and lack internal structure? To give an example: the last survey focused on a single optical illusion, the Hollow Mask. This survey used three different versions of the Hollow Mask in the hopes of removing noise and getting a higher-fidelity mask perception signal. This didn’t work at all. The correlations between the three masks were very low. For example, there was only an r = 0.09 correlation between being able to see the second mask illusion and the third mask illusion. While all correlations were significant, it doesn’t seem fair to conceptualize them as testing the same perceptual function.

Given that I couldn’t even get different versions of the same illusion to line up, you can guess that I didn’t get much correlation between different illusions. For example, the Tables Illusion correlated with average score on the Mask Illusion at about r = 0.06, p = 0.001. I was also able to replicate the correlation in the literature between autism and the Tables Illusion, r = 0.03 p = 0.01, but again this was very small.

Were there any actually large correlations? Surprisingly, the Surgeon Riddle produced some of the most impressive results on the whole survey. For example, it correlated at r = 0.18, p = 0.001 with ability to see “the the” as two separate words. Given that the Surgeon Riddle seems much less perceptually basic than the other things on here, I believe this is probably some kind of confounder, maybe amount of time people spend on each question or something like that.

There was an overwhelmingly strong effect of age on the Parentheses Riddle and nothing else. 62% of people in their 20s got it right, compared to 29% of people in their 70s. This was surprising enough that it might be worth its own post. I was unable to wring anything else out of this no matter how hard I tried.

I tried a factor analysis to draw some factors out of all the different illusions. SPSS came up with six factors, each of which explained 5-10% of the variance; given how many variables I put in, this doesn’t look that much better than chance. None of the factors correlated surprisingly well with anything else, nor was there an obvious pattern in which illusions they grouped together.

In conclusion, most illusions had very low (less than r = 0.1) but highly significant (less than p = 0.001) correlations with one another and with various mental illnesses. There was no clear pattern to the correlations, although they did generally replicate past observations and findings from the literature. I can’t really say for sure if there’s a real effect here or it’s all just confounders, and if there is a real effect I definitely can’t tease out its structure or say anything about it for sure. I encourage other people to look into these and see if they can do better. You can download the survey results here.

I have a couple other things I pre-registered to investigate but never got around to before, so here they are:

Political conflict theory was slightly correlated with various questions on the Tolerance of Ambiguity and Tolerance of Uncertainty tests, but not enough to be interesting. For example, when asked whether political extremists you don’t like (eg fascists) were making understandable mistakes or just evil, the answer was correlated at about r = 0.05 with various questions like “Small doubts keep me from acting” and “The ambiguities in life stress me out”. The more stressed people were by ambiguity, the more they thought extremists were evil. This was not a very large effect, it varied from question to question in an unpredictable way, and I wouldn’t have mentioned it if I didn’t feel obligated to tie up preregistered loose ends.

Contra my prediction, there was no relationship between autism and the likelihood of giving process-based (“meta-level”) responses to categorization questions as opposed to person-based (“object-level”) responses. For example, when asked to judge fascists beating up minorities vs. minorities beating up fascists, autistic and neurotypical people were about equally likely to base their responses on principles (ie “beating up people is wrong”) vs. on the groups involved (ie “fascists are bad”).

Contra my prediction, people with ADHD did not describe themselves as more ambitious. They did describe themselves as more risk-taking and more likely to prefer a buzzing-city aesthetic than a quiet-country aesthetic, but the results, although significant, were too low in magnitude to be interesting.

As I mentioned in a previous article, the results of the AI Persuasion Experiment two years ago did not persist.