SSC Journal Club: Friston On Computational Mood

A few months ago, I wrote Toward A Predictive Theory Of Depression, which used the predictive coding model of brain function to speculate about mood disorders and emotions. Emotions might be a tendency toward unusually high (or low) precision of predictions:

Imagine the world’s most successful entrepreneur. Every company they found becomes a multibillion-dollar success. Every stock they pick shoots up and never stops. Heck, even their personal life is like this. Every vacation they take ends out picture-perfect and creates memories that last a lifetime; every date they go on leads to passionate soul-burning love that never ends badly.

And imagine your job is to advise this entrepreneur. The only advice worth giving would be “do more stuff”. Clearly all the stuff they’re doing works, so aim higher, work harder, run for President. Another way of saying this is “be more self-confident” – if they’re doubting whether or not to start a new project, remind them that 100% of the things they’ve ever done have been successful, odds are pretty good this new one will too, and they should stop wasting their time second-guessing themselves.

Now imagine the world’s least successful entrepreneur. Every company they make flounders and dies. Every stock they pick crashes the next day. Their vacations always get rained-out, their dates always end up with the other person leaving halfway through and sticking them with the bill.

What if your job is advising this guy? If they’re thinking of starting a new company, your advice is “Be really careful – you should know it’ll probably go badly”. If they’re thinking of going on a date, you should warn them against it unless they’re really sure. A good global suggestion might be to aim lower, go for low-risk-low-reward steady payoffs, and wait on anything risky until they’ve figured themselves out a little bit more.

Corlett, Frith and Fletcher linked mania to increased confidence. But mania looks a lot like being happy. And you’re happy when you succeed a lot. And when you succeed a lot, maybe having increased confidence is the way to go. If happiness were a sort of global filter that affected all your thought processes and said “These are good times, you should press really hard to exploit your apparent excellence and not worry too much about risk”, that would be pretty evolutionarily useful. Likewise, if sadness were a way of saying “Things are going pretty badly, maybe be less confident and don’t start any new projects”, that would be useful too.

Depression isn’t normal sadness. But if normal sadness lowers neural confidence a little, maybe depression is the pathological result of biological processes that lower neural confidence a lot. To give a total fake example which I’m not saying is what actually happens, if you run out of whatever neurotransmitter you use to signal high confidence, that would give you permanent pathological low confidence and might look like depression.

This would explain a lot about depression. It would explain why depressed people have such low motivation. It would explain why their movements are less forceful (“psychomotor retardation”). It would even explain why sense data are less distinct (depressed people literally see the world in washed out shades of grey). I thought this was plausible, but said I’d wait for real scientists to say the same thing before believing it too much.

What Is Mood: A Computational Perspective by Clark, Watson, and Friston – is real scientists saying the same thing. Sort of. With a lot more rigor. Let’s look into it and see what they get.

Recent theoretical arguments have converged on the idea that emotional states reflect changes in the uncertainty about the somatic consequences of action (Joffily & Coricelli, 2013; Wager et al. 2015; Seth & Friston, 2016). This uncertainty refers to the precision with which motor and physiological states can be predicted. In this setting, negative emotions contextualise events that induce expectations of unpredictability, while positive emotions refer to events that resolve uncertainty and confer a feeling of control (Barrett & Satpute, 2013; Gu et al. 2013). This ties emotional states to the resolution of uncertainty and, through the biophysical encoding of precision, to neuromodulation and cortical gain control (Brown & Friston, 2012).

In summary, one can associate the valence of emotional stimuli with the precision of prior beliefs about the consequences of action. In this view, positively valenced brain states are necessarily associated with increases in the precision of predictions about the (controllable) future – or, more simply, predictable consequences of motor or autonomic behaviour. Conversely, negative emotions correspond to a loss of prior precision and a sense of helplessness and uncertainty about the consequences of action.

Here they’re saying that emotions – the day-to-day variation in whether we feel happy or sad – is meant to track what kind of environment we’re in. Is it a predictable environment that we should rush out to manipulate so we can harvest a big heap of utility? Or is it an unpredictable environment where we’re probably wrong about everything and should try to limit damage?

It’s not really clear from this quote, but later on they’re going to shift from happiness being “the world is predictable” to “the world is good”, which – sounds a lot more common-sensical. I think this has to do with Friston’s commitment to believing that uncertainty-resolution is the only drive, and every form of goodness is a sort of predictability in a way. See Monday’s post God Help Us, Let’s Try To Understand Friston On Free Energy – or don’t, for all the good it will do you.

Any hierarchical inference relies on hyperpriors. These furnish higher level predictions of the likely value of lower level parameters. From the above, one can see that important parameters are the precisions of prediction errors at high and low levels of the hierarchy (i.e. prior and sensory precision). These precisions reflect the confidence we place in our prior beliefs relative to sensory evidence. If emotional states in the brain reflect the precision of prior beliefs about the consequences of action, then distinct neuronal populations must also encode hyperpriors. In other words, short-term fluctuations in precision (i.e. emotional fluctuations) will themselves be constrained by hyperpriors encoding their long-term average (i.e. mood).

Here, we propose that mood corresponds to hyperpriors about emotional states, or confidence about the consequences of action. In other words, mood states reflect the prior expectation about precision that nuances (emotional) fluctuations in confidence or uncertainty. If emotion reflects interoceptive precision, and is biophysically encoded by neuromodulatory gain control, then this suggests that mood is neurobiologically encoded as the set-point of neuromodulator systems that determine synaptic gain control over principal cells reporting prediction errors at different levels of the interoceptive hierarchy. This set-point is the sensitivity of responses to prediction errors and has a profound and enduring effect on subsequent inference.

The traditional definition says that “mood is like climate, emotions are like weather”. I think they’re saying that mood – long-lasting states like being depressed or being a generally carefree person – are second-level priors about emotions, which themselves are first-level priors about actions.

So suppose you see a vaguely greenish piece of paper on the ground. If you’re happy, you have a prior for the world being good, and so you might be more likely to interpret it as possibly a dollar bill. And you have a prior for the world being exploitable, so you might be more likely to think you can reach down and take it and have an extra dollar. And if you do, and it really is a dollar bill, you might become happier, since you’ve gained a little evidence that your senses are trustworthy (you were right to perceive it as a dollar), the world is exploitable (your cunning plan to pick up the paper and gain $1 worked!), and you’re in the sort of high-reward environment where you should go off and do other exciting things.

On the other hand, if you’re sad, you have a prior for the world being bad, so you might expect it to be litter. You have a prior that you can’t really predict or affect the world, so it might not be worth bending down to pick it up – you might just end up disappointed. But if you did bend down to pick it up, and it did turn out to be a dollar bill, you might brighten up a little, just as the happy person would. You’ve gained a little bit of evidence that you’re in a nice part of the world where good things happen to you, and that you can execute a simple plan like picking up a dollar bill to gain money.

A depressed person would have the same prior that the world is bad and the paper is probably just litter. But if perhaps she did pick up the dollar, and feel tempted to conclude that the world was good and she should feel happy, a higher-level prior would kick in: even when it seems like the world is good, that’s wrong and you should ignore it. The world is never actually good. When good things happen that look like they should convince you that the world is good, those are just lies.

Friston et al bring up learned helplessness. Let’s say you shock a rat a lot. In fact, let’s say you’re even more cruel, and you constantly give the rat apparent escapes, only to close them off at the last second and keep shocking it. You give the rat what look like food pellets, but they turn out to just be rocks painted to look like food. You eventually gaslight the hell out of the rat. Finally, you stop doing this, and you give the rat some actual food and a way out, and the rat just doesn’t care. Yes, food and escape should be good things that make it feel lik the world is reward-filled and exploitable, but it’s been let down so many times before that it assumes anything seemingly-good is a mirage.

Here’s the picture they eventually draw:

Depression is a prediction of bad outcomes with high confidence. Mania is a prediction of good outcomes with high confidence. Anxiety (or “agitated depression”) is a prediction of bad outcomes with low confidence. There’s a blank space where it looks like there ought to be an extra emotion; maybe God will release it later as DLC.

Friston et al speculate that these hyperpriors over emotions can either be genetically encoded, or “learned” over very long periods of consistent stimuli. For example, if your childhood is unbearably terrible, that might be long enough to “burn in” a high-confidence hyperprior that the world is always bad.

(they don’t mention this, but if prediction and action are as linked as everyone always says, I wonder if this would explain why people with terrible childhoods are always mysteriously sabotaging themselves into have adulthoods that are terrible in the exact same way – eg someone with an abusive alcoholic father marrying an abusive alcoholic).

These hyperpriors can reach the level of a mood disorder when they become resistant to feedback. They present a couple of different arguments for how this might happen. In one, a depressed person doesn’t feel any positive emotions, since there’s such a strong prior on everything being terrible that these never reach the level of plausibility. Since positive emotions are a useful tool for figuring out what makes you happy and urging you to do it, depressed people aren’t motivated to make themselves happy, and so never end up contradicting their bias towards believing they’re sad all the time. This fits really well with “behavioral activation”, a common psychotherapy where therapists tell depressed people to just go out and do happy things whether they want to or not, and which often helps the depression resolve.

In another, all the brain’s predictions are so low-precision that it can’t even properly predict interoceptive sensations (the sensations received from organs, eg the heartbeat). Maybe it will think “I guess maybe my heart will beat right now”, but it’s not the sort of clear confident precision that really enters into its mental model. That means these interoceptive sensations are always predicted slightly incorrectly, and this keeps the brain feeling like it’s sick and confused and the world is unpredictable.

They don’t seem to mention this, but it also seems intuitively plausible that the strong prior on negativity could prevent the perception of positive factors directly. You see the piece of paper on the street, you think “the world is always terrible, so no way that’s a dollar bill”, you pass it by, and you miss an opportunity to feel lucky and give yourself a tiny bit of pleasure.

The rest of the paper is just a survey of some findings from biology and neuroscience that seem to support this, though they’re not all very specific. For example, the HPA axis is dysregulated, which fits with predictive processing, but it also fits with everything else. The main part I found interesting was this:

In healthy systems, mood should be affected by the valence of tightly controlled prediction errors. Recent animal work has shown that positive prediction errors (receiving more food than expected), show a strong positive correlation with dopaminergic change in the nucleus accumbens (Hart et al. 2014) with corresponding changes in functional brain activity in humans during a financial reward task (Rutledge et al. 2010). Similarly, it has been shown that signal change in the anterior insula is significantly related to the magnitude of prediction error (Bossaerts, 2010). The pharmacological manipulation of these networks was recently demonstrated where participants were given electric shocks (harms) in exchange for financial reward (gains), and offered the option of increasing the number of shocks in exchange for greater reward. It was shown that citalopram increased harm-aversion, while levodopa made individuals more likely to harm themselves than others (Crockett et al. 2015). This fits nicely with our notion that serotonin levels (and other neuromodulators) encode expectations about likely negative outcomes and encourage the fulfilment of these predictions through action (i.e. low levels promote behaviour with negative outcomes).

Focus on this sentence: “serotonin encodes expectations about likely negative outcomes and encourages the fulfilment of these predictions through action”. Also this one: “Low levels [of serotonin] promote behavior with negative outcomes”.

I don’t think I’m misunderstanding this – the authors cite some evidence that low serotonin causes self-harm, and yes, it certainly does. But what does it mean to have a system for promoting behavior with negative outcomes? Why have a neurotransmitter whose level corresponds to how much you should be trying to do negative-outcome behavior? Surely the answer is just “never do this”.

The only way I can make sense of this is through the paragraph above talking about the shocks-for-money game, where SSRIs decrease people’s willingness to get shocks. It sounds like maybe Friston et al are claiming that we have a “willingness to be harmed” lever so that we can calculate how willing we are to accept some levels of harm in exchange for a greater good. In that case, maybe self-harm is what happens when the “willingness to be harmed” lever is set so high that random noise, the chance of getting other people’s attention, or just passing the time presents some tiny reward, and your harm-for-reward tradeoff rate is so high that even that tiny reward is worth the harm.

More broadly, what should we think of this theory?

In retrospect, if you know Bayesian math, the idea of depression as a prior on bad outcomes seems pretty fricking obvious. I’m not even sure if it’s any different from the sort of stuff Aaron Beck was saying in the seventies. The big advance in this model is uniting “prior on bad outcomes” with “low precision of predictions / low neural confidence”. The low-precision part helps explain anergia, anhedonia, low motivation, psychomotor retardation, sensory washout, and probably (with a little more work) depression with psychotic features. Flipped around, it offers an explanation of psychomotor agitation, grandiosity, psychosis, and pareidolia in mania.

The only problem is that I still haven’t seen “prior on bad outcomes” and “low precision” really get unified. The authors seems to equivocate between “sadness means you’re in an unpredictable environment” and “sadness means you’re in a bad environment where everything sucks”. There is at least a little bit of work to add the hyperprior on top of the prior, so that at least we don’t get suspicious when we remember that depressed people are very confident in their depression. But it still seems like a world of low-precision predictions should be one where people just have no idea whether the paper in front of them is a dollar, not one where they’re really sure it isn’t. A world of high-precision predictions should look more like sitting in a bright room with a metronome, predicting each subsequent beat, rather than a world where everything is great and your life goes well. I’m not even sure this theory can explain why winning the lottery makes you happy rather than sad. It ought to make you think the world is really confusing and unpredictable (really? the thing you thought had a one in ten million chance happened?) – but in fact most lottery winners look pretty happy to me.

If this is confusing, at least it isn’t a new confusion. We know that a big part of the free energy research agenda is to try to unify desire-satisfaction with uncertainty-resolution, and claim that expectation and desire are (somehow, despite how it looks) the same thing. If we just assume that works, for the sake of argument, it allows this paper to be an impressive unification of several lines of research on mood disorder into a coherent and actionable whole.