The Emotional Landscape of Language Models: New Discoveries from GoodFire

www.goodfire.ai/research/stories-in-space#

Researchers at GoodFire continue to explore the so-called geometry of LLMs. Recently, they demonstrated that the thoughts of models are organized in the form of specific shapes. For instance, numbers are represented as circles, and when adding one number to another, LLMs are essentially summing numerous donuts.

Now they have discovered new intriguing details. It turns out that within the models exists a whole landscape of emotional states.

The scientists took LLama, had it read short stories, and observed the activation states. During reading, the model seems to move through a multidimensional space of states, 'feeling' various emotions depending on what is happening in the text at that moment.

If we visualize the states corresponding to different emotions, we find a structure that closely resembles the classical psychological model of human emotions. For example, joy is near interest, while fear and anger share a similar high intensity, and so on.

This map can also be used to predict the model's future responses. If we artificially nudge the activations towards a certain emotion, the mood and essence of the generations change.