AI
LLM Idea i have been exploring recently
An independent researcher has developed an experimental pipeline to recover recurring semantic computation graphs for concepts inside transformer models
Key takeaways
- Concepts inside transformers appear to be represented as distributed computational subgraphs across many neurons and layers rather than single neurons.
- Semantic divergence grows gradually through the network and becomes strongest in later layers.
- Repeated prompts about an entity produce surprisingly similar graph structures.
- These findings are an exploration rather than a definitive conclusion, and alternative explanations must still be ruled out.
An independent researcher has developed an experimental pipeline to recover recurring semantic computation graphs for concepts inside transformer models, rather than looking at single neurons or hidden states. By testing contrastive prompts about specific entities like India, France, and Japan, the pipeline captures residual, attention, and MLP activations to construct activation-based graphs. Initial observations show that semantic divergence grows gradually through the network, certain neuron groups consistently appear across prompts for the same entity, and different entities share parts of the graph while maintaining entity-specific branches.
In their words
“The computation appears to be distributed across many neurons and layers, which makes a graph representation much more natural than searching for a single "India neuron.”
Want this on autopilot?
We file the AI news, make the video, and post it for you.
Common questions
- marketing.blog.post.faq.what_happened
- Semantic divergence grows gradually through the network and becomes strongest in later layers.
- marketing.blog.post.faq.source
- marketing.blog.post.faq.source_answer
