Content-based routing under a fixed, norm-blind metric produces the same concentration signature on every axis.
The routes differ; the destination does not.
Semantic Gravity is an ongoing program built on one idea: when a model routes information by content under a fixed, magnitude-blind similarity metric, it pays for the mismatch by concentrating: collapsing its representations onto a low-dimensional subspace, leaning on a few anchor tokens, stratifying its norms. The same signature turns up whether the routing happens over tokens, nodes, time, or depth. These pages track where the idea has gone: what is published, what didn't pan out, and what is still open.
Paper 1, the diagnosis. Attention sinks, representation collapse, and norm stratification aren't quirks of attention. They're what content-based routing does under a fixed, norm-blind metric, on any axis. Shown across nine transformers and four non-attention routers (graph, state-space, recurrent, depth), with two within-model ablations pinning the cause to the routing itself.
The obvious follow-up: restore the magnitude term the metric throws away and see if performance improves. At toy scale it doesn't. A norm-aware score does most of the collapse relief for free, and handing the router more metric freedom is redundant or actively worse. Shelved as a performance play.
The reframe that fell out of that null result: maybe collapse isn't the disease but the cure, the model compressing its keys so a flat metric can still tell things apart. The decisive test is an anti-collapse control: regularize the collapse away and watch whether the loss gets worse. The current thread.
A way to read concentration across architectures: how much is inherited from the substrate (graph hubs, token frequency) versus generated by the routing dynamics. A lens for organizing the cross-architecture results, not a finished theorem.
A design that eases all four compensation axes at once: a learned-metric local attention beside a global state recurrence, wide heads, and cross-layer access, joined by a slow-waking gate so neither half drowns the other early. On paper for now.
The speculative end: metric curvature on the token axis and learned residuals on the depth axis as two faces of the same geometric enrichment of an otherwise flat network. A possible later paper.