Attention is the new RAM — and it's getting expensive fast

A new paper argues that the way LLMs process context is the bottleneck for the whole industry. Attention scales quadratically with input length — double the tokens, four times the compute. That means a 4K video transcript or 500-page book bogs the model down to a crawl. The authors propose sparse attention patterns that let models touch only the relevant parts of long context, cutting compute by 10–30x on documents. It's not a new idea, but this one is the most convincing case for why it matters right now.

The practical upshot: if you're building on top of a model and your prompt is getting longer every week, you're about to hit a wall. The models that win will be the ones that can read a whole manual without charging $100. This is why the open-weight models are interesting — you can swap in a sparse-attention variant and save money. The closed APIs won't let you do that.

Why this matters for us: if the models get cheaper to run, the side hustles that depend on them — the cousin who writes listings for $20, the shop that auto-generates Spanish/English signage — actually scale instead of bugging out on every long request.