Nucleus sampling: how AI keeps its head straight

When you ask an AI a question, it generates a bunch of possible next words, each with a confidence score. Nucleus sampling (called "top-p" in the docs) is a trick that keeps the AI from going off the rails.

The AI gathers up its most-likely answers until they add up to some percentage — say 90%. It throws out everything below that line and picks randomly from the rest.

Think of a bodega. The owner doesn't stock every item in the world. He stocks what people in the neighborhood actually buy. The rest of the shelf stays empty. Nucleus sampling does the same thing: it keeps the answers on the shelf and clears out the junk.

Without nucleus sampling, the AI can ramble, invent, or circle back to itself. With it, responses stay focused and grounded.

Why this matters for us: AI is getting used more in schools, government services, and customer support — places where answers that sound confident but are wrong can do real harm.

When you're testing AI outputs, ask: does it repeat itself, or does it keep coming up with new, relevant points? Repetition usually means the nucleus is too narrow.