Latency

How long you wait for the answer, measured in how it feels rather than how it's calculated.

You'll see it written as "p95 under 3 seconds." That means: out of every hundred requests, ninety-five come back inside three seconds. It's a promise about the bad ones rather than the average — because an average of two seconds still hides the one customer in twenty who waited fifteen and gave up.

Why it matters: speed is a feature, and one you can write into an agreement and test. Worth knowing that agents stack it: five AI steps one after another is five waits added together, so a chain where each step takes four seconds keeps somebody waiting twenty. That's a decision made early, not a surprise found late — and "what's the p95 on this?" is a perfectly reasonable question to put to whoever builds it.

Related: Streaming · Caching