6. Keeping it running in the real world

Rate limit

A cap on how often you're allowed to ask a service for something in a given stretch of time. Go over it and your requests get turned away until the clock resets.

The merchant will do you ten deliveries a day. The eleventh waits until tomorrow, however politely you ask.

Why it matters: this is the wall fan-out hits first. Set fifty agents running at once, all reaching for the same system, and you knock over your own service — an outage nobody had to attack you to cause. So pacing the fleet is part of running one. The upside: those same caps are what stop an agent stuck in a loop from spending your money all night. A limit is a brake as well as a ceiling.

Related: Fan-out · Token cost / cost discipline

Bence K. Csernak

Bence K. Csernak

Founder

Newsletter