Queuing Work

A throttle limits how quickly work begins by delaying or rejecting excess requests. A queue accepts work now and holds onto it for later processing.

Throttling is the right answer when the client needs an immediate response, but queuing is usually the better choice when the work can be done asynchronously.

For example, a Google search probably needs an immediate response, so excess requests should get a 429. A PDF report export, on the other hand, can be queued. When it's ready, the system can email the user a download link.

With queuing, throughput matters more than latency. It's "how many PDFs can I generate per hour," not "how many minutes does an individual PDF take to generate."

The simplest queuing system is a database table that tracks work to be done, plus a background worker that processes it. More sophisticated options like RabbitMQ or Redis Streams exist as well.