Prefetch

When you run a consumer, you may have assumed this process for message consumption:

1. Fetch a message from the queue (across the network, which can be slow)
2. Process the message
3. Acknowledge the message
4. Repeat

But that would slow everything down to a crawl due to the full network round trip for every message. Instead, RabbitMQ allows you to prefetch messages. When you prefetch messages, RabbitMQ will send you a batch of messages at once, the client library will store them in memory, and you can process them one by one. Much faster. The diagram shows 3 consumers each prefetching batches of 2.

By default, we were allowing one client to prefetch all 10,000 messages from the server! That means other clients couldn't get any messages until the first client had processed all 10,000. We need to limit the prefetch count.

Assignment

1. In src/internal/pubsub/consume.ts, update your consumption code to set the prefetch count to 10.

This will ensure that each client only prefetches 10 messages at a time. This will allow other clients to get messages while one client is processing.

2. Run the multiconsumer again. You should be able to consume around 100 messages per second when running 100 peril servers.
3. Run the servers until the queue is empty, then kill them with ctrl+c.

Run and submit the CLI tests with an empty game_logs queue.