Message queues

Throughout this book, we will be dealing a lot with message queues. You will also see it prevalent in the software we are developing. Messaging queues are how our ecosystem communicates, maintains separation of concerns, and allows for fluid and fast development. With that being said, before we get too far along into something else, let's spend some time discussing exactly what message queues are and what they do.

Let's think about the functionality of a message queue. They are two sided components; messages enter from one side and exit from the other one. Thus, each message queue can establish connections on both sides; on the input side, a queue fetches messages from one or more exchanges, while on the output side, the queue can be connected to one or more consumers. From the single queue point of view being connected to more than one exchange with the same routing key, this is transparent, since the only thing that concerns the message queue itself are the incoming messages:

Put another way...

The basic architecture of a message queue is simple. There are client applications called producers that create messages and deliver them to the broker (the message queue). Other applications, called consumers, connect to the queue and subscribe to the messages to be processed. A software can be a producer, or consumer, or both a consumer and a producer of messages. Messages placed onto the queue are stored until the consumer retrieves them:

And, breaking that down even further:

The preceding diagram illustrates the following process:

1. The user sends a PDF creation request to the web application
2. The web application (the producer) sends a message to RabbitMQ, including data from the request, such as name and email
3. An exchange accepts the messages from a producer application and routes them to correct message queues for PDF creation
4. The PDF processing worker (the consumer) receives the task and starts the processing of the PDF

Let's now look at some of the different message queue configurations that we can use. For now, let's think of a queue as an ordered collection or list of messages. In the diagrams that follow, we're going to use P to represent a producer, C to represent a consumer, and the red rectangles to represent a queue.

Here's our legend: