Competing consumers

A single consumer of messages works in a synchronous manner unless the application itself implements the logic of reading messages asynchronously. Competing consumers pattern helps in implementing a solution in which multiple consumers are ready to process the incoming message and they compete to process it. This helps in architecting solutions that are highly available and scalable. It is scalable because with multiple consumers it is possible to process a higher number of messages in the smaller period. It is highly available because there could be at least one or more consumers to process the messages even if some consumers crash.

This pattern should be used when each message is independent of other messages. The messages by themselves contain complete information for a consumer to complete a task. This pattern should not be used if there is any dependency among messages. The consumers should be able to complete the tasks in isolation. Also, this pattern is applicable if there is variable demand for services. Additional consumers can be added or removed based on demand.

A message queue is required for implementing competing consumers pattern. In this pattern, patterns from multiple sources pass through a single queue that is connected to multiple consumers on another end. These consumers should delete the message after reading it so that it is not re-processed again.