Delivery #

In a message’s journey in the RabbitMQ broker (message lifecycle), after successfully passing the publishing, routing, and Queueing stages, it arrives at the final gateway connecting the broker with our application’s business logic: the Delivery phase.

The Delivery phase is the stage where messages in the ready-to-consume status (Ready) are taken from the storage queue and delivered through TCP/AMQP channels to consumer applications. Although it sounds simple, this delivery process involves a complicated negotiation mechanism between the broker and consumers. To make the system run efficiently and reliably, RabbitMQ relies on a push-based delivery model, data flow control using prefetch QoS limits, acknowledgement status tracking, and redelivered flag handling. This article thoroughly discusses RabbitMQ’s internal message delivery mechanisms, performance comparisons between push and pull models, mathematical calculations for determining optimal prefetch settings, and Go code implementations for robust message consumption.

The Position of Delivery in the Message Lifecycle #

The Delivery phase acts as an active bridge between the storage queueing phase (Queueing) and the confirmation completion phase (Acknowledgement).

flowchart TD
    Queue["Queue (Message Status: Ready)"] --> Select{"Consumer & Quota Evaluation"}
    Select -->|Qualifies| Push["Push Message (basic.deliver)"]
    Select -->|Quota Full / No Consumer| Queue
    
    Push --> Track["Record to Unacked List"]
    Track --> Client["Consumer Processes Message"]
    
    Client --> Success{"Processing Success?"}
    Success -->|Yes| Ack["Send ACK (Message Deleted)"]
    Success -->|"No (Crash / NACK)"| Requeue["Requeue (Back to Queue with Redelivered=true)"]

When a message is in the delivery phase, its status transitions from Ready to Unacknowledged. While this status is active, the broker never deletes the message payload from its storage until the consumer application gives a clear confirmation response.

Push-Model vs Pull-Model Mechanisms: basic.consume vs basic.get #

RabbitMQ provides two AMQP 0-9-1 protocol methods for consumer applications to obtain messages from queues: the push model and the pull model. Understanding the fundamental differences between these two models greatly determines our data processing performance.

1. Push Model (basic.consume) — Recommended #

The push model is the default and highly recommended method in RabbitMQ. In this model, consumer applications subscribe to a specific queue through the basic.consume command.

• How It Works: Once a consumer is registered, the RabbitMQ broker actively pushes messages to the consumer’s network socket as soon as messages are available in the queue, without waiting for the consumer to request data.
• Advantage: Very low delivery latency (near zero milliseconds after the message is queued) and very efficient network resource usage because there are no unnecessary round-trip overhead calls.
• Control: This push speed is strictly controlled using the Prefetch Limit (QoS) setting so consumers don’t get overwhelmed receiving data.

2. Pull Model (basic.get) — An Anti-Pattern for Production #

The pull model is a manual polling method where consumers actively request a single message from the broker using the basic.get command.

• How It Works: The consumer sends a basic.get request to the broker. If there’s a message in the queue, the broker sends it; if not, the broker sends an empty response (no-message). The consumer must keep sending these requests in an infinite loop.
• Weakness: Every basic.get call triggers a full network round-trip cost (send request, open transaction, search data in queue, send response, close transaction). This causes throughput drops up to 100 times slower than basic.consume and drastically burdens broker CPU utilization.
• When to Use: Only for quick debugging, manual testing, or periodic batch processing where we only want to check one message without maintaining an active consumer connection.

Prefetch QoS Limits and Their Calculation Formula #

Because RabbitMQ uses the push model (basic.consume) by default, if there are 100,000 messages in a queue and we turn on one new consumer, the broker tries to deliver all 100,000 messages to the consumer application’s memory as fast as possible over the TCP socket.

This can cause the consumer application to run out of RAM memory (OutOfMemory Crash) or make other consumers starve (consumer starvation). To prevent this, we must apply the Prefetch Limit (Quality of Service - QoS).

How Prefetch Works #

Prefetch is the quota limit of unacknowledged messages allowed to be delivered to one consumer channel. If we set prefetch = 100, the broker only sends a maximum of 100 messages. The broker suspends sending the 101st message until the consumer sends at least one basic.ack signal to free the quota.

The Optimal Prefetch Calculation Formula #

Setting prefetch too low (e.g., 1) limits throughput because consumers must wait for the ACK to finish sending to the broker before receiving the next message. Conversely, setting prefetch too high (e.g., 10,000) can cause RAM waste and uneven load distribution.

We can calculate the optimal prefetch value using this simple mathematical formula:

$$\text{Prefetch} = \frac{\text{Consumer Round-Trip Time (RTT)} + \text{Message Processing Time}}{\text{Message Processing Time}}$$

Or more practically for parallel applications:

$$\text{Prefetch} \approx \text{Throughput Target (messages/second)} \times \text{Network RTT (seconds)}$$

For example:

• If the average message processing time of our application code is 50 milliseconds (0.05 seconds).
• The network latency (RTT) between the application server and the RabbitMQ broker is 10 milliseconds (0.01 seconds).
• Then, the optimal Prefetch per consumer thread is:

$$\text{Prefetch} = \frac{10\text{ms} + 50\text{ms}}{50\text{ms}} = 1.2 \approx 2$$

If our consumer uses a multi-threading system (goroutines) that can process 50 messages in parallel:

$$\text{Prefetch} = 50 \times 1.2 = 60$$

With a prefetch setting of 60, our channel always has backup messages in local socket memory to keep CPU utilization at 100% without network wait gaps, but not excessively burdening RAM.

Workload Distribution: Round-Robin vs Prefetch Scheduling #

When a queue has several consumers connected simultaneously, RabbitMQ distributes messages among them. This distribution mechanism is managed through two strategies:

1. Pure Round-Robin (Without Prefetch QoS) #

If we register several consumers without specifying a prefetch limit (or prefetch = 0, meaning unlimited):

• Logic: RabbitMQ divides messages evenly in turn (Round-Robin). Consumer A gets message 1, consumer B gets message 2, consumer A gets message 3, and so on.
• Problem: If message 1 takes 1 hour to process (e.g., heavy video processing) while message 2 finishes in 1 second, consumer A carries a large backlog in its local memory while consumer B sits idle after 1 second. This is called the starvation phenomenon and unbalanced load distribution.

2. Fair Dispatch (Prefetch Scheduling) #

When we set a prefetch limit (e.g., prefetch = 1 or prefetch = 10):

• Logic: RabbitMQ no longer sends messages blindly. The broker continuously monitors the Unacknowledged message count on every consumer.
• Impact: If consumer A is busy processing heavy messages and its prefetch limit is reached, the broker skips consumer A and directly routes the next messages to consumer B, which still has empty prefetch quota. This produces dynamic and fair workload distribution (fair dispatch).

Redelivered Flags and the At-Least-Once Delivery Guarantee #

RabbitMQ adheres to the At-Least-Once Delivery principle (messages guaranteed delivered at least once). To realize this guarantee, the broker must be ready to resend messages that failed confirmation.

How Does Redelivery Happen? #

When a message is delivered to a consumer, the broker tracks that consumer’s TCP connection lifetime. If:

1. The consumer’s TCP connection drops suddenly (e.g., application crash, server death, or network disconnect).
2. The consumer explicitly rejects the message with a requeue instruction (basic.nack or basic.reject with requeue = true).

The broker automatically returns that message to the Ready status in the destination queue.

The Redelivered = true Property #

When the broker resends that requeued message to a new consumer (or the same consumer after reconnecting), the broker attaches a special flag in the AMQP header: redelivered: true.

flowchart TD
    A["First Delivery"] -->|"basic.deliver"| B["redelivered: false"]
    B -->|"Connection Drops"| C["Message Resent"]
    C -->|"basic.deliver"| D["redelivered: true"]

The Importance of Idempotency at the Application Level #

The presence of the redelivered: true property is an early warning signal for our consumer applications. When our application receives a message with this flag active:

• The application must assume this message may have already been partially or fully processed by a previous consumer before the crash occurred.
• To prevent duplicate executions that damage data (e.g., deducting a user’s balance twice), our application must apply the Idempotency principle.
• Idempotency Solution: Use a unique key database (like Redis or a relational database index) to record the unique UUID of successfully processed messages. Before executing business logic, check whether that message UUID already exists in the success history list.

Go Code Implementation: Creating a Graceful QoS Prefetch Consumer #

Here is a complete implementation example in Go. This code declares a robust consumer by setting the QoS prefetch limit, using manual ACKs, checking the Redelivered flag, and applying system signal handling for graceful shutdown so in-flight messages aren’t cut off mid-process.

package main

import (
	"context"
	"log"
	"os"
	"os/signal"
	"syscall"
	"time"

	amqp "github.com/rabbitmq/amqp091-go"
)

func main() {
	// 1. Open a TCP Connection to the RabbitMQ Broker
	conn, err := amqp.Dial("amqp://guest:***@localhost:5672/")
	if err != nil {
		log.Fatalf("Gagal terhubung ke RabbitMQ: %s", err)
	}
	defer conn.Close()

	ch, err := conn.Channel()
	if err != nil {
		log.Fatalf("Gagal membuka channel: %s", err)
	}
	defer ch.Close()

	// 2. Configure the QoS Prefetch (Prefetch Limit)
	// We limit a maximum of 20 unacknowledged messages on this channel
	err = ch.Qos(
		20,    // prefetch count: message quota limit
		0,     // prefetch size: size in bytes (0 means unlimited)
		false, // global: false means the limit applies per active consumer on this channel
	)
	if err != nil {
		log.Fatalf("Gagal menyetel QoS Prefetch: %s", err)
	}

	queueName := "transaction-logs-queue"

	// 3. Register the Consumer (Push Model - basic.consume)
	msgs, err := ch.Consume(
		queueName,
		"",    // consumer tag (left empty so the broker auto-generates it)
		false, // autoAck: Must be false to guarantee manual ACK for data safety!
		false, // exclusive
		false, // noLocal
		false, // noWait
		nil,   // arguments
	)
	if err != nil {
		log.Fatalf("Gagal mendaftarkan konsumen: %s", err)
	}

	// Channel to capture OS termination signals (Ctrl+C, kill)
	sigChan := make(chan os.Signal, 1)
	signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)

	// Context to control the processing goroutine lifecycle
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	log.Println("[*] Menunggu pesan. Untuk keluar, tekan Ctrl+C")

	go func() {
		for {
			select {
			case <-ctx.Done():
				return
			case d, ok := <-msgs:
				if !ok {
					log.Println("[!] Channel pesan ditutup.")
					return
				}

				// 4. Check the Redelivered Flag for Idempotency
				if d.Redelivered {
					log.Printf("[Peringatan] Pesan dengan ID %s adalah pesan kirim ulang (Redelivered). Menjalankan pengecekan duplikasi...", d.MessageId)
					// Here we should verify in the database / Redis cache
				}

				log.Printf("Menerima pesan: %s", d.Body)

				// Simulate business logic processing (e.g., save to DB)
				time.Sleep(200 * time.Millisecond)

				// 5. Send the Manual ACK after successfully processing
				err := d.Ack(false) // multiple: false means only ACK this specific message
				if err != nil {
					log.Printf("Gagal mengirimkan ACK: %s", err)
				} else {
					log.Printf("✓ Sukses memproses pesan dengan ID %s", d.MessageId)
				}
			}
		}
	}()

	// Wait for the OS termination signal for Graceful Shutdown
	<-sigChan
	log.Println("[*] Menerima sinyal keluar. Memulai proses shutdown yang aman...")
	
	// Cancel the context so the goroutine doesn't read new messages
	cancel()
	
	// Give tolerance time for in-flight messages to finish processing
	time.Sleep(1 * time.Second)
	log.Println("[*] Shutdown selesai. Semua koneksi ditutup dengan aman.")
}

Anti-Patterns vs Practical Solutions in Production #

Avoid the following fatal message delivery configuration mistakes to maintain cluster stability:

1. Setting Auto-ACK (autoAck = true) to Speed Up Throughput #

Enabling automatic confirmation settings assuming our application code processing always runs successfully without errors.

Why is this wrong? #

With autoAck = true, RabbitMQ considers the message successfully processed exactly when the message is written to the consumer’s TCP network socket. If our consumer application crashes mid-way (e.g., running out of RAM while processing the payload, or the database connection drops), that message is lost forever from the broker queue with no recovery. This is the main cause of mysterious data loss.

• Solution: Always use manual confirmation (autoAck = false). Keep the message in the Unacknowledged status at the broker until our application business logic truly returns a success response and explicitly calls the d.Ack(false) command.

2. Using the Pull Model (basic.get) in the Main Processing Loop #

Writing consumer code that repeatedly calls basic.get inside an infinite for loop, arguing the code looks simple like reading data from a relational database.

Why is this wrong? #

Constant polling cycles burden the broker’s disk I/O and CPU because the broker must continuously process short transaction channel openings for every single message. This destroys broker scalability and spikes delivery latency.

• Solution: Use the push subscription model (basic.consume). Combine it with parallel processing goroutines and limit throughput using a measured Qos(prefetchCount) prefetch setting.

Summary #

• The Push-Model Is Superior — Using the basic.consume push model is far more efficient and has lower latency than the manual basic.get pull polling model.
• The Important Role of Prefetch QoS — Setting message prefetch limits prevents one consumer instance from being overwhelmed holding payload load in its RAM and facilitates fair dispatch.
• Prefetch Calculation — Determine the ideal prefetch value by calculating the ratio of network round-trip (RTT) latency to the real business logic processing time.
• Fair Dispatch Load Distribution — Prefetch QoS diverts message allocation from busy consumers (full quota) to other idle consumers.
• Evaluate Redelivered Flags — Always check the Redelivered header value to activate idempotency and data deduplication protection mechanisms on the application side.
• Manual ACK Safety — Using manual confirmation maintains data integrity so messages aren’t lost when consumer applications crash unexpectedly.

← Previous: Queueing   Next: Acknowledgement →