Acknowledgement #

In the message lifecycle inside the RabbitMQ broker (message lifecycle), after the message is published, routed, queued, and successfully delivered to the consumer application (Delivery), we arrive at the decisive stage that sets the final outcome of the message’s fate: the Acknowledgement phase.

The Acknowledgement phase is a crucial mechanism used by the broker and consumer applications to align data processing status. Through this phase, consumers can tell the broker whether the message was successfully executed so it’s safe to delete from physical disk/RAM storage, or whether it failed due to system constraints so it needs to be diverted to another path. Errors in designing this confirmation mechanism can be fatal for our system’s reliability, from mysterious data loss, RAM memory leaks from unconfirmed backlogs, to infinite failure loops that cripple CPU performance (poison message loops). This article thoroughly unpacks RabbitMQ’s internal confirmation semantics, the differences between success and failure acknowledgement instructions, message identifier management, and Go implementations to secure our system’s data integrity.

The Position of Acknowledgement in the Message Lifecycle #

The Acknowledgement phase is the absolute closing stage in a message’s lifecycle in the broker.

flowchart TD
    Deliver["Message Delivered (basic.deliver)"] --> StateUnacked["Message Enters Status: Unacknowledged"]
    StateUnacked --> Logic["Consumer Executes Business Logic"]
    
    Logic --> Result{"Execution Result Evaluation"}
    
    Result -->|Complex Success| Ack["Send basic.ack"]
    Result -->|"Failed (Wrong Format)"| RejectDLX["Send basic.reject (requeue=false)"]
    Result -->|"Failed (DB Connection Down)"| Requeue["Send basic.nack (requeue=true)"]
    
    Ack --> Delete["Broker Deletes Message from Disk/RAM"]
    RejectDLX --> DLXRouting["Message Diverted to Dead Letter Exchange"]
    Requeue --> RequeueQueue["Back to the Original Queue (Front Position)"]

While a message is in the Unacknowledged status, the broker holds that message in storage. The status transition from Unacknowledged to permanent deletion or requeueing is fully controlled by the confirmation instruction sent by the consumer back to the broker over the TCP socket.

AMQP 0-9-1 Confirmation Signal Types: basic.ack, basic.nack, and basic.reject #

The AMQP 0-9-1 protocol provides three main commands for consumers to respond to messages received from the broker. Each command has different implications for data existence in the queue.

1. basic.ack (Positive Acknowledgement) #

This instruction is sent by the consumer to tell the broker the message was successfully processed without issues.

  • Broker Action: Upon receiving basic.ack, the RabbitMQ broker permanently deletes that message’s metadata and body payload from RAM memory and disk. The consumer’s prefetch quota is released again.

2. basic.nack (Negative Acknowledgement) #

This instruction was introduced by RabbitMQ as an extension of the standard AMQP 0-9-1 protocol to signal that message processing failed.

  • Advantage: Unlike ordinary rejection instructions, basic.nack supports batch processing (bulk rejection). We can reject many messages at once in a single TCP command to save network latency.
  • Options: Has the determining requeue argument. If set to true, the message returns to the original queue; if false, the message is discarded or sent to a Dead Letter Exchange (DLX).

3. basic.reject (Rejection) #

The standard AMQP 0-9-1 command for rejecting failed messages.

  • Behavior: Identical to basic.nack in terms of error handling and the requeue option. However, basic.reject can only reject one single message per protocol call. It doesn’t support batch rejection.

The requeue Parameter and Its Implications for Data Flow #

When sending failure signals (basic.nack or basic.reject), the most crucial decision the consumer application must make is determining the value of the requeue parameter (a boolean true or false).

1. The Impact of requeue = true #

When we reject a message with the requeue = true parameter:

  • The broker places the message back in its original queue.
  • The returned message is placed as close as possible to the head of the queue position. This means the message is redelivered to the same consumer (or another consumer) almost instantly.
  • Usage Scenario: Perfect for transient failures, such as a momentarily disconnected database connection or a third-party API service experiencing RTO (Request Timeout).

2. The Impact of requeue = false #

When we reject a message with the requeue = false parameter:

  • The broker never returns that message to its original queue.
  • The message is deleted from the queue at that moment. However, if the queue is configured with a Dead Letter Exchange (DLX), the message payload is not physically discarded but diverted to that DLX for further analysis.
  • Usage Scenario: Must be used for hard failures, such as corrupted JSON payload formats (unparseable), business-invalid input data (e.g., a negative transfer value), or user IDs that aren’t found.

Delivery Tags and Channel Scope Limitations #

To match confirmations with delivered messages, RabbitMQ uses a unique numeric identifier called delivery_tag.

How Is a Delivery Tag Generated? #

  1. Every time the broker pushes a message to a consumer through a specific channel, the broker assigns a 64-bit integer sequence number starting from 1.
  2. This sequence number increases linearly for every subsequent message delivered in the same channel.
  3. This sequence number is unique only within that channel’s scope. If we open two different channels on the same TCP connection, each channel has its own delivery_tag sequence starting from 1.

Important Limitation: Channel Scope Lock #

All confirmation instructions (basic.ack, basic.nack, or basic.reject) must be sent using the same channel used when receiving the message.

sequenceDiagram
    autonumber
    actor Consumer as Consumer
    participant Ch1 as "Channel 1"
    participant Ch2 as "Channel 2"
    
    Ch1 ->> Consumer: "basic.deliver (DeliveryTag: 100)"
    Note over Consumer: Sends ACK
    Consumer ->> Ch2: "basic.ack (DeliveryTag: 100)"
    Note over Ch2: "Not the original channel! Triggers PRECONDITION_FAILED Channel Exception"

If our consumer application receives a message with delivery_tag = 100 on Channel 1, then due to a multi-threading (goroutine) design error our application sends the basic.ack(100) instruction through Channel 2:

  • The broker considers this a serious protocol violation.
  • The broker triggers a PRECONDITION_FAILED error check at the broker level.
  • The broker automatically force-closes (crash close) Channel 2 used to send that wrong ACK, and returns the original message on Channel 1 to the queue.

Batch Acknowledgement Mechanisms (Multiple Ack) #

To save network I/O load on high-throughput systems (hundreds of thousands of messages per second), RabbitMQ supports the Batch Acknowledgement feature using a custom parameter named multiple.

When a consumer calls the basic.ack function with the multiple = true parameter:

  • This instruction tells the broker: “Mark as successful and delete all messages with delivery_tag less than or equal to the tag I’m sending now.”
  • For example, if a consumer has received messages with delivery tags 1, 2, 3, 4, and 5 on one channel. If the consumer sends one basic.ack(5, multiple = true) command, the broker confirms and deletes all five messages at once from disk/RAM in a single transaction operation.
flowchart TD
    subgraph PesanDiterima["Received Messages"]
        direction LR
        T1["Tag 1"]
        T2["Tag 2"]
        T3["Tag 3"]
        T4["Tag 4"]
        T5["Tag 5"]
    end
    
    T5 -->|"basic.ack(Tag 5, multiple = true)"| Ack["Result: All five messages are collectively ACKed."]

Batch Ack Risks in Production #

Although it significantly increases throughput performance, using multiple = true has risks:

  • If the consumer application crashes mid-way while processing message 4 (before sending the batch ACK for message 5), the broker considers all messages from 1 to 5 as not yet confirmed. As a result, all five messages are redelivered to a new consumer, increasing the chance of duplicate data processing.

The Poison Message Phenomenon and Retry Loop Breaking Strategies #

One of the biggest threats to queue system stability in production is the Poison Message.

How Does a Poison Message Form? #

A poison message occurs when a producer publishes a message with corrupted data structures, empty values on required parameters, or triggers logical bugs in the consumer application code. When this message is consumed:

  1. The consumer application code errors while processing it.
  2. Due to immature error handling, the consumer sends a basic.nack(requeue = true) signal.
  3. The message returns to the original queue at the front position.
  4. The same consumer (or another parallel consumer) immediately reads that message again from the queue.
  5. The application errors again, sends another NACK requeue, and this cycle repeats endlessly (infinite retry loop).

This cycle consumes broker and consumer CPU utilization up to 100%, floods log files with millions of identical error lines, and clogs the queue so other normal messages never get a chance to be processed.

flowchart TD
    A["Consume (basic.deliver)"] -->|"Application Crash"| B["basic.nack (requeue=true)"]
    B -->|"Message Back to Queue Head"| A

Robust Error Management Design Solutions #

To break this poison cycle in production, we must combine the following strategies:

1. Dead Letter Exchange (DLX) and Dead Letter Queue (DLQ) #

Configure our queue with the x-dead-letter-exchange and x-dead-letter-routing-key arguments. When the consumer detects a permanent error (like an invalid data format), the consumer must send basic.reject(requeue = false). The broker automatically routes the corrupted message to a dedicated DLQ for isolation without blocking the main queue.

2. Retry Limit Tracking #

The RabbitMQ Quorum Queue has an internal feature called x-delivery-count that automatically records how many times a message has been redelivered to consumers. We can read this header on the consumer application side. If x-delivery-count exceeds a limit (e.g., 5 times), the consumer must stop requeueing and manually discard the message to the DLQ.

3. Delayed Retry (Delayed Message Exchange Plugin) #

If the failure is caused by temporary network disruptions (e.g., an external API is down), don’t immediately requeue the message instantly. Send the message to a delayed exchange to postpone redelivery for 5 minutes so the external system has time to recover first.

Go Code Implementation: Handling Manual ACK, Nack, and DLX Routing #

Here is a complete Go language implementation demonstrating safe manual confirmation handling. This code includes error type evaluation to decide whether a message should be ACKed, requeued, or sent to the Dead Letter Queue (DLQ).

package main

import (
	"context"
	"encoding/json"
	"log"
	"os"
	"os/signal"
	"syscall"
	"time"

	amqp "github.com/rabbitmq/amqp091-go"
)

// Transaction order message data structure
type Order struct {
	OrderID string  `json:"order_id"`
	Amount  float64 `json:"amount"`
}

func main() {
	conn, err := amqp.Dial("amqp://guest:***@localhost:5672/")
	if err != nil {
		log.Fatalf("Gagal terhubung ke RabbitMQ: %s", err)
	}
	defer conn.Close()

	ch, err := conn.Channel()
	if err != nil {
		log.Fatalf("Gagal membuka channel: %s", err)
	}
	defer ch.Close()

	// Limit the QoS prefetch
	err = ch.Qos(10, 0, false)
	if err != nil {
		log.Fatalf("Gagal menyetel QoS: %s", err)
	}

	queueName := "orders-processing-queue"

	msgs, err := ch.Consume(
		queueName,
		"",    // consumer tag
		false, // autoAck: set to false for manual confirmation!
		false, // exclusive
		false, // noLocal
		false, // noWait
		nil,   // arguments
	)
	if err != nil {
		log.Fatalf("Gagal mendaftarkan konsumen: %s", err)
	}

	sigChan := make(chan os.Signal, 1)
	signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)

	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	go func() {
		for {
			select {
			case <-ctx.Done():
				return
			case d, ok := <-msgs:
				if !ok {
					return
				}

				var order Order
				// Evaluate the payload data format
				err := json.Unmarshal(d.Body, &order)
				if err != nil {
					// PERMANENT ERROR (Corrupted JSON Format): Don't requeue!
					log.Printf("[ERROR] Payload rusak untuk Tag %d: %s. Mengirim ke DLX...", d.DeliveryTag, err)
					
					// basic.reject with requeue = false routes the message to the DLX
					rejectErr := d.Reject(false)
					if rejectErr != nil {
						log.Printf("Gagal reject pesan: %s", rejectErr)
					}
					continue
				}

				// Evaluate business logic validity
				if order.Amount <= 0 {
					// BUSINESS LOGIC ERROR (Negative transaction value): Can't be reprocessed!
					log.Printf("[ERROR] Nilai transaksi tidak valid (Amount: %.2f). Menolak pesan...", order.Amount)
					
					// Send to the DLX so it doesn't clog the queue
					rejectErr := d.Reject(false)
					if rejectErr != nil {
						log.Printf("Gagal reject pesan: %s", rejectErr)
					}
					continue
				}

				// Simulate transaction processing to the database
				dbSuccess := processOrderInDatabase(order)
				if !dbSuccess {
					// TEMPORARY ERROR (Database connection dropped): Requeue to retry later
					log.Printf("[WARN] Gagal menulis ke database. Mengembalikan ke antrean (requeue = true)...")
					
					// basic.nack with requeue = true
					nackErr := d.Nack(false, true)
					if nackErr != nil {
						log.Printf("Gagal nack pesan: %s", nackErr)
					}
					continue
				}

				// SUCCESS: Send the positive acknowledgement (basic.ack)
				ackErr := d.Ack(false)
				if ackErr != nil {
					log.Printf("Gagal mengirim ACK: %s", ackErr)
				} else {
					log.Printf("✓ Transaksi %s sukses terproses dan di-ACK.", order.OrderID)
				}
			}
		}
	}()

	<-sigChan
	log.Println("[*] Memulai graceful shutdown...")
	cancel()
	time.Sleep(1 * time.Second)
}

func processOrderInDatabase(order Order) bool {
	// Simulate a database function (return false if there's a simulated network error)
	return true
}

Anti-Patterns vs Practical Solutions in Production #

Avoid the following fatal mistakes in designing message confirmation flows:

1. Sending ACK Signals Using a Different Connection/Channel #

Receiving messages in the main socket reader loop, then splitting the database storage process into another goroutine thread, and sending the ACK signal using a new channel object dynamically declared by that thread.

Why is this wrong? #

As explained earlier, RabbitMQ locks delivery_tag usage per channel. Sending an ACK with the same tag on a different channel triggers a PRECONDITION_FAILED error and destroys the channel socket connection.

  • Solution: Make sure the original channel object that received the message is safely passed as an input parameter (dependency injection) into the processing goroutine thread. All ACKs must be called through that original channel object.

2. Requeueing Without Limits on Internal Application Code Failures #

Catching all exception errors in application code (including Null Pointer Dereference errors, division by zero, etc.) then returning messages to the queue using requeue = true hoping the bug will disappear by itself.

Why is this wrong? #

Code bugs in applications are static and won’t heal themselves without a new code fix release. Unlimited requeueing for code bugs only triggers pointless processing cycles that consume server CPU utilization.

  • Solution: Create a clear error type classification. Use disciplined try-catch handling. Only requeue (requeue = true) for external infrastructure errors (network disconnects). For internal code errors or data validation, route directly to the Dead Letter Queue (requeue = false).

Summary #

  • AMQP Signal Semantics — Use basic.ack for success, basic.nack for collective/batch rejection, and basic.reject for rejecting a single message.
  • Strategic Requeue Options — Set requeue = true only for temporary infrastructure failures, and use requeue = false (diverting to the DLX) for permanent business logic failures.
  • Delivery Tag Scope — Delivery tags are unique per channel. Sending an ACK on the wrong channel triggers a channel crash close by the broker.
  • Batching with Multiple Ack — The multiple = true setting ACKs all messages up to a specific tag at once, increasing throughput but risking large redeliveries if consumers crash.
  • Poison Message Danger — Unlimited requeueing for corrupted messages triggers infinite failure loops that block queues and burden CPU utilization.
  • DLQ & Retry Limit Mitigation — Use Dead Letter Queues for data isolation, delayed exchanges for retry postponement, and track the x-delivery-count metric to limit retry counts.

← Previous: Delivery   Next: At-most-once →

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact