At-most-once #

When designing message-based systems using RabbitMQ, one of the most fundamental architectural decisions is determining the message delivery guarantee level. At the lightest end of the reliability spectrum, we find the At-Most-Once guarantee.

The main principle of the At-Most-Once guarantee is extreme simplicity and speed: every message is delivered at most once by the broker to consumer applications. If the message is successfully processed, the system runs normally. However, if a network failure, power outage, consumer application crash, or broker crash occurs mid-journey, the message is lost forever without any retry attempt or automatic recovery. This article thoroughly discusses the internal mechanisms of At-Most-Once in RabbitMQ, why this model can deliver the lowest latency, the analysis of silent data loss risks, valid production scenarios for applying it, and Go code implementation examples.

The Position of At-Most-Once in the Message Lifecycle #

The At-Most-Once guarantee is achieved by cutting the entire confirmation and message status tracking flow, both on the producer and consumer sides.

flowchart TD
    Producer["Producer (Fire-and-Forget)"] -->|"Publish Without Confirm"| Exchange["Exchange"]
    Exchange -->|"Route Message"| Queue["Queue (Non-Durable / Transient)"]
    
    subgraph BrokerSide["Internal Broker Process"]
        Queue -->|"Push Message (autoAck = true)"| Delivery["basic.deliver"]
        Delivery -->|"Instant Delete from Storage"| Delete["Message Deleted from RAM/Disk"]
    end
    
    Delivery --> Consumer["Consumer Application"]
    Consumer -->|"Process Business Logic"| Result{"Success?"}
    
    Result -->|Yes| Done["Done"]
    Result -->|"No (Crash / Error)"| Loss["Data Lost Forever (No Requeue)"]

When this guarantee is active, messages never linger in the Unacknowledged status. Messages are deleted from broker storage memory the instant they are pushed to the TCP network socket, trimming the message lifecycle to be very short.

At-Most-Once Trigger Mechanisms in RabbitMQ #

In RabbitMQ, the At-Most-Once delivery guarantee isn’t activated through a single configuration switch. Instead, this guarantee is the result of combining several configuration choices on our producer, queue, and consumer.

1. Consumer Automatic Confirmation (Auto-Ack) #

Auto-Ack (Automatic Acknowledgement) is the biggest factor creating At-Most-Once behavior on the receiving side. In the AMQP 0-9-1 protocol, this is configured by setting the no_ack = true parameter (or autoAck: true in client libraries) when calling the basic.consume method.

• Broker Behavior: When a consumer is registered in Auto-Ack mode, the RabbitMQ broker considers the message fully processed as soon as the broker successfully writes the message frame to the TCP network socket buffer.
• Instant Deletion: Without waiting for a response from the consumer application, the broker immediately deletes that message from RAM memory and disk.
• Crash Disaster: If right after receiving that TCP packet our consumer application crashes (e.g., from OutOfMemory, or a SIGKILL interrupt from the Kubernetes orchestrator), the message is lost forever because there is no backup message copy left in the broker to requeue.

2. Eliminating Publisher Confirms (Fire-and-Forget) #

On the sender (producer) side, At-Most-Once occurs when the producer publishes messages to the broker without enabling the Publisher Confirms feature.

• How It Works: The producer sends the message using the basic.publish command and immediately assumes the message was successfully stored just because the local TCP socket write succeeded.
• Risk: If the Exchange misroutes the message (an unroutable message), or if the RabbitMQ broker experiences a power outage right before writing the message to disk/RAM, the producer will never know and that data disappears silently.

3. Using Transient Queues and Non-Persistent Messages #

To maximize At-Most-Once performance, developers usually combine it with volatile storage configurations:

• Declaring queues with the durable = false flag (queues only exist in RAM memory and are deleted if the broker restarts).
• Publishing messages with the delivery_mode = 1 property (Transient, messages must not be written to disk).
• Result: Eliminating disk I/O operations entirely ensures processing speeds equal to RAM memory speed, but with zero tolerance for hardware failures.

Performance Analysis: Why Is At-Most-Once So Fast? #

For systems processing massive data volumes (like billions of log events per day), every coordination operation between the broker, disk, and network is an expensive latency burden. At-Most-Once drastically eliminates all this coordination.

Let’s compare the internal broker workload differences between safe mode (At-Least-Once) and fast mode (At-Most-Once):

By eliminating disk write load, consensus replication, and RAM status tracking, the RabbitMQ broker can serve message delivery throughput up to 5 times higher with consistent latency under 1 millisecond. This makes it an attractive choice for systems prioritizing performance over data accuracy.

The Danger of Silent Data Loss and Diagnostic Difficulties #

The worst consequence of the At-Most-Once guarantee is its silent failure nature. When messages are lost under this model, the system doesn’t trigger any built-in alarms in RabbitMQ.

Why Is It Hard to Diagnose? #

1. The Broker Dashboard Looks Healthy: The RabbitMQ admin dashboard shows very clean queue graphs (zero backlog, zero Ready messages, and zero Unacked messages). However, this cleanliness is fake because incoming messages are deleted instantly, even though the consumers below are experiencing mass crashes and processing nothing.
2. Zero Traces in Logs: The broker doesn’t record any failures because for the broker, the delivery task to the TCP socket was successfully completed.
3. Producers Assume Success: The producer application keeps writing success status logs because the local delivery network socket doesn’t detect errors.

Therefore, if we apply At-Most-Once without designing strict application-level monitoring, we may only realize data loss has occurred after our application users file complaints or when our analytics database data shows severe inconsistencies.

Valid Production Use Cases #

Although it sounds dangerous for business transactions, the At-Most-Once guarantee is very useful and recommended for the following use cases with high data loss tolerance:

1. IoT Metrics and Telemetry Streaming #

Imagine an IoT sensor on a factory machine sending temperature metrics every 100 milliseconds.

• Analysis: If a momentary network disruption causes 5 metric messages to be lost, that’s not critical. The 6th message arriving 100 milliseconds later carries the latest valid temperature data for detecting overheating trends. Trying to retry data from 500 milliseconds ago is actually useless because that data is already stale.

2. Non-Critical Application Telemetry Logs #

Sending user activity logs (like mouse cursor coordinates, or clicked button tracking for UI analytics).

• Analysis: Losing a few click log lines won’t damage the payment system or cause financial losses. High throughput and minimal CPU load on web servers are far more valuable.

3. Idempotent Cache Invalidation #

Sending cache eviction signals to several application server nodes when there’s a data update in the main database.

• Analysis: If one server node fails to receive the cache invalidation signal due to an auto-ack crash, the cache data on that node naturally expires based on the configured cache TTL time, or gets deleted at the next data update.

Go Code Implementation: Configuring At-Most-Once (Auto-Ack & Transient) #

Here is a complete Go code example practicing the At-Most-Once configuration. On the producer side, we send transient messages without Publisher Confirms. On the consumer side, we register a consume function with the autoAck argument set to true.

package main

import (
	"context"
	"log"
	"os"
	"os/signal"
	"syscall"
	"time"

	amqp "github.com/rabbitmq/amqp091-go"
)

const (
	amqpURL   = "amqp://guest:***@localhost:5672/"
	queueName = "iot-sensor-metrics"
)

func main() {
	// 1. Open a TCP Connection
	conn, err := amqp.Dial(amqpURL)
	if err != nil {
		log.Fatalf("Gagal terhubung ke RabbitMQ: %s", err)
	}
	defer conn.Close()

	ch, err := conn.Channel()
	if err != nil {
		log.Fatalf("Gagal membuka channel: %s", err)
	}
	defer ch.Close()

	// 2. Declare a Transient Queue (Non-Durable)
	// We set durable = false so the queue disappears when the broker restarts
	_, err = ch.QueueDeclare(
		queueName,
		false, // durable: false (Transient Queue)
		true,  // auto-delete: true (deleted automatically if there's no consumer)
		false, // exclusive
		false, // no-wait
		nil,   // arguments
	)
	if err != nil {
		log.Fatalf("Gagal mendeklarasikan antrean: %s", err)
	}

	// Run the Producer (Publisher) asynchronously
	go startTransientPublisher(conn)

	// 3. Run the Consumer with Auto-Ack (no_ack = true)
	msgs, err := ch.Consume(
		queueName,
		"",    // consumer tag
		true,  // autoAck: true (THE MAIN KEY TO AT-MOST-ONCE ON THE CONSUMER!)
		false, // exclusive
		false, // no-local
		false, // no-wait
		nil,   // arguments
	)
	if err != nil {
		log.Fatalf("Gagal mendaftarkan konsumen: %s", err)
	}

	sigChan := make(chan os.Signal, 1)
	signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)

	log.Println("[*] Konsumen At-Most-Once aktif. Tekan Ctrl+C untuk keluar.")

	go func() {
		for d := range msgs {
			// Once the data is read from the Go channel, the RabbitMQ broker is GUARANTEED
			// to have permanently deleted this message from its memory.
			log.Printf("Menerima metrik: %s (ID Pesan: %s)", d.Body, d.MessageId)

			// Simulate fast processing (e.g., parsing sensor data)
			// DON'T put heavy processes or slow database calls here
			// because if this goroutine crashes mid-way, the data is lost forever.
			processMetrics(d.Body)
		}
	}()

	<-sigChan
	log.Println("[*] Shutdown aman diselesaikan.")
}

func startTransientPublisher(conn *amqp.Connection) {
	ch, err := conn.Channel()
	if err != nil {
		log.Printf("Gagal membuka channel produsen: %s", err)
		return
	}
	defer ch.Close()

	// DON'T CALL ch.Confirm(false)!
	// We intentionally eliminate Publisher Confirms for fire-and-forget performance

	ticker := time.NewTicker(500 * time.Millisecond)
	defer ticker.Stop()

	ctx := context.Background()

	for range ticker.C {
		payload := []byte(`{"sensor_id":"SNSR-JAK-01","temp":28.4,"timestamp":1781290382}`)
		
		// 4. Publish a Transient Message (delivery_mode = 1)
		err = ch.PublishWithContext(ctx,
			"", // Default Exchange
			queueName,
			false, // mandatory
			false, // immediate
			amqp.Publishing{
				DeliveryMode: amqp.Transient, // delivery_mode = 1 (Transient!)
				ContentType:  "application/json",
				Body:         payload,
				MessageId:    "msg_unique_uuid_sensor_data",
			},
		)
		if err != nil {
			log.Printf("Gagal publish metrik: %s", err)
		}
	}
}

func processMetrics(data []byte) {
	// Simple parsing logic
}

Anti-Pattern vs Data Safety Solutions #

One of the most common architectural mistakes found in production is unconsciously applying the At-Most-Once pattern due to a lack of understanding of client library default settings.

Anti-Pattern: Enabling Auto-Ack to Chase Performance Benchmarks #

A developer is required to increase the throughput of an application’s payment processing transactions. While reading documentation, he discovers that changing the autoAck setting to true can multiply processing speed up to 300%. He immediately enables it on the production server without changing the application’s error handling code.

Why is this wrong? #

Payment systems demand 100% data consistency guarantees. With autoAck = true, if the consumer application runs out of memory or the main database connection drops mid-transaction validation, RabbitMQ considers the transaction successfully delivered and deletes the payment message from the queue. Upstream will never know why that transaction data disappeared without a trace, causing real financial losses for the business.

• Solution: Change the delivery guarantee configuration back to At-Least-Once. Set autoAck = false, use manual confirmation (d.Ack(false)) after the database successfully commits the transaction, and control backpressure using QoS prefetch limits (e.g., prefetch = 100). Losing a little network RTT latency is far cheaper than losing our valuable financial transaction data.

Summary #

• At-Most-Once Definition — The guarantee that messages are delivered at most once. Messages can be lost if system failures occur, but duplicate processing is guaranteed never to happen.
• Main Configuration — Activated by setting Auto-Ack (no_ack = true) on consumers, eliminating Publisher Confirms on producers, and using non-durable queues with transient messages.
• Data Loss Causes — Data disappears instantly if consumers crash, run out of RAM memory, or the TCP connection drops right after the broker writes the message to the network socket.
• Performance Advantage — Eliminating disk I/O write costs (fsync), RAM memory status tracking, network confirmation negotiation, and cluster replication consensus produces the lowest latency.
• Blind Diagnostics — At-Most-Once failures leave no traces in broker logs, and queue monitoring dashboards always show deceptive zero-backlog status.
• Appropriate Use Cases — Ideal for high-frequency IoT sensor telemetry streaming, non-critical application debug log collection, and idempotent cache invalidation signals.

← Previous: Acknowledgement   Next: At-least-once →