Routing & Binding #

In message routing architecture using RabbitMQ, after a message successfully passes the Publishing gateway and is received by the exchange, the broker enters the next crucial phase in the message lifecycle: Routing & Binding Evaluation.

This is where the broker processes incoming binary data and answers fundamental questions: which queue should this message be delivered to, how should its route criteria be matched, and what should be done if the message has no destination. This evaluation process runs entirely inside the broker under the Erlang BEAM VM runtime’s control. The speed and efficiency of this stage determine our system’s overall throughput. This article thoroughly unpacks the Mnesia route table structure at the code level, Erlang’s RAM-saving mechanism through ProcBin binary pointers, message rescue via Alternate Exchanges, and the danger of cluster schema locks (Mnesia schema locks) from dynamically modifying bindings at runtime.

The Position of Route Evaluation in the Lifecycle #

Route evaluation acts as a logical bridge between the Exchange and destination queues. The Exchange in RabbitMQ is only a logical (virtual) entity defined as a metadata entry in the Mnesia database. Exchanges don’t have their own message storage queues (mailboxes).

flowchart TD
    Msg["Incoming Message (Routing Key: 'sales.order')"] --> Ex["Exchange (events.topic)"]
    Ex --> RouteEval{"rabbit_router:route/2"}
    
    subgraph MnesiaETS["ETS Memory Lookup (rabbit_route)"]
        RouteEval --> QueryETS["Query ets:lookup/2"]
    end
    
    QueryETS -->|Match 1| Pointer1["ProcBin Pointer (24 bytes)"] --> QueueA["Queue A (Mailbox)"]
    QueryETS -->|Match 2| Pointer2["ProcBin Pointer (24 bytes)"] --> QueueB["Queue B (Mailbox)"]
    
    subgraph SharedHeap["Global Off-Heap Memory (Refc Binary)"]
        Payload["Actual Message Payload (10 MB)"]
    end
    
    Pointer1 -.-> Payload
    Pointer2 -.-> Payload

The Route Evaluation Flow #

When the basic.publish Method Frame is processed by the broker channel:

  1. Header Extraction: The broker extracts the exchange name and routing key from the frame.
  2. Binding Lookup: The broker calls the internal rabbit_router:route/2 function. This function queries the ETS (Erlang Term Storage) memory table named rabbit_route to get all binding rules registered on that exchange.
  3. Route Matching: Based on the exchange type (Direct, Fanout, Topic, or Headers), the broker performs string comparison or Trie tree traversal to filter which queues are entitled to receive the message.
  4. Enqueueing: The message is then placed into the Erlang process mailboxes representing those destination queues.

Mnesia Route Table Queries and ETS Bags #

The RabbitMQ cluster binding metadata table is stored permanently in the distributed Mnesia database. However, querying Mnesia disk for every flowing message would cripple broker performance. Therefore, RabbitMQ projects that binding data into ETS RAM memory tables when the broker runs.

The rabbit_route Table Structure #

The rabbit_route ETS table is configured using the bag data type. The bag table type characteristic in Erlang lets us store several data entries (records) with the same key in parallel.

  • Key Format: The search key in the rabbit_route table is the source exchange name.
  • Record Structure: Every record in the table contains the information: {ExchangeName, BindingKey, QueueName, Arguments}.
  • Direct Exchange Lookup: For Direct Exchanges, the broker only needs to call ets:lookup(rabbit_route, ExchangeName), which runs with $O(1)$ time complexity. The broker compares the BindingKey value from the lookup result with the message Routing Key to instantly get the destination queue name.
  • Fanout Exchange Lookup: The broker takes all binding records for that exchange without caring about the BindingKey field. The message is directly duplicated to all registered queues.

Payload Replication and Erlang ProcBin Pointers #

One of RabbitMQ’s rarely known performance advantages is its ability to duplicate messages to many queues without consuming much broker RAM.

The Traditional Memory Duplication Problem #

If a producer sends a 10 MB message to a Fanout Exchange with 100 bound queues, a traditional system must copy that 10 MB payload 100 times to each queue’s memory, wasting 1 GB of RAM instantly for a single message. This triggers Garbage Collection churn that kills broker performance.

The Refc Binaries and ProcBin Pointer Solution in the Erlang BEAM #

The Erlang BEAM VM runtime solves this problem by separating binary storage by size:

  1. Heap Binaries (< 64 bytes): Very small messages are stored directly in each queue process’s local heap memory.
  2. Refc Binaries (> 64 bytes): Messages above 64 bytes are stored in global shared memory outside the process heap (off-heap memory), called Refc Binaries (Reference-Counted Binaries).
  3. Single Allocation: When a 10 MB message enters the exchange, RabbitMQ allocates the 10 MB memory only once in that shared off-heap memory.
  4. ProcBin Term: For every matching destination queue (e.g., 100 queues), the broker only sends a small 24-byte term named ProcBin to that queue’s Erlang process mailbox. This ProcBin term only contains a memory pointer address to the 10 MB payload in shared off-heap and increments the reference counter value.

When a consumer reads a message from one of the queues, it accesses the data through that pointer. After the message is successfully acked and deleted from the queue, the binary reference counter is decremented by 1. When the reference counter reaches zero, the Erlang runtime automatically frees that 10 MB memory. This mechanism makes mass (fanout) routing processes in RabbitMQ run very fast with minimal memory consumption.

Message Rescue via Alternate Exchange (AE) #

When a producer sends a message with mandatory = true and the message fails to route because no binding matches, the message is returned to the producer. However, if the producer doesn’t process message bounces in real time, or if mandatory is false, the message is silently discarded.

To rescue these unroutable messages without burdening the producer application with error-handling logic, RabbitMQ provides the Alternate Exchange (AE) feature.

How the Alternate Exchange Works #

An Alternate Exchange is a backup exchange bound to the main exchange through the "alternate-exchange" declaration argument.

flowchart LR
    A["Publish"] --> B["Main Exchange"] -->|"No Route"| C["Alternate Exchange"] --> D["Backup Queue (Orphan)"]
  1. First Evaluation: The message is published to the Main Exchange.
  2. Route Failed: The broker detects no matching queue binding on the Main Exchange.
  3. Route Diversion: The broker checks whether the Main Exchange has the "alternate-exchange" argument. If yes, the broker immediately moves the message to the Alternate Exchange.
  4. Acceptance: The Alternate Exchange (usually configured as a Fanout type) routes the message to a backup holding queue (an orphan queue) for bug investigation or audit needs.
  5. Priority: The diversion to the Alternate Exchange cancels the message bounce-back process to the producer, so the producer won’t receive a basic.return signal even if mandatory is true.

Go Code: Configuring an Alternate Exchange #

Here is a complete Go language example declaring a fanout-type Alternate Exchange, binding it to a topic-type main exchange, and verifying unroutable message rescue.

package main

import (
	"context"
	"log"
	"time"

	amqp "github.com/rabbitmq/amqp091-go"
)

func main() {
	// 1. Connect to RabbitMQ
	conn, err := amqp.Dial("amqp://guest:***@localhost:5672/")
	if err != nil {
		log.Fatalf("Gagal terhubung ke RabbitMQ: %s", err)
	}
	defer conn.Close()

	ch, err := conn.Channel()
	if err != nil {
		log.Fatalf("Gagal membuka channel: %s", err)
	}
	defer ch.Close()

	// 2. Declare the Backup Alternate Exchange (AE)
	aeName := "my-alternate-exchange"
	err = ch.ExchangeDeclare(
		aeName,
		"fanout", // AE is best as Fanout to hold all failed messages
		true,     // durable
		false,    // auto-delete
		false,
		false,
		nil,
	)
	if err != nil {
		log.Fatalf("Gagal mendeklarasikan AE: %s", err)
	}

	// 3. Declare the Backup Holding Queue (Orphan / Unrouted Messages)
	orphanQueue, err := ch.QueueDeclare(
		"unrouted-messages-queue",
		true,
		false,
		false,
		false,
		nil,
	)
	if err != nil {
		log.Fatalf("Gagal mendeklarasikan antrean unrouted: %s", err)
	}

	// Bind the backup queue to the AE
	err = ch.QueueBind(orphanQueue.Name, "", aeName, false, nil)
	if err != nil {
		log.Fatalf("Gagal mengikat antrean unrouted ke AE: %s", err)
	}

	// 4. Declare the Main Exchange linked to the AE
	mainExchangeName := "main-topic-exchange"
	mainExchangeArgs := amqp.Table{
		"alternate-exchange": aeName, // THE KEY CONNECTING THE MAIN EXCHANGE TO THE AE!
	}
	err = ch.ExchangeDeclare(
		mainExchangeName,
		"topic", // The main exchange uses Topic
		true,
		false,
		false,
		false,
		mainExchangeArgs,
	)
	if err != nil {
		log.Fatalf("Gagal mendeklarasikan Exchange Utama: %s", err)
	}

	// 5. Send a message to the Main Exchange with a Routing Key that will NOT match any queue
	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()

	payload := []byte(`{"error_report":"format_invalid","code":500}`)
	err = ch.PublishWithContext(ctx,
		mainExchangeName,
		"unmatched.routing.key", // Random routing key not bound to any queue
		true,                   // mandatory = true (the AE cancels the bounce)
		false,
		amqp.Publishing{
			DeliveryMode: amqp.Persistent,
			ContentType:  "application/json",
			Body:         payload,
		},
	)
	if err != nil {
		log.Fatalf("Gagal mempublikasikan pesan: %s", err)
	}
	log.Println("✓ Pesan dipublikasikan. Karena rute utama gagal, pesan dialihkan secara otomatis ke Alternate Exchange.")
}

Erlang Process Mailbox Flow & Backlog Handling #

After a matching route is found in the ETS table, the message must be delivered from the channel process (rabbit_channel) to the destination queue process (rabbit_amqqueue_process). This message-moving process uses the actor message passing architecture in the Erlang runtime.

Message Delivery to Actor Mailboxes #

At the internal Erlang level, every queue is represented by one isolated actor process with a memory mailbox. This mailbox is unbounded by default.

  • Mailbox Enqueue: The channel process sends the ProcBin pointer term to the destination queue process’s mailbox using the Erlang send instruction (!). This process runs asynchronously and very fast.
  • Backlog Bottleneck: If our consumer application is very slow, the queue process can’t process messages from its mailbox fast enough. Even though message payloads are safe in shared off-heap, the pointer pile-up in the Erlang process mailbox keeps growing, increasing RAM consumption and slowing the BEAM VM process scheduling performance.
  • Backpressure to Channels: To prevent fatal failures from mailbox bloat, RabbitMQ applies an internal coordination mechanism. If queue length exceeds a certain limit, the queue process sends control signals to all connected producer channels to slow down TCP socket reads, effectively suppressing the producer’s Publishing rate upstream.

Optimizing for Legacy Dynamic Binding #

Although dynamic binding is an anti-pattern to avoid in production, some legacy systems are forced to apply it. To minimize Mnesia schema lock impacts during dynamic binding, we can tune RabbitMQ startup parameters:

  • mnesia.dump_log_write_threshold: Sets the Mnesia transaction log write threshold before dumping to disk. Raising this parameter (e.g., to 50000) reduces the frequency of synchronous Mnesia disk writes during dynamic bind/unbind activity.
  • mnesia.dc_dump_limit: Controls the Mnesia dump file size limit to prevent sudden disk I/O freezes when Mnesia cleans up inactive binding route logs.

Anti-Pattern: Dynamic Binding Churn and Mnesia Schema Locks #

One of the most dangerous operational mistakes when designing RabbitMQ-based applications is treating Bindings as dynamic entities created and deleted at will during application runtime.

The Mistake Case: Dynamic Binding Churn #

For example, a web application creates a new binding to an exchange every time a new user WebSocket connection arrives, then unbinds that binding when the connection closes.

Why is this dangerous? #

  1. Mnesia Schema Locks: Every binding creation and deletion operation (QueueBind or QueueUnbind) requires the broker to do a schema modification transaction on the distributed Mnesia database. This transaction requires an exclusive schema lock across all cluster nodes.
  2. Cluster Throttling: If hundreds of users connect and disconnect randomly every minute, the RabbitMQ cluster spends most CPU cycles negotiating Mnesia schema locks and replicating Mnesia table updates to all nodes. During this process, normal message routing is temporarily blocked (throttling), causing delivery latency to spike drastically.
  3. RAM Degradation: Constant binding modifications damage the rabbit_route ETS table cache efficiency. The broker repeatedly empties and rebuilds search indexes in RAM.

Architectural Solution: #

Make sure all Exchanges, Queues, and Bindings are declared statically once (idempotent declaration) at application startup or through a separate infrastructure migration script before the application runs. If the application needs dynamic delivery to specific consumers, use a Topic Exchange with structured routing keys, where consumers connect using static exclusive queues listening to matching routing key patterns.

Summary #

  • Virtual Exchange — The Exchange in RabbitMQ is only a metadata entry representation in the Mnesia database and has no memory storage mailbox of its own.
  • ETS Bag Storage — Binding metadata is projected from the Mnesia database to the rabbit_route ETS memory table of type bag, which allows high-speed parallel lookups.
  • Erlang ProcBin Pointers — The Erlang BEAM VM prevents memory duplication when duplicating large messages to many queues by allocating binary data once in global off-heap memory and sending 24-byte ProcBin pointer terms to queue processes.
  • Alternate Exchange (AE) — Use the "alternate-exchange" argument when declaring the main exchange to automatically divert unroutable messages to a backup exchange.
  • Avoid Dynamic Binding — Dynamic bind/unbind operations at runtime trigger distributed Mnesia database schema locks (Mnesia schema locks) that can cripple cluster performance.
  • Route Evaluation Before Enqueue — The message routing process (rabbit_router:route/2) is completed on the broker side before the message is placed into queues.

← Previous: Publishing   Next: Persistent →

About | Author | Content Scope | Editorial Policy | Privacy Policy | Disclaimer | Contact