Message #

In asynchronous message-handling architecture, the Message is the smallest atomic unit of information exchange flowing across systems. All discussions about producer reliability, exchange routing flexibility, and consumer concurrency ultimately come down to how we design and manage the message itself. In RabbitMQ, a message is not just a raw data payload (like a JSON string) moved from one server to another. Technically, a message is a structured binary object wrapped by the AMQP (Advanced Message Queuing Protocol) protocol with various metadata, lifecycle rules, and physical implications for network and memory. This article dissects the physical anatomy of a message, frame segmentation efficiency on the network, serialization format comparisons, and operational message limits in production environments.

AMQP Frame Anatomy: The Physical Segmentation of Messages on the Network #

RabbitMQ communicates using the AMQP 0-9-1 protocol, which is a wire-level protocol. That means when a producer sends a logical message to the broker, the message is not delivered whole as one giant binary packet. The AMQP protocol physically splits the message into several small binary units called Frames.

flowchart TD
    subgraph Stream["TCP Socket Stream (Binary Stream)"]
        direction LR
        Frame1["Method Frame - basic.publish command"] --> Frame2["Content Header Frame - Metadata and Payload Size"]
        Frame2 --> Frame3["Body Frame 1 - Binary Data Fragment"]
        Frame3 --> Frame4["Body Frame N - Remaining Data Fragments"]
    end

This physical separation aims to prevent excessive memory buffer allocation on sockets and allows the broker to read data in a streaming fashion. There are three main frame types that make up one logical message:

1. Method Frame #

The first frame sent by the producer to tell the broker what action is being performed.

• Content: Contains the AMQP class (e.g., Basic Class), AMQP method (e.g., publish), the logical channel ID, and command arguments such as the exchange name and routing key.
• Function: Opens an instruction path for the broker to prepare to receive the next message data.

2. Content Header Frame #

After the method frame is sent, the producer immediately sends the content header.

• Content: Records the total payload size of the message in bytes, plus the built-in AMQP property table (such as delivery_mode, content_type, and the custom headers table).
• Function: Tells the broker how large a memory buffer it must allocate to read the message content.

3. Body Frames #

The final frames containing the actual payload data (message content).

• Content: Binary fragments of the message content.
• Mechanism: If the total message payload size exceeds the maximum frame size negotiated at connection initialization (Max Frame Size, default 128KB), AMQP cuts the payload into several consecutive Body Frames sent sequentially over the network.

These frame fragments are read sequentially by the Erlang process responsible for the TCP socket connection on the RabbitMQ node, reconstructed at the local memory level, and copied to the destination queue.

Frame Scheduling Management (AMQP Class-Method IDs) #

When the broker receives a Method Frame from a producer or sends one to a consumer, RabbitMQ must identify that specific instruction extremely quickly. To achieve this efficiency, the AMQP 0-9-1 specification maps every command into a structured binary number combination called the Class ID (2 bytes) and Method ID (2 bytes).

Inside RabbitMQ’s architecture, the Class ID defines the main functionality category, while the Method ID defines the specific action within that category. Here are some important Class and Method IDs we often encounter in the message lifecycle:

This structured division lets RabbitMQ’s network module (written in Erlang) do pattern matching directly at the binary socket stream representation level. For example, when RabbitMQ receives binary with class prefix 60 and method 40, the Erlang process responsible for the channel immediately knows the next incoming packets are the Content Header Frame and Body Frames containing the message, and immediately prepares its internal state without needing expensive string parsing.

Frame Size Limits and Large Payload Fragmentation Risks #

When a producer first performs the connection handshake with the RabbitMQ server, both negotiate the maximum single binary frame size configuration through the frame_max property. By default, RabbitMQ sets this limit to 131,072 bytes (128 KB).

What Happens if the Message Is Too Large? #

If a producer publishes a message of 1 MB:

1. The message is split into at least 8 Body Frames on the TCP network.
2. The Erlang VM on the broker must receive these 8 binary packets in sequence, allocate dynamic RAM buffers to reassemble them, and only then evaluate the routing key.
3. CPU and Memory Load (Overhead): The fragmentation and reconstruction process for megabyte-sized messages consumes significant CPU time. Worse, the Erlang VM uses a message copying model in memory (message copying). If a 10 MB message is duplicated to 5 different queues through a Fanout Exchange, RabbitMQ duplicates that 10 MB five times in Erlang Heap RAM, triggering an instant memory usage spike.
4. Garbage Collection Pressure: After large messages are consumed and deleted, the Erlang Garbage Collector must work hard sweeping those local memories, causing unstable CPU spikes.

Therefore, it’s very important to keep RabbitMQ message sizes as small as possible. Ideally, the average message size in production should stay under 100 KB, and it’s highly recommended to never exceed 10 MB.

Properties vs Headers: Built-in vs Custom Metadata #

AMQP separates message metadata into two main categories on the Content Header Frame: Properties (structured properties with a rigid schema) and Headers (flexible key-value pair tables).

1. Properties (Rigid Schema Metadata) #

Properties are fixed-size binary fields officially defined by the AMQP protocol specification. Because they have definite byte positions in the header frame, the broker’s parsing process for these properties runs very fast with minimal CPU consumption.

Here are the most important standard AMQP properties for operations:

• delivery_mode: Value 1 for transient messages (RAM only), value 2 for persistent messages (written to disk).
• content_type: Records the payload serialization type, helping applications avoid reading errors (e.g., application/json).
• message_id: The message’s unique ID. Producers must include it for audit logs and idempotency validation at the consumer level.
• correlation_id: A unique transaction ID, very helpful for distributed tracing to link logs from producer to consumer.
• reply_to: The destination queue name for sending reply messages (used in the Request-Response/RPC pattern).
• expiration: The message’s TTL (Time-To-Live) limit in milliseconds, calculated per individual message.

2. Headers (Custom Metadata) #

Headers is a dynamic dictionary table where developers are free to insert their own metadata (e.g., event version event_version: "v1.2", or sender identity source_app: "billing-service").

• Behavior: Unlike regular properties, Headers must be read dynamically from the byte stream. This requires a little more CPU time.
• Headers Exchange: We can use this Headers table as a message routing criterion using the Headers exchange type, where the broker matches key-values in the message headers against queue binding keys.

Data Serialization and Schema Contract Management #

Because RabbitMQ treats message payload content purely as an opaque binary stream (a binary stream with no known structure), our applications are fully responsible for defining how data objects are converted to binary bytes (serialization) when sent and back to objects (deserialization) when received.

There are three main serialization format options commonly used in production:

1. JSON (JavaScript Object Notation) #

• Advantage: Very human-readable, natively supported by almost every programming language, and flexible.
• Weakness: The resulting binary data size is relatively large because it repeats field names in text format. The character comparison process when parsing JSON consumes fairly high CPU time.

2. Protocol Buffers (Protobuf) #

• Advantage: A very compact and efficient binary serialization format. Protobuf parsing speed can be 5-10 times faster than JSON, and the resulting binary payload is much smaller (saving up to 70% bandwidth).
• Weakness: Pure binary data, so it cannot be read directly by humans during debugging without helper tools. Requires compiling .proto schema files.

3. Apache Avro #

• Advantage: Stores data in a very small binary format by including a schema reference. Very suitable for enterprise-scale event-driven systems.
• Schema Registry: Using Avro or Protobuf ideally combines with a centralized Schema Registry. This guarantees producers and consumers always share compatible data schema contracts (backward & forward compatibility).

The Impact of Serialization on Erlang Garbage Collection (GC) #

The serialization format choice on the producer and consumer sides has a direct, massive impact on the RabbitMQ broker’s internal performance. This is closely related to how the Erlang Virtual Machine (BEAM) manages heap memory and the Garbage Collection (GC) process.

Inside the BEAM VM, every Erlang process (such as the process responsible for a queue or channel) has its own isolated heap memory. When GC runs on a process, it only cleans that process’s local heap without stopping other Erlang processes (no global stop-the-world). However, this memory allocation behavior is strongly influenced by the characteristics of the message payload data type we pass:

1. Refc Binaries (Off-Heap Binaries) vs Heap Binaries #

Erlang divides binary data storage into two mechanisms based on size:

• Heap Binaries (≤ 64 bytes): Small binary data is stored directly in the Erlang process’s local heap. When a message moves from one channel to a queue, this binary data is physically copied (deep copy) between process heaps.
• Refc Binaries (> 64 bytes): Medium to large binary data is stored in a shared memory area outside the heap (global off-heap binary allocator). The local Erlang process only stores a small 24-byte reference object called a ProcBin on its local heap. When a message is routed to several different queues, RabbitMQ only copies the ProcBin (24-byte reference) to the destination queues, not the physical payload. This step prevents RAM data duplication and significantly saves memory I/O load.

2. Erlang GC Load on JSON vs Protobuf/Avro Parsing #

When we send a large JSON payload:

1. GC Pressure on Consumers & Producers: At the client application level, parsing JSON strings with many text keys forces programming languages (like Java, Python, or Node.js) to do thousands of small heap object allocations, triggering intensive GC on our application side.
2. Load on the Broker Side: Although RabbitMQ treats message content as a raw binary stream (opaque binary stream) without reading it, if we use a Headers Exchange with complex matching criteria on JSON header properties (or if the broker is forced to parse the payload for certain plugin needs), RabbitMQ must convert that text binary into Erlang term representations. Parsing JSON text forces the broker to create dynamic data structures in the Erlang heap, triggering more frequent Erlang GC cycles and increasing message processing latency (latency spikes).

Conversely, using binary serialization like Protocol Buffers or Apache Avro:

1. Compact Format: The binary size sent is very small, so it almost always falls into the Refc Binaries category. The broker only processes lightweight reference pointers.
2. Network Channel Optimization: Smaller payload sizes minimize the number of AMQP Body Frames that must be sent over the TCP connection. This reduces the frequency of socket buffer allocations on the broker, minimizes operating system interrupts, and keeps broker memory usage flat and stable even under very high traffic load.

Thus, the architectural decision to abandon JSON and switch to Protobuf/Avro in high-speed systems is not just about saving network bandwidth; it’s a vital tactic for keeping the RabbitMQ broker’s RAM memory and latency stable from fluctuations caused by Erlang Garbage Collection cycles.

Time-To-Live (TTL) and Message Expiration Lifecycle #

RabbitMQ lets us set message lifetime limits inside the system through the Time-To-Live (TTL) feature. If a message stays in a queue beyond the specified TTL limit without any consumer consuming it, the message is considered expired.

There are two ways to configure TTL in RabbitMQ:

A. Queue TTL (Queue-Level Lifetime Limit) #

• Mechanism: We set the x-message-ttl argument when declaring the queue.
• Behavior: All messages entering that queue automatically inherit the same TTL limit (e.g., 60 seconds). RabbitMQ can detect expiration linearly from the queue head efficiently because messages entering first will always expire first.

B. Message TTL (Per-Message Lifetime Limit) #

• Mechanism: The producer attaches the expiration property (in milliseconds) to each individual message header at publish time.
• Behavior: Each message has a different lifetime. This forces RabbitMQ to check expiration status periodically. Expired messages in the middle of the queue are not immediately physically deleted; they are only discarded when they reach the head of the queue and are about to be delivered to a consumer.

Expired messages are immediately deleted from the system, or automatically diverted to a Dead Letter Exchange (DLX) if the queue has been configured with the x-dead-letter-exchange parameter.

Anti-Pattern vs Solution: Sending Giant Binary File Payloads Directly #

One of the most fatal mistakes in designing message delivery systems is trying to use RabbitMQ as a medium for transferring large binary files.

Anti-Pattern Case: Sending Image / PDF Files in Message Payloads #

In this example, the producer reads a 15 MB PDF report file from disk, converts the entire binary file into a byte array, inserts it directly into the RabbitMQ message payload, and publishes it.

// ANTI-PATTERN: Sending large binary files directly in RabbitMQ payloads
func PublishLargeFileBad(ch *amqp.Channel, pdfPath string) {
    // Read the 15MB file into memory
    pdfBytes, _ := ioutil.ReadFile(pdfPath)
    
    // ✗ AVOID: Sending giant binary data directly to RabbitMQ
    _ = ch.Publish("document-exchange", "doc.upload", false, false, amqp.Publishing{
        DeliveryMode: amqp.Persistent,
        ContentType:  "application/pdf",
        Body:         pdfBytes, // 15MB payload!
    })
    
    // Problem: Causes fatal disk I/O on the RabbitMQ server, triggers OOM alarms,
    // and destroys the throughput of other small message deliveries.
}

Practical Solution: Using the Claim Check Pattern #

The best approach for handling large binary data is implementing the Claim Check Pattern. The producer stores the binary file in external object storage (such as Amazon S3, MinIO, or Google Cloud Storage), then only sends a small message containing reference metadata and the file’s access URL to RabbitMQ.

// CORRECT: Using the Claim Check Pattern to handle large files
type DocumentUploadedEvent struct {
    DocumentID string `json:"document_id"`
    StorageURL string `json:"storage_url"` // ✓ Only sends the reference URL
    FileSize   int64  `json:"file_size"`
}

func PublishLargeFileGood(ch *amqp.Channel, s3Client *S3Client, pdfPath string) {
    // 1. Upload the 15MB binary file directly to S3
    storageURL, docID, _ := s3Client.UploadFile(pdfPath)
    
    // 2. Create a small event payload containing only reference metadata
    event := DocumentUploadedEvent{
        DocumentID: docID,
        StorageURL: storageURL,
        FileSize:   15 * 1024 * 1024, // 15MB info
    }
    payload, _ := json.Marshal(event)
    
    // 3. ✓ SOLUTION: Publish the lightweight metadata message to RabbitMQ (Payload size < 1KB)
    _ = ch.Publish("document-exchange", "doc.upload", false, false, amqp.Publishing{
        DeliveryMode: amqp.Persistent,
        ContentType:  "application/json",
        Body:         payload,
    })
    
    // Consumers receiving this message download the binary file independently from S3
    // using the provided URL, freeing the RabbitMQ server from binary I/O load.
}

Summary #

• AMQP Frame Segmentation — The TCP binary flow that splits one logical message into Method, Header, and Body Frames to simplify socket RAM buffer allocation.
• Max Frame Size (128KB) — The negotiated frame limit that causes binary fragmentation when payload sizes exceed the limit, increasing Erlang GC computational load.
• Claim Check Pattern — A mandatory architecture pattern for handling large binary files by uploading them to S3/MinIO and only sending reference URL messages to RabbitMQ.
• Queue TTL vs Message TTL — Queue-level expiration limits evaluated linearly at the queue head are far more efficient than individual message-level TTL.

← Previous: Producer   Next: Exchange →