Asynchronous #

In modern software engineering, we often hear the advice to “use asynchronous processing” so our applications run faster and more responsively. But for many developers, the concept of asynchronicity is often equated with local multithreaded programming, using async/await libraries, or running background threads within a single machine.

When we jump to large-scale distributed system architecture (distributed systems), the concept of asynchronicity undergoes a fundamental paradigm shift. Asynchronicity at the distributed system level is no longer just about how the CPU manages our local threads; it’s about how independent systems communicate across networks without blocking each other (non-blocking), breaking time dependencies (temporal decoupling), and managing data consistency gradually (eventual consistency).

This article dissects the philosophy of asynchronous communication in depth, how non-blocking data flow works, the business performance advantages it produces, and the data consistency management challenges that accompany it.

The Philosophy of Asynchronicity and the Paradigm Shift #

To understand the power of asynchronous processing, we must compare it directly with the synchronous communication pattern. Synchronous communication is the default pattern of most web protocols today.

1. The Synchronous Pattern #

In a synchronous pattern, when Service A sends data to Service B, the executing thread on Service A enters a blocked state. That thread cannot serve other user requests, cannot perform CPU calculations, and cannot write data to its own local database. It is forced to sit idle waiting for the entire process on Service B to finish and the network response to be sent back.

If the network is congested, or Service B is slow, Service A’s thread stays held until timeout. This pattern forces all system components to act as one rigid unit of time.

2. The Asynchronous Pattern #

In an asynchronous pattern, we separate the action of sending data from the action of processing data. When Service A wants to send information to Service B:

Service A packages the data into a message.
Service A sends the message to a message broker (RabbitMQ broker).
RabbitMQ receives the message, writes it to memory/disk, and immediately sends a publisher confirm back to Service A within milliseconds.
Service A receives the confirmation, releases its worker thread back to the pool, and immediately continues serving the next user.
In the background, Service B pulls the message from RabbitMQ on its own when it has spare capacity.

This paradigm shift frees our system from the shackles of immediate availability. We no longer demand that Service B be actively stable at the same millisecond Service A sends data. Communication still succeeds even if Service B is completely down.

Asynchronous Message Passing #

The asynchronous message passing mechanism inside RabbitMQ is fully supported by the Erlang BEAM VM runtime architecture on the server side and non-blocking client libraries on our application side.

When we use an efficient RabbitMQ client library, the message-sending process does not block our application’s main thread. The client library uses a background write thread and an internal in-memory queue on the client. When we call the Publish function, the message is placed into the client’s in-memory queue and the function immediately returns control to the main thread. The client’s background thread then sends that data over the TCP socket to RabbitMQ asynchronously.

On the RabbitMQ server side, the Erlang BEAM VM uses an Event Loop mechanism based on epoll/kqueue at the operating system kernel level to detect incoming data on TCP sockets. Messages are read without blocking operating system threads, then copied to the mailbox of the Erlang process responsible for managing the destination exchange and queue. This architecture guarantees very high throughput with minimal internal processing latency.

Here is a sequence diagram comparing execution thread behavior between synchronous (blocking) calls and RabbitMQ-based asynchronous (non-blocking) calls:

sequenceDiagram
    autonumber
    actor User as User
    participant App as Upstream App (Service A)
    participant Broker as RabbitMQ Broker
    participant Worker as Downstream App (Service B)

    Note over User, Worker: Scenario 1: Synchronous Communication (Blocking Thread)
    User->>App: Click "Pay" (Request)
    Note over App: Thread Blocked
    App->>Worker: API Call (Synchronous)
    Note over Worker: Heavy Transaction Processing (3 seconds)
    Worker-->>App: OK (Response)
    Note over App: Thread Released (Unblocked)
    App-->>User: Success Screen (3+ seconds)

    Note over User, Worker: Scenario 2: Asynchronous Communication (Non-Blocking Thread)
    User->>App: Click "Pay" (Request)
    App->>Broker: Publish Event "order.paid" (Asynchronous)
    Broker-->>App: Publisher Confirm ACK (10ms)
    App-->>User: "Processing" Screen (15ms)
    Note over App: Thread Is Immediately Free to Serve Other Users!
    
    Note over Broker, Worker: Background Processing (Asynchronous)
    Broker->>Worker: Push Event to Consumer
    Note over Worker: Heavy Transaction Processing (3 seconds)
    Worker-->>Broker: Consumer ACK (Delete Message)

Temporal Coupling and Failure Isolation #

The main goal of an asynchronous architecture is to break Temporal Coupling. Temporal coupling occurs when the success of a business transaction process depends heavily on the instant availability of several external systems simultaneously.

In a synchronous request-response microservices architecture, if we have a call chain like:

Order Service $\rightarrow$ Payment Service $\rightarrow$ Notification Service

Then the operational availability of these three services is tied together as one. If the Notification Service experiences 1 hour of downtime due to an SMTP server problem, the Order Service indirectly also suffers 1 hour of downtime for transacting users, because its HTTP calls will time out and return errors.

By introducing asynchronicity with RabbitMQ, we isolate that failure (Failure Isolation):

The Order Service stores the order in its local database, publishes an OrderCreated event to RabbitMQ, then responds success to the user.
The Payment Service pulls the event, debits the balance, and publishes a PaymentSuccess event.
The down Notification Service does not affect the Order Service or Payment Service. Email/SMS events stay safely held in RabbitMQ’s send-notification-queue.
When the infrastructure team fixes the Notification Service and turns it back on, it immediately pulls the thousands of queued emails from RabbitMQ and sends them to users.

Users can keep shopping smoothly without ever realizing there’s a failure in our notification system. This significantly increases the system’s resilience.

The Data Consistency Challenge (Eventual Consistency) #

Although it delivers outstanding performance and fault tolerance, an asynchronous architecture brings a major challenge we must not ignore: Eventual Consistency.

In a traditional relational synchronous system, we can guarantee Immediate Consistency using database ACID transactions. If the order data storage fails, we cancel the payment in memory and the database state stays clean.

In an asynchronous distributed system, after the Order Service publishes a success event to RabbitMQ, the data state in the Payment Service’s database may not update at that exact same second. There is a time gap (ranging from a few milliseconds to several minutes if the queue is congested or network issues occur) during which data states between services are temporarily inconsistent.

The system is guaranteed to reach eventual consistency after all messages in the RabbitMQ queue are processed by the relevant consumer services.

To manage this eventual consistency safely in production:

Intermediate States: Design our business entity states to support asynchronicity. Don’t jump straight to binary states like Success or Failed. Use intermediate states like PendingPayment, Processing, or Validating.
Compensating Transactions: If a mid-process step fails (e.g., the debit transaction succeeds but inventory is out of stock), we cannot physically roll back another service’s database. We must write logic to publish a cancellation event (e.g., payment.refund) that the Payment Service consumes to return the money asynchronously.

Concurrency vs Parallelism in Asynchronous Communication #

Before we go further into the network level, we need to clear up a common misunderstanding about the difference between concurrency and parallelism, and how both manifest in distributed asynchronous communication.

Concurrency (Cooperative Concurrency): Concurrency is about dealing with many things at once. It relates to system structure. Inside our application runtime (such as the Erlang BEAM runtime on RabbitMQ or goroutines in Go), concurrency is achieved by cooperatively sharing CPU execution time among thousands of lightweight processes. Asynchronous here means the executing thread doesn’t wait for I/O to finish; instead, it hands control back to the scheduler so other processes can run.
Parallelism: Parallelism is about doing many things at once. It relates to physical execution on hardware. Parallelism requires more than one CPU core or several physical machines running simultaneously to truly execute instructions at the same millisecond.

In the context of RabbitMQ and distributed asynchronous architecture:

At the Node Level (Local): RabbitMQ leverages Erlang’s high-level concurrency to handle hundreds of thousands of TCP connections asynchronously. Every queue and every connection is managed by an isolated Erlang process. When a node runs on a multi-core server, the Erlang scheduler automatically maps these concurrent processes to different CPU cores in parallel.
At the Distributed System Level: Asynchronous communication with RabbitMQ lets us distribute workloads in parallel across many consumer servers (consumer instances). We can scale a consumer service from 1 instance to 10 instances horizontally. RabbitMQ distributes messages from the queue to those instances concurrently, so overall data processing runs in parallel across different servers.

OS Socket Buffers and the Backpressure Mechanism #

When we send millions of messages asynchronously, one of the biggest problems that can destroy a system is speed imbalance. What happens if the producer sends messages far faster than the consumer’s ability to process them? Without any control mechanism, system memory balloons and the system fails from out-of-memory.

This is where it’s important to understand the interaction between RabbitMQ and socket buffers at the operating system level (OS-level socket buffers) and the concept of backpressure.

1. The Role of SO_SNDBUF and SO_RCVBUF #

Every TCP connection established between our application and RabbitMQ is allocated memory buffers by the operating system kernel:

SO_SNDBUF (Send Buffer): Temporary storage for data sent by the application before it is transmitted to the network.
SO_RCVBUF (Receive Buffer): Storage for data received from the network before the application reads it.

When a producer sends messages asynchronously, data is written to our local SO_SNDBUF, transmitted over the network, and lands in the SO_RCVBUF on the RabbitMQ server. RabbitMQ’s internal processes then read from that socket buffer and process it.

2. How Backpressure Occurs #

If RabbitMQ detects that its memory allocation or disk space is nearly exhausted (past the alarm threshold), it stops reading data from producers’ TCP sockets. As a result:

The SO_RCVBUF buffer on the RabbitMQ server fills up completely.
The TCP protocol automatically lowers the TCP Window Size to zero in the ACK packets sent back to the producer.
The producer’s OS kernel sees a Window Size of zero, so it stops sending new data from the local SO_SNDBUF.
The SO_SNDBUF buffer on the producer side fills up.
The RabbitMQ client library on the producer side feels this obstruction because write function calls to the socket become blocking writes, or return a timeout error if using non-blocking I/O.

In this way, backpressure flows naturally from the busy RabbitMQ server back to our producer application without corrupting any intermediate memory.

3. Credit-Based Flow Control in RabbitMQ #

In addition to relying on TCP’s built-in flow control, RabbitMQ implements an internal flow control mechanism called Credit-Based Flow Control. In RabbitMQ’s Erlang architecture, every message-sending process must request “credit” from the receiving process before sending the next batch of data. If the receiving process (e.g., the queue process) is overwhelmed, it refuses to grant new credit. This holds back the sending process (the socket-reading process) from reading new data from the TCP socket, which ultimately triggers kernel-level TCP backpressure as described above.

The CAP Theorem and Availability in Network Partitions #

When designing an asynchronous distributed system with RabbitMQ, we cannot escape the CAP Theorem, which states that a distributed system can only guarantee at most two of the following three aspects simultaneously:

Consistency: Every read operation returns the most recent data or returns an error.
Availability: Every non-failing request receives a response, without guaranteeing that the response contains the most recent data.
Partition Tolerance: The system keeps running even when communication failures (network partitions) occur inside the system.

Because physical networks in the real world will inevitably experience disruptions sooner or later, we must choose Partition Tolerance (P). Our real choice is between CP (Consistency during partitions) or AP (Availability during partitions).

1. RabbitMQ’s Design Choice: Quorum Queues as CP #

In RabbitMQ’s modern features, the recommended queue type for critical data is Quorum Queues. These queues are based on the Raft consensus algorithm, which prioritizes data Consistency.

If a network partition splits the RabbitMQ cluster into two parts (minority and majority), the Quorum Queue on the minority side refuses to accept new writes from producers because they cannot reach a quorum (half the node count + 1).
This design chooses CP. RabbitMQ prefers to reject transactions to guarantee that no messages are lost or duplicated due to split-brain.

2. Availability (AP) from the Overall System Perspective #

Even though RabbitMQ applies the CP principle at its internal cluster level, using asynchronous communication actually helps us build application systems with a high availability level (AP from the end-user’s perspective).

Let’s compare:

Synchronous HTTP Scenario (Fragile CP/AP): The Order Service calls the Payment Service directly over HTTP. If the Payment Service is isolated by a network partition, the Order Service fails to respond to users. The entire system becomes Unavailable.
Asynchronous RabbitMQ Scenario (Robust AP): The Order Service only publishes an event to the local RabbitMQ broker within its network reach. The broker receives and stores the message. The Order Service can immediately reply success to the user. Even though the Payment Service is isolated on the other side of the network partition, the user’s shopping transaction still proceeds. Eventual consistency is achieved after the network partition heals and messages are consumed.

By shifting the integration point from direct API calls to asynchronous message queues, we increase the overall system’s resilience against partial network disruptions.

Anti-Pattern vs Solution: Async Transactions Without Intermediate States #

Let’s study a common developer mistake in designing data states for asynchronous workflows, along with its solution.

Anti-Pattern Code: Assuming Instant Status in an Async Workflow #

In the code below, the developer immediately marks the order as COMPLETED after publishing the transaction processing event asynchronously to RabbitMQ, assuming the background asynchronous processing will always succeed instantly.

// ANTI-PATTERN: Marking a status as completed before the async process is proven successful
func CheckoutOrderBad(w http.ResponseWriter, r *http.Request) {
    order := createOrderObject(r)
    
    // ✗ AVOID: Marking the order status directly as COMPLETED
    order.Status = "COMPLETED"
    saveToDatabase(order)

    // Publish the event for physical goods shipping processing asynchronously
    eventPayload := serialize(order)
    GlobalRabbitClient.Publish("shipping-exchange", "ship.order", eventPayload)

    // Return a success response to the user
    w.WriteHeader(http.StatusOK)
    w.Write([]byte(`{"status": "COMPLETED", "message": "Pesanan sukses dikirim!"}`))
    
    // Problem: If the Shipping Service actually fails in the background because the user's
    // address is invalid, our database has already written the 'COMPLETED' status
    // and the user thinks their goods were shipped. Fatal data inconsistency occurs.
}

Practical Solution: Implementing a State Machine with Intermediate States (Pending/Processing) #

The best approach is to use the intermediate PENDING_SHIPPING state and let the Shipping Service update that status to COMPLETED only after the physical async process is truly confirmed successful.

// CORRECT: Using intermediate states to support eventual consistency
func CheckoutOrderGood(w http.ResponseWriter, r *http.Request) {
    order := createOrderObject(r)
    
    // ✓ SOLUTION: Use the intermediate 'PENDING_SHIPPING' state
    order.Status = "PENDING_SHIPPING"
    saveToDatabase(order)

    eventPayload := serialize(order)
    err := GlobalRabbitClient.Publish("shipping-exchange", "ship.order", eventPayload)
    if err != nil {
        // Fallback if the broker is down
        http.Error(w, "Gagal memproses pesanan, silakan coba lagi", http.StatusInternalServerError)
        return
    }

    // Return a response with the accurate status to the user
    w.WriteHeader(http.StatusAccepted) // Status 202 Accepted: Request received for processing
    w.Write([]byte(`{"status": "PENDING_SHIPPING", "message": "Pesanan kita sedang diproses untuk pengiriman."}`))
}

// On the Shipping Service consumer side:
func ConsumeShippingEvent(msg amqp.Delivery) {
    order := deserialize(msg.Body)
    
    // Run the physical logistics process
    success, err := executeShippingLogistics(order.Address)
    if err != nil || !success {
        // Publish a failure event to trigger a compensating transaction
        GlobalRabbitClient.Publish("shipping-failures", "ship.failed", msg.Body)
        msg.Nack(false, false) // drop the message to the DLQ, don't requeue
        return
    }
    
    // ✓ SOLUTION: Update the final status in the database to COMPLETED from the consumer side
    updateOrderStatusInDB(order.ID, "COMPLETED")
    msg.Ack(false)
}

Summary #

Temporal Decoupling — Removes the requirement that all microservices be online simultaneously, isolating failure points so they don’t propagate upstream to higher services.
Non-Blocking Execution — Frees web server worker threads from waiting on slow downstream I/O processes, eliminating the risk of thread exhaustion.
Eventual Consistency — A paradigm shift from instant ACID transactions to gradual synchronization managed with intermediate states (pending/processing) and compensating transactions.
HTTP 202 Accepted — The best response for asynchronous APIs to tell users that their request has been safely accepted and is being processed in the background.

← Previous: Message Queue Next: Decoupling →