Multi-DC messaging is one of the most complex ActiveMQ architectural challenges. The WAN link between sites introduces latency, unreliability, and capacity constraints that are categorically different from local network conditions. The design decisions made here (push vs. pull forwarding, topology shape, duplicate detection, client failover configuration) determine whether the system behaves correctly under WAN failure, network partition, and datacenter outage scenarios.
This guide covers the five primary multi-datacenter deployment patterns for Apache ActiveMQ® and Apache Artemis™: Apache ActiveMQ® Network of Brokers hub-spoke and full mesh, Apache Artemis™ federation, Apache Artemis™ core bridges, and Apache Artemis™ AMQP broker connections, with configuration, tradeoff analysis, and when to use each.
The WAN Constraint: What Changes Across Datacenters
Before choosing a pattern, the WAN link characteristics define the design envelope:
| Factor | Intra-DC (same datacenter) | Cross-DC (WAN) |
|---|---|---|
| RTT latency | < 1ms | 5–150ms (varies by geography) |
| Bandwidth | 10–100 Gbps | 100 Mbps – 10 Gbps |
| Reliability | Very high (99.99%+) | Lower (planned maintenance, ISP outages) |
| Cost per byte | Negligible | Significant at scale |
These constraints have direct implications for pattern selection:
Synchronous operations across the WAN add RTT to every call. A cross-DC acknowledgment at 50ms RTT adds 50ms to every persistent message send that requires remote confirmation. Systems with strict latency SLAs must minimize synchronous cross-DC operations.
WAN links fail more frequently than LAN links. Every cross-DC broker connection must handle disconnection gracefully, buffering messages locally until the link recovers, then resuming delivery without message loss or duplication.
Bandwidth cost matters at scale. A 1KB message at 5,000 msg/s forwarded cross-DC consumes ~40 Mbps of WAN bandwidth continuously. Plan the WAN link capacity against the expected cross-DC message forwarding volume.
Pattern 1: Apache ActiveMQ® Network of Brokers (Hub-Spoke)
The hub-spoke topology places one broker as the central routing hub (typically in the primary or most connected datacenter) with spoke brokers in each additional site. All inter-DC message routing passes through the hub.
When to use: three or more sites with Apache ActiveMQ®; when central routing visibility is important; when spoke sites have relatively simple messaging requirements that don’t need direct spoke-to-spoke communication.
Topology: DC1 hub ↔ DC2 spoke, DC1 hub ↔ DC3 spoke. DC2 and DC3 do not connect directly. They route via DC1.
| <!– DC1 Hub: activemq.xml — connects to all spokes –> <broker brokerName=”dc1-hub” …> <networkConnectors> <!– Hub → DC2 Spoke: bidirectional (duplex=true) –> <networkConnector name=”hub-to-dc2″ uri=”static:(tcp://dc2-spoke.site2.internal:61616)” duplex=”true” messageTTL=”-1″ consumerTTL=”1″ networkTTL=”3″ conduitSubscriptions=”true” decreaseNetworkConsumerPriority=”true” userName=”${dc2.bridge.user}” password=”${dc2.bridge.password}”> <!– Only forward to queues that have consumers –> <dynamicallyIncludedDestinations> <queue physicalName=”>” /> <topic physicalName=”>” /> </dynamicallyIncludedDestinations> </networkConnector> <!– Hub → DC3 Spoke: same configuration –> <networkConnector name=”hub-to-dc3″ uri=”static:(tcp://dc3-spoke.site3.internal:61616)” duplex=”true” messageTTL=”-1″ consumerTTL=”1″ networkTTL=”3″ conduitSubscriptions=”true” decreaseNetworkConsumerPriority=”true” userName=”${dc3.bridge.user}” password=”${dc3.bridge.password}”> <dynamicallyIncludedDestinations> <queue physicalName=”>” /> <topic physicalName=”>” /> </dynamicallyIncludedDestinations> </networkConnector> </networkConnectors> </broker> <!– DC2 Spoke: activemq.xml — connects only to hub (not to DC3) –> <broker brokerName=”dc2-spoke” …> <networkConnectors> <!– Spoke → Hub only (hub-to-spoke is duplex from hub side) –> <!– With duplex=true on the hub side, the spoke does not need its own connector –> <!– Only add a spoke connector if duplex=false on the hub –> </networkConnectors> </broker> |
Key parameters for WAN links:
- messageTTL=”-1″: messages forwarded across the WAN do not decrement their TTL counter on each hop, which prevents messages from expiring in transit on a slow WAN link.
- consumerTTL=”1″: consumer subscription advertisements propagate only one hop. This prevents consumer demand signals from circulating unnecessarily across the full mesh in larger topologies.
- networkTTL=”3″: the maximum number of broker hops a message will travel. For a 3-site hub-spoke, 3 is sufficient (spoke → hub → spoke is 2 hops).
- conduitSubscriptions=”true”: the hub represents all consumers on a remote spoke as a single subscription rather than individual subscriptions. This significantly reduces subscription advertisement traffic across the WAN.
- decreaseNetworkConsumerPriority=”true”: local consumers get priority over remote (cross-DC) consumers. Messages are consumed locally first and only forwarded when no local consumer exists.
We covered the full Network of Brokers configuration parameters in our Network of Brokers Configuration post. The multi-DC context adds the WAN-specific parameter values described here.
Hub-spoke tradeoffs:
✅ O(n) broker-to-broker connections: manageable at scale
✅ Centralized routing visibility: easier to monitor
✅ Spoke failures are isolated: hub and remaining spokes continue
❌ Hub is a single point of failure: pair the hub with local HA (Master/Slave)
❌ Spoke-to-spoke latency is hub RTT × 2: DC2 → DC3 message traverses DC1 hub
❌ Hub bandwidth is the limiting factor for all inter-DC traffic
Pattern 2: Apache ActiveMQ® Network of Brokers (Full Mesh)
Every DC broker connects directly to every other DC broker. A 3-site full mesh has 3 connections; a 5-site full mesh has 10 connections.
When to use: two or three sites where spoke-to-spoke latency matters and the additional connection management complexity is acceptable; when you need direct routing between all sites without hub dependency.
| <!– DC1 Broker: direct connections to DC2 and DC3 –> <networkConnectors> <networkConnector name=”dc1-to-dc2″ uri=”static:(tcp://dc2.site2.internal:61616)” duplex=”true” messageTTL=”-1″ consumerTTL=”1″ networkTTL=”2″ conduitSubscriptions=”true”/> <networkConnector name=”dc1-to-dc3″ uri=”static:(tcp://dc3.site3.internal:61616)” duplex=”true” messageTTL=”-1″ consumerTTL=”1″ networkTTL=”2″ conduitSubscriptions=”true”/> </networkConnectors> |
Critical full-mesh parameter: networkTTL=”2″. In a full mesh, each broker connects to every other broker. Without a TTL limit, a message forwarded from DC1 to DC2 may be re-forwarded to DC3 via the DC2→DC3 link, even though DC3 already received it via the DC1→DC3 link, producing duplicates. networkTTL=”2″ limits forwarding to two hops maximum, preventing circular forwarding in the mesh.
Full-mesh tradeoffs:
✅ Lowest possible DC-to-DC latency (direct links, no hub hop)
✅ No single hub failure affects all sites
❌ O(n²) connections: scales poorly beyond 3 sites
❌ Complex subscription propagation: all brokers see all consumer subscriptions
❌ Potential for duplicate message forwarding without careful TTL configuration
Pattern 3: Apache Artemis™ Federation (Demand-Driven Cross-DC Routing)
Apache Artemis™ federation is the primary cross-DC pattern for Apache Artemis™ deployments. Unlike NoB (which pushes messages toward consumer demand), federation pulls messages from an upstream broker when local consumers request them.
Federation semantics: a federated queue on DC2 monitors local consumer demand. When a consumer subscribes to orders.main on DC2, the federation link activates and pulls messages from the orders.main queue on the DC1 upstream broker.
When the DC2 consumer unsubscribes, the federation link deactivates. Messages are only forwarded across the WAN when there is active demand. This is the critical efficiency advantage over NoB for WAN deployments.
| <!– DC2 broker.xml — federation pulling from DC1 upstream –> <configuration> <core> <!– Define the DC1 upstream broker –> <federations> <federation name=”dc1-federation”> <!– Upstream broker connection –> <upstream name=”dc1-upstream”> <static-connectors> <connector-ref>dc1-connector</connector-ref> </static-connectors> <ha>true</ha> <circuit-breaker-timeout>30000</circuit-breaker-timeout> <!– Queue federation policy: pull queues matching pattern from DC1 –> <queue-policy name=”cross-dc-queues” include-federated=”false” priority-adjustment=”-1″ transformer-class-name=””> <!– Pull these queue patterns from DC1 when local consumers exist –> <include match=”orders.#”/> <include match=”payments.#”/> <!– Exclude queues that should remain DC-local –> <exclude match=”*.local”/> </queue-policy> <!– Address federation policy: replicate topic messages from DC1 –> <address-policy name=”cross-dc-topics” max-hops=”1″ auto-delete=”true” auto-delete-delay=”300000″ auto-delete-message-count=”-1″> <include match=”events.#”/> </address-policy> </upstream> </federation> </federations> <!– Connector to DC1 hub broker –> <connectors> <connector name=”dc1-connector”>tcp://dc1-broker.site1.internal:61616</connector> </connectors> </core> </configuration> |
Queue vs. Address federation:
| Aspect | Queue Federation | Address Federation |
|---|---|---|
| Semantics | Pull messages from upstream queue when local consumers exist | Subscribe to upstream multicast address and receive copies |
| Message consumption | Message consumed from upstream (not replicated, moved) | Message replicated to local address (upstream copy survives) |
| Use case | Competing consumers across DCs (workload distribution) | Pub/sub fan-out across DCs (event broadcasting) |
| Duplicate risk | None (message is moved, not copied) | Possible if federation loops exist (set max-hops=”1″) |
Federation tradeoffs:
✅ Demand-driven: no cross-DC bandwidth consumed when no remote consumers
✅ Clean integration with Apache Artemis™ address model
✅ circuit-breaker-timeout prevents cascading failures from WAN outage
❌ Messages remain on upstream until pulled: consumers on DC2 see upstream latency when the link first activates
❌ Address federation supports only multicast (topics), not anycast (queues) for replication
Designing a Multi-Datacenter ActiveMQ Architecture?
Pattern 4: Apache Artemis™ Core Bridge (Guaranteed Cross-DC Forwarding)
A core bridge forwards every message from a local source queue to a target address on a remote Apache Artemis™ broker. Unlike federation (which only pulls when consumers demand), a core bridge is active continuously and guarantees delivery regardless of consumer state on the remote side.
When to use: guaranteed delivery to a specific remote destination is required (e.g., all regional orders must reach the central processing broker); the message flow is unidirectional (DC1 produces, DC2 processes); delivery order must be preserved.
| <!– DC1 broker.xml — core bridge forwarding orders to DC2 central processor –> <configuration> <core> <bridges> <bridge name=”orders-to-dc2″> <!– Source: local queue to drain –> <queue-name>orders.regional.dc1</queue-name> <!– Target: address on DC2 central broker –> <forwarding-address>orders.central</forwarding-address> <!– HA: bridge reconnects to DC2 backup on failover –> <ha>true</ha> <!– Duplicate detection: prevent message duplication on reconnect –> <!– Artemis adds _AMQ_DUPL_ID header; remote broker deduplicates –> <use-duplicate-detection>true</use-duplicate-detection> <!– Confirmation window: broker confirms after this many bytes delivered –> <!– Messages are re-sent from this point if connection drops –> <confirmation-window-size>10485760</confirmation-window-size> <!– Producer window: flow control for the bridge producer –> <producer-window-size>1048576</producer-window-size> <!– Retry configuration for WAN link failures –> <retry-interval>2000</retry-interval> <retry-interval-multiplier>1.5</retry-interval-multiplier> <max-retry-interval>30000</max-retry-interval> <initial-connect-attempts>-1</initial-connect-attempts> <reconnect-attempts>-1</reconnect-attempts> <!– Bridge user credentials (separate from client credentials) –> <user>${bridge.dc2.user}</user> <password>${bridge.dc2.password}</password> <!– Static connector to DC2 –> <static-connectors> <connector-ref>dc2-central-connector</connector-ref> </static-connectors> </bridge> </bridges> <connectors> <connector name=”dc2-central-connector”> tcp://dc2-central-broker.site2.internal:61616 </connector> </connectors> </core> </configuration> |
Duplicate detection and confirmation-window-size: these two parameters work together for exactly-once delivery across WAN failures. When the WAN link drops mid-delivery, some messages may have been sent but not confirmed.
On reconnect, the bridge re-sends from the last confirmed position. use-duplicate-detection=true adds an _AMQ_DUPL_ID header to each bridged message. The remote broker caches recently received IDs and silently discards messages it has already delivered. This combination provides at-least-once delivery at the bridge level with de-duplication at the remote broker.
Core bridge tradeoffs:
✅ Guaranteed delivery: every message on the source queue reaches the remote target
✅ Duplicate detection prevents double-processing on WAN reconnect
✅ Ordered delivery within the bridge connection
✅ Reconnects indefinitely: survives extended WAN outages, buffering locally
❌ Not demand-driven: the source queue accumulates messages even when the remote has no consumers
❌ One bridge per source-queue/target-address pair: multiple queues require multiple bridge definitions
Pattern 5: Apache Artemis™ AMQP Broker Connection (Mirror and Hybrid-Cloud Federation)
The Apache Artemis™ AMQP broker connection is the most flexible cross-DC pattern because it uses AMQP 1.0 as the transport, enabling broker-to-broker communication with any AMQP 1.0-compliant broker, not just other Apache Artemis™ instances.
Two modes:
Mode A: Mirror (Full Bidirectional Replication)
Mirror mode replicates all messages and acknowledgments between two Apache Artemis™ brokers over AMQP. Both brokers maintain identical message state, an active-active deployment where either broker can serve any client.
| <!– DC1 broker.xml — AMQP broker connection with mirror to DC2 –> <configuration> <core> <broker-connections> <amqp-connection name=”dc2-mirror” uri=”tcp://dc2-artemis.site2.internal:5672″ retry-interval=”500″ reconnect-attempts=”-1″ user=”${mirror.user}” password=”${mirror.password}”> <!– Mirror: full bidirectional message + ack replication –> <mirror durable-only=”false” queue-creation=”true” queue-removal=”true” message-acknowledgements=”true”/> <!– durable-only=true: only replicate persistent messages (WAN-efficient) –> <!– message-acknowledgements=true: ACKs replicated so consumers on DC2 don’t re-receive messages already consumed on DC1 –> </amqp-connection> </broker-connections> <connectors> <connector name=”dc2-amqp-connector”> tcp://dc2-artemis.site2.internal:5672 </connector> </connectors> </core> </configuration> |
Mode B: Federation over AMQP (Hybrid-Cloud)
AMQP broker connection federation enables demand-driven forwarding to/from any AMQP 1.0 broker, including Azure Service Bus, IBM MQ, Solace, and Amazon MQ:
| <!– broker.xml — AMQP broker connection to Azure Service Bus –> <broker-connections> <amqp-connection name=”azure-servicebus” uri=”amqps://your-namespace.servicebus.windows.net:5671″ retry-interval=”1000″ reconnect-attempts=”-1″ user=”RootManageSharedAccessKey” password=”${azure.sas.key}”> <!– Pull from Azure Service Bus queue to local Artemis queue –> <federation> <queue-policy name=”from-azure” include-federated=”false”> <include match=”azure-inbound.#”/> </queue-policy> </federation> </amqp-connection> </broker-connections> |
AMQP broker connection tradeoffs:
✅ Mirror: true active-active semantics with ack replication (strongest consistency)
✅ Works with any AMQP 1.0 broker: the only pattern that bridges to non-Apache Artemis™ platforms
✅ durable-only=true on mirror reduces WAN bandwidth by excluding non-persistent messages
❌ Mirror at high throughput doubles the WAN bandwidth requirement (every message crosses twice)
❌ Mirror does not prevent split-brain on WAN partition: both sides continue accepting writes; reconciliation on reconnect requires careful deduplication design
Pattern Decision Matrix
Client Failover Configuration for Multi-DC
Client-side configuration is the final piece of a multi-DC deployment. Clients must know about brokers in multiple DCs and must prefer the local DC broker while failing over to the remote DC broker when the local site is unavailable.
Apache ActiveMQ®: Failover Transport with DC Priority
| // ActiveMQ Classic client — DC-aware failover transport // DC1 is primary (randomize=false ensures DC1 is tried first) // priorityBackup=true prevents random reconnection to DC2 when DC1 recovers String failoverUrl = “failover:(” + “tcp://dc1-broker.site1.internal:61616,” + // Primary DC “tcp://dc2-broker.site2.internal:61616” + // Failover DC “)?” + “randomize=false&” + // Try URLs in order (DC1 first) “priorityBackup=true&” + // Prefer first URL; return to it when available “initialReconnectDelay=100&” // 100ms initial reconnect delay “maxReconnectDelay=30000&” // Max 30s between reconnect attempts “maxReconnectAttempts=-1&” // Retry indefinitely “startupMaxReconnectAttempts=3”; // Limited retries on initial connection ConnectionFactory factory = new ActiveMQConnectionFactory(failoverUrl); |
Apache Artemis™: Multi-DC Static Connector List
| <!– Artemis JMS client — static connector list for multi-DC –> <configuration> <jms> <connection-factory name=”MultiDCConnectionFactory”> <connectors> <connector-ref connector-name=”dc1-connector”/> <!– Primary DC –> <connector-ref connector-name=”dc2-connector”/> <!– Failover DC –> </connectors> <failover-on-initial-connection>false</failover-on-initial-connection> <reconnect-attempts>-1</reconnect-attempts> <retry-interval>500</retry-interval> </connection-factory> </jms> </configuration> |
AMQP Clients Without Failover Support
Many AMQP 1.0 client libraries (.NET, Python, Go, Node.js) do not natively support the ActiveMQ failover transport. For these clients, the recommended approach is a geo-aware layer in front of the brokers:
Option 1: DNS-based failover. Use a DNS name that points to the primary DC broker; update DNS on failover. Simple but limited (DNS TTL introduces delay).
Option 2: Load balancer with health check. A TCP load balancer (HAProxy, AWS NLB) with health checks on port 61616 can route AMQP clients to an available broker. Configure health checks to detect broker unresponsiveness within 10–30 seconds.
Option 3: Apache Qpid Dispatch Router. An AMQP 1.0 message router that provides intelligent routing across multiple broker endpoints, including geo-aware routing. More complex to operate but provides the richest routing semantics.
We covered TLS configuration for all inter-DC connections (both broker-to-broker and client-to-broker) in our Security Hardening Guide. All WAN links must use TLS; certificate-based mutual authentication between brokers is strongly recommended for production cross-DC deployments.
Unified Visibility Across All Your Datacenter Broker Instances
Split-Brain: The Multi-DC Failure Mode That Keeps Architects Awake
Every multi-DC active-active architecture must address split-brain: the scenario where the WAN link fails, and both sites continue accepting writes independently, producing divergent message state that must be reconciled when the link recovers.
Apache ActiveMQ® NoB on WAN partition: when the networkConnector link drops, each DC broker operates independently. Producers continue sending to their local broker; consumers consume from their local broker.
Messages produced on DC1 during the partition are not forwarded to DC2 (and vice versa) until the link recovers. On reconnect, NoB resumes forwarding based on current consumer demand. It does not replay missed messages. Messages produced during the partition remain on the originating broker’s queues.
Apache Artemis™ mirror on WAN partition: both brokers continue accepting writes. On reconnect, the mirror link re-establishes and continues from the current position. Messages produced on DC1 during the partition are not replicated to DC2, and vice versa. If consumers on both sides consumed different messages from their local copies of the same queue during the partition, the state is permanently divergent. There is no automatic reconciliation.
Design principle for split-brain tolerance: the safest multi-DC design is active-passive, not active-active. During normal operation, DC1 is active, and DC2 is standby. On WAN failure, a deliberate promotion procedure activates DC2. On WAN recovery, DC1 is re-synchronized from DC2 before clients reconnect. This design accepts planned failover latency in exchange for eliminating split-brain divergence.
For use cases that genuinely require active-active (e.g., geographically distributed producers that cannot tolerate the latency of routing to a single DC), design messages as idempotent (safe to process twice) and implement consumer-side deduplication using the _AMQ_DUPL_ID header.
Multi-DC Is a Feature, Not a Configuration Change
Deploying ActiveMQ across datacenters is an architectural commitment, not a configuration parameter. The topology shape (hub-spoke vs. mesh), the forwarding model (push vs. pull), the failure behavior on WAN partition, and the client failover configuration all interact. A change to one affects the others.
The patterns in this guide represent the well-understood, production-validated approaches for Apache ActiveMQ® and Apache Artemis™. Choose based on your broker type, message delivery guarantee requirements, WAN bandwidth budget, and split-brain tolerance. Then apply the TLS, retry, and monitoring configurations that make the chosen pattern robust.
We’ll cover zero-downtime maintenance procedures in the next post, including how to perform maintenance on individual nodes in a multi-DC topology without interrupting the broader message flow.
Get your multi-DC ActiveMQ architecture reviewed by our team → Request an Architecture Review
Frequently Asked Questions
Q: How do I deploy ActiveMQ across multiple datacenters? Apache ActiveMQ®: Network of Brokers with networkConnector elements linking sites. Apache Artemis™: federation (demand-driven), core bridges (guaranteed), or AMQP broker connections (mirror/hybrid-cloud). The right pattern depends on whether messages should be pushed or pulled, and whether guaranteed vs. demand-driven delivery is required.
Q: What is the difference between Network of Brokers and Apache Artemis™ federation? NoB pushes messages toward remote consumers proactively. Federation pulls messages from upstream brokers only when local consumers demand them. Federation is more WAN-efficient; NoB provides richer routing semantics for Apache ActiveMQ® deployments.
Q: What is an Apache Artemis™ core bridge? A persistent, guaranteed-delivery forwarder from a local source queue to a remote target address. Always active (not demand-driven). Uses duplicate detection headers to prevent double-delivery on WAN reconnect. Ideal for guaranteed cross-DC delivery pipelines.
Q: What is the Apache Artemis™ AMQP broker connection? Broker-to-broker communication over AMQP 1.0 in two modes: mirror (full bidirectional replication) and federation (demand-driven over AMQP). The only pattern that interoperates with non-Apache Artemis™ AMQP 1.0 brokers (Azure Service Bus, IBM MQ, Solace).
Q: How do ActiveMQ clients connect across multiple datacenters? Apache ActiveMQ®: failover transport with priorityBackup=true and DC-ordered URL list. Apache Artemis™ Core clients: static connector list. AMQP clients without native failover: TCP load balancer with health check, DNS failover, or Qpid Dispatch Router for intelligent AMQP routing.