Troubleshooting ActiveMQ Producer Flow Control Blocks

INFO | Usage(default:store:queue://orders.process:store) percentUsage=99%, usage=107374182400, limit=107374182400: Persistent store is Full, 100% of 107374182400. Stopping producer (ID:app-server-1:1:1:1) to prevent flooding queue://orders.process. See http://activemq.apache.org/producer-flow-control.html for more info (blocking for: 10800s)

Three hours of blocking. The answer was in the log the whole time.

This guide covers ActiveMQ producer flow control troubleshooting from first principles: what PFC is, how to diagnose it immediately from broker logs, what each resource trigger means and how to fix it, the catastrophic deadlock scenario that occurs when producers and consumers share connections, and the configuration changes that prevent recurrence.

Step 1: Read the Broker Log – The PFC Signal

Every producer flow-control event is logged at the INFO level in activemq.log. This is the primary diagnostic tool – no JMX query, no thread dump, no metric dashboard is more direct.

The PFC Log Message Anatomy

INFO | Usage(default:RESOURCE:DESTINATION_TYPE://DESTINATION:RESOURCE)       percentUsage=99%, usage=BYTES, limit=BYTES, percentUsageMinDelta=1%;       Parent:Usage(default:RESOURCE) percentUsage=100%, usage=BYTES, limit=BYTES:       RESOURCE is Full, 100% of LIMIT_BYTES.       Stopping producer (PRODUCER_ID) to prevent flooding DESTINATION_TYPE://DESTINATION.       See http://activemq.apache.org/producer-flow-control.html (blocking for: Xs)

How to read it:

Field	Where to find it	What it tells you
Resource type	Usage(default:store:queue://…)	store = KahaDB disk; temp = temp store; memory = RAM
Destination	queue://orders.process	Which queue/topic triggered the block
Percent used	percentUsage=99%	How full is the resource
Current usage	usage=107374182400	Actual bytes consumed
Configured limit	limit=107374182400	The systemUsage limit
Producer ID	Stopping producer (ID:app-1:1:1:1)	Which producer is blocked (trace to client)
Block duration	blocking for:10800s	How long PFC has been active (seconds)

The most important field is the resource type in the Usage() string. It identifies immediately whether this is a memory problem (memory), a disk problem (store), or a temp store problem (temp).

Real examples from production logs:

Memory trigger: Usage(default:memory:queue://orders.process:memory) percentUsage=98%… Memory Usage is Full…  Stopping producer… Store trigger: Usage(default:store:queue://orders.process:store) percentUsage=100%… Persistent store is Full, 100% of 107374182400.  Stopping producer… Temp store trigger: Usage(default:temp:queue://orders.process:temp) percentUsage=100%… Temp Store is Full (100% of 268435456).  Stopping producer…

Step 2: Identify the Root Cause

Root Cause 1: memoryUsage Exhausted

Symptoms: memory in the Usage() string. MemoryPercentUsage JMX metric at or near 100%.

What it means: The broker’s in-memory buffer for holding messages before dispatch (and for non-persistent messages that haven’t been spooled to disk yet) is full. Any new message sent blocks until existing messages are consumed and memory is freed.

The 64MB default problem: Apache ActiveMQ® ships with memoryUsage limit=”64 mb”. This is appropriate for development environments and completely inadequate for production workloads with any meaningful message volume or consumer lag. A single slow consumer holding 65,000 messages of 1KB each exhausts the entire default memory allocation. This is why newly deployed production instances trigger PFC almost immediately under real load.

Primary root causes:

1. Slow consumer: Messages accumulate faster than they are consumed, filling broker memory. This is the most common cause. We covered slow consumer detection and handling in our Slow Consumer Detection & Handling post.
2. Under-configured memoryUsage limit: The default 64MB is too small for most production workloads.
3. Large messages: even a few messages of 1MB+ can exhaust a small memoryUsage budget quickly.

Fix: Right-size memoryUsage

<!– activemq.xml — production systemUsage configuration –> <systemUsage>   <systemUsage>     <!– Option A: percentage of JVM heap (recommended — scales with JVM sizing) –>     <memoryUsage>       <memoryUsage percentOfJvmHeap=”70″/>     </memoryUsage>     <!– Option B: absolute value for environments with known heap sizes –>     <!– <memoryUsage><memoryUsage limit=”2 gb”/></memoryUsage> –>     <!– Persistent store: should reflect actual disk capacity –>     <storeUsage>       <storeUsage limit=”500 gb”/>     </storeUsage>     <!– Temp store: non-persistent message spooling –>     <tempUsage>       <tempUsage limit=”50 gb”/>     </tempUsage>   </systemUsage> </systemUsage>

percentOfJvmHeap is the correct approach for production: it automatically scales with JVM heap sizing changes, avoids the need to coordinate memoryUsage changes with JVM -Xmx adjustments, and reflects the actual memory available to the broker. The 70% figure leaves 30% for JVM overhead, message dispatch, threading, and KahaDB index cache.

Fix the slow consumer first: If the root cause is slow consumer accumulation rather than under-configuration, increasing memoryUsage only delays the next PFC event, it does not address the underlying consumption rate problem. Identify the slow consumer via JMX PendingQueueSize before treating the symptom with increased limits.

Root Cause 2: storeUsage Exhausted

Symptoms: store in the Usage() string. StorePercentUsage JMX metric at or near 100%.

What it means: The KahaDB persistent store has reached its configured disk limit. New persistent message sends block until disk space is freed through message consumption and journal file reclamation.

The storeUsage limit vs. actual disk space: There is a subtle trap here that catches many operators. The storeUsage limit is not automatically set to the available disk space, it is a configured cap. If you set storeUsage limit=”100 gb” but KahaDB is on a 50GB partition, PFC triggers from KahaDB filling the physical disk before the configured limit is reached. 

The log message will show the actual disk-capped limit (~50GB), not the configured limit (100GB), which confuses operators who believe they have 100GB configured.

From Apache ActiveMQ® 5.15.x: the total attribute on StoreUsage allows explicit configuration of the total available space so the file system is not queried. This is particularly important for cloud storage (EFS, NFS) where java.io.File.getTotalSpace() may return values exceeding Long.MAX_VALUE, causing overflow.

Primary root causes:

1. KahaDB journal pinning by slow consumer: An offline or slow consumer prevents journal file reclamation. The journal grows unboundedly until storeUsage is exhausted.
2. DLQ accumulation: messages accumulating in Dead Letter Queues pin journal files indefinitely.
3. Undersized storeUsage limit relative to actual disk space.
4. Missing KahaDB cleanup: long cleanup intervals allow file accumulation.

Fix: Diagnose what is pinning the journal

# Check actual KahaDB directory size vs. configured storeUsage limit du -sh /opt/activemq/data/kahadb/ # Via JMX: check which queues have messages # (look for queues with large QueueSize that are not consuming) # MBean: org.apache.activemq:type=Broker,brokerName=*, #        destinationType=Queue,destinationName=* <!– Fix 1: Right-size storeUsage to match actual available disk –> <storeUsage>   <storeUsage limit=”400 gb”/>  <!– 80% of 500GB disk –>   <!– From 5.15.x: use total= to prevent file system queries –>   <!– <storeUsage limit=”400 gb” total=”500 gb”/> –> </storeUsage> <!– Fix 2: Reduce KahaDB cleanup interval for faster journal reclamation –> <persistenceAdapter>   <kahaDB directory=”/opt/activemq/data/kahadb”           cleanupInterval=”15000″/>  <!– 15 seconds, down from default 30 –> </persistenceAdapter>

If the root cause is journal pinning by a slow consumer or DLQ accumulation, fixing the storeUsage limit is insufficient, it buys time until the limit is hit again. Consume or delete the DLQ messages, bring the offline consumer online, or configure mKahaDB destination sharding to isolate the slow destination. We covered both journal pinning and mKahaDB in our Message Persistence Strategies post.

Root Cause 3: tempUsage Exhausted

Symptoms: temp in the Usage() string. TempPercentUsage JMX metric at or near 100%.

What it means: The temporary file store, where non-persistent messages are spooled to disk when memory pressure builds, has reached its configured limit. This is less commonly encountered than memory or store exhaustion, but when it does, it is often mystifying because operators focus on memoryUsage and storeUsage while overlooking tempUsage.

The tempUsage < memoryUsage trap: If tempUsage is configured smaller than memoryUsage, the following sequence occurs: memory fills, the broker tries to spool non-persistent messages to the temp store to free memory, the temp store fills first because it has a smaller limit, and PFC triggers on temp exhaustion even though the configured memoryUsage limit has not been reached. The log will show tempUsage at 100% while memoryUsage is at 70-80%.

The additional complication: even a small number of non-persistent messages can exhaust a small tempUsage because KahaDB creates temp store journal files at the configured journalMaxFileLength (default 32MB) per file. If tempUsage limit=”50 mb” and the journal file size is 32MB, two journal files fill the entire temp store budget.

Fix: Size tempUsage relative to storeUsage and journalMaxFileLength

<systemUsage>   <systemUsage>     <memoryUsage>       <memoryUsage percentOfJvmHeap=”70″/>     </memoryUsage>     <storeUsage>       <storeUsage limit=”500 gb”/>     </storeUsage>     <!– Rule of thumb: tempUsage should be >= 2x journalMaxFileLength –>     <!– and should be sized for your expected non-persistent message volume –>     <tempUsage>       <tempUsage limit=”50 gb”/>     </tempUsage>   </systemUsage> </systemUsage>

If your broker has predominantly persistent messaging, tempUsage can be modest (10-50GB). If you have significant non-persistent message volume with consumers that may lag, size tempUsage generously, it is the spooling buffer that protects memoryUsage under burst conditions.

The Critical Failure Mode: PFC Deadlock via Shared Connections

This is the most dangerous PFC scenario, and it is not intuitive from reading the documentation. It requires a specific understanding of how ActiveMQ connection threading works.

The setup: A client application has a single Connection from which it creates both a Session with a producer (for sending messages) and a Session with a consumer (for receiving replies or acknowledgments).

The sequence:

1. PFC activates, a resource limit is hit
2. The broker blocks the producer’s send(), this blocks the thread holding the producer
3. PFC in Apache ActiveMQ® blocks the entire connection, not just the producing session
4. The consumer’s session is on the same connection, it is also frozen
5. Messages pending in the consumer’s buffer cannot be acknowledged
6. The broker is waiting for consumer acknowledgments to free memory
7. Consumer acknowledgments cannot arrive because the connection is frozen
8. The producer cannot unblock because the memory it needs never frees
9. Permanent deadlock requires a connection restart to break

The Apache mailing list has a precise description of this phenomenon: “PFC might be better named ‘producer connection flow control’ because the entire connection of the producer blocks, not just the one producer.”

The prevention rule: Always use separate connections for producers and consumers. This is the single most important architectural rule for avoiding PFC deadlock. A producer connection and a consumer connection operate independently. PFC on the producer connection cannot freeze the consumer connection.

// WRONG: producer and consumer on same connection ConnectionFactory factory = new ActiveMQConnectionFactory(brokerUrl); Connection sharedConnection = factory.createConnection(); sharedConnection.start(); Session producerSession = sharedConnection.createSession(false, AUTO_ACKNOWLEDGE); MessageProducer producer = producerSession.createProducer(orderQueue); Session consumerSession = sharedConnection.createSession(false, AUTO_ACKNOWLEDGE); MessageConsumer consumer = consumerSession.createConsumer(replyQueue); // PFC on producerSession will ALSO block consumerSession → DEADLOCK RISK // CORRECT: separate connections Connection producerConn = factory.createConnection(); producerConn.start(); Session producerSession = producerConn.createSession(false, AUTO_ACKNOWLEDGE); MessageProducer producer = producerSession.createProducer(orderQueue); Connection consumerConn = factory.createConnection(); consumerConn.start(); Session consumerSession = consumerConn.createSession(false, AUTO_ACKNOWLEDGE); MessageConsumer consumer = consumerSession.createConsumer(replyQueue); // PFC on producerConn cannot affect consumerConn

Configuring PFC Behavior: From Blocking to Exceptions

The default PFC behavior, indefinite blocking, is the worst outcome for production applications. An application that blocks indefinitely on send() is effectively frozen. It holds threads, consumes connection resources, and gives no signal to upstream systems that something is wrong. The correct production configuration replaces indefinite blocking with exception-based notification.

Option 1: sendFailIfNoSpace – Immediate Exception

<systemUsage>   <systemUsage sendFailIfNoSpace=”true”>     <memoryUsage>       <memoryUsage percentOfJvmHeap=”70″/>     </memoryUsage>     <storeUsage>       <storeUsage limit=”500 gb”/>     </storeUsage>     <tempUsage>       <tempUsage limit=”50 gb”/>     </tempUsage>   </systemUsage> </systemUsage>

When any resource limit is hit, the broker immediately throws javax.jms.ResourceAllocationException to the producer. The application must catch this and implement retry logic. Advantage: the application is never frozen. Disadvantage: brief, transient resource spikes (a GC pause, a momentary consumer lag) produce exceptions rather than resolving automatically.

Option 2: sendFailIfNoSpaceAfterTimeout – Timed Block (Recommended)

<systemUsage>   <!– Wait 30 seconds before failing: allows transient pressure to clear –>   <systemUsage sendFailIfNoSpaceAfterTimeout=”30000″>     <memoryUsage>       <memoryUsage percentOfJvmHeap=”70″/>     </memoryUsage>     <storeUsage>       <storeUsage limit=”500 gb”/>     </storeUsage>     <tempUsage>       <tempUsage limit=”50 gb”/>     </tempUsage>   </systemUsage> </systemUsage>

This is the recommended production configuration. The broker blocks for up to 30 seconds (configurable), giving temporary resource pressure time to resolve naturally. Only if the resource is still exhausted after 30 seconds does the broker throw ResourceAllocationException. Applications must still catch and retry, but brief spikes resolve without application-level errors.

Client-Side: Catching ResourceAllocationException

private void sendWithRetry(MessageProducer producer, Message message,                           int maxRetries, long retryDelay)         throws JMSException, InterruptedException {     for (int attempt = 1; attempt <= maxRetries; attempt++) {         try {             producer.send(message);             return; // Success         } catch (ResourceAllocationException e) {             if (attempt == maxRetries) {                 logger.error(“Send failed after {} attempts: resource exhausted”, maxRetries);                 throw e;             }             logger.warn(“Broker resource exhausted (attempt {}/{}), retrying in {}ms”,                 attempt, maxRetries, retryDelay);             Thread.sleep(retryDelay);             retryDelay = Math.min(retryDelay * 2, 60_000L); // Exponential backoff, max 60s         }     } }

Per-Destination Configuration (Apache ActiveMQ® 5.16+)

From Apache ActiveMQ® 5.16.0, sendFailIfNoSpace and sendFailIfNoSpaceAfterTimeout can be configured per destination via destinationPolicy. This allows critical queues to fail fast (immediate exception) while high-volume topics tolerate brief blocking:

<destinationPolicy>   <policyMap>     <policyEntries>       <!– Critical order queue: fail fast, never block indefinitely –>       <policyEntry queue=”orders.>”>         <deadLetterStrategy>           <sharedDeadLetterStrategy processNonPersistent=”true”/>         </deadLetterStrategy>       </policyEntry>       <!– Market data topic: allow brief blocking before failing –>       <policyEntry topic=”market.data.>”                   producerFlowControl=”true”>         <!– Per-destination sendFailIfNoSpaceAfterTimeout available 5.16+ –>       </policyEntry>     </policyEntries>   </policyMap> </destinationPolicy>

Apache Artemis™: address-full-policy

Apache Artemis™’s flow control model is fundamentally different from Apache ActiveMQ®. Rather than a broker-wide resource limit with a single PFC trigger, Apache Artemis™ uses per-address policies with four configurable behaviors:

<!– broker.xml — Artemis address full policy –> <address-settings>   <!– Critical queues: BLOCK producers when address is full –>   <address-setting match=”orders.#”>     <max-size-bytes>209715200</max-size-bytes>  <!– 200MB –>     <page-size-bytes>10485760</page-size-bytes>  <!– 10MB page files –>     <address-full-policy>BLOCK</address-full-policy>     <!– Producer is blocked until address size drops below max-size-bytes –>   </address-setting>   <!– High-volume telemetry: PAGE to disk, never block producers –>   <address-setting match=”telemetry.#”>     <max-size-bytes>104857600</max-size-bytes>  <!– 100MB –>     <address-full-policy>PAGE</address-full-policy>     <!– Messages paged to disk when limit is reached; producer never blocked –>   </address-setting>   <!– Low-priority data: DROP oldest messages when full –>   <address-setting match=”logs.low-priority.#”>     <max-size-bytes>52428800</max-size-bytes>  <!– 50MB –>     <address-full-policy>DROP</address-full-policy>     <!– New messages are discarded when limit is reached; producer never blocked –>   </address-setting>   <!– Default: PAGE for everything else –>   <address-setting match=”#”>     <max-size-bytes>-1</max-size-bytes>  <!– No limit –>     <address-full-policy>PAGE</address-full-policy>   </address-setting> </address-settings>

Apache Artemis™ address-full-policy options:

Policy	Behavior	Use Case
BLOCK	Block producer send() until the address drops below the limit	Transactional queues where message loss is unacceptable
PAGE	Spool messages to disk paging files; producer never blocked	High-volume queues with burst patterns
DROP	Discard new messages silently when full	Low-priority, non-critical data
FAIL	Throw an exception to the producer immediately when full	Critical queues where the application must handle backpressure

The key Apache Artemis™ advantage over Apache ActiveMQ®: PFC is per-address, not broker-wide. A full address blocks only producers sending to that address. Other destinations continue unaffected. This eliminates the Apache ActiveMQ® scenario where a slow consumer on one queue blocks producers on completely unrelated queues.

We covered the Apache ActiveMQ® vs Apache Artemis™ architectural differences, including flow control as a key migration consideration in our Classic vs Artemis: 2026 Definitive Guide.

PFC Root Cause Decision Matrix

Use this table to map your log message and symptoms to the correct remediation:

Log Shows	JMX Metric High	Primary Root Cause	Immediate Fix	Permanent Fix
memory resource	MemoryPercentUsage > 70%	Slow consumer / undersized memoryUsage	Increase memoryUsage limit	Fix slow consumer; use percentOfJvmHeap
store resource	StorePercentUsage > 85%	KahaDB journal pinning/disk full	Increase storeUsage limit or free disk	Fix journal pinning (DLQ, slow consumer); mKahaDB
temp resource	TempPercentUsage > 75%	Non-persistent message spooling overflow	Increase tempUsage limit	Size tempUsage ≥ 2× journalMaxFileLength
No PFC log	Application hangs anyway	PFC deadlock (shared connection)	Restart the application connection	Separate producer/consumer connections
PFC on NoB bridge	Multiple brokers block simultaneously	Bridge connection blocked by PFC	Clear consuming destination	Reduce message rate or add consumers

Monitoring PFC to Prevent Recurrence

PFC is a symptom of an underlying resource imbalance. The goal is not just to fix the current incident, it is to alert before the next one. The alert thresholds from our Monitoring & Alerting Setup post map directly to PFC prevention:

• MemoryPercentUsage > 70% (Warning) – PFC will trigger at 100%; this gives you a 30% buffer to act
• StorePercentUsage > 70% (Warning), > 85% (Critical) – store-triggered PFC is preventable with ahead-of-time alerts
• TempPercentUsage > 75% (Warning) – temp store exhaustion is the most frequently missed alert
• ConsumerCount = 0 on any critical queue (Critical) – the most common slow-consumer trigger for memory PFC

Adding a Prometheus alert for the PFC log message itself (via log scraping) provides the earliest possible warning:

# Alert on PFC log messages appearing in broker logs # (via Loki or similar log aggregation) count_over_time({job=”activemq”} |= “Stopping producer” [5m]) > 0

Fix the Root Cause, Not Just the Limit

ActiveMQ Producer flow control is not a bug to eliminate, it is a protection mechanism to understand and configure correctly. The default systemUsage values that ship with ActiveMQ are development defaults, not production defaults. Every production deployment needs explicit systemUsage configuration aligned with the broker’s JVM heap, available disk, and expected message volume.

The goal after resolving a PFC incident is not just to increase the limit, it is to identify the root cause (slow consumer, journal pinning, undersized memory, DLQ accumulation), fix it, and put monitoring in place so the next pressure event triggers an alert rather than a 3 AM application freeze.

Get your ActiveMQ systemUsage configuration reviewed by our team → Talk to an Expert

Frequently Asked Questions