Category: Apache Kafka®
Blog
The Challenges of Partition Rebalancing in Apache Kafka® Brokers and Effective Monitoring Strategies
Apache Kafka® seems easy at first. It quickly becomes the backbone of your data streaming strategy, making life smooth and simple. But as you grow, Apache Kafka® can become complicated quickly. One of the biggest headaches you’ll face is partition rebalancing between your brokers. Let’s dive into why it matters, and how to fix it.
Blog
Lessons Learned from Managing Apache Kafka® Costs
You probably have seen ads where someone claims that their app can save you money by finding subscriptions you forgot about. I have a hard time imaging someone with $100s of dollars of expenses they forgot about, but I have had the occasional one that was missed. The problem is that people are inefficient when
Blog
Evolution of Apache Kafka® and Advantages Over Messaging
Apache Kafka® has come a long way since its initial development at LinkedIn in 2010 and its release as an open-source project the following year. Over the past decade, it has grown from a humble messaging bus used to power internal applications into the world’s most popular streaming data platform. Its evolution is remarkable, and
Blog
Do you want to use Apache Kafka®? Or do you need a Queue?
Do you want to use Apache Kafka®? Or do you need a message broker and queues? While they can seem similar, they have different purposes. I’m going to explain the differences, so you don’t try to brute force patterns and concepts in Apache Kafka® that are better used with a message broker. Partitioned Log Apache
Blog
How to use OpenTelemetry for Apache Kafka® Monitoring
Apache Kafka® is a high-throughput, low-latency platform for handling real-time data feeds. Its storage layer is in essence a massively scalable pub/sub message queue designed as a distributed transaction log. It can be used to process streams of data in real-time, building up a commit log of changes. Apache Kafka® has strong ordering guarantees that
Blog
Just How Important Is Your Integration Infrastructure?
According to Gartner, the average cost of IT downtime is $300K per hour. Last year Facebook lost $79 million due to a prolonged outage. National Australia Bank had to pay $7.4 million in compensation due to an outage in its payment system. One of our retail customers does 60% of its annual business in 10 days. Having systems up and running over the holiday period means life and death to that company.