Performance Innovations Powered by Message Data in Your Integration Infrastructure (i2) Are Here to Stay

Message data is a digital treasure trove for enterprises that need to meet evolving operations performance, CX (customer experience), and even compliance goals.

“Message Data” includes critical information and context missing from log files and other data sources.  Getting it easier and earlier enables advanced analysis and improvements to begin without delay.

If this Integration Infrastructure (i2) messaging applications are in your environment, you’ve got a complex stack to be managed by the operations team. If your i2 (also referred to as the messaging middleware layer), includes IBM MQ or 2 or more of IBM MQ, IIB, ACE, TIBCO EMS, and/or Kafka, then you’re likely facing growing challenges in:

• Service Level Achievement for CX
• RCA (root cause analysis) and Problem Resolution
• Site Reliability Engineering (SRE) Goal Achievement
• Compliance-related Risk

The right data ingested and analyzed (or ”observed”) in a timely fashion is the key to addressing these four key challenges.  If you’re using IBM MQ v9.2.2 or earlier versions, there’s also a data ingestion problem which we’ll get back to shortly.

Observability & AIOps Platform

With Observability, you don’t need instrumentation. You should be able to view the external state of the system and infer potential issues from there. Key indicators to track include:

Latency – time to service a request

Traffic – demand placed on the system

Error rate – the rate of failed requests

Saturation – which resources are most constrained

To gain “observability” for a hybrid enterprise today, your Observability platform must:

• Respond to events. This includes the capability to raise an alert if a message queue was full or a channel was down.
• Continuously watch individual metrics (such as current depth) and do statistical analysis of trends to see whether a problem was coming.
• Dynamically build a holistic view of the entire system or critical business workflow, capturing data as components of a transaction passing through different servers and the i2 messaging middleware layer, store this in a time series database cluster, and run analytics in real-time to identify trends, predict and enable fixing issues before they occur.
• Provide AIOps (machine learning-powered ITOps automation).

Ingestion and Automation

Ingesting message data from these i2 apps (along with all forms of relevant machine data such as logs, metrics, transactions) enables your team to visualize relationships, find patterns and anomalies, gain intelligence into your mission-critical digital & business services.  Applying a wide range of machine learning algorithms can also present a predictive view of events, allowing problems to be identified and resolved quickly.

The AIOps capabilities of the Observability platform will ultimately bring the most productivity and performance gains for today’s hybrid enterprise. As enterprises evolve, it becomes increasingly likely that multiple-middleware variants will be in use. That often means the need to ingest message data from IBM messaging middleware, along with Kafka and even TIBCO EMS.  Enterprises can thank their mergers and acquisitions (M&A) team for making the i2 layer even more complex.

Before or After the Message Queue? (That is the question for IBM MQ shops)

Sooner is better for ingestion on an Observability Platform doing AIOps, and that’s a big reason the pressure has been on the IBM MQ team to enable Kafka-like message copy capability, enabling message data to be ingested for analysis without having to wait for it to exit the queue or disrupting MQ performing its primary function.

Enterprises serious about realizing all the benefits of Observability and AIOps have:

• Addressed the i2-shaped hole in their data feeds, and:
• Been looking to a new release of IBM MQ[1] to accelerate “data-driven” analysis, action, and automation for MQ and multi-middleware-powered operations. IBM MQ shops have been envying their Kafka-streaming brethren for years now because they want to stream messages for analysis immediately and without impacting MQ doing its work. See the footnote to see the latest on IBM MQ.

Getting Started and Realizing ROAI (Return on Observability/AIOps Investments)

Performance and complex transaction-based compliance innovations powered by message data in your multi-product i2 layer are at hand. All you need to get started is your own data and an Observability platform that can handle traditional data feeds as well as message data from your enterprise’s unique mix of IBM MQ, IIB, ACE, TIBCO EMS, and/or Kafka.

The Nastel Platform is one option to consider and there is a SaaS option to get you started at no cost.  If you’ve already invested in a platform but are missing the message data piece Nastel can help you there as well, managing the i2 messaging middleware layer has been Nastel’s specialty since before the first IBM MQ, TIBCO EMS, and Kafka messages were sent.

[1] What’s new in IBM MQ 9.2.3 for Multiplatforms – base and Advanced entitlement

About the Author: A business-to-business (B2B) innovator and products executive with 20 years of progressive experience, Steven Menges is a frequent industry author and speaker on enterprise computing, data analytics, managed service providers (MSPs), IT Security, regulatory compliance, and buyer’s journey-based engagement. Mr. Menges is also an Adjunct Instructor and Capstone/Thesis advisor at the NYU MS in Management and Systems (STEM) and MS in Integrated Marketing programs and is the co-developer of the Business-to-Business Marketing Maturity Model.

Steven has an MBA in Strategic Planning from the University of Pittsburgh’s Joseph M. Katz Graduate School of Business. Mr. Menges drives adoption and product efforts for Nastel Technologies and is the Director of the Nastel Technologies Advisory Board.