The Central Nervous System of Companies: Confluent at the Intersection of AI and Real-Time Data

The world is said to be moving faster than ever, especially in terms of technology and data. More and more essential services depend on the ability to collect and process data in real-time. This is where Confluent, a company recently acquired by IBM, comes into play. The company aims to position itself, as stated by CEO Jay Kreps, as the "central nervous system of businesses"—a theme that came up repeatedly during the Current conference held in London, which Edge9 attended.

Real-Time Data as the Foundation for AI

"Not everyone might like that we talk about AI here, but we must," Kreps began. Indeed, much of the conference centered around the use that AI can make of real-time data. One cannot help but think back to a time when services like ChatGPT referenced a snapshot of the web (and thus, we could philosophize, of the world) that was frozen in time at the moment the underlying model was trained. Then came the possibility of retrieving real-time information from the web, allowing for timely updates. The difference in utility and practicality between "frozen" models and "real-time" ones is vast: in the former case, for example, there could be partial or missing information (imagine asking for election results that had not yet occurred after the model was trained), while in the latter, the likelihood of this occurring is lower (though with highly variable results).

But we could take a step back and think more generally about the importance of "real-time" by looking at information related to an airport: imagine if the airport's own website did not report delays in departures and arrivals in real time, but only provided updates at predefined intervals, say every hour. We are now used to having information reflect the state of things without delays, and going back is simply not possible.

The same applies to artificial intelligence. The example of ChatGPT is merely to illustrate the difference between a "batch" approach (where information is processed in blocks, often at predefined intervals) and a "streaming" approach (where processing occurs in a continuous flow, without interruptions). The implications for AIs are vast and go beyond the more well-known chatbot: if the aforementioned airport does not give its AI chatbot access to real-time data, any questions posed by travelers would go unanswered or, worse, would receive incorrect answers.

This is why Confluent states that relying on data lakehouses, as has been done until now when discussing data for AI, is no longer sufficient. An additional step is necessary: to include real-time data, especially with the advent of AI agents.

"If you're trying to use data that's hours or days old, it really doesn't work. If you're trying to engage an agent with customers or people within your organization, perhaps to make decisions about business activities, you can't rely on an outdated version of reality," Kreps says.

Confluent's Innovations

During the presentation, Kreps highlighted how the shift to real-time data is reflected in the infrastructure layer, which is evolving to go beyond batch systems. For this reason, Confluent has invested many resources to make data streaming a generalization of the batch model: this way, it's possible to use real-time data without having to replicate data, rewrite all applications, or completely rethink the architecture. This is what the company announced last year with Flink; this year, Confluent announced an expansion with an MCP server, Confluent Intelligence, materialized tables, and Confluent Skills.

Through the MCP server, companies can connect their AI models to the data managed by Confluent and thus give them access to real-time information streams. Confluent Intelligence is the next step: it's a stack managed by Confluent designed to give AI applications simplified access to real-time data, without having to design everything from scratch.

One issue identified by Confluent is that applications only pull data after it's been entered into traditional lakehouses: this requires waiting several seconds for data to become available, which isn't compatible with all use cases. Therefore, the company has developed a cache that stores data in memory or on SSDs to speed up access. Cached data is available through the MCP server and is thus also accessible to AIs through simple SQL queries.

Materialized tables allow Flink to modify real-time SQL tables used to query data from applications and AI models without having to manually manage the complex underlying processes, which require stopping data flow in the old table and restarting it in the new one. This simplifies tasks such as changing the data available to AI models without stopping all processes and redesigning them.

Finally, the new skills for agents provide them with the necessary information to operate with the Confluent platform. In other words, they act as a sort of "instruction manual" to help agents connect to the Confluent platform using the MCP server.

From the Era of Business Intelligence to Artificial Intelligence

Kreps went further and made a very significant statement: "we're moving out of this world of just business intelligence, and entering a world of artificial intelligence." We decided to leave it in the original to maintain the contrast Kreps wanted to highlight between "business intelligence" and "artificial intelligence."

For this to be an important transition, it was explained to us by Peter Pugh-Jones, EMEA Field CDO at Confluent.

"Until now, I had to connect to my computer to get a report, so I could explore the data, looking for a pattern or anomaly in that data to help me decide, from a business perspective, what decision to make. I believe that in the future, the direction will be that when we talk about artificial intelligence, there won't be that report anymore. There won't even be a need to open the computer. There will be a notification on the phone that alerts me: there's something interesting," Pugh-Jones tells us.

"It's already like this with tools like LinkedIn, right? A notification comes indicating that that person has changed jobs or has had a birthday. And the example is not coincidental, because LinkedIn is where Confluent originates [Kafka, on which Confluent is based, was originally developed internally at LinkedIn by Kreps and others]. Confluent provides the necessary tools to bring information to AI in a better and smoother way, enabling better insights for those who need the information."

This is where the theme of the central nervous system returns. Pugh-Jones tells us that the ideal is to analyze data streams to detect anomalies: "I'm interested in receiving data from that sensor on that conveyor belt, but I'm not interested when everything is fine, only when there’s something different." Just think about how our sense of smell works: we only sense new odors that deviate from the situation we're used to, as our brain automatically filters out the "normal" signals (either because there is no odor, continuing with the smell example, or because the odor is constant and we have habituated) to focus on those that deviate from this norm.

The idea is to process data to mimic this type of behavior: there's a constant flow, but it’s the anomalies that get detected and are truly useful. And here converge real-time data, AI, and the central nervous system metaphor: companies already have a constant data flow; the problem is to separate the wheat from the chaff, that is, to intercept what truly matters and use it to allow AI to generate those insights that everyone is pursuing with multi-billion dollar investments in AI and data centers.

Beyond AI

There wasn't just AI at Current 2026. More traditional aspects were also discussed: for example, Confluent announced Kafka Copy Paste, a system that automatically manages the migration of an existing Kafka platform to Confluent and handles all steps automatically (detecting existing data flows, creating resources for their management on Confluent Cloud, migrating data, moving clients).

There was also discussion of private cloud, with Confluent Private Cloud: a solution designed for those companies that need (or prefer) a private infrastructure without giving up the ease of use of the cloud. According to Confluent, using its private cloud solution allows for a 50% cost reduction compared to public cloud and offers greater speed (up to ten times, according to the company’s estimates) in replication and broker management. Upcoming features include the ability to manage Confluent Private Cloud as a multi-tenant infrastructure, meaning multiple (internal or external) clients can manage their platform independently.

A Change in Atmosphere

Confluent was acquired by IBM at the end of 2025, in an 11 billion deal that was announced in December. A move that ensured IBM control over one of the leading platforms for real-time data management, increasingly relevant and valuable.

At Current 2026, IBM's acquisition was simultaneously an entirely negligible topic and a very important one. It was negligible because it was hardly mentioned—except for a citation at the beginning of the presentation by Kreps; on the other hand, it was very important for its effects on the conference itself. In 2025, many companies participated in the conference, including prominent names like MongoDB and Oracle. This year, the presence of partners was limited to small companies and entities specializing in creating tailored solutions for Apache Kafka (out of around fifteen entities present, five were independent Kafka dashboards).

The change in atmosphere was drastic, and speaking with some of the companies present, it seems that the IBM acquisition imposed it. It wasn't, as far as we understand, an explicit imposition, but more a change related to how Confluent is perceived. The feeling is that it has shifted from an event that Confluent itself claimed to sponsor but was dedicated to data streaming and "neutral" to a pure corporate event; from an event by and for the community (which remains relatively small) to one more focused on Confluent. Last year, actual competitors of Confluent were also present simply because the event was relatively neutral, whereas this year there was no trace of them.

Whatever the future of conferences may hold, Confluent has clearly recognized an emerging market need, significantly amplified by the advent of AI and AI agents: managing real-time data has become a necessity in many fields of application. The announcements made at Current 2026 show the direction the market is taking, but there is still a long way to go: often there is still a lack of awareness in companies that real-time data must be managed appropriately. Confluent will have its work cut out for it.