The concept of data products is evolving rapidly, reflecting both the growing sophistication of data users and the increasing capabilities of modern data platforms. In the early days, data products were simpler and more predictable; they served specific business needs, and the data required to create them was clearly defined. Agile methods in DataOps enabled these well-structured, easily understood products to reach production quickly, allowing organisations to generate immediate value from their data. In these cases, business users knew precisely what data sets they needed, and the features of these data products were straightforward.
However, as organisations recognise the potential of their data beyond well-defined use cases, a shift has occurred. Increasingly, data products are being created to enable exploration, allowing business users to enter a discovery phase where insights are neither obvious nor predefined. This shift has led to the creation of “second-generation” data products that emphasise flexibility, discovery, and adaptability.
Automated data preparation has been a key driver in this transformation. By using automated processes to ingest, clean, and prepare data, platforms can provide business users with ready access to vast amounts of well-organized data, ideal for exploratory projects. Automated preparation unlocks opportunities for these users to dive into “unknown-unknown” use cases, where the aim is to uncover patterns or relationships that may not have been apparent before. In this scenario, the platform, data, and business goals intersect to create a powerful environment where discovery flourishes.
New technologies, such as data fingerprinting, tagging, and profiling, have been crucial in enabling these exploratory products. Data fingerprinting and tagging help to surface relationships between data entities that would otherwise go unnoticed. Knowledge graphs, for example, can visually map these relationships, making it easy for users to explore connections and derive insights that go beyond traditional reporting. Additionally, by profiling and organising “dark data” (data previously underutilised or difficult to access), these techniques make it possible to reveal valuable information hidden within the organisation’s data ecosystem.
Despite these advancements, the usability of such sophisticated tools remains a challenge. Knowledge graphs, for instance, require users to understand tools or query languages like SQL, making them less accessible to non-technical business users who might not be skilled in querying. While these tools are highly effective, they can be complex for a general audience who may need to write SQL queries to access insights.
Today’s business users are accustomed to the ease and immediacy of tools like ChatGPT and AI-powered co-pilots, which have become valuable assets in everyday operations. With the rise of large language models (LLMs) in the market, there’s a growing demand for intuitive, conversational interfaces that allow business users to interact with data products without the need for specialised technical knowledge. These users want a ChatGPT or co-pilot-like interface that enables them to explore data simply by asking questions in natural language.
This demand for user-friendly interfaces has given rise to what we can call “next-generation data products.” These advanced products are no longer just data repositories but interactive, AI-enabled platforms that empower business users to extract insights seamlessly. By integrating LLMs and conversational AI into data products, these next-gen solutions bridge the gap between technical data capabilities and user accessibility. They make it possible for business users to interact with complex data structures, such as knowledge graphs, without needing SQL knowledge, empowering them to focus on decision-making rather than data retrieval.
Next-generation data products represent a shift in the role of data platforms. They’re transforming from passive tools to active enablers of insight, combining the power of automation, AI, and conversational interfaces to create a truly user-centric experience. As organizations embrace these advancements, data products will increasingly serve as intuitive collaborators, delivering value to the business and driving innovation in unprecedented ways.
