Use metric stores as main data servicing layer

In a single generation, we have to witness the internet revolution, the cloud revolution and it can be said we’re in the middle of the data revolution. Data has always been critical, but today its sheer size, speed and utility is reaching dizzying new heights.

Now that data applications and analytics are a permanent, essential, growing part of our work life we ​​can’t get enough. And that’s a problem.

A low-coding or no-coding data platform is going to be a critical component for data driven decision making and daily business operations.

By using a metrics store as the main servicing layer, data platforms by NLP (Natural Language Processing) and AI algorithms can reduce or even eliminate the dependency on SQL for business users.

While tremendous improvements have been made in the data engineering field, we still need to make the insight available to regular business users, not just power users. Power users understand the nuances of data well enough to successfully extract insight from it and typically accomplish this goal with SQL.

But this model doesn’t scale for most other businesses. Can your end users – store managers, sales reps, marketers, clerks – have that level of SQL skills? Can they download any dataset, spin up a database, create the table joins and run SQL queries to get to the insight?

Data platforms need to enable regular business users who don’t have deep SQL knowledge. That is the only way to get insight to everyone.

What does LC/NC mean for data platforms?

Analytical data is typically stored in a data lake/house. End users have to figure out how to query these data platforms with some sort of query language like SQL or through Python scripts.

To enable Citizen Analysts and Citizen Data Scientists, we need to reduce or eliminate this coding step and understand that they:

  • Don’t care about tables and columns
  • Only care about business information (sales volume, shipping cost, etc.)
  • Focus on business performance instead of arcane technical skills
  • Want information readily available when a question is asked
How to build an LC/NC data platform
Metrics Store as the main data service layer

Instead of servicing data from a data platform, enterprises should be creating metrics stores where business metrics are defined and curated. End users can simply drag and drop these metrics into their tools such as Excel, BI dashboards, web applications, etc.

In a Metrics Store business metrics are defined, calculated, and stored in a central location overlayed with appropriate governance processes. End users can then define and drive metrics that matter to their daily tasks (eg. sales figures by product, year over year sales growth, profit margin). These are the data points that users want to know. So why not define them once, calculate them correctly, and give everyone access across the company?

Natural language to help users ask questions

Instead of asking users to learn SQL, why not have them ask questions in plain English? With the advent of NLP technologies, we should expect today’s data platforms to understand the everyday language of everyday users.

We should also expect the platform to push NLP capabilities one step further with context awareness. Context awareness is critical for an interactive analytics experience. The ‘machine’ can have a conversation with the users in which follow up questions can be asked and answered.

With AI built into today’s data platforms, it will become vastly quicker and easier to analyze data and metadata, cross reference user behaviors, and optimize the way users get their questions answered. Today’s AI algorithms can even predict what questions users might ask and have the answers ready. AI can improve the user experience when they want to interact with their data. At the same time, AI in the data platform can also auto-optimize data storage to eliminate waste and reduce cost.

Magic happens when the most exciting technologies of the day converge. In the data world, going from unrefined data to insight and intelligence has been a massive undertaking. But like data itself, creating intelligence, advantage and insight must be viewed from a certain distance. From that distance we can now see that problem can be solved by leveraging metric stores, NLP, and AI to enable a Low Code/No Code platform.

Leave a Comment