Published: Nov 20, 2020

what it takes to be agile in data analytics - 3 steps to enable faster data insights

We have seen an increase in the adoption of data analytics in recent years. Yet, the ability to quickly yield actionable insights continues to be a challenge for most organisations. Some issues that have been most often cited include:

1. Poor data quality: While preparing data for analysis, users often discover that data elements are missing or erroneously captured at the point of collection. There is also often a lack of standardised definitions for data elements, which leads to the misinterpretation of results. Users end up spending more time correcting data integrity issues than deriving insights from the data.
2. Data stored in separate silos: Due to organic growth over time and the lack of a central data strategy upfront, data often resides in various operational and storage systems. Having data spread across multiple disjointed systems causes the blending and merging of data to be difficult and inefficient, inhibiting the discovery of correlations and hidden patterns in a timely manner.
3. Underestimating the effort required to deploy Machine Learning models: Except for a handful of advanced tech companies, few organisations have the expertise and scale to be able to automate the deployment of machine learning models into a production environment, monitor model performance over time and perform continuous re-training and model re-calibration to ensure that the model continues to be effective and relevant.
4. Lack of a common set of analytics tools: Organisations that allow the use of different data analytics tools may end up hindering collaboration and sharing of insights.
5. Over-reliance on data scientists: Many organisations rely on a limited number of data scientists to execute their analytics initiatives, resulting in a bottleneck that inhibits the rapid adoption of analytics.

In this article, we explore how we can take concrete steps to improve agility in analytics, so that we can reap the benefits of data and analytics for faster decision execution.

Data Virtualisation

Agility in analytics can be achieved by making access to data more frictionless. To facilitate and simplify access to disparate data sources, consider exploring technologies such as data virtualisation to add flexibility to an organisation’s existing data architecture. Data virtualisation allows users to access, query and integrate data from various sources, no matter if the data source is on-prem, on the cloud, or across various geographies. This approach creates a single, enterprise wide platform to enable connections to any kind of data source, combines various data types, and allows data to be accessed centrally and be consumed in various modes, including dashboards, reports or advanced analytics use cases. The data virtualisation layer can also provide data catalogue capabilities. This helps to reduce the time typically spent on hunting for data from various data silos and drive self-service, which in turn results in higher productivity.

In addition, with data virtualisation, new data sources can be added significantly faster than in the traditional Extract-Transform-Load (ETL) method. Virtualisation can also simplify the data model management process. According to Alex Hoehl, Senior Director at Denodo Technologies, data virtualisation solutions such as Denodo “simplify the data management process by providing a single access layer which allows data source access, data security and data governance to be managed from one single place.” This means that data access administration activities such as adding new users, changing access rights, monitoring audit trails etc., can now also be done faster and more easily.

Unified Data Analytics Platform

To harmonise the hodgepodge of analytics tools used across an organisation, consider implementing a consistent, unified data analytics platform. An ideal data platform should allow data preparation and data blending to be performed on it. It should also incorporate robotic process automation to automate certain manual processes, especially for mundane and time-consuming data preparation tasks. At the same time, the platform must be sophisticated enough for data scientists to collaborate on advanced analytics modelling, allowing machine learning models to be shared easily across the organisation. Most importantly, a unified data analytics platform should automate the deployment of the models, monitor model performance over time, and automate the re-training and re-calibration of the models for ongoing model maintenance.

Some platforms have the added bonus of providing a code-free or code-friendly environment that allows drag-and-drop functionality, making them as business-user friendly as possible. This is an important consideration in selecting the right analytics platform as we discuss the people impact in agile analytics in the next section.

We explored the topic of data democratisation and data literacy in an earlier article – Data Democratisation: Key to creating a truly data-driven organisation. To re-cap, in that article we stated that being data literate not only means being able to read and analyse data, but more importantly, being able to “argue” with data - challenging what the data means and using data to support a hypothesis. Data literacy is important not only for data scientists and CEOs but also for every staff member on the ground, because they understand the context of the data collected and can come up with unexpected insights into how it should be used.

There are more and more tools today that are geared towards “citizen data scientists” – business analysts who may not be armed with PhDs but who are just as capable at delivering actionable insights from a combination of enterprise and external data. Citizen data scientists are a direct result of the data democratisation movement and they are aided by an array of AI-driven tools and technologies.

We saw earlier the merits of having a unified data analytics platform. Having a platform that business end-users can easily use would further accelerate analytics agility because it reduces the reliance on specialised resources such as data engineers and data scientists. With some training and upskilling, data analysts and business analysts can tap into self-service capabilities that such a platform provides. Now, instead of waiting for data engineers to prepare the data; and data scientists to provide insights, business end users can be empowered to do these data-related tasks by themselves, thereby increasing end user productivity and agility.

Andy MacIsaac, Public Sector Solutions Marketing Director for Alteryx, a market-leading business user-focused analytics platform solution summarised it well: An effective unified analytics platform “provides data analytics, data science, and process automation capabilities across the entire digital transformation capability continuum and brings together business users, citizen analysts, and information consumers to accelerate organisation-wide outcomes.”

Data scientists reported that on average, 26% of their time is spent on data cleansing^[1]. This represents the amount of time that can be saved if there is data quality in the first place.

Broadly defined, data governance is the exercise of authority and control over the management of data assets. The proper management of data in accordance with well-established policies and best practices is key to empowering an organisation with information and analytics capabilities. Effective data governance helps to avoid inconsistencies and errors in an organisation’s data which could jeopardise the accuracy and completeness of the data insights required for sound decision-making. Data governance covers internal policies and procedures that ensure data is secure, trustworthy, well-documented, effectively managed, and periodically audited.

Data is a valuable asset to any organisation, and engendering trust in data is critical in the widespread adoption of data analytics initiatives. By allowing data to be managed with proper governance processes from the outset, organisations can provide staff with access to trusted, high quality data, giving users the peace of mind to make accurate, data-driven decisions without the need to spend time validating the accuracy and completeness of the data in the first place.

There is often not just one single reason an organisation is unable to achieve the promise of agility in its data analytics initiatives. The impediment is usually the result of a combination of various factors as mentioned at the beginning of this article. However, with the right set of data and analytics technologies, improvement in processes, and a shift in mindset towards data democratisation, we can take positive steps towards building more agility in data analytics.

^[1] The State of Data Science 2020: Moving from hype toward maturity; www.anaconda.com/state-of-data-science-2020