Published: Sep 09, 2022

are enterprises drowning in data?

With 500 million tweets, 65 billion messages sent on WhatsApp, and 4 terabytes of data created from Facebook, it’s estimated that by 2025, 463 exabytes of data will be created each day globally, which is the equivalent of 212,765,957 DVDs per day, according to statistics published by Raconteur in 2019. With this vast amount of data coming in increasing volumes and at a higher velocity, do we truly know its intrinsic value? And what could this look like as we enter an even more data-driven future? 

Since data is the fuel of machine learning and AI technology, businesses need to ensure that the quality of data is up to the mark.  

But having said that, many companies are still defaulting to a keep-everything mindset, archiving everything about their operations and customers in the hope that the data may become valuable someday. Log files and transaction records that could have been deleted are retained indefinitely, resulting in a landfill of data that ends up being irrelevant, messy, and costly.  

Poor quality data — that is, data that is inaccurate, incomplete, or out of date — is a substantial issue for any organisation. Among other things, it can cause an organisation to make wrong decisions or fail to deliver its services properly and ultimately, affect the bottom line. According to a Gartner article released last year, poor data quality costs organisations $12.9 million on average every year. 

It may seem like storing data is becoming cheaper, thanks to advances in storage technologies. However, storing data goes beyond the servers and hardware. One will need system administrators, database administrators, and other data professionals to organise the data in a safe and secure manner. If a company does not have proper data security measures, having more data will only exponentially increase the risk of a data security breach. 

Having more data also means greater efforts are needed to ensure the data is properly quality-checked and organised for access and use by the business. There is no point having all the data when it is not understandable, easily accessible, or trusted by the users. 

Then, there is also an environmental cost. As we see how much energy our data centres are consuming all the time, and how much electronic waste we are producing, the exponential growth in data does not necessarily bode well for our planet. 

According to Gartner analyst Nick Heudecker, the failure rate for big data projects stands at an astounding 85%. Success rates can be significantly improved if companies at the onset of their big data projects or transformation address two main questions:

• What are the business outcomes the company wants to achieve with data? 
• How can it collect and access the data that is most relevant to desired business outcomes? 

Only by being clear about what tangible business outcomes are to be achieved with data, will the company be able to:  

• Have a more purposeful data collection and DataOps strategy, avoiding a meaningless collect-all-data approach.  
• Efficiently build out the necessary data infrastructure to source, integrate, cleanse, master, protect, and deliver data. 
• Enable the development and operations of analytical models and other data-intensive applications that users need to meet the business outcomes. 
• More systematically discover alternative ways to monetise its data 

As companies increasingly turn to AI to build a competitive advantage, the importance of collecting and managing data will only increase. AI systems learn through data, and the accuracies of AI models depend heavily on the availability and quality of data.   

NCS has helped many of its enterprise clients put in place data lakes and extract, transform, and load (ETL) pipelines as well as analytic tools and techniques with which to manage and use their collected data effectively. In the emerging AI era, NCS is increasingly involved in data-intensive projects. Our work in this area ranges from training AI models with proper data sets, to ensuring that the data captured by the AI model is used in a completely fair and ethical manner that does not discriminate in terms of race, ethnicity, or gender.  

For more complex business decisions, NCS leverages machine learning and historical data to aid in decision making. For instance, in cyber security and financial transactions where there are millions of events a day, unsupervised machine learning techniques are used to detect anomalous activities that could indicate a cyber-attack or fraud. 

According to Royston Bok, the Director of AI Initiatives at NCS Group, AI has taken some of the pressure off decision making. In particular, it can help with the following: 

• Relatively straight-forward decisions that a regular experienced employee can make but could be time-consuming as continuous monitoring and decisions have to be made round-the-clock (e.g., determining when someone is in danger, when more taxis need to be at a taxi stand, or what products should be cross-sold to a customer, etc.) 
• More complex decisions where a lot of data must be analysed in order to reach a good, accurate decision (e.g., determining if an online activity is a cyber-attack, if a transaction is a fraud, how to schedule resources optimally, or to identify fake news, etc.) 

He also believes that AI is not just data-driven but also business-centric. Enterprises should work with AI service partners that are able to plan out the AI transformation roadmap with clients while keeping in mind the business values.  

Essentially, platforms, people, and processes must be looked at holistically to succeed in the AI transformation journey. 

Ravi Bedi, Head & Practice Lead, AI-Led Solutions, NEXT Solutions at NCS, outlined some of the key ways data is being used in different industries to create value and drive growth in times of uncertainty. He highlighted the following examples:    

1. Healthcare: Helping to monitor patients and dispense medicines 
   AI is able to make split-second decisions concerning emergency care while monitoring multiple patients without the need for rest. It is also able to ensure dosages are accurate and promptly administered at specific times.  

2. Financial Services: Helping to target customers more effectively 
   Financial institutions are using AI-powered advanced analytics to remain competitive in a sector that has undergone major disruption in recent years due to the emergence of innovative fintech start-ups, evolving government regulations, and growing demand from consumers for more personalised products and a better customer experience. These institutions use advanced analytics to generate insights from data that can help them more effectively target their products at the right customers. The use of advanced analytics ultimately makes customer acquisition and retention more efficient and cost-effective and enables an organisation to capture more market share.  

3. Airlines: Improving cross-selling and predicting demand  
   Airlines are increasingly harnessing the power of data to understand the needs of their customers more deeply and thereby tap opportunities for generating additional revenue through cross-selling. They are also using AI to better predict demand for flights whether it is based on seasonal trends or special events like music festivals or sports competitions. This allows them to streamline their operations and thereby stay afloat at a time when the airline industry is still reeling from the impact of the Covid-19 pandemic.  

4. Hospitality: Enhancing pricing strategies 
   Like air travel, the hospitality sector was hit hard by the pandemic. As a result, hospitality businesses are turning to AI to better navigate uncertainty. Hotels, for instance, are using AI to analyse booking data and adjust pricing strategies in line with customer behaviour.  

5. Telecommunications: Keeping up with evolving customer expectations Telecommunications companies have to constantly innovate to stay on top of the rapidly evolving expectations of their customers. Hence, many have found AI to be a valuable tool for analysing the wealth of data at their disposal to build up a holistic understanding of individual customers and offer service packages that are tailored to their specific needs.  

6. Manufacturing: Building adaptability and resilience 
   Manufacturing companies are overcoming supply chain constraints by using AI to analyse a wide range of internal and external data, including shipping timetables, warehouse capacity, and more, in order to more accurately forecast the supply of raw materials and components as well as the demand for specific products. This allows them to adapt more quickly to things like supply chain disruptions and thereby maximise profitability.   

By storing and making accessible certain persistent data features and attributes, some companies have been able to unlock significant value in their digital transformation journeys and enable innovation at-scale. 

There is a wide range of tools out there to help companies organise and harness the power of their data, once there is clarity about the relevant use cases and data attributes. 

C3 AI and NCS have established a Center of Excellence together to jointly explore new solutions that will accelerate the adoption of enterprise AI and develop custom-built offerings tailored to the business requirements of the Asia-Pacific market. 

AI models at-scale will be key to building the next generation of digital applications with enterprise AI embedded. A model-driven architecture fully addresses the requirements for the digital transformation software stack, providing a low-code/no-code AI and Internet of Things (IoT) platform that accelerates software development by a factor of 26 or more, reduces costs and risk, and delivers future-proof applications. 

Data and AI go hand in hand. With quality data, AI is able to learn and provide better outcomes that in turn reap more meaningful returns. Data quality is indeed a pre-condition for AI. Poor data quality leads to analytics and AI projects taking longer than expected which could lead to losses in revenue, resources, and reputation. Companies, regardless of sectors, ultimately have to be clear about what tangible business outcomes they would like to achieve. Only then can they build the right strategies, infrastructure, and processes to harness the power of data.