MARKET | Sep 4, 2019

Dark Data: how many data remain unused and why

More than half of the data collected by companies is unused: a Splunk report explains the reasons and the possible corrective actions.

Data as a resource or “as the key”, to resume an old saying, useful in order to understand the value of the information collected. A resource available to companies, often in large quantities given that we are talking about 7.5 million of GB generated every day, which unfortunately is still little known and used. According to a report released by Splunk, “The state of dark data”, which involved around 1,300 IT managers from seven different countries, 55% of the data held by companies is “dark”, obscure, not qualified and therefore not exploited. This is despite the fact that 90% of the leaders involved in the research believes that data analysis is fundamental for the success of companies.

What are dark data?

According to the report, dark data are “all the data generated by systems, devices and interactions, which are not known and not exploited within an organization”. Virtually all the information that companies are struggling to capitalize on, i.e. 75% of those collected according to the majority of respondents.

When we talk about Dark Data we can imagine two sub-categories: the data which companies know they have acquired but do not know how to use, and those which they are not certain they have. For this second category, another significant figure is that released by Gemalto, according to which 46% of executives believes that the companies they belong to do not know where sensitive or private information is stored.

Why do data remain buried “in a drawer”?

The main reason why companies do not use part of the data, according to the report, relates to the lack of useful tools and of the skills to analyze them. Where it is true, therefore, that digital transformation generates important flows of available data, it is equally true that companies, whilst being able to keep track of everything, do not have the human and economic resources to analyze the information, thus bearing the costs of storage without benefiting from the data themselves.

Another reason for the non-use should be attributed, instead, to the poor quality of the data collected and stored or to their incompleteness, which makes it impossible to use them to make decisions.

Another, but no less important, limit found by the respondents is that of increasingly having unstructured data available: Big Data which, as such, require more sophisticated and sometimes unknown instruments of analysis.

In addition to the more “technical” motivations, the report points out that 21% of respondents claims that dark data are not taken into consideration due to the lack of interest on the part of the top management of the organization they belong to.

How to turn Dark Data into useful data?

The report published by Splunk tries to answer the question of how to increase the quantity of data used. Among the solutions identified, 76% of the respondents believes training the people who work in the company in order to sensitize them on the topic of Data Governance would be useful; currently, 56% of the respondents considers this is only a slogan with no concrete application. “Every employee should know at least the basics of Data Analytics in the not too distant future”, according to the report.

However, we must not only envisage turning to internal resources, given that 70% of respondents considers it useful to hire external and specialized figures such as Data Scientists.

In addition to working on skills, according to the business leaders involved in the research, it is necessary to invest in digital tools and in the technologies required to exploit the possessed data. Skills combined with tools able to restore the right value to the many data which today, unfortunately, remain uselessly buried in a drawer.