Today there is much talk of Data Governance and almost always the focus is on two fundamental aspects: compliance with current and future legislation and security. But is it really only this? In reality, there is much more.
If we follow the approach of dama.org to data management, governance is the central element. There is no data management without it.
What is Data Governance?
Data Governance prepares us for the digital transformation process, because data are no longer locked in proprietary systems, or in a single data source or Data Warehouse, the complexity of organizations multiplies the amount of information and, especially now, “data are business”. There are companies that owe their full value to data such as Airbnb or Booking, which manage information and do not have real estate but represent the real tourism market.
Data Governance helps us make efficient decisions by providing credible and secure information. What is gross margin? How is it measured? Where can I find these data? What sources assure me these data?
Coming from Business Intelligence I have experienced moments of real frustration when, after endless processes of standardization, quality and increase in data, reports and analyses presented unreliable results. It has always been the fault of data: not traced, not consolidated, not real. Whoever deals with sales can say that a sale is when a transaction takes place, but for logistics who decides? Who controls? Who verifies? Welcome to the era of CDO. Data Governance allows us to improve processes and the tangible result is always the reduction of costs.
Data in times of digital transformation
An increasing number of data sources (Big Data, IoT, departmental ERPs with disparate technologies, data managed with Cloud) prevent us from working efficiently with the classic approach Kimball-Data Warehouse, which is why different platforms of virtualization and orchestration of data like Denodo, Querona or Talend are growing.
On the one hand data-hungry analysts, on the other CIOs, CDOs, CTOs always attentive to resources, the use of machines and information flows. Without orchestration or a tool of Data Governance it would be difficult to consistently comply with increasingly restrictive regulations (GDPR, RDA) and the data-driven mission. Without forgetting that it is becoming increasingly easy to change technology platforms or work with multiple technology platforms at the same time. In both scenarios, it is essential to monitor information assets with a unified view.
A practical help in decision-making
Working with data that have been defined unambiguously is a decisive factor for analysis. Some think that this is an almost vertical process starting from the different departments of the company, others declare its completely transverse nature within the organization and defend its full adoption right from the first day.
It is necessary to control a precise definition of information, such as the ability to explain the process of building an indicator, the sources of information and the operations that have been carried out in the transformation process. All this ensures success in giving credibility to the information provided. Analysts must have reliable data and go through an internal certification process.
In order to work with the necessary data, it is essential to observe the evolution of the results of information quality with continuous improvement and verification processes. If the level of data on turnover drops from 90% to 40%, will it still make sense to analyze the evolution of sales? What credibility will a predictive analysis fed by poor quality data have?
It is necessary to be able to obtain business value that can be quantified by information, recognizing data as an additional asset of the organization, or as the most important asset.
The process of transforming data into relevant and secure information
Creating a link between the user and the data engine reduces solution times and eliminates reprocessing, so it is necessary to generate agreements and define responsibilities for eliminating duplication of functions as well as misunderstandings about the role and responsibilities of each. This is despite the variety of data we have in our systems, with clear adoption of a system of personal data and metadata. The result is a substantial decrease in the time spent on research and understanding information, which drastically reduces the effort of working groups.
Working with governed data means fewer replications of data, fewer processes, fewer reports, less computing capacity and therefore less structure.
Maximizing the potential for data revenue generation and exploiting Data Lineage
A data management policy and a data acquisition system allow the transformation of data into information. By taking advantage of Data Lineage and the management of metadata, we obtain not only better performance but also a winning strategy. If data are structured and governed they become credible and are used. Otherwise, all this work will have been for nothing.
Typically, data governance and management determine the scope of Data Lineage according to regulations, the company’s data management strategy, data impact, reporting attributes and critical elements of the organization.
Data lineage provides an audit trail of data points at the highest granular level, but lineage presentation can be done at various levels of depth in order to simplify vast amounts of information. Data Lineage can be visualized at various levels based on the granularity of the view. At a very high level, Data Lineage identifies which systems interact with data before reaching the destination. As granularity increases, it reaches the data point level where it can provide details on historical behavior, attribute properties, trends and data quality.
What is the future of Data Management?
According to Gartner, “requests for data are constantly increasing within organizations: ranging from the request for easier and more flexible access to data, through greater data governance, to the hope of being able to quantify the value of data and sell them. These diverse expectations in an increasingly complex and distributed landscape are shifting the attention of organizations to managing metadata, hoping that if data are unmanageable, metadata will be easier to manage”.
These are the challenges that the tools must face:
- variety and scope of metadata supported
- extension of the field of application of metadata through automation (Machine Learning) and capacity for semantic search, standard processes and crowdsourcing
- semantic formalism (also known as formal ontology) for improving interoperability
- shared understanding among multiple domains
- new methods of acquisition and visualization of metadata (preparation of data for analysis is the basis of this requirement)
- transfer of metadata ownership from the CIO to the CDO, or a similar function.
In MDM, for example, there is no standard for data migration between different tools. Within the scope of the activities of dama.org, the intention is to promote the use of some data formats such as XML so that it is possible to manage all this easily.
One thing is certain: having a governance tool is not an option, it is a natural evolution in a scenario of multiple data sources that virtualization itself has encouraged.