The IT Nerd

Infogi today revealed its fourth annual list of trending challenges and opportunities in data management.

Every year, Infogix’s global experts and influencers identify top data trends based on their decades of knowledge and experience working with clients worldwide.

Below are the six trends Infogix has identified for 2020.

Real-Time Data to Disrupt the Future

Massive amounts of data are generated from a diverse set of industry domains, including social networks, e-commerce, transactions, IoT devices and web applications, requiring organizations to react quickly to extract value from that data. Traditional batch processing, where data is sent on a schedule from system to system, will not meet the demands of the changing data landscape. Companies are increasingly turning to event-driven architectures to handle growing volumes of streaming data. They are using distributed streaming platforms like Apache Kafka, ActiveMQ, Apache Pulsar, Amazon Kinesis and many others to provide high-throughput, low latency real-time streaming, flexible data retention, redundancy and scalability. In a world that demands lightning-fast speed-to-insights and real-time access to data, data quality has never been so important. Organizations must enlist vendors who can safeguard data quality to prevent data assets from becoming liabilities and provide validation at a speed and scale to match their data-in-motion.

Cultural Change through Data Governance

More and more organizations are embracing data governance as a means to improve enterprise data understanding and create a data-driven culture. Yet many still struggle to bridge the technical/business divide. Business-focused data governance encourages collaboration between business and technical stakeholders to build user-friendly tools, like data catalogs, that explain technical data in a business context and include critical institutional business knowledge. A business-oriented approach prioritizes business user understanding, empowering them to quickly turn data assets into actionable business insights. Business users won’t use or depend on data they don’t trust, making data quality a critical element of any data governance effort. Data governance that includes end-to-end data quality monitoring and metrics gives both technical and non-technical users a 360-degree view of data that will lead to increased revenue, customer retention and competitive advantage.

Conquering Bad Data

Even though data quality is one of the most persistent and pervasive challenges in data management, historically organizations only prioritized quality when revenue, reputation or mission-critical data was at risk. But that is changing. Complex regulatory compliance and the ever-increasing speed and scale of data have prompted organizations to prioritize data quality as a critical component of their enterprise data governance initiatives. By building a data quality-powered data governance framework, organizations improve enterprise data value and resolve data quality issues before they proliferate across systems. They understand they can’t wait for “data quality horror stories to provide evidence that poor data quality is having an impact on your organization,” as this article notes. By then, the damage is done.

Maturing Data Privacy Laws

The European Union’s General Data Protection Regulation (GDPR) was implemented nearly two years ago, serving as a global catalyst for data privacy legislation. In the U.S., states like Nevada and California have already passed sweeping legislation to protect the personal data of consumers, with many other states poised to follow suit. Noncompliant companies risk both significant financial fines and reputational damage, prompting many organizations to evaluate and address any potential compliance gaps. Businesses need strong data governance to identify and protect personal data, control data access and track lineage as data moves from sources to systems and processes, but data quality also plays a critical role in mitigating compliance risk. Poorly maintained data and poor quality data can both easily result in compliance violations that impact an organization’s brand and bottom line.

Self-Service Technologies on the Rise

Tools and technologies with machine learning (ML) and automation capabilities that enable self-service data analytics took off in 2019. Still, we often see these tools leveraged as part of a departmental project, rather than an enterprise program. To scale enterprise-wide, organizations must encourage data literacy among users so self-service analysis yields accurate and actionable results. Organizations must also establish policies for data access and usage, and ensure the accuracy of high-value data with key capabilities including timeliness, completeness and integrity checks. Only quality data will yield quality business insights.

The 2020 Buzzword: Automation

In the coming year, expect everyone to be talking automation! From hyper-automation using machine learning and AI, to workforce automation that eliminates jobs, to IoT building automation for physical plant efficiency—automation will be a top focus in data and technology. In analytics, companies will have to take self-service to the next level, not just empowering business users to analyze data, but completely automating data science tasks so they can focus more on leveraging insights than generating them. With automated data and analytics, data integrity will be even more critical, demanding automated data quality detection, monitoring and improvement.

To learn more about these data management trends for 2020 and beyond, visit http://www.infogix.com or @infogix.