Data quality 'chasm' looms

insightsoftware has released research that shows data quality is the biggest challenge organizations face when making data-driven decisions. The 2022 State of Analytics: Quest for Data Fluency asks businesses about their attitudes toward data literacy, the problems they face, and how close they are to reaching “data fluency”, the ability to fully understand data and information and garner insights to enhance the decision-making process. Large-scale adoption of cloud technology solutions over the last few years has led to higher volumes of data, but with it comes an urgent need to make sense of it and drive critical business decisions.

According to the report, the first hurdle for businesses is a lack of data quality. More than a third (39%) of respondents citing “completeness”, or a lack of comprehensive data, as an issue. Almost the same number see consistency (38%), and accuracy (35%) as major challenges. The next challenge is the inability to understand data through existing analytics tools, with complexity being a challenge for half (50%) and a lack of training (43%) for almost as many. This need for self-service capabilities is further emphasized by the top priorities for organizations seeking to improve end-user data literacy, which were “making tools more intuitive” (77%) and “enhancing custom reports” (66%).

“As workforces shift to hybrid environments, businesses will continue to adopt cloud technology and as-a-service solutions,” said Monica Boydston, Vice President, Product Management at insightsoftware. “When these new solutions operate in tandem with legacy systems, the result is a large pool of data from multiple sources—"polluted” data that requires cleaning. Doing this takes time away from the critical work of gathering business intelligence. We can’t eliminate errors and inconsistencies, but there are tools that can make things easier, paving the way for teams to achieve data fluency.”

Predictive analytics, which requires people, tools, and data to work together, represents the final stage of data fluency, and should be a goal for any data-driven business. Many organizations are not there, yet. Thirty-seven percent of businesses say they cannot extrapolate data to forecast future outcomes, while only 27% of organizations say most users are able to forecast using current analytics tools. However, despite these significant challenges, organizations do clearly value the concept of data fluency—and have an appetite to achieve it. Seventy-four percent believe that it is important for users to become more data literate, while 61% believe that working toward a data fluent state is desirable. Support from business leaders also increased 30% from last year.

“The growing recognition of the value of data fluency within organizations is positive, but also puts pressure and creates expectations across all departments. Employees need to be properly trained, be able to piece different data concepts together, and possess soft skills to communicate that widely across departments. Quality of data, however, remains the most critical concern,” Boydston added.