Organisations burying heads in the sand over Hadoop

Almost half of organisations have no intention of working with Hadoop to address their data integration needs over the next 12 months, while less than one in ten already run a Hadoop cluster in a production environment. This is evidence of a dangerously tentative approach to innovation at a time when the spiralling demands of Big Data are putting considerable strain on traditional tools, according to Nejde Manuelian, Director of EMEA Operations at Syncsort.

45 per cent of organisations questioned as part of Syncsort’s EMEA Big Data and Hadoop Trends survey[1] said they would not be working with Hadoop over the coming year. 25 per cent are currently experimenting with the open source platform, while a further 15 per cent have plans in place to begin using Hadoop within the next 12 months. Only eight per cent are currently in the process of selecting a Hadoop distribution, and just seven per cent are actively using a Hadoop cluster.

A far-sighted minority are putting themselves in a strong position to manage the challenges of Big Data, while those organisations that are ignoring Hadoop risk leaving themselves exposed, says Manuelian: “As organisations find themselves collecting and generating more and more data to analyse in the quest for competitive advantage, many are finding that their traditional data integration tools are struggling with the volume and variety of data they are having to handle. It is, therefore, crucial that organisations take steps to identify alternatives quickly.

“It is still very early days for Hadoop as a data integration platform, and it would be fair to say there is still plenty to be done to ensure it is mature and feature-rich enough for the enterprise. But the open source community, with contributions from vendors, is making rapid strides in this direction – and more widespread adoption will only help accelerate this progress,” Manuelian continued.

Of those organisations already using Hadoop or planning to do so in the next 12 months, nearly nine in ten (88 per cent) see Hadoop as a complement or extension to their existing architecture, rather than a fully-fledged replacement.

“It is perhaps understandable that most organisations are being somewhat tentative in their deployment of Hadoop, given the progress that remains to be made with the platform. But organisations that are dipping their toes in the water will at least be prepared when traditional tools cease to be fit for purpose. Waiting for those tools to break before exploring the alternatives will be too late – organisations need to pull their heads out of the sand,” Manuelian concluded.