Can the cost of Big Data outweigh the benefits?

There is growing recognition within the corporate sector that businesses face significant and increasing “energy risk” due to the huge pressures on energy suppliers and distributors around the world. This cost problem is particularly acute where local power generation is reliant on coal or on gas from unreliable sources, where nuclear plants are being decommissioned, or where governments are investing heavily in renewables and passing the costs onto businesses.

The issue for businesses is that as we continue our reliance on fossil fuels, all-in energy costs are for the most part high and rising, even in underperforming economies. This is especially true in Europe, where electricity prices have steadily and relentlessly risen over the past several years. The risks associated with rampant energy price inflation are particularly high for knowledge-oriented businesses which are critically reliant on power intensive IT and data centres. As more businesses use more modern technologies like cloud services and Big Data to drive efficiency and better decision making, the demand for the data centre facilities where these applications live has soared.

Big Data defined
Perhaps the best definition of Big Data comes from the IT industry analyst firm IDC, which says that “ultimately, Big Data solutions exist to improve decision making and to provide greater insights, faster, to decision makers at all levels of the organisation.” Fundamentally, Big Data tools provide a way for organisations to make use of the vast quantities of data that they are constantly collecting. A wide range of data - from customer buying patterns to stock levels, to lighting efficiency - can be analysed to streamline operations and drive business growth. However, this data needs to be stored and then analysed. A huge volume of the structured and unstructured data needs to be crunched and refined in order for it to be a relevant piece of information that companies can then act upon, and ultimately this is what is increasing the pressure on data centres.

Hidden cost of Big Data
By unlocking business intelligence from growing troves of data held within organisations, Big Data projects often promise to have a positive and substantial impact on a company’s profitability. Indeed, there have already been some notable success stories. For example, Avis Budget has reported an impressive return on investment from one of its Big Data projects. By analysing the data amassed on its 40 million customers and their behaviours, the company increased its revenue by $200 million. To provide a sense of scale of the amount of data being generated and stored to support Big Data, every two days we create the same amount of information as we did from the very beginning of time up until to 2003. Big Data is also being used by firms to cut costs. Tesco uses Big Data analytics to ensure its lighting and refrigeration levels are running at optimum levels at all times, cutting its energy costs by £3.9 million. Married to this explosion in data creation are ever more powerful analytical techniques and algorithms, so it is of little surprise that forward thinking businesses are embracing Big Data technologies to improve their financial results and compete more successfully.

Yet, many companies are blinded by the potential benefits and pay inadequate attention to the rising electricity cost associated with powering Big Data platforms. Alternately, they feel resigned to simply accept it as an inevitable cost of doing business. Despite advances in technology, the servers which store and process Big Data require vast quantities of power to both run and cool them. If housed in traditional urban European data centre facilities, which primarily draw electricity from high priced, fossil fuel powered grids, the cost of running these Big Data initiatives can come as a big surprise. Of equal concern, any failure in the delivery of energy supplies to the data servers can force these vital applications offline, with severe consequences for everyday business operations. For example, in a report from Insurance company Allianz, it predicted that during a blackout, a financial trading company could suffer costs of around £5 million per hour, emphasising the need for systems to be running at all times. However, Ofgem, Britain’s energy regulator, recently forecast that the spare margin – the amount of energy available for consumption at times of peak demand – will fall to as low as 2 per cent in 2015, increasing the risk of blackouts6, and this problem isn’t unique to the UK. Countries all across Europe are struggling to juggle the need to secure a more adequate and renewable power supply, whilst also trying to avoid shocks to the system such as nuclear plant shutdowns and threats to gas supplies from Russia.

That said, as companies of varying sizes chose to implement Big Data projects, the cost associated with these ventures will differ accordingly, While this makes it difficult to accurately pinpoint the full cost of powering the IT equipment, reasonable estimates put this figure at approximately 15 percent of the total cost for most data infrastructure systems. For some deployments this figure is much higher. For example, NASA7predicts that its ‘Discover’ super computer system, used to model and simulate the Earth’s climate, could consume enough power to fuel 16,000 homes.

Energy sources
Businesses need to begin to prioritise conversations regarding the sourcing of electricity and concentrate on discovering ways to mitigate the risk that unreliable energy and its suppliers can cause. Given that power prices vary significantly among world regions – and fossil fuel prices and green tariffs remain the primary driver of electricity prices in most markets – businesses should understand their overall exposure to power costs, including IT related power.

There are tools available to reduce a company’s exposure to energy risk, and ultimately provide them with more stable costs. Traditionally, companies have built their data centres in major metropolitan areas close to their headquarters, allowing them quick access if things need attention. However, modern initiatives like cloud computing enable the use of remote data centres with no decrease in performance or security. One solution to the power quandary is to base power-hungry IT projects in data centres in northern climates with unfettered access to dependable and low-priced renewable energy sources such as hydroelectric and geothermal. In Iceland for example, the energy produced is 100% renewable, and the country only uses a small percentage of its total energy capabilities. Furthermore, customers can procure long-term, fixed-price contracts, providing their organisations with long-term visibility into energy costs. This allows companies to completely evade the issue of volatile which are prevalent in carbon-rich jurisdictions.

With sluggish economic growth, the issue of sustainability has often been sidelined by businesses and governments alike, but as we attempt to bring more renewables into fossil fuel driven grids and reduce carbon emissions, we cannot ignore the resulting energy inflation. Electricity pricing is already on a tear: a 2012 paper8 from the EU Commission confirmed that energy prices for industrial customers increased by 27% in real terms between 2005 and 2012, due in large part to environmental protection taxes. Furthermore, energy consultancy Baringa recently investigated the risk around price increase of energy in the UK, and they forecast that large-scale user pricing will rise from £132 per megawatt-hour in 2014 to over £200 per megawatt-hour by 2025. There are scenarios where this electricity price inflation will be even more severe.

Organisations keen to ride the wave of the current technological revolution must take steps to manage the rising and unpredictable cost associated with powering technology. Fortunately there are options which allow businesses to develop IT and data strategies based on affordable, and importantly, sustainable, power.