Big data

Data could be the new river of gold for organisations that harness its power.

Consider some claims about “big data”, which refers to enormous data / information sets that are almost impossible to process with conventional technologies:

  • More than 90 per cent of global data was created in the past two years, says IBM.

  • The volume of business data worldwide is expected to double every 1.2 years, according to eBay.

  • Data production will be 44 times greater in 2020 than in 2009, says Wikibon.

  • Annual spending on big data services will reach US$48.6 billion in 2019, predicts International Data Corporation (IDC)

  • By 2020, half of all business analytic software will incorporate predictive analytics, says IDC.

  • The amount of high-value data worth analysing will double by 2020 and 60 per cent of information delivered to decision makers will be actionable, predicts IDC.

  • Big data will be the next trillion-dollar industry, says Dell.

These are just some of the many big data predictions. Technology trends, of course, are prone to hyperbole and nobody knows for sure how big data will play out. But it certainly has the potential to be transformative and the epicentre of so many other emerging technology trends.

In time, big data might be considered the fuel for the fourth industrial revolution.

Big data is inextricably linked to the Internet of Things (IoT), which refers to objects that have embedded technology that allows them to send and receive data via the internet and connect with other devices. The Governance Leadership Centre examined the IoT megatrend in June 2016.

Cisco Systems estimates 50 billion hardware devices on the planet by 2020 – a trend that will drive almost unimaginable growth in the creation of big data as more devices talk to each other via the internet and data is captured and analysed.

Big data is also closely linked to cloud computing. Organisations are increasingly storing and managing vast data banks offsite through cloud-computing – another booming IT trend. Cloud computing is a general internet-based technology that allows companies to store information in services and access it on-demand as a software service.

As companies capture more data through the IoT, and manage it via cloud-computing, powerful software algorithms are being developed to analyse this information. These algorithms will transform decision-making in many industries, quicken the development of robots and machine learning, and boost demand for workers with data-analytic skills.

These combined trends could redefine labour markets as technology does higher-level tasks and replaces more white-collar workers across industry.

Organisations investing in big data

More companies are investing heavily in big data and analytics. After initially experiencing modest returns on investments, firms are now deriving substantial benefits for big-data investment, according to a recent McKinsey survey.

The firm’s survey of 714 companies worldwide found the profitability and productivity-enhancing effects of big-data were similar to those experienced in the earlier period of intense information technology investment. Big data, it seems, is following the pattern of other major technology trends, which disappointed at first, before stronger gains.

Tech researcher Gartner says investment in big data is starting to slow as the technology becomes the ‘new normal’ in business. Nevertheless, about three quarters of companies are investing in or planning to invest in big data in the next two years, according to Gartner’s June 2015 survey of 437 clients.

IBM says companies are extracting more value from big data in five ways. First, they are using big data to visualise information, share it across the organisation, and improve decision making. Second, big data is providing organisations with an enhanced 360-degree view of the customer: why they buy, what they buy and when, and what makes them tick.

Third, big data is helping organisations lower risk, detect fraud and monitor cyber-security threats in real-time. That has been an important focus of boards that are ensuring their organisation uses big data to help mitigate risks.

Operations analysis is the fourth focus for big data. Companies are analysing machine and other operational data to improve transparency and productivity. Big data’s fifth high-value use is helping firms modernise warehouses and create other efficiencies.

Big data and boards

Governing through this new era of big data is a huge board challenge. The concept of ‘information governance’ – how information is managed to support the organisation’s obligations and meet its outcomes – is deservedly gaining more traction in board circles.

But big data extends well beyond information. It will affect everything from the organisation’s strategy, skills and capabilities to its culture and values. Boards must ensure that structures and frameworks are in place for the organisation to take advantage of the opportunities that big data presents, and manage through its many threats.

Here are eight considerations for directors with big data (the role of big data in organisation strategy and business models is a separate topic in its own right).

  1. Does the board have sufficient skill to understand big data issues? Australian boards continue to upgrade their technology skills by recruiting technology executives, visiting overseas technology clusters and forming advisory panels or technology committees. They need structures to ensure that the right information on the organisation’s big data capabilities, and the capabilities of others in its industry, are presented and considered in

  2. Can the board use big data in its decision making? Most commentary has focused on executive teams using big data to make decisions. Less considered is whether the board can use big-data visualisation technologies to make key governance decisions, for example, whether to approve management’s proposal to invest in a new project.

  3. Does the executive team have sufficient big data skills? An essential board responsibility is to recruit, incentivise and monitor the right CEO. Increasingly, CEOs and their direct reports will need strong data-analytic skills, or an ability to manage those who have them. Boards need to know that the organisation’s executives are capable of adapting to big data and working with data scientists who will help shape decisions.

  4. Will the organisation’s skill and capabilities accommodate big data? Predictions abound of a coming shortage of data scientists and managers with deep data analytic capabilities. Demand for data scientists will be 60 per cent greater than supply, estimates McKinsey. Accenture also found that 90 per cent of its clients intended to hire data scientists, but 40 per cent said a lack of talent in this area was a key concern. Boards will need to spend more time with the human resources executives to understand how the organisation is developing its data analytics capabilities and its ability to recruit more people in this field.

  5. Can the organisation’s culture facilitate big data? Boards must understand whether their organisation’s culture is sufficiently flexible and adaptive to respond to big data trends. A culture that encourages lifelong learning and embraces rapid change will be even more valuable as big data redefines business models and reshapes global industry.

  6. What is the framework to assess return on big data investments? Many boards will be asked to approve larger investments in big data projects in coming years. They will need to understand and measure the return on this investment – arguably a more challenging task compared to traditional investments in physical assets.

  7. How is the organisation managing security risks from big data? Big data potentially creates as many threats for organisations as it does opportunities. Organisations that capture more data on customers, store it offsite, and allow more employees to access it, have a delicate task to safeguard this information and ensure they meet their ethical, social, regulatory, legal, environmental and governance obligations.

  8. Does big data meet the organisation’s values test? More organisations could find themselves conflicted as big data provides unprecedented customer insights and technological advances. Boards must know that their organisation has appropriate policies and reporting systems that provide checks and balances on how big data is being used, so that the right decisions are being made.