Business leaders show reliance on intuition as they struggle to gain confidence in data

While today’s business executives across the world make decisions based on data, they are still pitting this against their intuition, a new report finds. Published recently, Decisive action: How businesses make decisions and how they could do it better, authored by the Economist Intelligence Unit and commissioned by Applied Predictive Technologies (APT), examines the beliefs and approaches to decision-making amongst business leaders, and outlines how organisations can improve the decision-making process.

In particular, the study exposes a gap between business executives’ understanding and beliefs about the importance of data, and the extent to which they actually employ – or are able to employ - data and analytics in the decision-making process today. Despite the obvious value placed upon data, the findings suggest that many executives still lack confidence, often not fully trusting the outcomes of their data analysis and choosing to re-analyse.

Executives highly value data, but there’s a conflict with gut feel
It is clear that business executives place a high value on data for decision-making: 59% describe their approach as data-driven or empirical, whereas just 10% say they primarily use their intuition in making decisions.

However, this belies a conflict between the role of data and that of intuition. It seems that executives haven’t left ‘gut feeling’ behind as they incorporate data into the decision-making process: almost three-quarters of executives surveyed for the study say they trust their own intuition when it comes to decision-making, and 68% believe they would be trusted to make a decision that was not supported by data. One contributor to the study pointed out the danger of putting too much emphasis on intuition: “When there is a high degree of novelty, our insights and experience are limited at best or at worst deeply misguided,” says Stefan Thomke, William Barclay Harding professor of business administration at Harvard Business School.

It appears that, while executives highly value data, they are using it to confirm that their gut feeling and pre-existing beliefs are correct, rather than being guided by what the data tells them. Indeed, if the data available to inform a decision contradicted an executive’s gut feeling, 57% of respondents would re-analyse the data, and 30% would collect more data – just 10% would take the course of action suggested by the data. This seems to imply that they are seeking confirmation from the data that they are taking the right action, or using data as a ‘hygiene factor’ in the process, rather than fully trusting the data to tell them what they need to know from the outset.

Better ability to test ideas and analyse the results could improve decision-making
Despite the high value placed on data in decision-making, business executives aren’t yet able to use this to the best of its potential. When asked what they believed would improve decision-making within their organisation, the most popular factor amongst respondents was ‘better ability to analyse data’, with 53% of responses, and 22% cited ‘running more trials or tests before making a decision’. This argument is lent weight with 45% of respondents who agree that their company is growing faster than the competition also say they can predict decision outcomes by analysing tests and trials. Among those who are not growing faster than their peers, that figure is just 10%. However, 64% said they don’t believe that they are currently able to predict decision outcomes by effectively analysing the results of tests and trials.

Accountability remains a problem – and collaboration can create problems
One serious challenge, highlighted by the study, is accountability for decision-making. Remarkably, 19% of those surveyed said that decision-makers at their organization are not held accountable at all. This may be compounded by a lack of transparency around the decision-making process: nearly two-thirds of executives (64%) said that information about who made certain decisions, and why, is limited to sufficiently senior employees.

Collaboration in decision-making divided opinion: exactly the same proportion of respondents, 38%, agreed or disagreed that ‘the more people involved in making a decision, the better it will be’, and a further 24% remained neutral. It seems there is a danger that including more stakeholders into the decision introduces new problems to the process and makes it fraught with politics: worryingly, almost half of those surveyed (43%) believe that ‘company politics trump evidence when it comes to management decisions’.

“Data and experimentation is the solution to these problems,” said Jim Manzi, chairman of APT. “Taking the personal angle out of data, by running tests and basing decisions on clear, empirical evidence, ensures transparency as to why a particular course of action was taken. The study shows that executives clearly understand the value of data – but some of the problems that have traditionally plagued decision-making persist.”

Manzi added, “It is clear that a significant proportion of executives are taking action and evolving their approach to move towards the next frontier of analytics: testing and experimentation. 17% of those surveyed for the study have already adopted this kind of empirical approach to decision-making, and I expect to see this continue to grow as organisations realise intuition simply isn’t enough in today’s fast-changing and complex business environment.”

The Economist Intelligence Unit surveyed 174 senior managers and executives from organisations around the world to explore how they make decisions, as well as identify their beliefs about the best way to do so. Over half of the executives surveyed were of C-level seniority, and 49% represent organisations with over $500 million in annual revenue. In-depth interviews were also conducted with seven senior executives and industry experts, including Gerry Grimstone, chairman, Standard Life; and Robin Tye, chief operating officer, EY.