Data Analytics and the Confirmation Bias

“Wow!”, “Those graphs are cool.”, and “I didn’t know x was our most profitable customer.”, are examples of the responses that are typical when we deliver an Analytics solution to our customers. The rich functionality and speed of change is amazing in both Oracle’s and Microsoft’s analytics solutions: technologies such as Oracle Analytics Cloud, Oracle Autonomous Data Warehouse, Oracle Machine Learning, Microsoft Power BI, and Azure Machine Learning Service, never-fail-to-deliver solutions that blow the socks off our customers.

So it is too often, and always disappointing, when months later the customer has lost some of their initial transformation enthusiasm and is only using the analytics for transactional purposes rather than driving real decision-making and innovation. If this has a pang of truth in your business, read on.

What is Confirmation Bias?

Part of the issue, I believe, is human. And is epitomised by one of my favourite quotes on data;

“Most people use statistics like a drunk man uses a lamp post; more for support than illumination”.

The reason why statistics makes us drunk is explained by the confirmation bias. The confirmation bias is the tendency to search for, interpret, favour, and recall information in a way that confirms our pre-existing beliefs. It is a type of cognitive bias and a systematic error of inductive reasoning.

We display this bias when we gather or remember information selectively, or when we interpret it in a biased way. Who me? Yes! All of us, and the effect is stronger for desired outcomes, emotionally charged issues, and for deeply entrenched-beliefs. (Think Brexit…)

A fascinating exploration of the confirmation bias is explored in Nigel Haidt’s book on morality, The Righteous Mind: Why Good People are Divided by Politics and Religion, in which he explains, humans rely on intuition not reasoning (or data!) for our opinion, “intuitions come first, and logical reasoning second.” This means that we all have quick gut feelings which point our thinking in one direction.

We then send our reasoning out to search for confirming evidence only (aka “the confirmation bias”). We almost always find the data we need to support our initial inclination.

Thus, we love a new analytics solution, as we can slice and dice the data to confirm our opinions. However, when you have already made up your mind you don’t really need analytics; yes, it’s nice to look at the data to confirm your stance, but once confirmed the pace of business makes it futile to keep proving what you already know. At this point the analytics settle as transactional reporting rather than a transformational driver.

How do we move forward?

So how do we move from confirmation to transformation? Well, we rarely take the time to confirm our opinion in the first place, excepts perhaps indirectly, i.e. sales went up or sales went down. Having the discipline to back up our decisions with data is therefore key, as this is a giant leap to becoming data driven. Or as my maths teacher would have insisted, “show your working out”.

I believe “showing your working out” can lead to a culture of curiosity and change rather than just confirmation. This practice can show us if we are making any cognitive leaps in a decision, as it exposes any areas where the data doesn’t quite hold all the water, as it were. This can lay the intuitive foundation that our opinion is just that - an opinion, rather than a fact.

Taking the time to back up our decisions with data doesn’t just help us make better initial decisions. Even more value is gained when one of our decisions eventually doesn’t turn out the way we wanted.  When we don’t get the outcome we want, having our “working out”, that is, the data behind the decision, can be crucial in making a new decision, as the data can now be looked at with hindsight to help us interpret the real story that our previous confirmation bias hid from us. This pattern can then help us use the data to do two transformational things; learn and make better decisions.

I believe human tendencies like the confirmation bias show us that successful analytics projects are as much about culture as they are about technology. I believe companies that adapt their culture or already have a culture of iterative learning from data driven decisions, who show their working out, will be an order of magnitude more successful that the rest.

For more information about Data Analytics and how DSP-Explorer can help you to use your cultural and technical resources most effectively, contact us via the DSP-Explorer website.