In 1943, the Royal Air Force looked for quick fixes to improve the probability that their bombers could complete their missions and successfully return home. One option was to add armor-plating, but what was the optimal placement which would ensure maximum protection without compromising the aircraft’s performance?
The accepted thinking, based on the data collected from returning bombers, was that the wings, nose, and fuselage should be armored up. They, after all, had the most bullet holes.
It took a gifted mathematician, Abraham Wald, to highlight that the best solution would, in reality, be to armor the three areas that never seemed to get hit: the cockpit, the engines, and the tail.
He had correctly hypothesized that the bullet patterns would be evenly distributed and, therefore, the planes that did not return had been taken down due to weaknesses in these three key areas.
This is a wonderful story as it highlights how, if you want to be effective using data for strategic decision-making, you have to think deeply about both the data you have and the data you don’t.
Companies, surprisingly, despite their significant investment in technology, still seem to succumb to their own version of this data selection bias.
For while they do not have to worry about the negative consequences of suboptimal armoring, they do need to optimize the placement of resources, to ensure they provide maximum customer value without compromising their ability to scale profitably.
The data they consistently seem to ignore when making these types of resource decisions is that which exposes the real value they bring to their clients.
I’ll use the case of mid-size Asian bank, DBS Bank, as an example. For years they had managed their business using key performance indicators (KPIs). These KPIs helped them to focus on efficiently distributing their products to millions of retail customers.
Call centers, for example, would typically focus on reducing call abandonment, queuing times, handling times, and improving first call resolutions.
And I suspect that over time DBS were able to improve productivity and offer a more streamlined service to their clients.
But although these metrics no doubt helped the company reduce its costs, it reportedly added extra ones for their customers.
And the cost that seemingly annoyed their customers the most was the extra hours they had to waste, dealing with the bank.
So it was Paul Cobban and his team who realized they were managing using the data they had, not the data they needed. So they pivoted and reset the strategy to focus on improving just one metric: reducing the time customers spend dealing with them. Now, their teams had to think more deeply about how to ensure their customers didn’t have to call centers, write emails, travel to branches, or wait in long queues.
Cobban estimated that this process was able to “take out at least 250 million wasted customer hours per year”, as per a 2016 article in Forbes.
This helped DBS transform from being recognized as one of the worst banks in Singapore to becoming one of the best and most loved.
It must have taken courage for the team to stop worrying about internal metrics and start trying to develop customer-led ones.
Especially when business schools and consultants are still extolling the virtues of balanced scorecards based on the assumption you can’t manage what you can’t measure.
The truth is, these management systems tend to reinforce selection bias. This occurs as executives often shape strategies around what is straightforward to manage and quantify, rather than exploring the often ethereal data sets that help identify where the real client value lies.
Aligning compensation with easily quantifiable metrics such as revenue, profit margins, or operational efficiency is much simpler than assessing the value a client derives because you saved him time, or amplified his brand, or increased his profits, or made him happier, or reduced his stress levels.
Still, working out the bullet patterns on planes that are hundreds of meters below the sea wasn’t easy either.
Predicting how best to serve your customers, without understanding the actual value you bring them may have worked in a world where change is slow and the future is predictable.
That world, I’m afraid, has gone.