Appearing in In The Black magazine, April 2016
This one tool will drive the next era in business
By Derek Parker
“There is a revolution going on in business,” says Ram Charan, former Harvard professor and author of seminal business books such as The Attacker’s Advantage, Execution and Boards That Deliver. “Never before has so much mental power been computerised and made available so broadly. Algorithms and their related sophisticated software, coupled with new tools capable of collecting and storing huge amounts of raw data, can predict patterns and changes in everything from consumer behaviour to the maintenance requirements of machinery.”
Charan believes that the use of algorithms and maths has become a critical source of competitive advantage. “Google, Facebook and Amazon were created as mathematical corporations, communicating with their customers through analytical systems,” he says. “The advantage for such businesses is that they can deal directly with buyers, without intermediaries, and can personalise the customer’s experience.”
For the corporation, this means a much flatter organisation, with tiers of middle management jobs simply disappearing. It also requires a rethinking of what performance metrics are appropriate, what qualifications are desirable for recruits, and how expertise can be accessed. For senior managers and directors, it raises the question of how they can make decisions based on advice they might not fully understand.
Big Data gets bigger
“It has taken a while for the maths area to be picked up in Australian business,” says Jodie Sangster, CEO of the Association for Data-Driven Marketing and Advertising and the Institute of Analytics Professionals of Australia (IAPA). “A key turning point was when the term Big Data began to appear, about three years ago – that put a useful tag on the concept. There was a realisation that there was a huge amount of data on consumer behaviour that could be collected, analysed and utilised. But there is still a very long way to go, and a lot of companies – and C-level people as well – haven’t even begun to invest in it.”
Sangster nominates the banking and finance sector as the most prominent user of analytics for marketing, followed by the travel industry. But the retail sector and the consumer goods sector are lagging. Franchise food chains have also been slow to enter the field.
Already, people with expertise in maths and business are in great demand, with salaries around the $200,000 mark being offered, according to the 2015 survey of the IAPA. In many cases, companies will provide additional training in analytics to employees who show an aptitude for the field.
“It’s about finding that space between the theory of maths and the practicality of marketing,” Sangster says. “Someone who understands both is worth their weight in gold.”
She is pleased to see the development of specialist Masters degrees, such as in Business Analytics and Data Science, but notes that as they are very new it is too early to see how they will function in the marketplace. These degrees, even though expensive, are usually over-subscribed. Sangster notes that the training courses offered by her organisations are also extremely popular.
“An area we see as crucial,” she says, “is training for C-level officers and directors. We are currently exploring options for this, including partnering with business associations. Greater awareness of the value of maths by senior people could really take things to the next level.”
Rachel Edwardes, Head of Marketing at Forethought, a consulting firm specialising in business analytics, acknowledges that Australian business is, in general, a few years behind US competitors.
“But having said that, there are some critical areas where Australia is leading the world,” she notes. “Forethought holds US patents on algorithms for modelling rational and emotional consumer behaviour, for example. Another advantage is that Australian companies, when presented with solid data, are willing to move much faster to implement new strategies than their US counterparts.”
A particularly significant project done by Forethought involved department store chain K-Mart, which not too long ago seemed to be locked into a pattern of decline. Forethought used detailed survey data to model the drivers of acquisition and retention by customers, with the information then being used to design a marketing campaign (for more information on the K-Mart turnaround and the use of analytics, see the award-winning article in the November 2015 issue of Marketing Science magazine).
“Using analytics to model consumer behaviour for marketing is the difference between shooting at a target and shooting in the dark,” says Edwardes. “K-Mart saw clear benefits within six months, and the upward trajectory has continued. We have continued to work with them to model progress and changes.”
Bottom line impact
Marketing might be the most obvious way for companies to use sophisticated data analysis but there are plenty of other avenues emerging. Consulting firms specialising in maths have begun to appear, offering solutions across a range of activities. The key is to link the expertise of maths experts with real-world business requirements. One organisation to do this is the Centre for Industrial and Applied Mathematics, part of the University of South Australia.
“Mathematical models can show how changes to one variable affect the whole,” says Professor John Boland, Director of the Centre. “We have advised electricity generators, for example, on the impact of a drop-out, how to integrate non-traditional energy sources such as solar, and how to allocate load according to changes in consumer demands.”
The Centre has also helped the mining sector by showing how to optimise truck traffic in and out of mine sites. It has advised the transport industry on reducing energy costs by adjusting the speed of long-haul freight trains.
“Making these improvements can generate savings of millions, even hundreds of millions,” says Professor Boland. “A problem with high-level maths is that it can be difficult to explain to people without a maths background. Clear cost savings make the senior managers of a company, or the board, take notice.”
But many industry sectors have not yet fully grasped the potential of algorithmic modelling. The construction industry has begun to use maths in a limited way, such as environmental add-ons to architectural designs, but not for fundamental issues such as the best way to site a building in order to minimise energy use and to maximise solar gain for electricity production. Professor Boland works on this type of problem with practitioners from several disciplines in the Cooperative Research Centre for Low Carbon Living.
There are other areas which could benefit from maths input but no connection has yet been made.
“Hospitals, for instance, are still doing staff rostering old school, in terms of shift allocations and customer demand, even though there are gains to be made through optimisation,” Professor Boland says. “The great advantage of modelling is that you can try out a wide range of options and mixes to see what works and what doesn’t. But some organisations either don’t understand what can be done, or are too invested in their existing arrangements. In those cases, the maths industry should do more to communicate the exciting opportunities it can offer.”
Professor Ujwal Kayande, director of the Centre for Business Analytics and the Master of Business Analytics degree at the Melbourne Business School, notes that communication is a key element of the course, with one day of each week being devoted to effectiveness development.
“It’s particularly important when you realise that some maths-based conclusions are likely to go against the gut instinct of senior managers,” he says. “We have tried to teach the students how to conduct difficult conversations, to build a common language. That is something that came out of our early discussions with business leaders.”
On the other side of the equation, Professor Kayande explains that the MBS has already conducted executive education programs customised to particular organisations to show senior managers the potential of business analytics.
“These short courses have been very well accepted, and we our using what we have learned from them to design an open course, which we hope to start up in the near future,” he says. “There would be good export opportunities for that sort of course as well. The constraint is not demand but the supply of people who could teach it. This is still a very young field so there isn’t a stock of experienced people.
“Aside from senior managers, there is huge potential in re-training across organisations. Even in specialised fields like HR there are gains to be had from for maths input. The business community has only just begun to understand the benefits that advanced maths can provide.”
Ram Charan underlines the point. “Regardless of how young or old your company is, you must make the use of algorithms part of your vocabulary, as much as profit margins and the supply chain are today,” he says. “A company that is not already doing it, or is unable to tap into it, is already a legacy company.”
Predicting the vote
Barack Obama is a politician who has grasped the value of advanced maths as a campaigning tool. In the 2012 election, his chief scientist Rayid Ghani used machine learning to predict the answers to four questions for each individual swing voter, using all the data about them they could find. The questions were: How likely were they to support Obama? To show up at the polls? To respond to reminders? And to change their mind about the election based on a conversation about a specific issue? Then, based on the results of the modelling, they regularly ran a program called “the Optimizer” to choose which voters to target. In contrast, Mitt Romney’s campaign used standard polling and targeted broad demographic categories like “suburban middle-aged woman”. On election day, Obama carried all but one of the swing states and won the election.
Opening the black box
“If algorithms are a black box to you, you have no control over where they will take you. Think of a car as an analogy: only engineers and mechanics need to understand exactly how the engine works, but you need to know how to use the steering wheel and pedals,” says Pedro Domingos, University of Washington professor of computer science and one of the key figures at the intersection of maths research and the business world. “Courses for executives are important, but it is also the responsibility of experts to explain what we do in terms that non-specialists can understand.”
Domingos’ most recent contribution to building bridges is the book The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World (Allen Lane, $50). The book is designed as a primer on machine learning – that is, computers learning by themselves by generalising from data instead of having to be programmed by people.
“Google uses machine learning to decide which Web pages to show you, Amazon and Netflix to recommend books and movies, Twitter and Facebook to select posts for your feed,” Professor Domingos told In The Black. “Siri uses learning algorithms to understand what you say and predict what you want to do. Retailers use it to decide which goods to stock and how to lay out their stores. Online dating sites use machine learning. Machine learning is involved in pretty much everything we do these days.”
Machine learning makes it possible to understand much more complex phenomena than before. Biologists are using machine learning to build models of the cell based on data from DNA sequencers, and neuroscientists are using it to build detailed maps of the brain, literally neuron by neuron. Social scientists are using it to study large social networks, with millions or billions of people.
Within the machine learning field there are competing schools of thought. Symbolists take ideas from philosophy, psychology, and logic. Connectionists are inspired by neuroscience and physics. Evolutionaries draw on genetics and evolutionary biology. Bayesians have their roots in statistics. Analogisers are influenced by psychology and mathematical optimisation. A Master Algorithm would combine the strengths of all of them, providing – in theory – the capability of discovering all knowledge from data.
“This is an incredibly powerful idea,” says Professor Domingos. “When will we find it? It’s hard to predict, because scientific progress is not linear. It could happen tomorrow, or it could take many decades.”
Many of Professor Domingos’ award-winning research ideas have been implemented in software, and are freely available. They include:
- Alchemy: Algorithms for statistical relational AI alchemy.cs.washington.edu
- VFML: A toolkit for mining massive data sources http://www.cs.washington.edu/dm/vfml/
- NBE: A Bayesian learner with very fast inference http://www.cs.washington.edu/ai/nbe
- BVD: A bias-variance decomposition for zero-one loss http://www.cs.washington.edu/homes/pedrod/bvd.c
- RISE: A unified rule- and instance-based learner http://www.cs.washington.edu/homes/pedrod/rise.c