The past year’s devastating drought might have been mitigated if information about previous droughts had been used to predict it and to prepare, says Bhekisipho Twala, director of the Institute for Intelligent Systems at the University of Johannesburg.
Data scientists could also help Eskom predict the where and when of electricity use, easing its task of keeping the lights on when power supply is constrained.
“The whole point of data mining or machine learning is finding hidden information that can make business (or other) decisions easier,” says Obakeng Moepya, one of the three founders of Johannesburg-based Isazi Consulting, which uses advanced mathematics and statistics to solve difficult problems.
Machine learning — also known as artificial intelligence — is a field of computer science that gives computers the ability to learn without being explicitly programmed. Data mining is the analysis step in machine learning, with the goal of extracting patterns and knowledge from large amounts of data.
“There’s endless potential to do things we couldn’t before,” says Daniel Schwartzkopff, co-founder of Data Prophet. “It’s just what (PayPal co-founder) Peter Thiel was saying: a lot of what we do is repeat things we have done before, we need to create something new that has the potential to do good.”
Cape Town-based Data Prophet uses computers and algorithms to crunch data sets that were previously too large and unwieldy for the identification of patterns to make sense of problems and to find solutions for them.
There’s loads of information available, but the newness to SA of the concept of machine learning is probably the biggest challenge to companies like these, Twala says. And companies are often wary of sharing their data.
“Maybe things will change as people realise the importance of doing research,” he says.
Isazi Consulting devised a way for teachers to collect thousands of facts about their pupils, and it offers teachers advice on how best to teach concepts to individual pupils. It was offered to the Department of Basic Education, which declined it.
The wariness about sharing data is one of the reasons Data Prophet, which started up two years ago, is looking offshore to expand its business.
“Our experience in SA is that it takes six months and an untold number of meetings (to finalise a deal),” Schwartzkopff says. “It can take three months to sign a non-disclosure agreement.”
To work with “companies as nimble as us”, Data Prophet is setting up shop in the US technology mecca: San Francisco. In Silicon Valley, a data scientist can command a salary of $100,000 a year. “We can work at a quarter of the cost,” Schwartzkopff says.
Isazi Consulting is growing its brand in SA and Africa.
“Then we will try to go global,” Moepya says. “We live here; we know the conditions, and the problems.”
Isazi Consulting launched in 2012 when Moepya, Ashley Anthony, and Dario Fanucchi, confronted with a newspaper article that claimed SA’s school pupils were bottom of the class in an international mathematics ranking, wondered how this could be true when they all had advanced mathematics degrees and used their knowledge daily to solve social and economic problems.
Data Prophet has its origins in the return to SA from the US of Richard Craib, who studied machine learning at Stanford University, abstract algebra and finance at Harvard, and game theory at the University of California, Berkeley.
Craib and friends Schwartzkopff, a chemical engineer, and Frans Cronje, an actuary, set up Data Prophet because they saw a gap in the domestic market for using machine learning to solve problems.
They started with the call centre industry, using machine-learning technology to better match agent and product to consumers, improving profit up to 34%.
The company has since developed a mobile application for the vehicle insurance industry that can produce an instant quote using photographs of the vehicle and the driver’s licence.
There is nowhere in SA to study machine learning, Twala says.
“You have to go abroad to study, and where are you supposed to get funding for that? This is a very interesting area. You can do a lot with it.”
It can provide solutions to myriad problems: more accurate tumour classification; help online casinos retain clients; assist drones searching for poachers in the bush; help police fight crime by recommending sites for roadblocks.
Twala has proposed a master’s programme in “cognitive science” to the University of Johannesburg and it is being reviewed. The institute he directs is a first for Africa. Data Prophet is developing a one-year postgraduate course in data analytics for the University of the Western Cape.
Criticism of this type of modelling includes that it uses historical data to pinpoint patterns and predict future behaviour, but Twala points out that data can — and should — be updated continually so that the system keeps learning. It is also important, he says, continually to validate the data. Other challenges to this new industry are costs (better machine-learning software is developed predominantly in the US, making it expensive), red tape, and a lack of support.
Also, while a lot of data is collected in SA, it is often “messy”, Twala says. “There are errors, there is duplication, and there is missing information. You spend 70% of the time cleaning the data, and 30% on modelling.”