How valuable is big data? What endless possibilities does big data have? Steve Lohr does an exceptional job at diving deep into big data and its applications in his book, Data-ism. It illustrates how it can help businesses big and small improve operations while also exploring its limitations. We get several real-world applications, and more importantly, where humans fall into the whole equation.
Excerpt on “McKesson Simulation”
Lohr begins his examples with the operations of McKesson Corporation, which distributes one third of all pharmaceuticals in the U.S. to 26,000 customer locations, including roughly 240 million pills a day. The company accumulates massive amounts of data through its shipment tracking systems and sensors. This data provided the necessary building blocks for IBM to build its decision-making simulator. How does this decision simulator work? By using data to predict profit and losses for every product, supplier and customer. In other words, the data describes what is happening today. But it can also help — and perhaps the key word there is “help” — figure out the future.
For example, one of these simulations was to predict the impact of centralizing the distribution of expensive drugs. Although centralizing involved higher travel costs initially, the simulation predicted huge savings. When the project was pilot tested, inventory costs were cut in half, resulting in net savings. “The software gave McKesson the clarity and the confidence to go ahead,” Lohr writes. “McKesson’s success illustrates where the big-data approach shines today. It shows data really being used to guide decisions and to make better decisions, ones that trump best guesses and gut feel, experience and intuition.”
Applications and Ideas
- What data do you have about clients, or elsewhere, that could potentially help you simulate future costs or profits?
Excerpt on “The Girl and The Butterfly”
A fundamental challenge for data-driven artificial intelligence (AI) is described through the example offered by Tom Mitchell, chairman of the machine-learning department at Carnegie Mellon.
The example involves two sentences:
The girl caught the butterfly with the spots.
The girl caught the butterfly with the nets.
Only one word in the two sentences is different. Humans reading those two sentences understand that girls don’t have spots and butterflies don’t have nets. Thus, the spots in the first sentence refer to the butterfly, while the nets in the second sentence are associated with the girl. The advantage of humans in this example can be summarized in one word: context.
Therefore, the greatest challenge for AI is to be able to put the data it accumulates in contextWhile big data can recompile all of the information, computers lack the ability to properly context the information and having information without context does not add any value. Context can be achieved in two ways, Lohr explains: correlation, which connects data to action or behavior in the real world, and association, the more sophisticated ability of computers to draw context from putting words together.
The startup ZestFinance uses correlation to determine the risk of payday borrowers. Its sophisticated algorithms allow ZestFinance to reduce the risk of default by its borrowers by 40 percent. Through their algorithm they were able to discover that by looking at how people enter their name in websites they could determine how likely they are to repay a loan. Zest might not know why the way they write their name has something to do with their likelihood of repaying a loan, nor does it care since it is able to use the data regardless of the reason.
Applications and Ideas
- Does “why” ZestFinance’s correlation predict the way it does matter? It’s a heated debate in these circles of big data experts
- Casserly Consulting uses correlation and association in cybersecurity every day. But tools only go so far and this is why having a team to check alerts is crucial — to disentangle the noise from real threats.
Data-ism is one of the most balanced examinations of the potential of big data. The author balanced the novel either by pointing out the limitations of big data or giving voice to the skeptics and critics of a data driven world. Will machines take over? Will machines ever be able to understand context like humans do? Only time will tell if we will one day see computers taking over and thinking like human beings. However, the value of big data is undeniable for businesses, understanding massive amounts of data points and finding correlations is no new way of conducting business. What is new is the technology that processes these data points, how they are processed, and the correlations that can be found between otherwise seemingly unrelated variables.
