Big Data: Big Results from Marginal Gains August 2014
Want more free featured content?
Subscribe to Insights in Brief
A recent opinion piece in the Economist argues that organizations should aim for lots of small gains from big-data applications. For example, QuantumBlack, a data-analysis firm, helped an engineering multinational improve team performance by analyzing dozens of variables about team composition. QuantumBlack claims that the multinational was able to create a 22% productivity improvement by combining a series of small improvements that were mostly in the 0.5%-to-1%-improvement range.
The Economist rightly points out that internet companies lead the way in marginal-gains approaches, saying that "every pixel" on Amazon's homepage has had to justify its existence through repeated testing. Facebook's experiment to change the sentiment of user pages was very controversial but in fact produced only a very small effect in posting behavior (a difference of less than 0.1%, according to Facebook researchers). But, as the Facebook researchers explain, "given the massive scale of social networks such as Facebook, even small effects can have large aggregated consequences" (see "Experimental evidence of massive-scale emotional contagion through social networks" in the June 2014 PNAS).
In recent years, professional sports teams have become experts in achieving marginal gains through data analysis. Formula 1 team McLaren is even reselling its data-analysis expertise—for example, to help IO Data Centers forecast demand for its computing services.
Some businesses have understood the power of data analysis and marginal gains for decades. Delivery companies have optimized their operations through many small improvements, and retailers finely tune store layouts and product lines according to data analysis. Supply-chain managers and manufacturing staff are also familiar with optimization, continuous improvement, and marginal gains. As examples from professional sports illustrate, sensors and the Internet of Things are enabling an increasing number of physical-world applications to benefit from marginal-gains approaches.
Despite these examples of marginal gains in action, some CEOs, senior managers, and politicians remain big-picture people, more likely to be enthused by high-profile, high-impact initiatives than by the day-to-day drudgery of marginal gains. People with responsibility for big-data strategies in their organizations need to ensure that senior staff are not waiting for the "needle" to emerge from big-data projects and instead are preparing their organizations for the multitude of continuous small changes that will provide big data's real value.
In the long term, big-data analytics could run the day-to-day operations of many organizations. Analyzing and optimizing countless variables could become the norm—and software could far outstrip the capabilities of people in such tasks. The addition of artificial intelligence, robotics, and the Internet of Things may enable analytics software to take action as well as provide recommendations, eventually leading to the automation of some day-to-day management.