Data versus Intuition Featured Pattern: P0762 April 2015

Author: Rob Edmonds

Businesses are increasingly turning to data to drive their decision making.

Abstracts in this Pattern:

In 2012, a team of researchers led by the Massachusetts Institute of Technology's (Cambridge, Massachusetts) Andrew McAfee and Erik Brynjolfsson conducted a study and found that the more companies described themselves as data driven, the better their financial and operational results tended to be. According to Drs. McAfee and Brynjolfsson, "Throughout the business world today, people rely too much on experience and intuition and not enough on data" (see SC-2012-12-05-089).

In recent years, the popularity of data-driven decision making has grown. As a result, demand has increased in Silicon Valley, California, for microeconomists who use detailed data analysis to solve problems. For example, Smarter Travel Media's (TripAdvisor; Newton, Massachusetts) Bryan Balin helped create an algorithm that uses data such as click speed and website-visit history to calculate the potential value of website visitors. And hiQ Labs (San Francisco, California) chief data scientist Genevieve Graves employs algorithms that analyze data to predict which workers might leave an employer and why they might do so.

The investment-finance industry's appetite for data continues to grow. Various hedge funds use data from Orbital Insight (Mountain View, California), which developed software that analyzes satellite images to make interferences about the condition of the construction industry. Investors also use data from Dataminr (New York, New York), which analyzes tweets about investment-related information on Twitter's (San Francisco, California) social network. And in agriculture, Monsanto (Creve Coeur, Missouri) is increasingly focused on helping its clients make data-driven decisions. Monsanto subsidiary The Climate Corporation (San Francisco, California) claims that its software has all US agricultural land mapped with soil and climate data to a 10-meter-by-10-meter resolution. The software can offer farmers detailed recommendations such as how much water and fertilizer they should use.

Although data-driven decision making has its limits (for example, for radical innovation), more organizational decisions likely could benefit from the systematic use of data. For example, US home builders have been steadily increasing the average size of family homes, but a few years ago, a study by University of California, Los Angeles (Los Angeles, California), archeologists and anthropologists found that much of the floor area in many houses remains unused because residents prefer to spend most of their time in certain rooms. These findings suggest that the use of data could enable the design of homes that have much less wasted space.