Monthly Archives: September 2012

Big Data for business: be careful what you ask for

This post on the temptation of data raised an interesting aspect of Big Data: that we run the risk of being overwhelmed by data and that organisations and investors are looking too hard for the one piece of data that will be the key to success. Drowning in data is a real risk to an enterprise and something that great leaders are aware of ( lesson 3 in this awesome leadersip slidedeck from Colin Powell – “Experts often posses more data than judgement”) but being swamped by the data is not the only problem.

Another problem with casting the net of big data wide in this way is that eventually you will end up finding some sort of pattern, any sort of pattern if you keep looking hard enough (there’s an XKCD for this). The human brain is a fantastic pattern recognition system and is easily fooled (see Pareidolia) and it’s far too easy to make mistakes like confusing correlation with causation.
Continue reading