Mining business data: Get ready to drill and excavate

The IT industry loves buzzwords and the phrase coming into fashion is “big data”. Forget “social media” or “cloud computing”, much of what you’ll be reading about in columns like this over the next few years will be about mining the information piling into our businesses.

Big data’s power is illustrated in yesterday’s report that Mac users will pay more for hotels than those on Windows systems. So travel site Orbitz now plans to headline more expensive options to visitors using Apple computers.

While social media and cloud computing are falling out of favour, we shouldn’t discount those old-fashioned terms as they are two of the main drivers of big data. Cloud computing makes it possible to crunch data cheaply, while social media is generating even more data for people to play with.

Sydney business Roamz is a good illustration of how social media, cloud computing and big data come together. Born out of founder Jonathan Barouch’s desire to find local activities for his young child, Roamz pulls together data from Facebook, Twitter and Foursquare to build a picture of what’s interesting in your neighbourhood.

It’s no coincidence direct marketing giant Salmat has invested in Roamz, as the data being gathered allows companies to paint a comprehensive picture of what their customers like.

Another business making sense of big data is Kaggle, set up by former Reserve Bank economist Anthony Goldbloom. Kaggle is a crowdsourcing service which runs data analysis competitions on business problems through to things like HIV research, chess ratings and dark matter exploration.

Like most things in the computing industry, these services were only available to those who could afford supercomputers a few years ago, today they’re available to anyone with a credit card and internet connection.

The era of big data might help us overcome Pareto’s Principle – otherwise known as the 80/20 rule – that 80% of your profits come from 20% of your customers. We can also be sure that 80% of our problems come from a different 20% of clients.

Having the ability to crunch numbers quickly means we can identify the good payers and problem customers before we do work for them. It might also mean we beat another business truism by finally being able to identify the 50% of our advertising budget that is being wasted.

Even if you don’t think your business is ready to dive into the world of big data, it’s worthwhile at least having a look at your website analytics to see what your customers are looking at.

Who knows? Like Orbitz, you might identify those cashed up Mac users who love spending money.

Paul Wallbank is one of Australia’s leading experts on how industries and societies are changing in this connected, globalised era. When he isn’t explaining technology issues, he helps businesses and community organisations find opportunities in the new economy.