The future is big data

“Plastics” was the career advice to uni students in the 1967 movie The Graduate. Today the same advice to a smart young entrepreneur would be “big data”.

Big data is the current buzzword for the IT industry, we’re seeing start-ups with cool tools popping up and whole new job descriptions to manage it, while big and small businesses ponder how to use another technology in their operations.

At the end of the month, the third of the City of Sydney’s 2012 Let’s Talk Business series will see SmartCompany’s James Thomson among others discussing how data drives business.

How we use data in our business is something we’ve had to come to grips with for ages, but many of us haven’t really started to find those nuggets of value in our databases.

We’ve actually been in the era of big data for decades since computers were introduced in the workplace. One thing that PCs do very well is gather and store information.

Today computerised point-of-sales systems, database software, loyalty programs and web-tracking tools mean we have a massive amount of data about our clients at our fingertips.

As computers get more powerful and cloud-based services start making detailed data analysis more available, we’re going to see even more data pouring into our businesses.

Social media services add to the data deluge as they gather, giving even more intelligence about our markets, individual customers and the performance of our businesses.

The problem is that many of us are already overwhelmed by what we have. The thought of even more data we can’t use causes many managers and business owners to hide under their desks and weep.

An article in the MIT’s Technology Review about Peter Fader, co-director of the Wharton Customer Analytics Initiative at the University of Pennsylvania looked at this problem.

Professor’s Fader’s view is that most businesses have enough data – the problem is managing what we have, along with the risk of trying to extrapolate too much from historical information.

To deal with this overload we’re seeing companies like Kaggle starting-up to help us mine this data and get useful information about our businesses and customers.

What these data-mining companies are promising is the ability to see the patterns in what appears to be just a mass of confusing data.

Already we’re seeing businesses that can connect the dots get a head start on their slower competitors who don’t appreciate the value locked in their databases and CRMs.

Making sense of the data we’re accumulating is the real challenge. If we’re not paying attention to what we already have then there’s little point in gathering more.

Tickets for How Your Customer Data Can Drive New Business at the Sydney Town Hall on May 29 are still available.

Paul Wallbank is one of Australia’s leading experts on how industries and societies are changing in this connected, globalised era. When he isn’t explaining technology issues, he helps businesses and community organisations find opportunities in the new economy.