Big Data – the New New not-so-New Thing

The cry sounds very familiar: “Big Data. We’re doing Big Data”. Substitute ‘the Internet” for “Big Data” and it could be 1997.

In 1997, it was very common to meet people who were “doing the Internet”. It was by far the most exciting game in town. There was often not so much understanding of how “doing the Internet” was going to generate value. There were new business models and new future titans being built. It was also imperative for all existing big corporates to be doing it.  They didn’t have the in-house skills, so a raft of product and service start-ups mushroomed to fill the as-yet-not-so-defined gap.

There is a similar rush now with Big Data. And as in the internet rush, while there are some people that are building and investing in companies that generate and look set to monetise Big Data, there are also many that are looking to take advantage of and fill the gap that the lack of maturity in this discipline has opened up.  In many cases, it’s not obvious what kind of business is actually being created.

In a way, this is to be expected. As in the early days of any emergent field, there is a surge of new entrants. This is particularly helped by the fact that much of the underlying technology that is being used to manipulate Big Data is open source. The cost of participating is low, while the perceived need is great.

When looking at companies that want to provide services to enterprises, one major decision is between whether to offer a platform or an industry-specific solution. At RIG, we have come across quite a lot of companies that are in the middle – they have a basic platform that they are trying to use to solve problems for a range of customers. Essentially they are offering tool-driven consultancy, taking advantage of the gap – as for the internet – where core mass data handling skills are not yet embedded in the organisations that have the data and which are facing the expectation of being able to do something interesting with it.

This can be a starting point, but we would strongly recommend that they quickly move to focus on a cohesive set of industry-specific problems, preferably for an industry whose problems they understand well.  As my colleague David Gates’ wrote in his recent blog post, having market focus sharpens a company, improves its product, and ultimately allows it to offer and extract more value.

Sometimes early success can get in the way of finding a better path of evolution. One company we spent time with had won a couple of big name customers, but was having to spend a huge amount of time providing consulting and customisation services to deliver. The business plan showed a move to a self-serve product and ramping sales numbers, but it seemed that much of their value was actually in their ability to consult.

Ultimately, Big Data will go the same way as the internet. It will spawn a host of new technologies and tools. It will become a core competence for most large corporates; there will be some excellent ‘pure Big Data’ companies. And it will create a raft of new kinds of jobs. While we should expect that most of the current Big Data startups will not last, their existence now is important as part of launching the movement to imagine and unlock the value of data.