By Felix Rios
One of the frequent questions we get from curious market research (MR) clients is: “What is big data?” The answer they are really looking for is not the Wikipedia definition that you get from Googling it, but what they actually want to know is: “What’s the deal with big data and market research?” To answer this question purely in the context of MR, we start by telling them what big data is not.
- Big data is not the solution to all your problems.
- Big data is not a black box where magic things happen.
There is this weird idea that big data is about overloading a machine with information and then expecting it to answer all your questions. The reality is that a lot happens behind the curtains before you get to the question-answering stage. Big data is about organizing large sets of data so it can be processed faster. The most important aspects about big data are that it only makes sense if the data sets are large enough, and that it needs to be done with a specific, defined business need in mind. Your 1,000 gen-pop surveys per month do not meet this criteria.
At Ugam, one of our solutions is intended to cater to the retail industry. Our retail team crawls the Internet daily, collecting gigabytes of pricing information for products and then uses the data for our price intelligence solutions (amongst other things). Because we know what we are going for, we let our robots loose to get this for us and bring it in with certain structure. The volume of data we collect is so large that we need tools to help us aggregate and query that data. This is one practical example of big data in action. Every day more and more data is being collected, and our data gets richer because we are collecting it with a purpose.
The Market Research Thing
Market research needs to evolve, we all agree, but it is not only about making surveys shorter (which is needed) or mobile surveys (which is happening) or passive monitoring (which will get better). We have to get better at understanding and merging other sources of data. For example, one area that requires our attention is using our clients’ transactional data. Clients have loads of transactional data sitting inside their systems that is very valuable for understanding their customers. The answers to many of their questions could already be within that data.
In the spirit of evolving our industry, it is also critical that we figure out a meaningful way to utilize democratic data (that which is freely available on the internet). I think we have passed the stage where we ignore it (maybe because it scares us?) and are now in the stage where we underestimate it. I keep hearing, “This is not representative,” or “It’s nice, but you really can’t do anything with it.” Eventually, we will take it seriously and give it the attention it deserves. For the researchers of tomorrow, this will be the most important source of data to understand their consumers in new and deeper ways.
New Times Require New Measures
Until not long ago, the only option for storing digital photos was DVDs. A few weeks ago, Google gave us all the free access we need to store an unlimited amount of photos on their cloud servers. No more stress about DVDs failing and losing your most precious memories. Today we can be confident that our photos are safer in the cloud than anywhere else. Similarly, for big companies, in the past, it was difficult to collect and store massive amounts of data – let alone make use of it.
Today it is different because technology has given us the tools to capture more data, storage is only getting cheaper, and we keep gaining more and more processing power .All these technological advancements are making processing and querying this ocean of data possible. If at this point you have already realized that this data needs to be structured while you store it, then your future clients will see the benefits from this knowledge. You will know more about your customers than any of your competitors ever can.
The time is now to start advising our clients about this. Structuring data is not a task that MR firms typically would give advice on, but if we are able to guide our clients about how to organize all this internal data, it will make future market research integrations a lot easier. In the end, our goal is to help them understand their customers in new ways. In this new world of multiple sources of data, big data is simply the underlying technologies and tools used to unearth insights and make all these instruments sing together.
Felix Rios is Technology and Innovation Manager at Ugam
