Big Data Primer
“Everybody complains about the weather, but nobody does anything about it.” Charles Dudley Warner
Similar to Warner’s observation about the weather, enterprises (and the marketers who work in them) are aware of “Big Data” and how important it probably is, but very few enterprises and marketers can articulate a cohesive strategy to leverage big data into their operational initiatives. According to a recent McKinsey report, “Big data: What’s your plan?” the authors note that although “The payoff from joining the big-data and advanced-analytics management revolution is no longer in doubt, most companies don’t have a plan for what to do with that data”.
Given this, it probably makes sense to align on an easy-to-grasp definition of Big Data. Big Data is essentially a reflection of the digital life around us; over a billion users on Facebook pouring their lives on-line, over 5 billion mobile subscribers interacting with both each other and on-line content and services, and the much bigger M2M (machine to machine) data play which is starting to interact with the consumer data play via products such as Google Glass, healthcare informatics, telemetry systems, geo-location targeting, etc. Tracking and operationalizing data on this level requires a different approach to analytics and multichannel marketing.
The upside? Marketers will now have access to unlimited data about their customers.
The downside? Marketers will now have access to unlimited data about their customers.
The challenges associated with Big Data can be categorized along the following sequence:
• Capturing the data
• Storing the data
• Accessing the data in real time for analytic purposes
• Displaying the data in meaningful ways to a non-technical audience
• Operationalizing the data so relevant functional groups can make use of it
• Iterating the use of Big Data in a continuously improving customer engagement cycle
Capturing the data—we live in a multi-channel world. Think about your day-to-day and how you use technology: your mobile phone is probably your alarm clock, can be used to purchase a wide range of products and services, helps you get around (along with the navigation screen in your car), and it can also make phone calls. Your tablet has probably replaced newspapers, magazines, television and possibly books, and we still use laptops when we need to create anything significant. Billions of us move back and forth across these devices without giving it a second thought, and all of these elements are data capture points; anytime a user is interacting with any on-line resource there is data to be captured (and it is). There is also the angle of our moving through a grid network of sensors that track our movements (street level video surveillance being the most obvious example), Wi-Fi hotspots are now expected at nearly any location where people congregate, etc. All this information is streaming in at a stunning rate, all of it comes in via disparate formats, and it all needs to be captured, categorized, and integrated, which goes to the next step.
Storing the data—Facebook is currently uploading 500 terabytes of user data per day . Amazon has commercialized the concept of effectively infinite storage on-demand. Oracle has essentially ceded the mobile market in order to focus on Big Data. Some of the biggest, wealthiest companies in the world are moving aggressively to infrastructure around this trend. How do enterprises open themselves up to this volume of data in a controlled fashion, or more euphemistically, how do you fill a wine glass from a firehose? Technical issues such as data architectures and the budget impact of how to re-align your organization to absorb this much information become critical path decision points.
Real-time access to data-Now that you have zillions of bytes of data at your disposal, what makes you think you can get access to it? Or more importantly, how do you get access to the information you really want, that is, what is the customer doing right now, and how can you leverage this into a sale? How does the system know what data is relevant, and wouldn’t that depend on who is asking the question? Also keep in mind real-time means right this instant, not 10 minutes ago.
Displaying the data—A highly trained analyst can look at the matrix and recognize what’s there, but for the rest of us without advanced technical degrees, how do you visualize data on this level? What does 500 terabytes of data even look like, and how can you extract something meaningful from it? Also keep in mind you will have a wide range of data sets that need to be interconnected if they’re going to be meaningful, and how does that work? What tools do you invest in, and whose needs are going to be dealt with first? Finance, Marketing, and Sales may all be looking at the same data set, but like the six blind men and the elephant, each will interpret and therefor require something completely different.
Operationalizing the data—So you’ve captured it, stored it, accessed it, and looked at it in a way that makes sense. Now what? How can you take this hard-won knowledge and use it to drive marketing initiatives that enable you to provide a truly personalized experience to your customers and prospects? How do you integrate cross-functionally so that customer support and marketing are aligned as a new product rolls out? What tools and processes do you need in order to tie the data together? How do you make this data not only strategic, but more importantly, transactional?
Iterating the data—Then the final step, pulling this data bounty into a closed iterative loop that allows you to fine-tune your execution in real-time in order to optimize your customer’s experience. You, and your customer, and their customers, etc. are in a constant production cycle, whether its cars or cookies or on-line services. Presumably you want your customers coming back for more (as do they for their customers), and also presumably you’re not the only one pursuing your customer (don’t forget the competition, because they won ‘t forget you).