I was thinking about the gap between the promise of technology and the realities of retail the other day. What triggered this was a pretty technical discussion with a system designer about how to move retailers beyond thinking of their businesses in purely transactional terms (i.e. what sold, when, and where) are more towards thinking of themselves in terms of their relevance to consumers (i.e. how consumers react to the total Brand value offering). That conversation inevitably led to if and how retailers are actually using all the “Big Data ” analytics that companies like RSR have been talking about for a decade.

The question is, how far has the industry progressed since the industry-wide discussion of new data/new analytics began several years ago? The answer is, “not as much as you think ” (figure). When RSR recently asked retailers about the status of capabilities related to capturing and analyzing “Big Data “, we found that no capability was reported to have been “implemented ” by up to 50% of retailers, and less than ¬º of retailers claimed to be “satisfied ” with what they had implemented.

Not Nearly As Far Along As You Might Think

Source: RSR Research, 2016

The industry has a “half empty or half full ” problem. The optimistic view is that it is still very early days when it comes to truly leveraging non-transactional data, and that even those who have implemented new capabilities may need to rethink their approach due to the swiftly changing nature of the new data being created.

But whether retailers take that positive perspective or the other, more pessimistic point of view that retailers are just too slow to change, “what’s next ” is just the same. It’s time for retailers to get moving- but to what end?

Will The Retail Model Change Because Of Big Data?

Retail is a famously reactive business, and in the past companies have reacted to changes reflected in aggregated product sales in quarterly, monthly, or weekly totals. Up until relatively recently, it was very difficult for retailers to react to changes detected in daily sales much beyond determining unusual out of stock conditions. At least as far as the stores are concerned, retailers have focused on closing the gap between perceiving a change and reacting to that change, or put another way, on enabling a fast reaction to events as soon as possible after they occur.

Enter “Big Data “. Here’s the thinking: retailers now have the opportunity to virtually eliminate the lag time to reaction by predicting events before they occur and being ready when they do occur. Of course that approach depends heavily on the quality of the data (both internal and external) used to model a future-state, and the model itself. Predictive analytics represent a huge leap forward for most retailers because they take into consideration not only internal transactional data, but also consumer path-to-purchase and social data, as well as exogenous data like weather, competitive, consumer psychographic profile, calendared events (such as concerts or sporting events), and manufacturer and supply chain data.

That sounds like a real “win “! But whether slow or fast, Retail remains a fundamentally reactive business. Is that different than being “adaptive “? In science, the term “adaptive system ” typically refers to a “closed loop ” environment (a famous example is a fresh water pond) that senses changes and then responds accordingly. In the 1980’s, the Santa Fe Institute defined such a closed loop system environment as a “Sense-Interpret-Decide-Act cycle “, where “decide ” meant “reflexive choices, programmed choices, learned and instinctual responses, and conscious choices. ” In the 1990’s IBM Research Fellow Stephen Haeckel applied that thinking to business organizations, and the notion of an “agile ” business was born.

According to Haeckel, businesses need to be organized to handle unpredictability. To do so, a business must be able to:

Know earlier the meaning of what is happening now;

Rapidly and effectively reconfigure the response via “modular response capabilities “; and

Achieve a major reduction in the number of elements that must be predicted in advance by organizational leadership (or in other words, boil down the array of possible variables to a subset that maters to the business).

Knowing something is happening in the moment, and doing something about it, are two different things. There are certain things about the Retail industry that probably won’t change anytime soon; for example, the best way to offer a good price to consumers is to achieve economies of scale on the supply chain side, and that implies “buying big ” – which in turn means that demand is aggregated. That is the essence of the mass merchandising model, which it by its very nature isn’t very quick about handling sudden shifts in demand. But consumers want more relevance, and that also means not having to slog through mountains of merchandise they don’t want to get to the items they do want. That’s what localization of the offer is all about.

When the industry talks about relevance, as often as not the focus turns to 1-to-1 marketing. But 1-to-1 isn’t the right answer for everyone. Retailers need to answer these questions:

What level of “intimacy ” is needed to know what you need to know to be relevant? There is no one “right ” answer to this question. Some needs are highly predictable, others much less so. The granularity of customer-specific data needed to make a highly relevant offer is closely related to the specificity of the need (for example, offering basic commodities doesn’t require much customer specificity);

What level of relevance is needed to be relevant? This point is closely related to the question about the level of customer “intimacy ” required, but relates to the granularity of the offer itself. Extreme localization isn’t necessary for every Brand;

How much data do you really need? Or, how much analytical sophistication do you need to have to apply what you know to be relevant to customers? This gets to Haeckel’s 3^rd point about reducing the array of possible variables needed to make a decision.

So will “big data ” cause retailers to change their model? In a way, I think that the world of possibilities that “big data ” represents might be distracting retailers from the real question, “what value are you trying to deliver? ” Getting an answer to that has to come first, and you don’t need technology to answer it. But you MAY need technology to deliver it, and DO need technology to tell you or not whether you’re getting the job done in a way that will satisfy customers and generate profits for you. That’s when your version of “big data ” becomes important.