Notes From The Big Data Frontier: Esri User Conference 2016
I’ve seen plenty of eye-rolling in the retail industry around big data – “No one knows what big data is until they’ve had to deal with retail data, ” that kind of thing. Well, after attending Esri’s annual user conference, I can tell you that retail knows nothing about big data. But the industry will soon find out.
There are two elements to what Esri does that will change the way retailers should be thinking about big data. First, obviously, Esri specializes in Geographic (increasingly Geospatial) Information Systems, or GIS for short. Retailers with brick and mortar stores have cared a lot about location in the past, but usually only so far as to decide where to locate a store. Once that decision is made, it’s not like the store moves a whole lot, so understanding the store’s location over time becomes pretty useless.
But we no longer live in a world where retail’s relationship with geography is defined by store location. And stores really should no longer be the most important asset in a retailer’s portfolio – customers should be the most important thing. And unlike stores, customers move around. A lot. I’m not saying that retailers should be tracking the real-time location of their customers every second of every day. That would be creepy. Don’t do that.
But I am saying that retailers once bought 99-year leases for some of their store locations, and we’re now in an age where some retailers consider a 5-year lease to be a long-term commitment. Why? Because customers move. And a lot of location data about customers that is more transient than where they live – like which stores they shop at, where have they been when they purchased online, and where did they have their orders delivered – is becoming much more central to understanding customer behavior.
In other words, customer is becoming the central location – at any given point in time – much more so than stores. Which means that the location-based information that a retailer collects and uses is getting much more complex, and needs to be updated and managed much more frequently.
It also needs to be translated into analysis that the average executive can understand. In one sense, Esri has a distinct advantage here – people are inherently visual, and map-based analytics are by definition a visual depiction of analysis. But there’s a lot that goes into how to translate analysis into an insight that literally leaps off the page. There really is an art to it, not just in understanding which relationships in size and color and shape are important, but in creating a kind of visual appeal that makes people want to dig deeper and learn more.
If you want an example of what I’m talking about, here’s my favorite map that I saw from the conference. It instantly conveys meaning – and it has a certain intrinsic beauty too. And sorry, Paula, looks like the Miami weather forecast contains a high risk of Great Whites and a lesser risk of Hammerhead sharks.
Retail, more than almost any other industry, should resonate to this idea instantly, that the visual display of information has a lot of power. It’s exactly what the science and art of merchandising is based on. And yet there are far too many retailers that are still resistant to the idea of dashboards and insist on scrolling through reports to “get into the data “. As big data gets bigger, that’s just not going to be an option.
Add in all of the digital information about customers that is out there – and Esri, through its Tapestry service, has already brought together a lot of different kinds of information about consumers – and you rapidly reach a level of complexity around customer information that retailers need to understand, and even more importantly, need to be able to act upon.
Big data, indeed.
Second, there is the whole IoT angle. You think retail has big data? Try this on for size. In one session, a commercial construction company presented on a project they are doing at an airport. It requires tunneling under the air traffic control tower for the airport, and naturally the airport managers were a bit concerned about that. So the company wired sensors to the whole building, to measure vibration, the angle of the building – and whether different sides of the building were at different angles or not for tilt vs. sheer.
This project generates 1 Terabyte of data every month. For one building. Try shifting through that for some insight – you can’t. Machine intelligence must play a role in examining all of this data and identifying patterns of possible importance that people need to pay attention to.
I can hear the resistance from retailers now: “There’s no way I’d ever need that level of detail about my buildings. ” Yes, probably true. But retailers are trying very hard to peel back the roofs of their stores to take a very detailed look inside, and it won’t be too long before the technology to do so cheaply is here – where every shelf is, every product on that shelf, every shopping cart, every customer as they move through the store. Every interaction between customer and shelf, customer and cart, and customer and product. It really won’t be long before that level of detailed capture – and subsequent analysis – is cheap enough that retailers could afford to put the capability in every store, and monitor it all day long, 365 days a year.
But only if they have a way of intelligently shifting through all of that data in order to derive insights. Which is why the rise of machine learning is more important than you may realize. The power of machine learning won’t really be in intelligent chatbots. It will be in being able to learn and surface patterns in IoT data that humans just won’t be able to see through all the noise.
And we’re still talking about huge amounts of data. Another session I attended at Esri involved a street mapping technology. If you think this has nothing to do with retail, think again – LiDAR, or laser-based radar mapping systems, are exactly the systems that make self-driving cars, and delivery trucks, a working technology. One presenter shared his company’s project work with the city of Detroit, mapping area roads. The maps created from the process are being tested to see if they can be used to predict when roads will need to be repaired or resurfaced. But the technology is just an application of the same kind of LiDAR that guides Google’s cars. It’s just instead of using it on a real-time basis, the company is doing longer-term trend analysis with it.
This company, because it only focuses on roads, not on other objects moving on or around the roads, employs two laser/camera hardware combinations, both pointed down at the road’s surface. Scanning a road doesn’t take very long – that’s part of what makes the technology so revolutionary, if you care a lot about road maintenance. But it collects 1 Gigabyte of data per road lane per kilometer. Per scan. So think about if your store was laid out as multiple “lane ” passes by a scanner. How “long ” would your store be? A big box store – I’m thinking that might be close to a kilometer. And if you wanted to do that scan every second or even every minute? 60 minutes in an hour, a store open 10 hours per day, times 30 days per month? Now you’re getting solidly into that monthly Terabyte territory pretty quickly.
Yeah, retailers have a lot of SKUs. And they have a lot of locations and even more customers. And they have struggled in the past to turn all of that data into something meaningful. But after seeing the kinds of problems Esri is trying to solve, I have a feeling retail knows nothing about big data. But it will. Very soon.