Data Collection for Store Location Analytics: Picking a Technology Winner

Data collection technology is at the heart of in-store customer location analytics. In my past two posts, I’ve described some of the cool analytics and measurement that our second release of DM1 brings to the enterprise. And in a way, this is the only stuff that matters. It’s what you use to solve problems. But you can’t solve those problems and DM1 can’t give you the measurement you need, without a workable data collection technology: a technology that’s reliable, accurate, and cost-effective to deploy. In digital analytics, tagging was that technology. And while tagging can seem mysterious, a basic tag is really nothing more than 20 lines of javascript code that any competent programmer could write in a day. For in-store location analytics, it’s a lot more challenging. It’s so challenging, in fact, that we’ve struggled to find data collection technologies that meet our needs. We’ve engineered the DM1 platform to be hardware and collection neutral. We take data from a variety of sources and we’ll engineer the best possible measurement from that source. But we do have a favorite – and it’s a solution that’s become our go to suggestion for MOST clients. It’s called iViu.

The iViu technology uses passive network sniffers. These little devices track smart-phones (and potentially other electronics like Smart Watches). They triangulate on the signal the device sends out to position the phone. And because they can identify the phone, they are able to track the full shopper journey from just outside the store to cash-wrap or exit. Like almost all in-store tracking devices, they can’t identify who somebody is, though they can track the same device over time. So the iViu data does let us track same-store usage (at least outside of the EU where, to be fully compliant with EU guidelines we throw away device signatures at the end of each day) and even the same shopper at different locations under your real estate portfolio. But it doesn’t tell us who the shopper is or give us a natural join key to household or digital data.

In laying out the basics, I’ve glossed over all the complexity involved – and there’s a lot. In store location analytics, data collection technologies compete along several critical dimensions: coverage, accuracy, journey measurement, demographics, privacy, associate tracking, ease of implementation and cost. Each technology and each vendor has its own unique strengths. I’m going to cover each of these factors, explain where iViu fits in, and summarize why we usually end up choosing their technology. The three main location analytics technology contenders are Camera, Passive Network Sniffers (iViu) and off the shelf WiFi access points.

Population Coverage: Ideally, a counting system will measure every shopper who comes in the store. If a counting system doesn’t collect everyone, it’s important to understand the breadth of its coverage and whether it introduces any deep bias into the measurement. In terms of population coverage, you can’t beat camera systems. They are the best technology around for getting 100% coverage. They rarely miss anyone and if anything, their pitfall is that they can be prone to overcounting.  All electronic mechanisms are limited to tracking shoppers with smart-phones. In the U.S. (and most of the world), that isn’t much of a problem. It does mean you likely won’t be counting smaller kids. Of more significance, however, is that the phone must be an emitter – with either its Bluetooth or WiFi turned on. Best estimates are that about 15% of people don’t enable those signals. So an iViu device will typically get signals from about 80-85% of the population. And we think that’s a relatively unbiased group – probably a more accurate sample than what we get in the digital world using cookies. One big advantage to the iViu system versus other electronic systems is that iViu does a much better job tracking iPhones or other devices that employ MAC randomization. Why are iPhones an issue? Beginning with iOS8, Apple started randomizing the MAC address of the device when it pings out to the world. WiFi access points and most electronic detectors use the MAC address as the device identifier. So every time an iOS device pings a typical collector, it will look like a different device. In practice, this means that WiFi based coverage will lose all iOS devices that aren’t connected to your network. That’s a pretty huge problem. In addition, iViu devices are dedicated to measurement and they do a much better job of listening than standard WiFi access points. In side by side tests, iViu devices pick up more shoppers, more consistently.

So for Population Coverage, we see it this way:

Camera: Best

iViu: Good

WiFi: Poor

Accuracy: There are lots of different ways to think about accuracy and many different use-cases and data quality problems. I’m going to focus here on basic positional accuracy – the ability to locate the shopper at particular place in the store. Positional accuracy isn’t vital for applications like door-counting – but our DM1 platform absolutely depends on it. We map location to the store and report and segment based on shopper interest. If the mapped location is wrong, our analytics are wrong. With camera systems, each camera covers a specific area of the store and it’s relatively easy to map the location of the person to the specific part of the area the camera covers. For electronic tracking, it’s more complicated. Most electronic systems work by using either (or both) the relative signal strength detected and a triangulation of the signal across devices. But while the methods used are similar, the end result is quite different depending on the implementation and the vendor. Using the iViu devices, we can usually get an accuracy of location down to about 1.5 meters. That’s almost always good enough for the type of measurement we do with DM1. With off-the-shelf WiFi systems, the accuracy is more like 10 meters (and that’s often best case). We can live with that level of accuracy if we’re doing measurement on a mall, stadium, airport or a resort. But for a store, it just isn’t good enough to work with.

So for accuracy we see it this way:

Camera: Good

iViu: Good

WiFi: Poor

Journey Measurement: All the interesting questions in shopper and store optimization involve the journey – the ability to track the shopper visit across the store. It’s fundamental to DM1 and big part of what our analytics bring to the table. In theory, all the data collection technologies should be able to track the journey. In practice, however, we’ve found that electronic systems do this vastly better than the current crop of camera systems. Electronic systems have the fairly easy task of distinguishing one phone from another. Camera systems have to track people. And while there have been dramatic improvements in facial recognition, those improvements are challenged by real-world measurement situations and often haven’t found their way into workable/available technology. A typical camera only covers a 20×20 foot area. So even a modest mall store will require a goodly number of cameras to track its full footprint. As shoppers move from zone-to-zone, the system has to be able to determine that it’s the same person. Camera systems suck at this. Suppose a camera system gets a zone crossing right 90% of the time. That sounds pretty good, until you realize that you have about a 50% chance of following a shopper across 100 feet of your store. It’s because of this limitation that so many camera-based systems are, essentially, zone counters. They count the shoppers in an area and their linger time. They don’t count journeys. It’s not a software problem. It’s a collection problem.

Camera: Poor

iViu: Good

WiFi: Good

Demographics: We’re behavioral analysts, but while we tend to believe that real behavior trumps demographics, it doesn’t mean we think demographics don’t matter. Age and gender are nearly always interesting analytic variables and it’s a distinct advantage to be able to collect them. The scorecard here is simple. Camera does a pretty good job of this. No electronic system does this at all.

Camera: Good

iViu: No

WiFi: No

Privacy: There’s an undeniable creep factor involved with in-store tracking and it can be a legitimate barrier to measurement. All of the technologies involved here do essentially the same thing and all of them do it anonymously. Some of our clients have preferred video to electronic measurement on privacy grounds. I frankly don’t understand that thinking, but privacy is an area where the arguments turn more on perception than reality. Both technologies are providing the same basic measurement and the only significant difference is that electronic measurement provides an opt-out mechanism and video doesn’t. For electronic measurement, there’s a national, online opt-out registry (and, of course, you can always turn off phone WiFi too). There is no equivalent system to opt-out of video measurement and if there was, I imagine it would involve sending in your face – which kind of sucks. I do think video benefits from the fact that most stores have already deployed it for security purposes (though the camera’s you use are usually different), but it’s hard to understand why it’s better to measure people one way than another when the implications for them are identical. I’m going to call this one a wash.

Camera: Ok

iViu: Ok

WiFi: Ok

Associate Tracking: Coming from the digital world, our focus when we started Digital Mortar was all about shoppers. But we quickly realized that tracking associates was critical. First, because associates are a huge part of the customer experience. You can’t really measure shopper journeys unless you can measure when and if they talked to an associate. But there’s also real value in understanding whether you had enough staff on the floor. If they were in the right places. If the type of associate, their training or experience or tactics, made a substantial performance difference. With electronic tracking, it’s pretty easy to measure associates (they just have to carry a device). In most cases, you don’t even have to register that device. DM1 automatically detects devices with employee behaviors and classifies them appropriately. We do that to minimize compliance issues. With camera, it’s a different story. Camera systems either conflate employees with shoppers (which is a disaster) or use supplementary electronic means to remove them from the data. Not only does this introduce complexity into the system, it makes it much harder to track and measure interactions. We’ve also seen minor compliance issues (a few associates forgetting to pick up their tags occasionally) have significant negative implications on measurement quality. It’s worth mentioning here that if you only want to measure associates, there are other technologies worth considering that require code on a mobile device but which will provide VERY accurate and detailed associate tracking.

Camera: Poor

iViu: Good

WiFi: Fair

Ease of Implementation: Let’s face it, having to put hardware into stores is a hassle. One of the big benefits to WiFi based measurement is that it can take advantage of existing access points that were put there to provision WiFi to customers or to support store functions. Every other form of measurement takes new hardware and store installation. But there is a pretty big difference in the level of effort required. It takes a lot of cameras to cover a store. The cameras have to be in the ceiling and they have to precisely placed. It’s real work to get right and it’s often expensive, involves some degree of retrofitting and is time consuming. An iViu device will cover something like 10x the area of a camera – so you need a lot less of them. It’s easier to install. It doesn’t have to precisely placed and it doesn’t have to ceiling mounted. We’ve put iViu devices under tables, on top of or behind displays, on pillars and even in drawers. They do require power (plug or PoE), but they’re a snap to setup – it’s pretty much plug and play – and we’ve found that we can install in most locations without huge difficulty.

Camera: Poor

iViu: Fair

WiFi: Good

Cost: You know how athletes who sign huge new contracts always say “It wasn’t about the money” and you’re thinking – “Of course it was about the money”? Money matters. Realistically, a store can only afford to spend so much on measurement. Worse, the more you sink into the hardware, the less you can spend on the stuff that actually makes a difference – the analytics software and the people to drive it. At Digital Mortar, we’re believers in comprehensive measurement. We want to measure every store – not one store out of a hundred. And if you’re trying to measure Associates, optimize locally, or do real-time interactions, measuring a single store just doesn’t cut it. So having a measurement technology that’s cheap enough to go fleet-wide? We think that’s priceless. From that perspective, you can’t beat WiFi since it’s usually already in place and even if you have to provision it, the cost is reasonable and there are extra benefits. But, as noted, there’s pretty limited analytics you can do with a 10 meter margin of error. For a system that provides robust measurement, we like the fact that iViu devices are very cost-effective. The hardware for most stores costs less than $5k (that’s to buy the devices not an annual cost). Even very, very large stores will cost less than $20K per store. Camera systems are often far more costly – to the point that they are generally impractical for very large stores and make deploying to a large number of stores impossible.

Camera: Poor

iViu: Good

WiFi: Very Good

As you can see, there isn’t one solution that’s perfect in every respect. And in the real world, we often find reasons to deploy each technology. In Part 2 of this post, I’ll summarize the findings, explain why – given it’s overall profile – iViu makes sense for most retail stores, and also talk a little bit about the ways that you can blend technologies to get the best of each world.

A Peek Into the Future of Store Analytics

We just did our first non-incremental release of the DM1 store analytics platform since we brought it to market. It brings new analytics views to the Workbench, a host of UI and analytic tweaks, new cloud options and, best of all, real-time and full-store playback functionality to the product. Real-time creates a bevy of opportunities to operationalize measurement in both operations and marketing. So DM1 can drive more value, faster. What’s next? At the end of my last post, I described some of the juicier features slated for upcoming release: a real-time, dedicated Store Managers console, full pathing and even some machine learning applications. But I want to step back from a feature list and talk a bit about where we see the DM1 platform headed and how we try to balance and prioritize new functionality as we shape the product. It’s hard to do because we love all the new features.

From a personal perspective, no part of building Digital Mortar is more interesting or more intellectually challenging than building DM1. On the one hand, building SaaS systems in the cloud today is incredibly gratifying. You can build powerful, beautiful stuff so much faster and easier than back in the ‘80s when I first started programming or even in the late ‘90s when Semphonic took an abortive shot at building a web analytics tool. But an embarrassment of riches is still an embarrassment. Throwing stuff at a wall doesn’t make for a coherent product road-map. So when we think about new feature prioritization for DM1, we start with our core product vision.

DM1 is designed to be the measurement backbone of the store. We see the store as a learning machine with the core methodologies we brought from digital: continuous improvement through test & measure driven by analytics based on behavioral segmentation (what people actually do) and the ability to break-down shopper journeys into discrete, analyzable steps.

That core vision shaped our initial DM1 release (what Valley folks love to call the MVP – an acronym that is surely designed to suggest the sporting world’s Most Valuable Player award but actually stands for Minimally Viable Product). When DM1 went live, just about every piece of it was specifically targeted to this core vision. It provided direct access to a bunch of journey metrics that described how the store performed, it included basic shopper segmentation to analyze cross-sell patterns and do simple day-time parting, and it included a pretty robust funnel tool for breaking down shopper journeys into individual (step) components.

Let’s call this basic shopper-journey, store measurement system DM1’s core. It’s the engine that drives and integrates every other aspect of what the product might eventually do. Coming out of the digital analytics world, we tend to map a lot of our thinking into that model. The DM1 core is the equivalent of Adobe Analytics in the broader Adobe Marketing Suite. It’s the analytic and measurement engine.

Right now, most of our focus will continue to be on building that core engine.

Of the significant features we have slated for short term development, here are the ones that contribute directly to the core function of the program:

New and More Comprehensive Associate Reporting: Track individual and team performance on the floor with optional integration to VoE, employee meta-data, and VoC from in-store visits. DM1 already includes a lot of generalized Associate analytics, but this report will distill that into a set of reports that are much easier to digest, understand and act on.

D3 Integration: DM1’s current charting capabilities are pretty basic. We use an off-the-self package and we provide straightforward bar and line charting. Probably the best part of the charting is how seamlessly DM1 picks the best chart types, intelligently maps to separate axes, and lets you easily combined “like indices” in a single chart. But we’re far from pushing the envelope on what we can do visually and by using D3 for our charting package, we’ll be able to considerably expand the range of our visualizations and support even deeper on-chart customization.

Full Pathing: We’ve been tinkering since day 1 with ways to bring full pathing to store analytics. On the one hands, it’s not really all that hard. The amount of data is much less than we’re used to in digital. Our engine passes the data exhaustively with every query, so full pathing isn’t going to strain us from a performance perspective. But stores don’t have discrete waypoint like pages on a Website which makes each shopper’s path potentially a snowflake. We’ve tried various strategies to meaningfully aggregate paths within the store and I think we’ll be able to produce something that’s genuinely interesting and useful in the next few months. This will supplement the funnel analytics and provide richer and more varied analysis of how shoppers flow through the store.

Segmentation Builder: DM1’s current segmentation capabilities are limited to basic filtering on a set of pre-defined types. It does provide a pretty nice ability to segment on uploaded meta-data, but you can’t build more complex segments using Boolean logic or Regex. Not only do I think that’s important for a lot of analytic purposes, it’s also something we can support fairly easily.

Machine Learning for Segmentation: On that same theme, I’m a believer in data-driven segmentation. Data-driven segmentation uses more data, is richer, more reflective of reality, and usually more interesting than rule-based segmentations even if produced in a fairly rich builder. Both GCP and Azure offer pretty amazing ML capabilities that will allow us to build out a good data-driven segmentation capability for DM1. I think the harder part is doing the UI justice.

Store Groups: DM1 handles lots of stores, but right now, the store is the ultimate unit of analysis. We don’t support regions or fleet-wide aggregations. There are a lot of analytic and reporting problems that would be solved or made much easier with Store Groups. It’s a capability we’ve considered since Day 1 and sooner rather later I except it to be in the product.

Fully Integrated Dashboard: V2 didn’t do much to evolve the dashboard capability of the product, but we have a pretty clear direction in mind. In the next release, I expect the Dashboard to be capable of containing ANY Workbench view. That’s a simple elegant way to let analysts customize the dashboard to their taste and produce exactly what they need for the business. I remember a computer scientist from the original deep-blue chess program saying something to the effect that “Exhaustive search means never having to say your sorry”. No matter how much capability we build out in the dashboard, analysts are always going to want something from the Workbench if it does more. So I think it just makes sense to unify them and let the Dashboard do EVERYTHING the Workbench does.

Not everything we have in mind is about the core though. In the next few months, we plan to release a Store Manager Console based on the new real-time capabilities. The Store Manager Console is a whole new companion capability for DM1 targeted to a fundamentally different type of user. DM1 core is for the corporate analyst. It’s a big, powerful enterprise measurement tool. It’s definitely more than most Store Managers could handle.

But while the centralized model works really well in digital analytics (since Websites are wholly centralized), it’s less than ideal in the store world. There are a lot of decisions that need to happen locally. DM1’s Store Manager console will continuously monitor the store. It will keep track of shopper patterns, monitor queue times, alert if shoppers aren’t getting the help they need, and make it easy for Store Managers to allocate staff most effectively and message them when plans need to change.

It’s a way to bring machine smarts and continuous attention to the Store Managers iPad. Most of the capabilities we’re baking into the Store Manager Console (SMC) were actually delivered in V2. The real-time store tracking, simulator and Webhooks for messaging are the core capabilities we needed to deliver the SMC and were always a part of that larger vision.

As I hope our rate of progress has already made clear, we’re ambitious. Software design usually embodies deep trade-offs between functionality and ease-of-use or performance. Those trade-offs are challenging but not inevitable. We’ve seen how digital analytics tools like Google Analytics and data viz tools like Tableau have sometimes been able to step outside existing paradigms to deliver more functionality side-by-side with better usability. Most of what we’ve done so far in DM1 is borrow creatively from two decades worth of increasing maturity in digital analytics. Still, tools like our Funnel Viz and – particularly – our Store Layout Viz have tackled location/store specific problems and genuinely advanced the state-of-the-art. As we tackle pathing and machine learning, I hope to do quite a bit more of that and find ways to bring more advanced analytics to the table even while making DM1 easier to use.

Bringing Real-time Analytics to the Store

When we released the first full production version of DM1 in May, it was a transformational leap in customer location analytics. Now, six months later, we’re releasing the first major upgrade to DM1 – and it’s a doozy. We’re adding powerful new approaches to improving the store and driving actionable results while putting the competition even further in the rear-view mirror (they’re back there somewhere). Some of the coolest new features include:

Real-time View:  See and track what’s happening in the store as it happens

Real-time Messaging: Integrate with Associate and Shopper Mobile devices based on what’s happening right now

Playback: See what happened in the store using a time-lapse simulation of actual shopper behavior

Week-Time: A cool new visualization that shows a Day-Time part analysis for ANY metric or segment

Flexible Cloud Hosting: Azure or Google Cloud, your choice.

And that’s just the big stuff. There have been a host of small tweaks, fixes, performance enhancements and visual improvements in the intervening months.

That initial version of DM1 had a pretty remarkable feature set for a V1 product in a brand new market. It delivered a revolutionary store visualization tool that mapped its powerful metrics into the store at any level of abstraction – from Display to Section to Department to Floor to Store. And the metric set if provided shattered existing door-counting paradigms by delivering real journey metrics. What shoppers did first. Where they spent the most time. Every place they shopped. What got the most consideration. What converted. What didn’t. And DM1 V1 integrated a rich set of Associate tracking metrics into the basic product as well. Associate presence, intra-day STARs, interaction rates and times, and associated lift. DM1 also delivered powerful grid-based reporting and charting, a really cool funnel analytics tool and a powerful side-by-side comparison tool.

With the new features, we’ve focused on adding a set of capabilities that extend DM1 deeper into the store and make it easier to integrate into broader store marketing and operations efforts. The biggest part of that is, of course, real-time.

We’ve always collected store data in real-time – and all of our collection technologies support near real-time data about the shopper. But in the V1 release, everything we did was batch processing and next-day analytics. That isn’t just because batch processing is easier (though that did matter to us). It was also because many years of experience in digital analytics convinced us that the applications for real-time analytics were fairly rare. There was a time when real-time reporting was a huge part of the digital analytics sales process. But in the end, the dominant tools in digital analytics deliver intra-day but far from real-time analytics. So we figured if a very mature market like digital analytics could do without real-time, we didn’t need it for our V1 release.

But the store is different beast than a Website, and we’re finding that having a real-time view of what’s happening makes it possible to add value at the Store Manager level AND support both CRM and queue management applications that help improve the customer experience.

So real-time became a central piece of our V2 release. And once we built out the core capability, we took advantage of it in multiple ways.

The Realtime view show’s what’s happening right now in the store. Associates and Shoppers are shown (using different symbols) and you can quite literally track exactly where they are and where they are going. We even color-code Shoppers to make it easy to identify how long they’ve been in the store.

Check it out:

realtime, location analytics, store analytics, store traffic, DM1, Digital Mortar

Pretty cool. The Playback capability provides the same view but processes historical data. Since it’s historical, it can collapse time periods into a time-lapse view. So you can see an hour or a day in five minutes.

Being able to see real-time data is cool and highly useful for folks on the ground, but I won’t claim the analytic implications are enormous. What is enormous, though, is the capability to tie DM1’s tracking and measurement into real-time messaging systems. We’ve built a straightforward Webhook messaging interface that lets you get three kinds of data out of DM1: current metric data (for uses like queue length), real-time shopper data, and real-time Associate data.

The real-time shopper data can be based off any customer key we have. And it provides a CRM view that includes Entry Time, Total-Time in Store, Presence and Lingers by Section, Current Location, Time since Interaction, and Associate ID of last interaction. You can use this data to generate real-time messages via your mobile application or in-store beacons to the shopper.

As cool as this shopper interface is, I’ve long been a believer that messaging Associates is even more fundamental and important. With shoppers, opt-in is always a challenge. And barring a truly compelling application, I think it’s tough to get enough opt-ins for messaging to make it impactful. But Associates are increasingly being armed with digital tools that allow them to do more (like distributed PoS) and serve the customer better. Being able to optimize Associate interactions with real-time data and positioning is a huge leap forward in operational sophistication.

In addition to real-time, we’ve made a boatload of analytic enhancements too. And one of my favorite new views is the WeekTime report. We’d already build a custom report around staffing that laid out STARs (Shopper to Associate Ratio) by day of week and time of day (down to the hour). But that report wasn’t a thing of beauty and, in any case, it was specific to the STARs metric.

Because Day-time parting is so fundamental to the store, we wanted a generalized analytics tool that would do the job. The WeekTime tool lays out any metric by day-of-week and time-of-day.

Here are some examples of the WeekTime tool. The first view shows when the store had the most shoppers. The second view shows when shoppers spent the most time in the store.

Shopper Traffic by Day-TimeAvg. Shop Time by Day-Time
Day-Time Traffic View for DM1 retail analyticsDM1 Avg. Time by Shopper for Store Analytics

And like all DM1 workbench tools, the Weektime tools is driven by our big data event-level engine so it supports integrated segmentation across any time-period. It’s a really nice analytic addition to the Workbench and the Dashboard view. It makes it much easier to quickly visualize and understand how day-time parts are driving performance on any measurement.

One of the biggest changes we made in V2 isn’t functional but environmental. We built DM1 on Azure and it’s been a great platform. But we’ve seen that our clients are going in all sorts of directions in an incredibly competitive cloud marketplace. And if the rest of your infrastructure is on Google Cloud (GCP), then it just makes sense that DM1 live there too. So in V2, we offer the choice of cloud provider. We’ve ported the entire platform into GCP and – as a bonus gift to ourselves – made the deployment process a lot more automated and easier for us. Microsoft Azure or Google GCP – it’s now your choice. And it’s just part of making sure DM1 is the most technologically sophisticated AND seamless platform around.

V2 is another big step for us. But we’re just getting started. I keep an ever-growing Trello board of new features and some of the most exciting stuff to come includes a full real-time Store Manager tool, a much more comprehensive and beautiful Associate performance report, a store-change report that automatically shows you the impact of every store change in a period of time, the integration of D3 into our charting capability, a full pathing tool, a robust segmentation builder, and even an initial foray into machine learning.

Can hardly wait!

Connecting Marketers to Machine Learning: A Traveler’s Guide Through Two Utterly Dissimilar Worlds

Artificial Intelligence for Marketing by Jim Sterne

There are people in the world who work with and understand AI and machine learning. And there are people in the world who work with and understand marketing. The intersection of those two groups is a vanishingly tiny population.

Until recently the fact of that nearly empty set didn’t much matter. But with the dramatic growth in machine learning penetration into key marketing activities, that’s changed. If you don’t understand enough about these technologies to use them effectively…well…chances are some of your competitors do.

AI for Marketing, Jim Sterne’s new book,  is targeted specifically toward widening that narrow intersection of two populations into something more like a broad union. It’s not an introduction to machine learning for the data scientist or technologist (though there’s certainly a use and a need for that). It’s not an introduction to marketing (though it does an absolutely admirable job introducing practical marketing concepts). It’s a primer on how to move between those two worlds.

Interestingly, in AI for Marketing, that isn’t a one way street. I probably would have written this book on the assumption that the core task was to get marketing folks to understand machine learning. But AI for Marketing makes the not unreasonable assumption that as challenged as marketing folks are when it comes to AI, machine learning folks are often every bit as ignorant when it comes to marketing. Of course, that first audience is much larger – there’s probably 1000 marketing folks for every machine learner. But if you are an enterprise wanting two teams to collaborate or a technology company wanting to fuse your machine learning smarts to marketing problems, it makes sense to treat this as a two-way street.

Here’s how the book lays out.

Chapter 1 just sets the table on AI and machine learning. It’s a big chapter and it’s a bit of grab bag, with everything from why you should be worried about AI to where you might look for data to feed it. It’s a sweeping introduction to an admittedly huge topic, but it doesn’t do a lot of real work in the broader organization of the book.

That real work starts in Chapter 2 with the introduction to machine learning. This chapter is essential for Marketers. It covers a range of analytic concepts: an excellent introduction into the basics of how to think about models (a surprisingly important and misunderstood topic), a host of common analytics problems (like high cardinality) and then introduces core techniques in machine learning. If you’ve ever sat through data scientists or technology vendors babbling on about support vector machines and random forests, and wondered if you’d been airlifted into an incredibly confusing episode of Game of Drones, this chapter will be a godsend. The explanations are given in the author’s trademark style: simple, straightforward and surprisingly enjoyable given the subject matter. You just won’t find a better more straightforward introduction to these methods for the interested but not enthralled businessperson.

In Chapter 3, Jim walks the other way down the street – introducing modern marketing to the data scientist. After a long career explaining analytics to business and marketing folks, Jim has absorbed an immense amount of marketing knowledge. He has this stuff down cold and he’s every bit as good (maybe even better) taking marketing concepts back to analysts as he is working in the other direction.  From a basic intro into the evolution of modern marketing to a survey of the key problems folks are always trying to solve (attribution, mix, lifetime value, and personalization), this chapter nails it. If you subscribe to the theory (and I do) that any book on Marketing could more appropriately have been delivered as a single chapter, then just think of this as the rare book on Marketing delivered at the right length.

If you accept the idea that bridging these two worlds needs movement in both directions, the structure to this point is obvious. Introduce one. Introduce the other. But then what?

Here’s where I think the structure of the book really sings. To me, the heart of the book is in Chapters 4, 5 and 6 (which I know sounds like an old Elvis Costello song). Each chapter tackles one part of the marketing funnel and shows how AI and machine learning can be used to solve problems.

Chapter 4 looks at up-funnel activities around market research, public relations, social awareness, and mass advertising. Chapter 5 walks through persuasion and selling including the in-store journey (yeah!), shopping assistants, UX, and remarketing. Chapter 6 covers (you should be able to guess) issues around retention and churn including customer service and returns. Chapter 7 is a kind of “one ring to rule them all”, covering the emergence of integrated, “intelligent” marketing platforms that do everything. Well….maybe. Call me skeptical on this front.

Anyway, these chapters are similar in tone and rich in content. You get the core issues explained, a discussion of how AI and machine learning can be used, and brief introductions into the vendors and people who are doing the work. For the marketer, that means you can find the problems that concern you, get a sense of where the state of machine learning stands vis-à-vis your actual problem set, and almost certainly pick-up a couple of ideas about who to talk to and what to think about next.

If you’re into this stuff at all, these four chapters will probably get you pretty excited about the possibilities. So think of Chapter 8 as a cautionary shot across the bow. From being too good for your own good to issues around privacy, hidden biases and, repeat after me, “correlation is not causation” this is Pandora’s little chapter of analytics and machine learning troubles.

So what’s left? Think about having a baby. The first part is exciting and fun. The next part is long and tedious. And labor – the last part – is incredibly painful. It’s pretty much the same when it comes to analytics. Operationalizing analytics is that last, painful step. It comes at the end of the process and nobody thinks it’s any fun. Like the introduction to marketing, the section on operationalizing AI bears all the hallmarks of long, deep familiarity with the issues and opportunities in enterprise adoption of analytics and technology. There’s tons of good, sound advice that can help you actually get some of this stuff done.

Jim wraps up with the seemingly obligatory look into the future. Now, I’m pretty confident that none of us have the faintest idea how the future of AI is going to unfold. And if I really had to choose, I guess I prefer my crystal ball to be in science fiction form where I don’t have to take anything but the plot too seriously. But there’s probably a clause in every publisher’s AI book contract that an author must speculate on the how wonderful/dangerous the future will be. Jim keeps it short, light, and highly speculative. Mission accomplished.

 

Summing Up

I think of AI for Marketing as a handy guidebook into two very different, neighboring lands. For most of us, the gap between the two is an un-navigable chasm. AI for Marketing takes you into each locale and introduces you to the things you really must know about them. It’s a fine introduction not just into AI and Machine Learning but into modern marketing practice as well. Best of all, it guides you across the narrow bridges that connect the two and makes it easier to navigate for yourself.  You couldn’t ask for a wiser, more entertaining guide to walk you around and over that bridge between two utterly dissimilar worlds that grow everyday more necessarily connected.

 

Full Disclosure: I know and like the author – Jim Sterne – of AI for Marketing. Indeed, with Jim the verbs know and like are largely synonymous. Nor will I pretend that this doesn’t impact my thoughts on the work. When you can almost hear someone’s voice as you read their words, it’s bound to impact your enjoyment and interpretation. So absolutely no claim to be unbiased here!

 

An Easy Introduction to In-Store Measurement and Retail Analytics with DM1

My last post made the case that investing in store measurement and location analytics is a good move from a career perspective. The reward? Becoming a leader in a discipline that’s poised to grow dramatically. The risk? Ending up with a skill set that isn’t much in demand. For most people, though, risk/reward is only part of the equation. There are people who will expend the years and the effort to become a lawyer even without liking the law – simply on the basis of its economic return. I’m not a fan of that kind of thinking. To me, it undervalues human time and overvalues the impact of incremental prosperity. So my last and most important argument was simple: in-store measurement and location analytics is fun and interesting.

But there’s not a ton of ways you can figure out if in-store measurement is your cup of tea are there?

So I put together another video using our DM1 platform that’s designed to give folks a quick introduction to basic in-store measurement.

It’s a straightforward, short (3 minute) introduction to basic concepts in store-tracking with DM1 – using just the Store Layout tool.

The video walks through three core tasks for in-store measurement: understanding what customer’s do in-store, evaluating how well the store itself performed, and drilling into at least one aspect of performance drivers with a look at Associate interactions.

The first section walks through a series of basic metrics in store location analytics. Starting with where shoppers went, it shows increasingly sophisticated views that cover what drew shoppers into the store, how much time shoppers spend in different areas, and which parts of the store shoppers engaged with most often:

retail analytics: measuring store efficiency and conversion with DM1

The next section focuses on measures of store efficiency and conversion. It shows how you can track basic conversion metrics, analyze how proximity to the cash-wrap drives impulse conversion, and analyze unsuccessful visits in terms of exit and bounce points.

DM1 Layout Overview Video

Going from what to why is probably the hardest task in behavioral analytics. And in the 3rd section, I do a quick dive into a set of Associate metrics to show how they can help that journey along. Understanding where associates ARE relative to shoppers (this is where the geo-spatial element is critical), when and where Associates create lift, and whether your deployment of Associates is optimized for creating lift can be a powerful part of explaining shopper success.

retail analytics with dm1 - analyzing associate performance, STARs and lift with DM!

The whole video is super-quick (just 3 minutes in total) and unlike most of what I’ve done in the past, it doesn’t require audio. There’s a brief audio introduction (about 15 seconds) but for the rest, the screen annotations should give you a pretty good sense of what’s going on if you prefer to view videos in quiet mode.

I know you’re not going to learn in-store measurement in 3 minutes. And this is just a tiny fraction of the analytic capability in a product like DM1. It’s more of an amuse bouche – a little taste –  to see if you find something enjoyable and interesting.

I’m going to be working through a series of videos intended to serve that purpose (and also provide instructional content for new DM1 users). As part of that, I’m working on a broader overview right now that will show-off more of the tools available. Then I’m going to work on building a library of instructional vids for each part of DM1 – from configuring a store to creating and using metadata (like store events) to a deep-dive into funnel-analytics.

I’d love to hear what you think about this initial effort!

Check it out:

Building a Career Around In-Store Measurement and Location Analytics

Over the nearly two decades I spent in digital analytics, I did a lot of selling. More than I ever wanted to. But during that time, I saw the process of selling digital analytics transformed. When I started, way back in the ‘90s, selling web analytics was evangelical. I had to convince potential clients that the Web mattered. Then I had to convince them that analytics mattered to the Web. If I got that far, I just to convince them that I was the right person to buy analytics from. But since there were only about five other people in the world doing it, that last part wasn’t so hard!

Over time, that changed. By 2005, most companies didn’t need to be convinced that the Web mattered. The role of analytics? That was still a hard sell. But by around 2012, selling digital analytics was no longer evangelical. Everyone accepted that analytics was a necessary part of digital. The only question, really, was how they would provision it.

I didn’t miss the evangelical sell. It’s a hard path. Most people are inherently conservative. Doing new stuff is risky. Most organizations are pretty poor at rewarding risk-taking. It’s great to suggest that analytics is powerful. That it will do the organization good. But for someone to take a risk on a new technology and process, there needs to be real upside. Think of it this way – just as VC’s expect out-sized returns when they invest hard-cash in risky startups, so do decision-makers who are willing to go outside the well-trodden path.

Well, with Digital Mortar, I’m right back in the evangelical world. I have to sell people on the value of in-store customer measurement and analytics – and often I have to do it within environments that are significantly disrupted and challenged. So here’s the question I ask myself – what’s in it for an influencer or a decision-maker?

I think that’s a surprisingly important question and one that doesn’t often get asked (or answered).

If you’re thinking about in-store measurement and analytics, here’s the personal questions I’d be asking if I were you (and my best guess at answers):

1. Is there a future career in this stuff?

There was a time when understanding how to create digital analytics tags was a really critical skill. That day has passed. Tagging is now a commodity skill often handled by offshore teams. In technology and analytics, in-demand skills come and go. And it’s critically important to keep building new skills. But which skills? Because there’s always lots of possible choices and most of them won’t end up being very important.

It’s pretty obvious that I believe location analytics has a big future or else I wouldn’t have started Digital Mortar. Here’s why I think this stuff matters.

I saw how compelling analytics became in the digital world. With increasing competition and interaction between digital and physical experiences, it’s just implausible to believe that we’ll continue in a world where online experiences are deeply quantified and physical experiences are a complete mystery. Every digital trend around customer centricity, experimentation and analytics is in-play in the physical world too – and all of them drive to the need for location analytics.

The thing is, measurement creates its own demand. Because once people understand that you can measure something, they WANT to know.

It will take a while. Change always does. But I have no doubt that in a few years, measurement of the physical customer journey will be well on its way to being the kind of table-stakes must-have that digital analytics is in the web world. That means new roles, new department, new jobs and new opportunities. Which brings me to…

2. Is there a real benefit to being an early adopter?

The people who got into digital analytics early carved out pretty admirable careers. Sure, they were a smart group, but in a new analytics domain, there is a real premium to early adoption. When that field starts to get traction, who gets to speak at Conferences? Who gets to write the books? It’s the early adopters. And if you’re the one speaking at conferences or writing the books, you get real opportunities to build a unique career. Being an early adopter of a technology that pans out is a huge win for your personal brand. It almost guarantees a set of terrific career options: leading a consultancy, having a cush job as an evangelist at a place like Google, getting recruited by a technology unicorn, or managing a large group at a premium company. All good stuff.

And by the way, it’s worth pointing out that this type of measurement isn’t limited to retail. You think resorts, arenas, and complex public spaces don’t need to understand the customer experience in their spaces? Location analytics won’t be in every industry. But it will be every WHERE.

But none of that stuff will happen unless you have some success.

3. Can I be successful right now?

How much success you need is easily exaggerated. Early adopters (and this is a good thing) are like fisherman. We mostly know how to tell a good story. But getting real success IS important. And fortunately, location analytics systems are good enough to do interesting measurement. The capture technologies have plenty of issues, but they work. And a platform like DM1 lets you do A LOT with the data. Best of all, if you’re already experienced with digital analytics, you know a bunch of what’s important about dealing with this data. That makes early adoption a little less frightening and a lot more likely to be successful.

There are real use-cases for this technology. Use-cases that have been hidden by the generally awful analytics capabilities of previously existing systems. This kind of measurement can identify and help solve line and queue management problems, answer questions around store and location design, resolve issues around staffing and associate optimization and feed better forecasting and allocation models, and drive powerful enhancements to customer CRM and personalization efforts.

4. How risky is it?

Middling. This stuff is still pretty new. But it’s starting to mature rapidly. The technology is getting better, the analytics software just got MUCH better, and the needs just keep growing. As with most analytics – the hard part is really organizational. Getting budget, getting authority, driving change – those are always the hardest tasks no matter how challenging things are on the data collection and analytics front. But no one’s ever seen this kind of data before. So the bar is incredibly low. When people have spent years living with hunch, intuition and door-counting as their sole metrics, you don’t have to provide world-beating analytics to look like a star.

5. Is it interesting (because no one wants to spend their life doing boring stuff)?

Yep. This stuff is deeply fascinating. Customer experience has long been one of the most interesting areas in analytics. People are great to study because their behavior is always complex. That makes the analytics a challenge. And because its people and behavior and the real-world, the problem set keeps morphing and changing. You’re not stuck analyzing the same thing for the next ten years.

Even better, identifying problematic customer behaviors is the table-set for actual business change. Once you’ve found a problem, you have to find a way to fix it. So the analytics drives directly into thinking about the business. I like that a lot. It means there’s a purpose to the measurement and the opportunity to brainstorm and design solutions not just analyze problems. If you enjoy doing digital analytics (or have always thought you might), this is an even richer and more complex set of analytic problems.

Yeah. It’s fun.

Which brings me to the bottom line. Risk is risk. A lot of businesses fail. A lot of technologies don’t take off. But I’m pretty confident that in-store journey measurement and location analytics will become a significant discipline in the next few years. If I’m right, there will be real dividends to being an early adopter. Both for the companies that do it and the people who drive it. And along the way there’s some fascinating analytics to be had and a whole bunch of really interesting stuff to learn. That doesn’t seem like such a bad deal.