The Dunning-Kruger Effect

In the realm of faking-it-till-you-make-it, the undisputed heavyweight champion is Malcolm Gladwell (pictured above, right of my boss, Hartland Ross). While undoubtedly an influencer in business, science and psychology, Malcolm is merely an author with a history degree. Don’t get me wrong, I’ve enjoyed his page-turners as much as the average adult stuck in an airport. Hell, I’ve been known to throw out an anecdote or two from his bestsellers ‘Blink’, ‘Outliers’, and ‘The Tipping Point’ in conversation to sound smarter than I am. Have you ever hear about that daycare that fined parents for being late to pick up their kids after school?

Inspired by my fellow Canadian Mr. Gladwell, today’s blog entry muses over a psychology concept I’m woefully unqualified to speak to: The Dunning-Kruger Effect. But at least I’ll be look at it through the lens of marketing analytics, something I do know a little bit about.

What is the Dunning-Kruger Effect?

The Dunning-Kruger Effect is best explained visually:

Source: William Poundstone

I bet this curve is relatable for a lot of readers. I’m sure we’ve all gone through a similar progression while learning a new discipline. I am reminded of learning to drive, when emboldened by naivety, I had my share of reckless and dangerous moments as a teenager (something the stats show is unfortunately common). This was largely due to high confidence and low competence as characterized by the left-most peak of the curve. But with experience, I’ve moved into the middle of the curve becoming a cautious and defensive driver. And that’s where I’ll likely stay; I doubt I’ll ever develop enough driving expertise to navigate reckless and dangerous situations with confidence again.

The Dunning-Kruger Effect applies to confidence and competence with marketing analytics too.

Marketing Analytics Beginners

When you first start learning about marketing analytics, it’s rather impressive how many different metrics are available. Google Analytics alone has many dozen default reports to peruse. You can sift through details on everything from visitors’ genders, to the model of cell phones they use, to the municipality they’re located in. So when the beginner first gets exposed to all this data, they take its accuracy for granted, and put a lot of stock in the metrics as they are stated.

A challenge that a lot of marketers face, is that the person they are accountable to, be it a boss or a client, is often a relative beginner with marketing analytics. Especially in the case of upper management, this person may have a proclivity to understand business through numbers. So they tend to want to understand marketing through numbers too. They especially want to see an ROI calculation. This makes the marketer tend to prioritize marketing tactics whose returns are measureable more so than tactics that are just plain good.

To beginners, marketing measurement is precise, accurate, and all-encompassing. Revenue gets tracked, tagged to specific marketing initiatives, and ultimately an ROI is calculated down to the penny. If only it were that easy.

Marketing Analytics Intermediates

As the beginner gains experience, they get exposed to the many shortcomings of marketing analytics.

Often the name of the metric gives an unrealistic expectation about what’s actually being measured. Take “time-on-site” for example. That sounds pretty self-explanatory eh? But consider how it’s actually calculated. Google measures the difference in time between when you arrived on a site and the last action you performed on the site (like clicking). So if you don’t click anything, your session gets recorded as zero seconds, even if you’d spent minutes reading the page. Or say if you go to the bathroom and come back 10 minutes later like most office workers, Google assumes you were reading the whole time.¹ So “time-on-site” could be more accurately described as “time before last action assuming no-one went to the lou”.

Another shortcoming of marketing analytics is the lack of a universal attribution model. For example, I am generally a proponent of using retargeting to serve ads to visitors to your website who’ve left without purchasing/converting. When done well, retargeting has a significant impact in terms of improving conversion rates for B2B brands, and can be implemented in a way that minimizes the creepy factor. But retargeting providers are a bit sneaky with how they attribute conversions. At first glance, what you see is a straight-up “Conversion” number. But this is a sum of both “click-thru” and “view-thru” conversions. A click-thru conversion is when someone clicks an ad, arrives on a site, and converts in that same session. A view-thru conversion is when someone is shown an ad and then converts at some point later on, perhaps days or weeks after seeing the ad. Most advertisers place a greater value on click-thru conversions than view-thru conversions. Retargeting providers benefit from beginner advertisers likely not knowing the difference.

Retargeting dashboard showing nineteen conversions (1 CTC and 18 VTC).

Some of these types of issues like calculating time-on-site and attributing conversions may seem like nit picking, but they can have  significant impacts especially for SMBs. When you don’t have a lot of website traffic, a handful of sessions with bad data one way or another can throw the totals off in a major way. An example of this is when your own staff’s sessions are recorded. I once had a client with a low volume of website traffic inquiring about a retargeting campaign. They mentioned their average time on site, which was something like four minutes, which would’ve been excellent if accurate. It would’ve been indicative of high quality traffic, which made retargeting that audience an attractive tactic. But upon further inspection, I noticed there were a bunch of sessions being tallied originating from a password protected backend system only staff could access. These sessions were quite lengthy, ranging from 30 to 60+ minutes. When these staff sessions were stripped away, the average time-on-site dropped well below a minute. Suddenly the client’s priorities shifted from spending on a retargeting campaign to a web dev engagement to improve site engagement.

B2B companies in particular face challenges with marketing analytics that are more significant than what B2C companies face. With B2B sales cycles there tends to be multiple decision makers, using multiple devices, in multiple locations, and the sales cycle can be multiple months or longer. The number of permutations really starts getting up there, and it’s impossible to encapsulate everything. Sure, there are MarTech tools that can help capture a lot of data by using tactics like cookies, fingerprinting, and by referencing vast databases on B2B decision makers. And eBridge can help you with an engagement like that if you are interested. It’s never perfect though. You’ll always be dealing with incomplete information, which isn’t exactly confidence inspiring. Intermediate marketers know they are never going to be able to calculate an accurate ROI, regardless of the expectations of their boss or client.

Marketing Analytics Experts

True marketing analytics experts are hard to come by. There are lots of charlatans out there over-promising and under-delivering. If you’ve taken the time to read this blog, sorry, you probably don’t fall into the Experts category. Most people, myself included, should instead feel humbled by the impossible complexities of the internet, the vast amounts of data available from tracking online activity, and the countless ways that data can be analysed and the resulting insights applied. A true expert is able to find ways to mitigate the issues that beginners overlook and intermediates stress over. Experts can make sense of imperfect data. They can strategize about ways to gain new insights. Still they may not be able to calculate an accurate ROI, but they’ll have a repertoire of tools to tease useful information out for the given situation.

eBridge’s analytics expert is Frank Siano. Over the years, Frank has helped countless companies set-up their analytics to track important KPIs, and has experienced the full gamut from simple integrations to very complex. I asked Frank for his impressions about the state of marketing analytics:

Q1: How accurate are companies’ marketing analytics generally speaking?

A1: Typically not very accurate at all. The tracking systems have become more complex in many ways, and this has caused tracking to become easier in some ways, but much harder in others.

Q2: How do you feel about companies who desire to measure ROI as a primary KPI?

A2: I do think it’s wise to try to measure ROI on a few different levels. The most basic level of tracking is crucial, while the most advanced level is reserved only for companies who have enough traffic to gain meaningful results which justify the investment. Far too often, companies desire complex analytics implementations, only to find out that they don’t have enough traffic to support it, or they have not yet gotten the basics down first. It’s important to understand what the company wants to track exactly, but it’s often different than what should or can be tracked. Determining what helps spend budget effectively to acquire customers can be measured at so many levels, not just ROI. The basic levels should always be covered first, and then more advanced reporting incorporated out of the need to be more efficient with budgets as a whole. The goal is to first track ALL traffic, and then figure out how to slice and dice it later.

Q3: Any thoughts or predictions on the future of marketing analytics?

A3: Machine learning, artificial intelligence, and predictive analytics will become more helpful in making assumptions and estimated guesses with data. But too much hype around AI can easily lead you to run tests or do things with your budget that may not always be the best path. That’s why it’s important to track everything and constantly be measuring the numbers to make sure you are using your budget most wisely.

Forecast

Reading the tea leaves about how things are progressing regarding data privacy, while in the future we will have better data analysis techniques available, it seems there will be less information for marketers to work with. There’s an increasing concern about personal privacy, and we’ve seen legislation like Europe’s GDPR trying to curb which data is being collected and stored. We’ve also seen the rising adoption of technologies like encryption, ad blockers and VPNs, all of which limit the ability to effectively track the results of marketing campaigns. Recently, Mozilla announced that future releases of the Firefox web browser will block all tracking by default, a move that likely will be adopted by other browsers too eventually. These are all strong indicators that ethical marketers will have less analytics data than they did in the recent past.

Learning From The Past

Marketers indeed have been spoiled with arguably too much data in recent years. But it wasn’t that long ago when companies relying on what we now think of as “traditional” marketing tactics had very limited metrics. Yet B2B companies still advertised to promote awareness, keep churn at bay, and stake their claim to a piece of an expanding pie.

Source: Mike Wong, Pinterest

I remember as a kid, a couple times we participated in listenership surveys for radio. We carried around booklets, and whenever we listened to the radio, we wrote down which stations and the listening duration. You can imagine skipping through the channels and whether a child would be diligent enough to record that activity accurately. The radio stations would tally these listener surveys, extrapolate the results to the entire local population, and use that as their estimated listenership. So essentially they were relying on childrens’ data entry skills for their marketing metrics. Yet companies still advertised on radio and found ways to get the most important conversion data, typically using a custom phone number or referral code. Both of those these techniques can be utilized with digital marketing too, notably in podcast advertising, which is one of the trendiest mediums to advertise on for B2B brands.² These sorts of imprecise marketing measurements from the past will likely become more popular again. Sure, this way you don’t get to automatically capture the person’s gender, ethnicity, affinity for certain topics, or any of the other goodies that modern analytics offer. Sure, it requires a little bit more risk tolerance and comfortability with the unknown – likely more than your boss or client desires from their marketing. Sure, you may not be able to accurately measure that coveted positive ROI without cobbling together a bunch of loose assumptions. But you can still get a pretty good idea of a campaign’s effectiveness using traditional methods without being creepy. And unless you’re an absolute expert, you’ll have to take solace being only somewhat confident that your marketing is generating a positive return.

“Ten thousand hours is the magic number of greatness.” – Malcolm Gladwell

In his 3^rd book ‘Outliers’, Malcolm Gladwell famously alluded that it takes 10,000 hours to become an expert at something, which is roughly equivalent to 10 years’ worth of hard practice. We aren’t going to question the validity of this oft-repeated idea, instead, I’ll conclude by simply asking, do you want to spend 10,000 hours becoming a marketing analytics expert? eBridge’s expert Frank has already done his time, and he’d be happy to talk to you about solving your marketing analytics challenges.

Call Frank at 888 436.5262 or reach out to Frank through our site!

References:

¹ https://support.google.com/analytics/answer/1006253?hl=en
² https://martechtoday.com/listen-state-podcasting-marketing-207269