Advertising Wisely: The Art of Dodging Confirmation Bias 

No more vanity metrics. It’s time to accurately measure ad success. Audience Group co-founder and Managing Director, Tom Evans provides a playbook for avoiding confirmation bias in advertising measurement.

They say: ‘Never assume’. You know why they say that. Well here’s another reason: it can cause you to make unwise media-buying decisions and negatively impact your advertising campaigns. 

Confirmation bias is the tendency to interpret information in ways that confirm one’s existing beliefs. It can be a comfort, but it’s a trap. Today we’re taking a sobering look at confirmation bias in advertising measurement, and how to use evidence as your best defence against this common pitfall. You’ll only know which of your ads actually work if you accurately measure ad success.

“Evidence is your best defence against confirmation bias.”

– Tom Evans, Managing Director, Audience Group

Caution: Advertising Measurement Mistakes Ahead

Imagine this scenario: ‘Company A’ ran a month-long radio advertising campaign in May. During that time, sales numbers soared compared to the previous month and the same period last year. Company A is pleased, convinced that its radio campaign is the secret sauce. 

But let’s pause for a moment. Where’s the concrete evidence? Just because sales went up in May doesn’t automatically mean the radio campaign is the hero. It might be a coincidence, or there could be other factors at play. What if the overall monthly sales represent an improvement, but the sales peak occurred only in the first two weeks of May and then dropped off? What if the outstanding month was actually a result of a successful TV campaign in April, with the radio campaign simply amplifying the effect?  

Unless the original campaign was designed with metrics to accurately measure ad success and show the direct correlation, basing budgetary decisions on this assumption and building future campaign tactics around it – such as increasing Company A’s budget for radio advertising – would be a gamble. 

Company A might find out, after a number of unsuccessful radio campaigns and wasted budget, that it wasn’t the radio advertising after all – at least, not on its own, or that it isn’t a repeatable success.  

Evidence: The Countermeasure to Confirmation Bias 

Thankfully, we live in an age of abundant data. It’s high time to capitalise on this wealth of information and protect your advertising investment decisions against confirmation bias. 

Here’s your playbook: 

• Begin with the Intention to Avoid Falling for Confirmation Bias
  Measurement, attribution and calculation of specific ROI can be challenging but well worth doing well. Set the team’s over-arching mindset to avoid confirmation bias and spread that directive to your agency partners. Your advertising strategy needs to be assessed with objectivity, leveraging the right data sets and digging deep to uncover uncomfortable or inconvenient truths.   
• Look at the Bigger Picture
  Resist the urge to focus solely on short-term successes. Sometimes, the real measure of success lies in the long run. Look beyond monthly or quarterly data and consider the larger context. In the example above, maybe the radio campaign boosted sales temporarily, but it had no impact on Company A’s market share – an essential distinction for future planning. 
• Seek Clear Evidence
  The key is to seek clear evidence. What does the data unequivocally tell you? Is the outcome something you can expect to happen consistently, or was it just a one-time event? By striving for evidence-based advertising, you can anticipate what your business should realistically expect.
• Reimagine Campaign Tracking
  That likely means you’re going to need to change the way you and your agency partners track and evaluate advertising campaigns. Campaigns should be meticulously designed with metrics that unearth precise connections to avoid superficial correlations. Tying campaign objectives to measurements that truly matter to your specific business will help to make these connections. Data analytics and data science skills, required. 

Pursue Measurement Truths, Not Assumptions

You’re investing advertising dollars to drive sales and business growth, so understanding the effectiveness of each campaign is crucial. However, it’s important to recognise that campaign success isn’t always as clear-cut as a spike in profits. Often, it can be difficult to determine whether a specific campaign was truly responsible for a boost in sales. This is where confirmation bias can come into play. 

By diving deeper into the data in search of evidence, it’s possible to avoid this common pitfall and gain a more realistic understanding of what drives success. 

Contact us if you’re interested in evidence-based advertising. 

Did you know “confirmation bias” was first coined in the 60s?