How to solve a problem like ROI

To start, let’s go back to first principles.

The marketing funnel attempts to represent the consumer’s journey, from marketing activity and brand or product awareness, through to purchase and, hopefully, ongoing loyalty and product advocacy - something like this (with credit to Adam Cohen’s blog).

The key to understanding marketing effectiveness is to measure the links between each of these dimensions.

In an ideal world, your company would have a trove of weekly brand tracking scores for awareness and consideration, as well as robust weekly sales data and customer survey data (for loyalty). Hey, they’d even have an experienced WOM practitioner to measure their advocacy.

With all this sweet data, it would be a simple matter of finding a technically proficient econometrician to model the mathematical relationships between each dimension, all the way from marketing investment to sales and beyond. Voila, marketing effectiveness measured.

Data matters

One problem is information. In a world replete with it, in one form or another, there is sometimes not enough of quality when it comes to marketing (quality being defined as data that is consistent in definition, unbroken in time sequence, and sufficiently long in time series).

Measuring marketing well is an evolutionary process, where the first step is, often, capturing the right information properly.

Another problem is method. It is widely recognised that econometric modelling (especially of the marketing mix variety) is one of the most sophisticated ways of measuring return on marketing investment, and it is - in the right hands.

A science – and an art

It is not just a case of rubbish data in, rubbish results out. Even with solid data, modelling is a practice that requires much skill and experience to build a robust model and tease out actionable marketing applications. Don’t believe anyone who tells you marketing mix modelling (or MMM) is purely a science, it requires measures of art and flair and creative spark, too. The same brush can produce a Rembrandt or a  Picasso, depending on the hand that’s holding it.

When marketing mix modelling works, it is probably the only way of determining marketing ROI with any level of objective assurance. Most importantly, it doesn’t just calculate average ROI but incremental or marginal ROI – that is, the return (whether that be in terms of sales units or revenue or awareness points) for each dollar spent. This distinction is important because the ROI for marketing channels change for different levels of spend. What is the most efficient marketing mix for the first one million dollars, is not necessarily the most efficient mix for the next million dollars.

When all else fails
 

There are alternative ways to measure marketing effectiveness, some more effective than others.

• Brand and ad tracking. Using surveys to determine the impact of marketing on consumers in terms of longer term brand measures or the shorter term impact of a particular campaign. The major flaw with this approach is there is no link from brand and ad scores to a financial outcome. Advertising may increase awareness but we have no idea of how this concretely affects sales.

• Direct response tracking (offline media). Using a unique phone number of URL for each media channel means you can see how many people called or visited that number or URL. If you then divide the cost of advertising by calls or visits, you can work out the cost per call or visit and have some sort of way of measuring the relative effectiveness of each media channel. Great. But how does online advertising impact this equation? Is a site visit worth the same as a call? How do calls and visits relate to sales? What about the interactive effect of media channels, when cost per call or click is only picking up the last action of the interaction? What about marketing outside media? Direct response is a nice try but raises more questions than it answers.

• Last click allocation (online media). Suffice to say it suffers similar drawbacks to its direct response sibling above, though exuding the pretense of greater rigour because, in some cases, there is a link made to sales.

• Structured judgement. Building a systematic, structured process to leverage the knowledge within any organisation, to determine marketing ROI. A competition was run in Times Square NYC for passers-by to guess how many jelly beans there were inside a car. No one was remotely close but when the mean was taken of all responses, it was only one away from the correct number. In the effectiveness context, you can use your crowd of talented people to ‘guess’ their way to marketing ROI, with adequate structure to ensure some sort of rigour, in terms of process and reasonableness. This method may seem flaky but it can make sense in marketing, where data can be scarce and the subjective nature of consumer whimsy not prone to data capture anyway.

In conclusion, there is no one way to accurately measure marketing effectiveness with absolute certainty 100 per cent of the time, but there are ways to improve your current attempts to measure effectiveness, if attempts are being made at all. In short, there is plenty of upside. Don’t reel back at the intellectual uncertainty, embrace it, and then use your own common sense, to make greater sense, of it all.