Simon Francis: is there an ideal way to measure marketing effectiveness?

Simon Francis, CEO of integration consultancy Flock Associates, assesses the various options available for measuring integrated marketing effectiveness – from digital dashboards to agent-based modelling.

Before we look at the various options for measuring marketing effectiveness let’s consider the ‘dream’ scenario so we can compare each of the options against it.

This is to have a holistic measurement system that has a common currency of measurement across all marketing channels for costs, outputs, and outcomes. Outputs can be considered as a measure of scale, eg. impressions, and outcome is the effect of the outputs eg. awareness or clicks.

Think of the London Underground Tube Map (below).

It is a great metaphor for a brand or business, where each of the ‘lines’ represents a different marketing channel. For example the brown line, may be YouTube and the red line, your website; the light blue line, your stores etc. People are on a series of dynamic journeys through your brand on the various lines. At different times they have the opportunity to change at ‘stations’. This is where a consumer, in real life, moves from one marketing channel to another.

The ideal would be to have this map for each of your brands with data on how many people are joining each marketing channel with each marketing intervention (or station), swapping to different marketing channels (at each station), or leaving your brand. You would then need to understand the effect of each channel on actions (purchase, clicks etc.) and attitudes (awareness, preference).

So what are the various measurement options?

Digital dashboards

There are a host of digital dashboards available that allow clients or their agencies to monitor many aspects of their digital performance.

Chartbeat is a good example of the genre. Chartbeat shows a lot of detail about your website, what media has taken to it, and so on.

The problem is with digital dashboards like Chartbeat, Piwik, Mint and KISSMetrics, Woopra, Reinvigorate, Mixpanel and the rest is that they are ..err.. digital – so you cannot link your TV, OOH, retail, vending, PR and so on in comparable forms.

Adobe Web Analytics offers a comprehensive set of site reports, but also links to campaign management, and media mix monitoring tools. As such, it does offer a bit more insight than some of the others tools. The same is true is true of Oracle’s Eloqua.

And, then there is Google Analytics. It is deep and broad and allows you to measure everything digital. It also allows you to monitor attribution.

Most services tell you when someone clicks on an ad, and visits your website, and so that ad gets the click attributed to it. However, a consumer might have seen a social media post just before seeing the ad, and the social media post might have actually triggered the interest in the brand while the ad just provided a call to action. But, using most services you cannot give some of the ‘attribution’ to the social media post, which was actually very effective.

Google Analytics allows you to set up models and ‘rules’ to manage this attribution. There are other services that help model and manage attribution, such as London start-up  Shomei.

Also, Google has created something really neat called ‘Events’ that could really transform cross channel measurement. Rather than explain in words have a look at the video. It is pretty geeky, and one of the presenters is a bit scary, but it’s well worth a look.

Regression Modelling

If you have lots of historical data across your marketing channels, but in inconsistent forms, you may be able to use regression analysis to model your marketing effectiveness. Historically, this route has been used by many sophisticated advertisers and has the huge advantage of being able to look at price and promotional information, and be able to model and predict outcomes.

For those clients with relatively simple marketing ecosystems and categories, and with stable, predictable markets, econometric modelling is a great tool.

There are however some things for an advertiser to consider. Firstly, what resource are you going to use to do the modelling?

In an ideal world you would do the work in-house, using software like MarketingQED. This means that you as a client own your data, the model, and no-one understands the data and decision-making better than you.

However, you may not have the skills in-house and may need a consultancy to help. In any case, it’s useful to consider the different resource options against the following criteria:

*Independence & Objectivity

*Ability to make insights actionable

*Understanding of business, brand, creativity, and other tactics

*Access to data

*Network

*Cost

But the problem with regression-based modelling is that it requires consistent historical data, and relatively stable markets. Imagine you were a VC-backed start-up: you have no historical data, you are creating a new market, so how can you effectively plan and then evaluate your marketing channels and ROMI?

Agent-based modelling.

Agent-based modelling is where an ‘agent’ is created by computer. The agent usually represents an individual consumer. The agent has ‘rules’ ascribed to it that dictate how the agent will behave in different circumstances. Agent-based modelling was used to crack the code of how birds flock, which appeals to us, of course.

Take this example; a car company is launching a new car. A car-buying agent is created. Rules can be ascribed to the agent about what competitor car companies do, what economic situations occur, what life changes might occur (family, house purchase etc.), distribution, behaviour based upon pricing, and so on. Scenarios can be run and assessed and decisions taken accordingly.

Companies like Concentric and others may provide solutions for certain clients/situations. However, as with some regression models, having your precious data outside your company and having to pay by the scenario, may prove a stumbling block or at least a concern.

Independent Common Currencies

One of the biggest challenges of multi-channel marketing effectiveness work is to gain a ‘common currency’ for outcome, across all channels. There is no multi-market industry standard, so primary research is normally required. This can prove expensive if you operate in multiple markets and in multiple categories.

There are, however, research techniques that evaluate each channel and allow cost accounting methods to be applied to evaluate ROI by marketing channel. Amongst these companies Integration IMC have been long established and have a depth of study and experience that is hard to argue with.

One of the major benefits of studies like Integration IMC’s MCA is that they allow comparable studies across channels, markets, and categories which is particularly useful for global best practice transfer.

Whilst research like MCA does not actually link directly to sales, a simple correlation analysis can show the statistical confidence of the predictions, and in many cases the MCA models and market share have proved to be very closely correlated.

So, what is the best way to measure integrated marketing effectiveness? The only answer I can give is “it depends”! There are a host of great options but first you have to identify your needs, skills, and budget.