How do the changes in Barcelona Principles 3.0 affect your measurement program, Impact's Durgesh Garg explains

Undoubtedly, measurement remains an important aspect of the communications stream. AMEC has been carrying out a critical role in spreading awareness about measurement for several years. In 2010, AMEC announced path-breaking Barcelona Declaration of Research Principles at its annual summit in Barcelona, Spain. Since then, these principles have been revised with the evolving needs of the industry. 

Recently, AMEC announced Barcelona Principles 3.0 during its Annual Summit 2020 which, for the first time, was successfully held online due to COVID19 pandemic.

In 2015, when AMEC announced Barcelona Principles 2.0, I wrote about what had changed and how would it impact us as measurement professionals. Now, in 2020, when AMEC announced Barcelona Principles 3.0, I decided write and understand the changes and implications.

To start with, I noticed a significant change (which I completely agree with) in the latest Barcelona Principles. The principles have again, and rightly so, put equal emphasis on outputs with Principle No. 2 declaring “Measurement and Evaluation Should Identify Outputs, Outcomes, and Potential Impact”. So far, Barcelona Principles advocated measuring the effects of outcomes over measuring outputs. Now, Barcelona Principles 3.0 have placed equal emphasis on "Output, Outcome and Potential Impact". Outputs do perform an important role in shaping the outcomes and potential impact. Therefore, holistic evaluation of outputs and outcomes across the audience journey is the key to successful measurement and evaluation.

Let’s understand it with a basic example of a new mobile handset launch by Company X. The communications’ team’s immediate objective would be to ensure good and maximum coverage, in the target media. They would aim at:

1. OUTPUTS: Higher number of media stories that talk about the product and its features positively within the target media.
2. OUTCOMES: Target Audience is aware of the new product, understands how it is better than competition and is willing to consider the phone for purchase.
3. POTENTIAL IMPACT: Phone sells well.

Success of the communications team’s immediate objective could be measured by analyzing the media stories (Output) on the new product across target media. This, in the short run, could help understand gaps and misses and do relevant course correction for desired result (Impact), if required. Therefore, measuring outputs is as important as measuring the outcomes or potential impact. 

Please note that output does not mean looking at AVEs. AVEs are a complete no-no as discussed in this article below.

I also observed increased emphasis on setting measurable goals and making them an "Absolute Prerequisite to Communication Planning, Measurement, and Evaluation" (Principle No. 1). Any measurement program can only be successful if it starts with pre-defined measurable goals. I have seen several clients reaching out for undertaking measurement services. However, when we enquire about their objectives and how do they plan to use measurement and evaluation in their overall communication program, 9 out of 10 clients struggle to clearly articulate their objectives or what would success mean to them.

AMEC has also broadened the scope of Principle No. 3 (Outcomes and Impact Should Be Identified for Stakeholders, Society, & the Organization). The revised principle takes it to the subsequent level, from the previous version, by adding the effect on stakeholders and society and not only on organizations. 

It highlights the need to think beyond sales and services provided and to look at impact on other stakeholders like employees (retention, satisfaction, etc.), influencers, policymakers, customers (change in behaviours, attitudes, etc.). It is important to comprehend how communication efforts affect society and organization and create a measurement plan around the same.

In view of increasing avenues for communication, revised principle no. 6 asserts the importance of including all the channels in the measurement program compared to merely social media in the previous version. The principle says, “Holistic Communication Measurement and Evaluation Includes All Relevant Online and Offline Channels.” Analysis should be deployed by collecting data and analyzing performance across all communication channels. Therefore, as a communication professional, we need to ensure that our measurement program includes the following:

• Online and offline earned media content
• Owned social channels (Twitter, Facebook, Instagram, YouTube)
• Owned websites and microsites
• Organic and paid search
• Paid advertising (programmatic, display)
• Surveys, interviews, bulletin boards
• Events
• Influencers (Paid and earned)
• Email Marketing
• Ecommerce, Sales and CRM

The goals and outcomes should be consistently defined across all these channels.

I noticed no, or minor, changes in rest of the principles, though they remain equally significant and need to be followed for a successful measurement program:

Principle No. 4

Communication Measurement and Evaluation Should Include Both Qualitative and Quantitative Analysis

Both quantitative and qualitative analysis are necessary and can be employed depending on the objective of measurement and evaluation program. The AMEC Valid Matrix Framework captures its essence beautifully. During the awareness stage, only quantitative measures may be enough like number of articles, impressions, shares and likes. As we move further in our communication journey qualitative analysis may occupy center stage by looking at messaging and sentiment.

Principle No. 5

AVEs are Not the Value of Communication

Enough has been said and discussed about AVEs not being the value of communication. Its usages have certainly come down over years, but we nevertheless come across clients using AVEs to measure their communication programs. The rational is my management understands ROI only in Dollars and if not AVEs then what? They even use multipliers to increase this AVE value and dub it PR value since “it is earned media and thus has more value than paid media.” My advice is, please go through this article on AMEC’s website 22 reasons to say no to AVEs.

Principle No. 7

Communication Measurement and Evaluation Are Rooted in Integrity and Transparency to Drive Learning and Insights

Any measurement program can thrive if it is founded on integrity, transparency, openness and ethics. For success of the program, it is critical everything is clearly documented be it data sources, analysis methodology as well as any relevant variable that could affect results. 

To read more about the Barcelona Principles 3.0, please visit https://amecorg.com/resources/barcelona-principles-3-0/

Durgesh Garg has over 20 years’ experience in Media Measurement and currently works with Impact Research & Measurement Pvt. Ltd. as VP - Operations. He also holds an International Certificate in Measurement and Evaluation from AMEC. The views expressed here are personal.