Theory Meets Reality: Implementing PR & Social
Media Measurement Standards
NJ Social Media Summit Katie Delahaye Paine CEO Paine Publishing April 10, 2015
www.painepublishing.com
@queenofmetrics
Paine Publishing: Providing communications’
professionals the knowledge and information they need to navigate their path to perfect measurement
Newsletters
Training Courses
Consulting painepublishing.com
Katie Delahaye Paine: Helping communications professionals define and measure success for 25 years.
Founder of:
The Delahaye Group
KDPaine & Partners
Paine Publishing
Author of:
Measuring the Networked Non-Profit
Measure What Matters
Measuring Public Relationships [email protected]
Measuring Attribution in an era of all media everywhere
Web Analytics
e-commerce
iCloud &
Google+
Map My Run/Social Sharing
CRM/Convio
Market Factors
Investments are going up:
On average, top marketers expect
to devote 9% of their budgets to
social media spend in 2014, and
16% by 2018.
Return is going down:
ROI & Vanity metrics are out
NPS, Engagement, relationship
building are in
We need a new Attribution Model
ROI
Other Paid
Marketing
Digital/On-line
Media Buy
Print/TV Media
Buy
ROI
Media Relations
/Social Media/PR
Digital/ Online Media
Buy
Print/TV
Media Buy
Media
Relations/
Social
Media/PR
The Barcelona Principles, The Conclave &
Industry Standards
1. Importance of Goal Setting and Measurement
2. Measuring the Effect on Outcomes is Preferred to Measuring
Outputs
3. The Effect on Business Results Can and Should Be Measured
Where Possible
4. Media Measurement Requires Quantity and Quality
5. Earned Media Value/AVEs are not the value of Public Relations
6. Social Media Can and Should be Measured
7. Transparency and Replicability are Paramount to Sound
Measurement
All standards are available on
http://www.painepublishing.com/st
andards-central
The Conclave
Standards have been published for:
Content Sourcing & Methods 1
Reach and Impressions 2
Engagement 3
Influence & Relevance 4
Opinion & Advocacy 5
Impact & Value 6
4/13/2015
The Transparency Table:
#2: Standards for Reach & Impressions
Reach: the scope or range of distribution that a given communication product has in a
targeted audience group; broadcasting, the net unduplicated (also called “deduplicated”)
radio or TV audience for programs or commercials as measured for a specific time
period.
Impressions: Number of people who might have had the opportunity to be exposed to a
story that has appeared in the media (aka opportunity to see (OTS))
Impressions do not equal awareness. Awareness needs to be measured using other
research tools. Impressions are indicative of the OTS. Consider OTS as an alternative
nomenclature to better clarify what impressions really means – [the] potential to see/read.
Multipliers should never be used
OTS must be specific to a particular channel – i.e. For Twitter OTS is the number of first
line followers. For Facebook it is the number of fans to a page.
Eyeball counting
HITS Outcomes
MSM Online Social Media
Impressions are not awareness. Where’s the “So What”?
11
#3: Standards for Engagement
Engagement is an action that happens in response to content – i.e. when someone
engages with you. (not “about you”)
Engagement manifests differently by channel, but is typically measurable based on effort
required and how it is shared with others.
Engagement can be desirable or undesirable
Any measure of engagement must be tied to the goals and objectives for your program
Engagement occurs both off and online, and both must be considered if you intend to
integrate your metrics with other marketing or communications efforts.
Engagement includes such actions as likes, +1, shares, votes, comments, links, links,
retweets, Facebook's “Talking about you” etc.
Engagement actions should be counted by % of the audience that responds.
Likes Are Not Engagement
13
Advocacy Commitment Trial/Consideration
Followers Likes Impressions
4/13/2015
968 1027
914
208
297 261 269
215 222 145
328
204
510
188 222 234
41 71 50 36 25 56 30 71 45
107
Share & Likes Over Time
Average of PostLikes Average of PShares
GWA’s, Photos Drive Shares & Likes
Mia Farrow
David Beckham & Dierks Bentley
Video
• Influence happens when you are persuaded to
change behavior or opinion that would
otherwise not have changed.
• Must include:
• Reach
• Engagement around individual
• Relevance to topic
• Frequency of posts around the topic
• Audience impact as measured by the
ability to get the target audience to change
behavior or opinion.
• If an individual scores a 0 on one element, they
have no influence
• Influence cannot be expressed in a single score
or algorithm
#4: Influence & Relevance Image courtesy www.getlittlebird.com
#5 Standards for Measuring Tone/Sentiment
Sentiment is overrated
Messaging and Positioning &
Recommendations are more useful
Whatever process is defined and
applied, it must be used
consistently throughout any
analysis.
Validity testing is critical
Comparison of Tonality Between Vendors
33.12%
58.00%
7.84%
11.95%
26.53%
13.00%
0.32%
9.46%
40.35%
30.00%
92.11%
78.00%
SDL
Beyond
NetBase
UberVu
% positive %negative %neutral
Impact & Value
ROI is: Cost savings
Greater efficiency
Lower legal costs
Faster time to implementation
Lower cost per impact vs. other marketing channels
Lower cost per message exposure vs. other channels
ROI is not: Impressions
Earned media value or AVE
Number of shares, views or likes
The biggest challenges
• Getting from personal justification to a single source of
truth
• Finding valid, relevant data
4
What we learned
1.Consensus on how PR/SM contributes to
the business is critical
2.Once you have that defining success &
metrics is easy
3.Tracking down the data and climbing in
and out of silos is hard
4.Produce a bespoke “Standards
Document” telling everyone, including
agencies and foreign offices how to report
is easy, getting it adopted is hardest of all
5
Custom Standards Standards for Media Reporting
Web Analytics
5
-0.40%
-0.20%
0.00%
0.20%
0.40%
0.60%
0.80%
1.00%
1.20%
1.40%
8500
8600
8700
8800
8900
9000
9100
9200
9300
9400
9500
5/1/2014 5/11/2014 5/21/2014 5/31/2014 6/10/2014 6/20/2014 6/30/2014
Follower and Follower Growth Since May 1
Followers Follower Growth
What we had: “XYZ Tweets Generated the Greatest Increase in
Followers”
#Tweet 5
Tweet 3
Tweet 2 Tweet 1
Tweet 4
Social Media Engagement Index
Action Score
Like/Follow/Opens/+1 .5
Favorite or Opens or Views 1
Comment 1.5
Share content 2
Signs up to receive email or other owned content 2
Contains a quote from one of our content experts 1.5
Shares a link to an owned site 2.5
Total 10
24
Implementation Steps
1.Pull together a dashboard with the best
data you can find and point out the gaps
2.Put on your bullet proof vest & explain to
EVERYONE
3.Make changes, make more changes
4.Produce the first report
5.Repeat Step 2
6.Draw conclusions, make decisions.
7.Take it to the board
5
The biggest challenges
• Getting from this to this
• Finding valid, relevant data
4
• Geographies vs Languages • Lack of consistent competitors list • Different definitions of Products/Brands Covered • Time frames • Media Types • Search strings and filters
3
Testing validity of criteria
The Theory: 6 steps to Standards-
Compliant measurement
Step 1: Define your goal(s).
What outcomes is this strategy or tactic going to achieve?
What are your measurable objectives?
Step 2: Define the parameters
Who are you are trying to reach? How do your efforts
connect with those audiences to achieve the goal.
Step 3: Define your benchmarks.
Who or what are you going to compare your results to?
Step 4: Define your metrics.
What are the indicators to judge your progress?
Step 5: Select your data collection tool(s).
Step 6: Analyze your data.
Turn it into action, measure again
26
Six Steps
to Success
1
2
3
4
5
6
Step 1: Define the goals
What return is expected? – Define in terms of the
mission.
Define your champagne moment. If you are
celebrating complete 100% success a year from
now, what is different about the organization?
27
Why do we communicate?
4/13/2015 28
Outtakes
(Intermediary Effects)
• Awareness
• Knowledge/Education
• Understanding
Outcomes
(Target Audience Action)
• Revenue
• Leads
• Engagement
• Advocacy
Activities How does what you do
contribute to the bottom line?
Dashboard Framework
Support the business
% increase in engagement which drives…
% increase in video starts, % increase in ratings, market share
Increase engagement
1. Increase in % of audience that is engaged
2. Engagement Index:
• Shares
• Comments
• Favorites
• Repeats vs. uniques
• Time on site
• Pages per visit
Definitions of “Success” Workshop defined the criteria
All criteria linked back to the goals:
High Quality Media Coverage
Intent to visit
Improved reputation
Visits/Non-gaming revenue
Step 2: Understand the parameters.
What management’s priorities?
Who are you are trying to reach?
How do your efforts connect with
those audiences to achieve the
goal?
What influences their decisions?
What’s important to them?
What makes them act?
31
Goal: Get the cat to stop howling
Options: Local? Cheap? Convenient?
Strategy: Buy cat food
What are you measuring?
Paid – Google Adwords, Facebook
Ads, popups, banners etc.
Owned – #njsms15
@queenofmetrics
http://www.pinterest.com/wsbtv/co
x/
Earned -- Everything else
(Including Shared)
32
Digital/Advertising/Marketig
Social Media Team
PR
Who’s got the data?
Step 5: Pick the right measurement tools
If you want to measure messaging, positioning,
themes, sentiment:
Content analysis
If you want to measure awareness, perception,
relationships, preference:
Survey research
If you want to measure engagement, action,
purchase:
Web analytics
If you want predictions and correlations
you need two out of three
Step 5: Selecting a measurement tool
34
Objective KBI Tool
Increase message
communications
Increase percent of items containing one or
more messages
CyberAlert, Prime Research
Increase
awareness/preference
% of audience preferring your brand
to the competition
Survey Monkey, Survata
Engage
marketplace
% increase in engagement on website and/or
social sites
Simply Measured, Unmetric,
Google Analytics, Site Catalyst,
Network Analysis
Become a thought
leader on a specific
issue/market/topic
% increase in desirable share of voice on
issue/market/top
Little Bird, Trackr, CyberAlert
Step 6: Be Data Informed, not Data Driven
Rank order results from worst to best
Ask “So What?” at least three times
Put your data into an overall framework consistent with C-Suite expectations
Find your “Data Geek”
Compare to last month, last quarter, 13-month average
36 Page 36
The ROI of Emily
37
Red line
indicates media
impressions
35,152,789 OTS
6,253,852 OTS
So What = Revenue
38
Without ACA events, OCS Scores for Atlantic City would
have been significantly lower
4
3.25
2.75 2.99
3.65
2.96 3.36 3.24
2.34 2.37 2.43
1.30
-1.24
0.37 -0.05
0.28 0.28
-1.56
4.91
3.92
2.99
3.58
4.14 4.1 4.27 4.12 4.29
2.78 2.56
1.53
-0.29
0.61 0.20
1.44 1.77
0.63
-2
-1
0
1
2
3
4
5
6
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June
Ave
rage
OC
S Sc
ore
Average OCS Score Over Time
Atlantic City OCS without ACA Atlantic City OCS with ACA
The red line represents coverage of Atlantic City minus all mentions of ACA and its programs
When ACA programs received media coverage,
engagement, preference and traffic followed
0
50
100
150
200
250
300
0
20000
40000
60000
80000
100000
120000
140000
Relationship between ACA Program Mentions and Site Visits
Site Visits Program Mentions
Photo Event
Best practices: Resource use vs Results High Engagement i.e. Video Starts
Resource Use
Low
Hig
h
Med
ium
Ver
y h
igh
Total Volume of Coverage
Ver
y H
igh
Med
ium
Hig
h
Low
High Resources
Low Engagement
Low Resources
Webinar
Status update
Link
Ultimate Road Trip
Google + Chat
Media Day
Corporate Video
10 Ways to find the Money for Measurement
1. Don’t call it measurement – It’s Research
2. Consolidate - Social? IR? Agencies? Research?
3. Crowdsource – Who else has tools or surveys?
4. Spread the cost over 2 fiscal years
5. Reach out to universities
6. Take advantage of free stuff
7. Monitor only what matters
8. Narrow your Top Tier media list to what you CAN afford
9. Google analytics
10. Facebook Insights
11. Twitter Analytics
Remember These Points
It’s not about the media, it’s about the business and the customer 1
It’s not about Big Data, but about how you use it. 2
You need to be data informed, not data-driven 3
It’s not how loud you’re shouting it’s about relationships. 4
Standards are a reality not an excuse to hide behind 5
Thank You!
For more information on measurement, read my blog:
http://www.painepublishing.com/blog
For a copy of this presentation or to subscribe to our newsletter, give me
your card or email me at [email protected]
Follow me on Twitter: @queenofmetrics
Friend me on Facebook: Katie Paine
Or call me at 1-603-682-0735
44