Theory Meets Reality: Implementing PR & Social

Media Measurement Standards

NJ Social Media Summit Katie Delahaye Paine CEO Paine Publishing April 10, 2015

www.painepublishing.com

@queenofmetrics

measurementqueen@gmail.com

Paine Publishing: Providing communications’

professionals the knowledge and information they need to navigate their path to perfect measurement

Newsletters

Training Courses

Consulting painepublishing.com

Katie Delahaye Paine: Helping communications professionals define and measure success for 25 years.

Founder of:

The Delahaye Group

KDPaine & Partners

Paine Publishing

Author of:

Measuring the Networked Non-Profit

Measure What Matters

Measuring Public Relationships measurementqueen@gmail.com

Measuring Attribution in an era of all media everywhere

Web Analytics

e-commerce

iCloud &

Google+

Map My Run/Social Sharing

CRM/Convio

Market Factors

Investments are going up:

On average, top marketers expect

to devote 9% of their budgets to

social media spend in 2014, and

16% by 2018.

Return is going down:

ROI & Vanity metrics are out

NPS, Engagement, relationship

building are in

We need a new Attribution Model

ROI

Other Paid

Marketing

Digital/On-line

Media Buy

Print/TV Media

Buy

ROI

Media Relations

/Social Media/PR

Digital/ Online Media

Buy

Print/TV

Media Buy

Media

Relations/

Social

Media/PR

The Barcelona Principles, The Conclave &

Industry Standards

1. Importance of Goal Setting and Measurement

2. Measuring the Effect on Outcomes is Preferred to Measuring

Outputs

3. The Effect on Business Results Can and Should Be Measured

Where Possible

4. Media Measurement Requires Quantity and Quality

5. Earned Media Value/AVEs are not the value of Public Relations

6. Social Media Can and Should be Measured

7. Transparency and Replicability are Paramount to Sound

Measurement

All standards are available on

http://www.painepublishing.com/st

andards-central

The Conclave

Standards have been published for:

Content Sourcing & Methods 1

Reach and Impressions 2

Engagement 3

Influence & Relevance 4

Opinion & Advocacy 5

Impact & Value 6

4/13/2015

The Transparency Table:

#2: Standards for Reach & Impressions

Reach: the scope or range of distribution that a given communication product has in a

targeted audience group; broadcasting, the net unduplicated (also called “deduplicated”)

radio or TV audience for programs or commercials as measured for a specific time

period.

Impressions: Number of people who might have had the opportunity to be exposed to a

story that has appeared in the media (aka opportunity to see (OTS))

Impressions do not equal awareness. Awareness needs to be measured using other

research tools. Impressions are indicative of the OTS. Consider OTS as an alternative

nomenclature to better clarify what impressions really means – [the] potential to see/read.

Multipliers should never be used

OTS must be specific to a particular channel – i.e. For Twitter OTS is the number of first

line followers. For Facebook it is the number of fans to a page.

Eyeball counting

HITS Outcomes

MSM Online Social Media

Impressions are not awareness. Where’s the “So What”?

11

#3: Standards for Engagement

Engagement is an action that happens in response to content – i.e. when someone

engages with you. (not “about you”)

Engagement manifests differently by channel, but is typically measurable based on effort

required and how it is shared with others.

Engagement can be desirable or undesirable

Any measure of engagement must be tied to the goals and objectives for your program

Engagement occurs both off and online, and both must be considered if you intend to

integrate your metrics with other marketing or communications efforts.

Engagement includes such actions as likes, +1, shares, votes, comments, links, links,

retweets, Facebook's “Talking about you” etc.

Engagement actions should be counted by % of the audience that responds.

Likes Are Not Engagement

13

Advocacy Commitment Trial/Consideration

Followers Likes Impressions

4/13/2015

968 1027

914

208

297 261 269

215 222 145

328

204

510

188 222 234

41 71 50 36 25 56 30 71 45

107

Share & Likes Over Time

Average of PostLikes Average of PShares

GWA’s, Photos Drive Shares & Likes

Mia Farrow

David Beckham & Dierks Bentley

Video

• Influence happens when you are persuaded to

change behavior or opinion that would

otherwise not have changed.

• Must include:

• Reach

• Engagement around individual

• Relevance to topic

• Frequency of posts around the topic

• Audience impact as measured by the

ability to get the target audience to change

behavior or opinion.

• If an individual scores a 0 on one element, they

have no influence

• Influence cannot be expressed in a single score

or algorithm

#4: Influence & Relevance Image courtesy www.getlittlebird.com

#5 Standards for Measuring Tone/Sentiment

Sentiment is overrated

Messaging and Positioning &

Recommendations are more useful

Whatever process is defined and

applied, it must be used

consistently throughout any

analysis.

Validity testing is critical

Comparison of Tonality Between Vendors

33.12%

58.00%

7.84%

11.95%

26.53%

13.00%

0.32%

9.46%

40.35%

30.00%

92.11%

78.00%

SDL

Beyond

NetBase

UberVu

% positive %negative %neutral

Impact & Value

ROI is: Cost savings

Greater efficiency

Lower legal costs

Faster time to implementation

Lower cost per impact vs. other marketing channels

Lower cost per message exposure vs. other channels

ROI is not: Impressions

Earned media value or AVE

Number of shares, views or likes

The biggest challenges

• Getting from personal justification to a single source of

truth

• Finding valid, relevant data

4

What we learned

1.Consensus on how PR/SM contributes to

the business is critical

2.Once you have that defining success &

metrics is easy

3.Tracking down the data and climbing in

and out of silos is hard

4.Produce a bespoke “Standards

Document” telling everyone, including

agencies and foreign offices how to report

is easy, getting it adopted is hardest of all

5

Custom Standards Standards for Media Reporting

Web Analytics

5

-0.40%

-0.20%

0.00%

0.20%

0.40%

0.60%

0.80%

1.00%

1.20%

1.40%

8500

8600

8700

8800

8900

9000

9100

9200

9300

9400

9500

5/1/2014 5/11/2014 5/21/2014 5/31/2014 6/10/2014 6/20/2014 6/30/2014

Follower and Follower Growth Since May 1

Followers Follower Growth

What we had: “XYZ Tweets Generated the Greatest Increase in

Followers”

#Tweet 5

Tweet 3

Tweet 2 Tweet 1

Tweet 4

Social Media Engagement Index

Action Score

Like/Follow/Opens/+1 .5

Favorite or Opens or Views 1

Comment 1.5

Share content 2

Signs up to receive email or other owned content 2

Contains a quote from one of our content experts 1.5

Shares a link to an owned site 2.5

Total 10

24

Implementation Steps

1.Pull together a dashboard with the best

data you can find and point out the gaps

2.Put on your bullet proof vest & explain to

EVERYONE

3.Make changes, make more changes

4.Produce the first report

5.Repeat Step 2

6.Draw conclusions, make decisions.

7.Take it to the board

5

The biggest challenges

• Getting from this to this

• Finding valid, relevant data

4

• Geographies vs Languages • Lack of consistent competitors list • Different definitions of Products/Brands Covered • Time frames • Media Types • Search strings and filters

3

Testing validity of criteria

The Theory: 6 steps to Standards-

Compliant measurement

Step 1: Define your goal(s).

What outcomes is this strategy or tactic going to achieve?

What are your measurable objectives?

Step 2: Define the parameters

Who are you are trying to reach? How do your efforts

connect with those audiences to achieve the goal.

Step 3: Define your benchmarks.

Who or what are you going to compare your results to?

Step 4: Define your metrics.

What are the indicators to judge your progress?

Step 5: Select your data collection tool(s).

Step 6: Analyze your data.

Turn it into action, measure again

26

Six Steps

to Success

1

2

3

4

5

6

Step 1: Define the goals

What return is expected? – Define in terms of the

mission.

Define your champagne moment. If you are

celebrating complete 100% success a year from

now, what is different about the organization?

27

Why do we communicate?

4/13/2015 28

Outtakes

(Intermediary Effects)

• Awareness

• Knowledge/Education

• Understanding

Outcomes

(Target Audience Action)

• Revenue

• Leads

• Engagement

• Advocacy

Activities How does what you do

contribute to the bottom line?

Dashboard Framework

Support the business

% increase in engagement which drives…

% increase in video starts, % increase in ratings, market share

Increase engagement

1. Increase in % of audience that is engaged

2. Engagement Index:

• Shares

• Comments

• Favorites

• Repeats vs. uniques

• Time on site

• Pages per visit

Definitions of “Success” Workshop defined the criteria

All criteria linked back to the goals:

High Quality Media Coverage

Intent to visit

Improved reputation

Visits/Non-gaming revenue

Step 2: Understand the parameters.

What management’s priorities?

Who are you are trying to reach?

How do your efforts connect with

those audiences to achieve the

goal?

What influences their decisions?

What’s important to them?

What makes them act?

31

Goal: Get the cat to stop howling

Options: Local? Cheap? Convenient?

Strategy: Buy cat food

What are you measuring?

Paid – Google Adwords, Facebook

Ads, popups, banners etc.

Owned – #njsms15

@queenofmetrics

http://www.pinterest.com/wsbtv/co

x/

Earned -- Everything else

(Including Shared)

32

Digital/Advertising/Marketig

Social Media Team

PR

Who’s got the data?

Step 5: Pick the right measurement tools

If you want to measure messaging, positioning,

themes, sentiment:

Content analysis

If you want to measure awareness, perception,

relationships, preference:

Survey research

If you want to measure engagement, action,

purchase:

Web analytics

If you want predictions and correlations

you need two out of three

Step 5: Selecting a measurement tool

34

Objective KBI Tool

Increase message

communications

Increase percent of items containing one or

more messages

CyberAlert, Prime Research

Increase

awareness/preference

% of audience preferring your brand

to the competition

Survey Monkey, Survata

Engage

marketplace

% increase in engagement on website and/or

social sites

Simply Measured, Unmetric,

Google Analytics, Site Catalyst,

Network Analysis

Become a thought

leader on a specific

issue/market/topic

% increase in desirable share of voice on

issue/market/top

Little Bird, Trackr, CyberAlert

Step 6: Be Data Informed, not Data Driven

Rank order results from worst to best

Ask “So What?” at least three times

Put your data into an overall framework consistent with C-Suite expectations

Find your “Data Geek”

Compare to last month, last quarter, 13-month average

36 Page 36

The ROI of Emily

37

Red line

indicates media

impressions

35,152,789 OTS

6,253,852 OTS

So What = Revenue

38

Without ACA events, OCS Scores for Atlantic City would

have been significantly lower

4

3.25

2.75 2.99

3.65

2.96 3.36 3.24

2.34 2.37 2.43

1.30

-1.24

0.37 -0.05

0.28 0.28

-1.56

4.91

3.92

2.99

3.58

4.14 4.1 4.27 4.12 4.29

2.78 2.56

1.53

-0.29

0.61 0.20

1.44 1.77

0.63

-2

-1

0

1

2

3

4

5

6

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June

Ave

rage

OC

S Sc

ore

Average OCS Score Over Time

Atlantic City OCS without ACA Atlantic City OCS with ACA

The red line represents coverage of Atlantic City minus all mentions of ACA and its programs

When ACA programs received media coverage,

engagement, preference and traffic followed

0

50

100

150

200

250

300

0

20000

40000

60000

80000

100000

120000

140000

Relationship between ACA Program Mentions and Site Visits

Site Visits Program Mentions

Photo Event

Best practices: Resource use vs Results High Engagement i.e. Video Starts

Resource Use

Low

Hig

h

Med

ium

Ver

y h

igh

Total Volume of Coverage

Ver

y H

igh

Med

ium

Hig

h

Low

High Resources

Low Engagement

Low Resources

Webinar

Status update

Link

Ultimate Road Trip

Google + Chat

Media Day

Corporate Video

10 Ways to find the Money for Measurement

1. Don’t call it measurement – It’s Research

2. Consolidate - Social? IR? Agencies? Research?

3. Crowdsource – Who else has tools or surveys?

4. Spread the cost over 2 fiscal years

5. Reach out to universities

6. Take advantage of free stuff

7. Monitor only what matters

8. Narrow your Top Tier media list to what you CAN afford

9. Google analytics

10. Facebook Insights

11. Twitter Analytics

Remember These Points

It’s not about the media, it’s about the business and the customer 1

It’s not about Big Data, but about how you use it. 2

You need to be data informed, not data-driven 3

It’s not how loud you’re shouting it’s about relationships. 4

Standards are a reality not an excuse to hide behind 5

Thank You!

For more information on measurement, read my blog:

http://www.painepublishing.com/blog

For a copy of this presentation or to subscribe to our newsletter, give me

your card or email me at measurementqueen@gmail.com

Follow me on Twitter: @queenofmetrics

Friend me on Facebook: Katie Paine

Or call me at 1-603-682-0735

44