NT Cities Measurement

Martyn Price / Eleanor UnderhillNT Cities Sounding Board Sub Group

Measurement

13 June 2012

2

Context

Objectives of NTCitiesProgramme Level MeasurementArea MeasurementProject MeasurementSummary / Next Steps

3

4

Context: What the group identified

• NT Cities is more about areas and populations and less about properties (with a few notable exceptions) than we’re used to…

• Lots of our existing measures and mechanisms are about our properties

• So a balance between using what we already measure alongside some new methods.

• And a need to cut some of our existing data differently.• Also a need to achieve a balance NT Cities programme-

wide measures, and local projects + pilots measures.• Perhaps opportunity for NT Cities programme to support

pilot measurement of new local community relevance KPI

5

What to Measure

Programme

Area/City

Projects

Workstreams

6

Base: All interviewed 28/02/11 - 26/02/12: (5732); Urban 15 (2201); Rest of Country (3531)

49

20

35

2332 32

25 29

51

25

3928

34 32 28 29

0

20

40

60

80

ANY TYPE OF VISIT(House, Garden,

Coastal,Countryside)

Walking on aNational Trust

Property

National Trustcountryside location

National Trustcoastal location

National TrustHouse

National TrustGarden

National Trust shop National Trustcafé/restaurant

Urban 15 Rest Of Country

Penetration (Claimed NT Activities undertaken in past 12 months)All respondents interviewed : Urban 15 vs. Rest of the Country

52

16

3 3 38 7 3 4

55

18

2 3 210 6 2 3

0

20

40

60

80

Overall NTPenetration (Any

activity/visitsundertaken in past

12 months)

Made a charitabledonation to theNational Trust

Gone on aNational Trust

working holiday

Attended aNational Trust

centre orassociation

meeting

Done volunteerwork for the

National Trust

Attended aNational Trust

event

Cycled on aNational Trust

property

Stayed in aNational Trust

holiday cottage

Camped onNational Trust

property

Urban 15 Rest Of Country

NT Activities Taken Part In Over Past 12 Months – URBAN 15 vs. REST OF COUNTRY

NT Places Visited in past 12 months – URBAN 15 vs. REST OF COUNTRY

VISITING-SPECIFIC

Q. Please can you tell us when you last took part in any of the following National Trust activities? : - Past 12 months

And marginally less penetration in the Urban 15 areas…

7

78

35

59

44

54

75

33

57

42

52

80

36

60

46

55

0 20 40 60 80 100

Awareness

Knowledge

Positivity

Relevance

Penetration

%

All respondents

Urban 15

Rest of Country

Slightly subdued EFLAM scores amongst the Urban 15 compared to the Rest of the Country...

EFLAMAll respondents interviewed 28/02/11 - 26/02/12

Base: All interviewed 28/02/11 - 26/02/12: (5732); Urban 15 (2201); Rest of Country (3531);

For Knowledge: All asked about National Trust first: (4986); Urban 15 (1915); Rest of Country (3071) 7

8

Programme

Programme

Scorecard Approach Around Urban 15AWARENESS: Thinking about organisations which look after and provide access to our built heritage…which can you think of that provide access to…coastlines/historical buildings and sites of interest/Countryside/Gardens/charitable organisations you can think of…?KNOWLEDGE: And now, thinking specifically about the National Trust again, how much do you feel you know about the National Trust? - % Know a lot/a reasonable amount about NTPOSITIVITY: Please think about how close you feel to these brands; if you feel they are close to you, put them close to you in the centre of the screen. If you feel that are distant, put them far away.PENETRATION: Thinking specifically about destinations for days out, which of these organisations have you visited in the past 12 months/6 months/month?RELEVANCE: And thinking about the sorts of people who support these types of organisations, are they…”For People Like Me”?NPS:LOCAL RELEVANCE:

9

Programme

Programme

KPIsVisitor NumbersMembership RecruitmentMembership RetentionMembership PenentrationVE ScoreCommercial Performance

Aggregate to a programme levelAll Data in houseCan be cut several ways

10

Areas

Areas of Focus:London, Manchester, Birmingham, Bristol and South Wales

Local Survey – awareness, knowledge, penetration, local community relevance etc. Annual measurement.

Mapping – Membership retention/recruitment/penetration. Annual snapshots.

Local visitor numbers – aggregate by area.

2012/13 = base-lining.

Area/City

11

NT Cities area measurement

INDICATIVE COSTS

Telephone

Face to Face

PERCEPTIONS / AWARENESS / CLAIMED INTERACTIONS

200 respondents10 minute phone survey

= c. £6,000 + VAT

200 respondentsOn street/at venue survey

= c. £7,500 + VAT

3 areasSingle dip= c £16,000 +VAT

Multiple areasSingle dip= possible cost efficiencies

12

Projects

Local measurement, done locally

Toolkit approach – DIY.

Recommended approaches Suggested indicative measures to trackAgency roster

VE & VCI upskilling – longer-term… But Consultancy support required.

Set their own targets and measures.

NOT ALL PROJECTS WILL HAVE MEASUREMENT

All Outcomes to Victoria

Projects

13

Cross-Cutting Workstreams

To be defined.

i.e. how do we cluster affiliate/similar activities across different cities. E.g. cross-city partnerships; Dyffryn style partnerships; digital engagement; special exhibitions.

Working with Victoria to share evaluation across workstreams and places/cities

14

Where Insight can help

Programme

Area / City

Projects

InsightInvolvement

15

Consultancy Involvement

Programme

Area / City

Projects

InsightInvolvement

ConsultancyInvolvement

Not All Projects will be measured

16

Summary

Programme - Area - Local Projects

What the measures will tell us:

• City by City how well things have worked• How urban peoples attitudes have changed• Better insights with urban audiences and what

works• Should tell us what we should do next in urban

areas

17

Measure matched to Objectives

Visitor Numbers

VE

Visitor Numbers

Recruitment,Penetration,Retention

Staff Survey,Volunteer Survey

Tracker,Focus

Groups

Tracker,Focus

Groups

Awareness, Knowledge, Penetration, Positivity,

Relevance, NPS

18

Next Steps

Agreement of Measures – Today

Establish NT Cities Area of Measurement - Soon

Establish costs – Soon

Baseline – by 2013

Enhanced Guidance – by December 2012

Agreement on Local Relevance – Soon

Measure and Target 2013

19

END

20

Project Measurement

21

78

85

87

77

68

67

77

78

81

74

89

91

82

79

64

61

0 20 40 60 80 100

TOTAL

Active Retirement

Alpha Territory

Careers and Kids

Claimant Cultures

Elderly Needs

Ex-Council Community

Industrial Heritage

Liberal Opinions

New Home Makers

Professional Rewards

Rural Solitude

Small Town Diversity

Suburban Mindsets

Terraced Melting Pot

Upper Floor Living

%

Base: All interviewed 28/02/11 - 25/09/11 (3314);

Active Retirement (109); Alpha Territory (109); Careers & Kids (219); Claimant Cultures (125); Elderly Needs (71); Ex-Council Community (278); Industrial Heritage (263); Liberal Opinions (212); New Home-Makers (181); Professional Rewards (281); Rural Solitude (106); Small Town Diversity (292); Suburban Mindsets (530); Terraced Melting Pot (189)

AWARENESS (Total Spontaneous Awareness of National Trust)By Mosaic Groups

Awareness of the Trust is highest amongst the Rural Solitude and Professional Rewards Mosaic groups; and lowest amongst Upper Floor Living

22

Small Project Measurement

-Consider whether investment in measurement is worth it-Potential to do local measurement yourself- Requires

Recruiting and retaining

members

Growing our audiences

Tell Stories that will engage

urban audiences

Champion Urban green space and heritage

Developing internal

learning

- Membership Recruitment either Property or Residence- Membership Deletions by residence (Who’s member is it?)

-Penetration Brand Tracker or Actual- Car Park Income?-Catering Income?-Number of Visitors to Urban Properties and projects

-VE scores at properties-Programme of Digital Take up?- Volunteer Recommendation (Urban properties)?-??

-Brand Tracker (Enables you to enjoy the outdoors)

-??

EFLAM Measures – Awareness, Knowledge, Positivity, Relevance

GOAL: To help people

love special places

in cities and close to where

they live, by championing

and promoting urban heritage, green space, nature and

beauty.

VISION: That everyone in

England, Wales and Northern

Ireland will feel like a member of the National Trust and, by

2020, five million people

will be.

NTCities - Measures

24

Urban area measurement (1)

• Limitations in NT brand tracker – online reach is insufficient for area by area sampling (except perhaps for sample in London).

• For local measurement more likely to require:

Telephone

or

Face to Face

ExpensiveQuestion limitationsGood reach

ExpensiveLess limited on questionsGood reach

PERCEPTIONS / AWARENESS / CLAIMED INTERACTIONS

25

Urban area measurement (3)

• Our own internal data may be available to cut on an area by area basis.

• We could use visitor survey data at urban properties.

Membership recruitment; retention;

Visitor numbers for NT properties

Local visitors/postcode data

VE measures for existing NT properties [+ potential measurement at new sites (£1k)]

MEMBERSHIP / VISITS / VISITOR EXPERIENCE

26

Urban Sounding Board

What do we present at Urban Sounding Board in June?

NT Cities: Measurment

28

Recommended Approach

29

Programme Wide

Scorecard? Main measures. Linked to objectives.Contextual measurement – Urban 15. Not targets.

30

Where is Urban?

31

Rest of the Country

62%

Urban 1538%

The Urban 15 definition constitutes over one third of the total sample.

Sample Breakdown: Urban 15 vs. Rest Of The CountryAll respondents interviewed 28/02/11 - 26/02/12

Base: All interviewed 28/02/11 - 26/02/12: (5732) 31

34

Polesdon Lacey Case Study

Web based Questionnaire150 respondents 45 mins drivetime from Polesdon Lacey10-15 mins to completeAgree own set of questionsFairly quick turn aroundConsumer Insight Agency Used£1500 for one off

Issues:Limited questionsMay be difficult to recruit enough people