Pride Scorecard TrainingDecember 2013
Agenda• Big picture (10 min)• Overview of the new measures
- Ariela on capacity (45 min)- Brian on constituency (20 min)
• Recap the scoring system (15 min) • Highlight upcoming events (5 min)
Outcomes• Understand the new measures• Understand how the PSC can inform next steps for your
campaign• Understand how to assess a campaign based on the criteria
and assigning a color score
Agenda & Outcomes
• Big picture• Overview of the new measures
- Ariela & Team on Capacity- Brian on Constituency
• Review the scoring system• Highlight upcoming events
Agenda
(Succinct) Timeline of Pride Scorecard
~Feb20122011 2012 -
TodayJuly
2013Dec. 13
Feb14
Laun
ch o
f the
New
C
ampa
ign
Trac
ker
Ref
ine
Que
stio
n on
Goo
gle
Doc
s
Beg
an s
olic
iting
form
al fe
edba
ck
from
TM
T, S
ocia
l Mar
ketin
g, M
MT
•U
sed
vers
ion
on G
oogl
e D
ocs
•C
ount
less
twea
ks
Form
al a
ppro
val b
y R
egio
nal
VP,
Dal
e
Pilo
ted
in 3
regi
ons
for 3
mon
ths
Last
dra
ft re
view
ed b
y TM
T, S
ocia
l m
arke
ting,
MM
T
~Jan2012
Commitment is to create at least one opportunity per year to make substantive changes.
What the Pride Scorecard Measures
The Pride Scorecard measures the progress of a campaign based on the objectives of the 3Cs using a set of action-oriented indicators. Using the 3Cs as the framework allows us drive improvements in the campaigns and to assess the likelihood of continued success after the campaign has ended.
Objectives of the 3Cs: • Build the Capacity of the Rare Fellow(s) and IP to have the
willingness and ability to continue the campaign successfully on their own.
• Develop a Constituency for the campaign that ensures the community's commitment to conservation, their adoption of the desired BC as a social norm and ownership of threats.
• Achieve the campaign's Conservation goals via new behavior adoption, threat reduction and successfully monitoring impact.
1. Improve Campaigns – Action oriented indicators are designed to drive decisions to maximize the impact of our campaigns
2. Guidance - Support staff in identifying what the key issues are for their campaigns
3. Transparency - Create a standard indicator for global conversations about campaigns to foster a global conversation
4. Accountability – we all should know our key roles in driving campaign improvement
Why We have a Pride Scorecard
Who Uses the Pride Scorecard and WhyWho WhyDale • Part of Dale’s monthly reporting pack to assess the progress of each cohort, where to
invest resources, etc.• Liz sends a Status Report with PSC scores included to Cohort Director at same time as to the
team member who assembles the full reporting pack• Cohort director adds 3 to 5 bullet points to the report and sends them to VP
RVP • To assess cohort progress and how to allocate resources across campaigns• To understand the needs of the team
Cohort Director
• To assess the progress and impact of their cohort, where it is on track and where it needs support
• To identify what actions are needed to ensure each campaign achieves its long-term objectives
• To make recommendations about investing or cancelling campaigns
Board • To get updated on Rare’s programmatic progress and impact
Rare • To measure our performance and impact using the BSC
Regions • To measure regional performance and impact using the DBSCs
PPMs • To systematically document successes and achievements of their campaigns• To identify specific steps and actions needed to improve their campaigns• To keep an eye on upcoming actions and steps that need to occur next to stay on
track
TMT • To assess areas where training needs to be re-enforced
MMT • To track the progress of their monitoring efforts, and adapt accordingly
QMI • To share campaign and cohort progress in reports to other teams• To flag challenges early, to collect and share bright spots• To ensure the global minimum standard of quality for campaign progress is be met
Where the Scorecard Fits In
Pride Scorecard
Weekly Flash Reports
Regions have own management
processes
Quarterly BSC Reports
Select BSC Measures are in
your IBSC
Monthly Status Reports
• Big picture• Overview of the new measures
- Ariela & Team on Capacity- Brian on Constituency
• Review the scoring system• Highlight upcoming events
Agenda
Refining Rare’s ‘Capacity’ Metrics
Define what we want to measure
Deliverables or skills/capacities?
Pride Skills Assessment
TMT Meetings and 1:1
Conversations
Training Department Strategic Planning
Final 5 Capacities
Inventory of Existing Tools
Scorecard, Alumni Surveys, etc.
Questions
Succ
ess
Indi
cato
rs
Input from TMT & Global Programs
Training Management Team Meeting
Validate Success Indicators and Draft
Tools with Alumni
Roadmap for Developing the 5 Capacities, Success Indicators, and Metrics
TMT Feedback on Learning Outcomes
Revise Success Indicators with PPM
and TMT Input
Monitoring Team Meeting
Input from Monitoring Team on Success Indicators
Share Metrics Framework with LT
LT Meeting
RareWorld Communication
Email Metrics Framework to
RareWorld
5 Key Leadership Capacities
Alignment of Metrics Tools to Kirkpatrick’s Model of Training Evaluation
During the course of the campaign, what are some of the key Strategic Planning skills that the Rare Fellow(s) has displayed?
Rare Fellow(s) is consulting qualitative and/or quantitative research to inform campaign decisions.
Rare Fellow(s) is using SMART Objectives of the site-level Theory of Change to drive campaign strategies and/or work plan.
Rare Fellow(s) is adapting strategies and/or Research Plan and/or work plan(s) to account for new information and/or unforeseen circumstances.
Capturing ‘Behavior’ in the Scorecard: “Presence” or “Absence”
• Big picture• Overview of the new measures
- Ariela & Team on Capacity- Brian on Constituency
• Review the scoring system• Highlight upcoming events
Agenda
Stepped BC Metrics
What percentage of the campaign’s SMART Objective for the TA’s enabling step in BC has been achieved so far? Sustainable Fisheries campaigns: Fishers regularly attend and participate in meetings, trainings and workshops about fisheries.ARA campaigns: Targeted members of the Upstream TA are taking steps towards signing an ARA.
Campaign is achieving at least 20% to 39% of this BC SMART Objective
Campaign is achieving 40% to 59% of this BC SMART Objective Campaign is achieving 60% to 79% of this BC SMART Objective Campaign is achieving 80% to 99% of this BC SMART Objective Campaign is achieving 100% of this BC SMART Objective
Stepped BC Metrics
What percentage of the campaign’s SMART Objective for the TA’s intermediate step in BC has been achieved so far? Sustainable Fisheries campaigns: Fishers register their boat and fishing gear.ARA campaigns: Targeted members of the Upstream TA have signed an ARA with a minimum duration of 5 years.
Campaign is achieving at least 20% to 39% of this BC SMART Objective
Campaign is achieving 40% to 59% of this BC SMART Objective Campaign is achieving 60% to 79% of this BC SMART Objective Campaign is achieving 80% to 99% of this BC SMART Objective Campaign is achieving 100% of this BC SMART Objective
Stepped BC Metrics
What percentage of the campaign’s SMART Objective for the TA’s ultimate step in BC has been achieved so far? Sustainable Fisheries campaigns: Fishers acquire fishing licenses.ARA campaigns: Members of the Upstream TA who have signed ARAs, isolate the agreed upon conservation objectives in their properties for conservation and got a minimum and renewable period of 5 years
Campaign is achieving at least 20% to 39% of this BC SMART Objective
Campaign is achieving 40% to 59% of this BC SMART Objective Campaign is achieving 60% to 79% of this BC SMART Objective Campaign is achieving 80% to 99% of this BC SMART Objective Campaign is achieving 100% of this BC SMART Objective
• Big picture• Overview of the new measures
- Ariela & Team on Capacity- Brian on Constituency
• Review the scoring system• Highlight upcoming events
Agenda
Recap: Scoring SystemOld New
• Pride Scorecard scores were an average of the scores for each measure
• Color assignment was automatic based on the 5 pt. scale
• Scores were sometimes artificially low or high due to averaging
• Pride Scorecard scores are the summed total points of each measure
• PPMs assign a color to each Pride Scorecard score, which is approved by Cohort Director
• Campaign progress is portrayed more accurately throughout the cohort timeline
Indicative Pride Scorecard Dashboard
Indicative target for cohort at completion
Period
Key Points to Keep in Mind for Scoring Baseline
• Created by hypothetically scoring a TOP campaign using each measure for each month
• This baseline is a starting off point for us to test and improve• Anything in the scoring baseline being presented can change
and be negotiated as needed
Points Possible Over TimeMonth Possible Points Descriptions
Month 1 23 The end of the first month Rare Fellow(s) has been in the field after 1st Uni Phase
Month 4 40 4 months after Rare Fellow(s) has been in the field, midpoint of the Planning Phase
Month 9 63 The end of the first month Rare Fellow(s) have been back in the field after 2nd Uni Phase
Month 12 84 4 months after Rare Fellow(s) has been in the field after 2nd Uni Phase, near or at public launch time
Month 22 103 Final months of Implementation Phase, all essential SM, BC, BR, TR, CR actions in place
Color Assignment GuidanceEach region can set their own standards for green, yellow, red
Month Possible Points
Green Range(80% or more of points possible)
Yellow Range(50% to 79% of points possible)
Red Range(49% or less of points possible)
Month 1 23 18 – 23 12 – 17 0 -11
Month 4 40 32 – 40 20 – 31 0 – 19
Month 9 63 50 – 63 32 – 49 0 – 31
Month 12 84 67 - 84 42 – 66 0 – 41
Month 22 103 82 - 103 52 - 81 0 - 51
The most important thing is that if you need management assistance, flag the campaign as red
Green = Progress is on track to achieve objectivesYellow = Progress is lower than expected but does not require immediate senior manager actionRed = Progress is off track and/or requires senior management attention to resolve critical issues
Submission & Approval Process
1. PPM, Training Manager/Director, Monitoring Specialist and BR Specialist score measures
2. PPM assigns color before submitting to Cohort Director3. Cohort Director reviews and approves Pride Scorecards as
before, and approves PPM’s color assignment4. Cohort Director assigns a rank to each campaign
• Big picture• Overview of the new measures
- Ariela & Team on Capacity- Brian on Constituency
• Review the scoring system• Highlight upcoming events
Agenda
Next StepsComplete December Pride Scorecards (due 12/20/13) in Google with updated measures and new scoring system• Share feedback on measures, month ranges for points and point ranges asap• Feedback will be incorporated into January Pride Scorecards
Complete January Pride Scorecards in Google• Share feedback again on updates made asap• Feedback will be incorporated but not by the February Pride Scorecard due to
Campaign Tracker launch
Get training on the new Campaign Tracker (late January)• Learn how to upload and approve milestones• Learn how to blog and upload multimedia files• Complete the Pride Scorecard using the CampaignTracker (February)
Appendix: Questions about the CampaignTracker TransitionWhat’s in the CampaignTracker• Milestones page• Prompts on upcoming milestones• For Rare/EDF/UCSC staff
• Weekly flash report• Pride Scorecard• Campaign administration
What’s NOT in the Campaign Tracker• No more groups• No more personal pages
What happens to content that is there• Active groups are being migrated to new online tools• Campaign content will be ported into the Campaign Tracker automatically• Personal pages and blogs will be archived
When will it happen?The transition will begin in mid January and be largely complete by the end of February
Who has access to the Campaign Tracker• Unlike RarePlanet the site will ONLY be accessible to Rare/EDF/UCSC staff and Implementation partners
How does this link to the upcoming relaunch of Rare.org• Campaign blogs will link to Rare.org• Select campaign information to provide an overview and some deliverables will be linked• People will learn about programs, cohorts and campaigns via Rare.org
Appendix: Status ToolsWeekly Flash
ReportPride
ScorecardCohort
RankingD-BSC
Purpose Shows the progress the campaign is making against this month’s goals in the project plan.
Helps to track progress in achieving campaign goalsHelps to provide guidance about how to achieve those goals
Ranks the intrinsic importance of a site
Tracks progress on achieving annual objectives
Primary User PPM & Cohort Director
Cohort Team & Cohort Director
Cohort Director, VP, Dale
All
Indicative actions
• Quickly escalate at risk campaigns
• Highlight key issues
Highlight key opportunities and challenges
Ensure that we invest in the most important campaigns
Allocate regional priorities to achieve regional (not just cohort) objectives
Frequency Weekly Monthly Monthly Quarterly
During the course of the campaign, what are some of the key Strategic Planning skills that the Rare Fellow(s) has displayed?
Rare Fellow(s) is consulting qualitative and/or quantitative research to inform campaign decisions. Rare Fellow(s) is using SMART Objectives of the site-level Theory of Change to drive campaign strategies and/or work
plan. Rare Fellow(s) is adapting strategies and/or Research Plan and/or work plan(s) to account for new information and/or
unforeseen circumstances.
During the course of the campaign, what are some of the key skills associated with Behavior Change Interventions that the Rare Fellow(s) has displayed?
Rare Fellow(s) is accounting for audience’s perceived benefits and barriers in the design and/or implementation of their Marketing Strategy.
Rare Fellow(s) is aligning Social Marketing and Barrier Removal strategies to accelerate the adoption of the new behavior.
Rare Fellow(s) is selecting and/or implementing activities and materials to progress the target audience through the Stages of Behavior Change (as aligned to K, A, IC, and BC).
Appendix: All Updated Capacity Measures
During the course of the campaign, what are some of the key Effective Communication skills that the Rare Fellow(s) has displayed?
Rare Fellow(s) is communicating the goals and/or strategies of the campaign in terms that are appropriate for the intended audience.
Rare Fellow(s) is using Brand, messages, activities, and/or materials to communicate the Benefits Exchange. Rare Fellow(s) is motivating key influencers and/or groups to take concrete actions to support the campaign.
During the course of the campaign, what are some of the key skills related to Conservation Science and Management that the Rare Fellow(s) has displayed?
Rare Fellow(s) is addressing site-specific threats to the conservation target through the campaign strategies and/or work plans.
Rare Fellow(s) is designing and/or implementing a Barrier Removal strategy that addresses site-specific threats. Rare Fellow(s) is tracking the measurement and monitoring of campaign impact (TR and CR).
During the course of the campaign, what are some of the key elements of Skills Transference that the Rare Fellow(s) has displayed?
Rare Fellow(s) is sharing lessons learned within and/or across cohorts. Rare Fellow(s) is communicating and/or sharing key capacities (Strategic Planning, Behavior Change Interventions,
Effective Communications, and Conservation Science and Management) with implementing partners and/or local agencies.
Rare Fellow(s) is taking actions and/or considering decisions that will contribute to the sustainability and/or continuation of the campaign.
Appendix: All Updated Capacity Measures