Prioritizing Obesity Strategies

Obesity Integration Steering Committee Meetings2.29.12, 3.19.12, 4.2.12

Prioritization Process

When Purpose/Actions

Meeting 1 (2/29) • Getting Started• Agreements For Moving Forward

Between Meetings(e-input)

Survey Coming Tomorrow (due 3/9)• Review/Provide Input on Proposed

Criteria• Identify State Roles for Strategies

Meeting 2 (3/19)9:30am -12:30pm

• Review/Finalize Criteria• Clarify Strategies• Clarify State Roles

Between Meetings(e-input)

Prioritization Survey (sent 3/21, due 3/26)• Rate Strategies Using Final Criteria• Refine Concise Statement

Meeting 3 (4/2)9:30am – 12:30pm

• Share Results of Prioritization • Gather Additional Input for Ex Committee

Agenda ReviewWelcome and Overview

Review/Finalize Criteria

Break

Clarify Strategies and State Roles

Next Steps and Closure

Discussions and DecisionsDiscussion: All encouraged to participate

Decisions: For Today & Prioritization Survey (btwn Meetings 2 & 3)

60% super majority vote, motioned by a steering committee member

One vote per Steering Committee Member

Executive Committee Members, and other observers, do not vote

Steering Committee Voting MembersSector Team Leads or Designee (8 reps from

PSD)

Healthy Eating (2 reps from PSD)

LHAs (5 total: 1 rep each from El Paso, Weld, Boulder, Pueblo, West-Central Partnership)

External Organizations (3 total: 1 rep each from Live Well, Kaiser, Health Foundation)

Review/Finalize Criteria

The “Ask” Regarding Criteria11 prioritization criteria proposed

Asked to make 2 judgments:RATE each criteriaRANK top five criteria

Opportunity to suggest other criteria

19 people responded

Criteria Ratings - Results

Not at all Well Somewhat Well Fairly Well Well Very Well

Rating TakeawaysAll criteria would do fairly well or

better at helping during the prioritization process.

Could potentially combine some criteria: Population Impact AND Expected

Reach? Community Support, Capacity to

Implement, AND Opportunity for Leverage?

Could consider the following additional criteria: Alignment with National Priorities Evidence Level

Criteria Rankings - Results

Ranking TakeawaysRanking a more effective way to

distinguish preferences among criteria that are all helpful.

“Least Helpful” ranking could be misleading – only least helpful among 5 favorites.

At least 60% of responding members chose the following as one of their top 5 criteria: Likelihood of population impact Capacity to Implement

Considerations

Evidence is being published every day.

Executive committee will get: the full list of strategies with their ranks from

the prioritization survey.

Future implementation teams will consider:Applicability to ColoradoResources needed

PSD will keep you informed.

Demonstration of Prioritization Scoring

No right way ~ rather match how you want to make the decision

Rate each strategy against the criteria

Criteria scale: 1 = “Little” to 5 = “Great”

Create a prioritization score from your ratings

= add the ratings on each criteria,

divide by the number of criteria,

divide by the number of respondents (raters)

Expect ties

Demonstration of Prioritization Scoring

Paint your house

Criteria A: True Blue

Criteria B: Easy to clean Sum

Priority Score

Rank using 2 criteria

Rank using 3

Rank with weighting

Rank w/

multiplier

Strategy A 1 3 4 2.0 6 6 6 2

Strategy B 2 3 5 2.5 4 2 2 3

Strategy C 3 2 5 2.5 4 5 5 3

Strategy D 4 5 9 4.5 1 1 1 1

Strategy E 5 3 8 4.0 2 2 3 6

Strategy F 3 3 6 3.0 3 4 3 5

Demonstration of Prioritization Scoring

Criteria A: True Blue

Criteria B: Easy to clean

Criteria C: Longlasting Sum

Priority Score

Rank using 3 criteria

Equal Weight 1 1 1Strategy A 1 3 1 5 1.7 6Strategy B 2 3 5 10 3.3 2Strategy C 3 2 3 8 2.7 5Strategy D 4 5 5 14 4.7 1Strategy E 5 3 2 10 3.3 2Strategy F 3 3 3 9 3.0 4

With 3 criteria

Demonstration of Prioritization Scoring

Criteria A: True Blue

Criteria B: Easy to clean

Criteria C: Longlasting

Criteria C: Longlasting

Priority Score Rank

Weight 1 1 1 1

Strategy A 1 3 1 1 1.5 6

Strategy B 2 3 5 5 3.8 2

Strategy C 3 2 3 3 2.8 5

Strategy D 4 5 5 5 4.8 1

Strategy E 5 3 2 2 3.0 3

Strategy F 3 3 3 3 3.0 3

With 3 criteria and weighting

Demonstration of Prioritization Scoring

With multiplier

Paint your house

Criteria A: True Blue

Criteria B: Easy to clean

Priority Score

Evidence (multiplier)

Final Score Rank

Strategy A 1 3 2.0 proven (4) 8.0 2Strategy B 2 3 2.5 Likely effective

(3)likely effective (3)

7.5 3

Strategy C 3 2 2.5 7.5 3Strategy D 4 5 4.5 promising (2) 9.0 1Strategy E 5 3 4.0 emerging (1) 4.0 6Strategy F 3 3 3.0 promising (2) 6.0 5

Discussion & DecisionDiscussion: Can some of the potential

prioritization criteria be combined based on similarity?

Could potentially combine some criteria:Population Impact AND Expected ReachCommunity Support, Capacity to Implement, AND

Opportunity for Leverage

Decision: Vote on combination(s) suggested.

Discussion of Additional CriteriaCould consider the following additional criteria:

Alignment with National PrioritiesEvidence Level

In a nutshell…Levels of Evidence in PSDProven: systematic or narrative reviews; considers study

design and execution, external validity, body of evidence, and results

Likely Effective: peer review articles in scientific literature; considers study design and execution, external validity, body of evidence, and results

Promising: written program evaluation without formal peer reviews; considers summative evidence of effectiveness, theory, and formative evaluation data

Emerging: ongoing work with little evidence so far, but sound theory and evaluation in place

Not Recommended: evidence of effectiveness is conflicting and/or of poor quality and/or suggestive of harm

Discussion & DecisionDiscussion: How many criteria should be

included in the prioritization rubric? Top 3Top 5All

Decision: Vote on number of criteria to be included in final prioritization rubric.

Discussion & DecisionDiscussion: Should any of the prioritization

criteria be more heavily weighted than others?

Decision: Vote on weighting.

Clarify Strategies and State Roles

The “Ask” Regarding Strategies and CDPHE Roles 58 strategies presented (only included those

with some level of evidence)

Asked to select appropriate CDPHE roles and identify information gaps for each strategy:

Three purposes: Info about CDPHE role may provide information

that will be helpful at a later stage in the EBPH process.

Can we eliminate any strategies based on no CDPHE role?

What information is needed before we can move forward?

Pages 4-13

Notes:

1. Evidence level noted inaccurately on survey for 2 Built Environment Strategies:

• Transportation policy/access to transit

• Open space preservation

2. Diabetes Prevention Program has now been designated as “Proven”

Takeaways

Four roles were most commonly noted for CDPHE: Influence state-level policy with regard to this strategy Provide funding to local or state partners to implement this

strategy Provide guidance and/or technical assistance to local or

state implementation partners Coordinate activities with other state agencies

No strategies met elimination criteria, but the specific CDPHE role was unclear for 8 strategies (see page 5).

Members desire more information about 8 strategies (see pages 6-13)

Literature Review ResultsSECTOR EVIDENCE RATINGS

Schools 7 Likely Effective; 2 Promising; 5 Emerging

Child Care 4 Likely Effective; 3 Emerging; 2 Not Recommended

Food Systems 2 Likely Effective; 3 Promising

Health Systems 2 Proven; 3 Likely Effective; 1 Promising; 1 Emerging

Worksites 5 Proven; 1 Likely Effective

Media 1 Likely Effective; 2 Promising; 2 Emerging; 1 Not Recommended

Community 1 Proven; 5 Likely Effective; 1 Emerging

Built Environment

5 Likely Effective; 2 Emerging

Discussion & DecisionDiscussion: Should strategies where no clear

agreement was reached about a CDPHE role be included or excluded from the prioritization process?

Decision: Vote on whether to eliminate 8 strategies in final prioritization rubric.

Discussion & DecisionDiscussion: Should any strategies not previously identified

by the sector teams be included in the prioritization process?

For each new strategy discussed, need: Description Evidence: Individual Study? Narrative Review? Systematic Review? Potential State Role

Decision: Vote on whether or not there is a state role for each new strategy proposed. If so, the strategy moves to the list to be prioritized.

Final Tally of # of Strategies Moving Forward for

Prioritization

Next Steps:Prioritization Process

When Purpose/Actions

Meeting 1 (2/29) • Getting Started• Agreements For Moving Forward

Between Meetings(e-input)

Survey Coming Tomorrow (due 3/9)• Review/Provide Input on Proposed

Criteria• Identify State Roles for Strategies

Meeting 2 (3/19)9:30am -12:30pm

• Review/Finalize Criteria• Clarify Strategies• Clarify State Roles

Between Meetings(e-input)

Prioritization Survey (sent 3/21, due 3/26)• Rate Strategies Using Final Criteria• Refine Concise Statement

Meeting 3 (4/2)9:30am – 12:30pm

• Share Results of Prioritization • Gather Additional Input for Ex Committee

Thank You and ClosureOn the index card provided let us know

What is still on your mind?

Leave note sheets on table before you leave

For those on the phone: email your responses to: Laurie.schneider@ucdenver.edu