Date post: | 05-Jan-2016 |
Category: |
Documents |
Upload: | may-henderson |
View: | 213 times |
Download: | 1 times |
Update onPerformance Measures Pilot
and Development of the Cancer Plan Index
Presented byDeborah Porterfield, MD, MPH
RTI International
Presented atCCC Program Directors Meeting
Atlanta, GA, May 13, 2008
RTI International is a trade name of Research Triangle Institute
3040 Cornwallis Road ■ P.O. Box 12194 ■ Research Triangle Park, NC 27709Phone 919-541-1293 e-mail [email protected] 919-541-6683
Outline
Performance measures pilotBackground and purposeMethodsExamples of preliminary results and issues raised in
the pilotNext steps
Cancer Plan IndexBackground and goalsMethods and where we areNext steps
Background: Performance Measures
Since 1998, the Division of Cancer Prevention and Control (DCPC) has provided funding to states, tribal organizations, and jurisdictions to conduct comprehensive cancer control (CCC) in their areas.
Evaluation was part of the Building Blocks and has been stressed by CDC from the beginning
In 2003 the DCPC initiated a process of developing an evaluation framework and performance measures for all CCC programs
Now CDC is asking programs to submit information on performance measures
Developed by CDC to assess recipient’s progress toward reaching required Program Activities One assessment tool for the entire US, including
Tribes and Territories, and USAPI Jurisdictions
Performance is judged by CDC through various methods including a Performance Measurement Assessment tool
“Measures of Effectiveness” are to be listed on the work plan and can be whatever programs decide. CDC does not “judge” them in any way.
Purpose of Performance Measurement
To establish accountability for public investments in cancer prevention and control;
To document the outcomes and accomplishments of funded programs and partners; and,
To facilitate a quality improvement process at the program level.
Further Purposes of Performance Measures for CDC
General information on CCC nationallyEx: The number of sectors, organizations and
individuals meeting around the CCC tableEx: Funds and in-kind other than from CDC
General knowledge of the program to help in technical assistanceEx: Sharing of satisfaction surveysEx: Proportion of interventions that are truly
“evidence-based”
Further Purposes of Performance Measures for CDC
Monitoring of the most important activitiesEx: Reviewing data at specified intervalsEx: Coalition representing “all” of your people
Documenting outcomes that we can be used to show that CCC “works”
At this point, results from the Performance Measurement worksheet are not being used for funding decisions
Methods
Advisory workgroupWithin CCC, both Program Evaluation and
Scientific Support teams represented
Framework based on CCC logic model
Draft performance measures
Extensive review
Development of a worksheet
Pilot of worksheet in 2008
Preliminary Results and Issues Raised
Year 01 Performance Measures
Response Rate
48 of 51 worksheets for states and D.C.
6 of 6 worksheets for tribes
8 of 10 worksheets for USAPI jurisdictions
Total of 62 of 67 worksheets were received
Update on Analysis
Focused review of USAPI jurisdictions and tribes already complete
Data entry underway for states
Numbers presented today represent 45 of the states, tribal organizations and USAPI jurisdictions
Response Burden
Median of 10 hours to complete the worksheet (range of 3–100 hours)
Some Potential Performance Measures Appeared to Go Smoothly
1.3 Assessment of members’ satisfaction with aspects of the partnership was conducted
62% of programs YES
2.1 Assessment of cancer burden using specified data sources and appropriate periodicity was conducted
96% of programs YES
6.1 The program has a written evaluation plan 64% of programs YES
And Some Additional “Informational” Measures Appeared to Go Smoothly
3.1 Amount of non-CDC funds Mean $2.3 million
Median $158 k
3.2 Amount of in-kind contributions Mean $642 k
Median $31 k
And Some Questions Need Work
5.1 Proportion of partner organizations implementing interventions directly related to the plan
How to know?
What counts?
5.2 Proportion of plan objectives being addressed
How to know?
5.3 Proportion of interventions that are evidence based
Definition of evidence based needs more discussion
7.1 Number and list of policy changes Need criteria for what “counts” — perhaps also in-process activities?
General and Measure-specific Revisions
Refinements in instructions
Calculation of proportionsEven if the performance measures is a proportion,
the worksheet will not ask it as such
Suggestions for additional sectors in 1.1
Next Steps
Analysis of all worksheets
Engage CCC Branch to review the data and suggest revisions to the worksheet
Continued involvement of programs in refinement of measures and worksheets for Year 02
Ongoing work of the CDC/RTI workgroup to refine the worksheet
Determination of benchmarks or standards
Inclusion of revisions in next funding announcement in fall/winter 2009
Conclusions
Performance measurement is an important tool for public health agencies to establish accountability and a culture of quality improvement.
With further refinement of the measures and evolution of a system of routine measurement, this activity has the potential to stimulate improvements in CCC at the program level and to document nationwide progress in CCC efforts
Cancer Plan Index
Background
A State Plan Index was developed by the Obesity program at CDC to review existing plans and provide guidance for new plan development
2 articles published in Preventing Chronic Disease
DCPC interested to adapt this methodology for cancer plans
Goals of the Cancer Plan Index Activity
Describe the current status of plans
Inform guidance from the DCPC to programs on CCC plans
Provide information to programs who are currently writing or revising their plans
Provide information for one of the performance measures in the current funding announcement
Methods for the Cancer Plan Index
Advisory workgroup
Program volunteers’ extensive review
Workgroup reworked components and added items
(currently) Review by larger group of program volunteers
Next Steps
Pilot in summer 2008
Review of all existing cancer plans
Outputs:Report with descriptive information about current
plansRevised instrument that will be available to
programs Guidance from CCC on plan content
Future performance measure for CCC programs
Questions?