Post on 15-Nov-2014
description
transcript
The ISOR Project
No health without research.
And no research without evaluation
Panel Session - HTAi 2012, Bilbao
Maite Solans-DomènechCatalan Agency for Health Information, Assessment and Quality
(CAHIAQ)
Outline of the presentation
1. CAHIAQ call in clinical and health
services research
2. The ISOR project
3. Impact on informing decision-making
Outline of the presentation
1. CAHIAQ call in clinical and health
services research
2. The ISOR project
3. Impact on informing decision-making
• Mission: to fill local knowledge gaps and inform clinical and healthcare decision-makers
• Research topic identification and priority-setting designed and implemented to ensure that the knowledge gaps of decision makers are addressed
• Ex-ante, ongoing and ex-post evaluations (13 years experience)
• There is a scientific committee that monitors the whole process.
• Predominance of oriented research with local relevance and informing local decision-making
• 7 calls from 1996 to 2008
Call in clinical and health services research
Call in clinical and health services research
CallPrioritised
topicsFunded projects
NAmount granted
(€)Average Euros/
project
1996 15 18 510,860.0 28,381.1
1998 20 19 540,911.0 28,469.0
2000 30 25 841,417.0 33,656.7
2002 35 25 841,417.0 33,656.7
2004 30 22 1.021,721.5 46,441.9
2006 29 18 1.021,721.5 56,762.3
2008 20 14 1,021,214.0 72,943.9
TOTAL 179 141 5,779,261.0 37,622.4
Outline of the presentation
1. CAHIAQ call in clinical and health
services research
2. The ISOR project
3. Impact on informing decision-making
Closing the assessment cycle: The ISOR Project
Objectives:
To carry on the evaluation of the impact of research
funded by CAHIAQ (accountability)
To develop a methodology and/or tool for the
assessment of research projects (transfer methods and
capacity-building)
To contribute to the spreading of these assessment
practices beyond CAHIAQ (advocacy)
Adoption of the Canadian CAHS-ROI model:
Useful common framework for any different type of health research (from basic research to health services)
Useful for any stakeholder’s view (from funder to evaluator)
Provides a set of generic indicators in the five impact levels: −Advancing knowledge−Capacity building − Informing decision-making −Health benefits−Broad economic and social impacts
Closing the assessment cycle: The ISOR Project
Catalan Health Research
Economic and Social Prosperity
Determinants of Health
Improvements in Health and Well-being
Healthcare Appropriateness,
Access, etc.
Prevention and Treatment
Health Status, Function, Well-being, Economic conditions
Initiation and Diffusion of Health Research Impact
Global Research
Research capacity Impacts feed back into inputs for future research
Government
Research Agenda
Interaccions
/Feedback
Other industries
Health industry
Advancing Knowledge
Capacity Building
Informing Decision Making
Health Benefits
Economic Benefits
Rese
arch
Res
ults
Know
ledg
e P
ool
Inte
racti
ons/C
olla
bora
tions
• Clinical
• Health Services
• Population and Public Health
Public Information, Groups
Canadian Academy of Health Science.
Return on Investment framework 2009
Closing the assessment cycle: The ISOR Project
PAYBACK
FRAMEWORK
Topic identificationInputsProcess
PrimmaryOutputs
Dissemination
SecondaryOutputs
Adoption Final Outcomes
Outline of the presentation
1. CAHIAQ call in clinical and health
services research
2. The ISOR project
3. Impact on informing decision-making
Bottom-up assessment of CAHIAQ Calls
Descriptive
[based on questionnaires to 70 PI]
Case study (respiratory disease research)
[based on semi-structured interviews to 8 decision-
makers and 15 researchers related to 6 projects]
Impact on informing decision-making
Questions to PI (n=70) n (%)Types of changes carried out in practice based on the research outcomes Clinical 29 (58.0) Organizational -
centre/institution12 (24.0)
Patient´s behaviour 5 (10.0) Public Health management 2 (4.0) Legal/regulations 2 (4.0)Unknown / No changes in practice. 28 (40.0)
Stakeholders or recipients who have used or taken into account the research outcomes Managers and planners 24 (41.4) Clinical professionals 17 (29.3) Investigators/researchers 12 (20.7) Scientific Societies 5 (8.6)
Not taken into account/ unknown 13 (18.6)
70% (n=46)
•Stakeholders in clinical and health services
•Scientific community
•Scientific societies.
60% PI: (n=40)
Real changes
Clinical and/or organizational setting.
Impact on informing decision-making
Almost all projects have induced or may have induced changes in clinical practice or in the organization of health services
Changes
Direct / indirect
Contribute to a complex change in clinical practice or healthcare organization
“After two years ... it [home-based hospitalization] grew as a real service and an integrated care cross-sectional unit was created within the hospital”
“Yes, it has all generated a culture... We are essentially a clean unit…”
… some “good stories”:
Impact on informing decision-making
Scientific publications: scientific articles, communications in congresses
Direct transmission to potential users: clinical sessions, …
Unconventional or poorly studied ways for the knowledge transfer
Scientific societies - as authoritative sources
Connection between managers and research teams – direct/indirect participation and collaboration
The channels used to transfer new knowledge into clinical practice are complex
“... the participation of hospital and primary care specialists in charge was beneficial [to change]”
… dissemination….
Impact on informing decision-making
Structural barriers
Lack of channels for the translation of research
Organizational dynamics
Individual barriers
Personal factors: reluctance to change
Nature of research: gap between researchers and practitioners
“Overall, I would say that researchers are forced to take on many roles. I am ready to be a researcher, but not to make an electoral pamphlet. In this regard, I think the relationship with healthcare policies and health plans is not as it should be.”
“Because hospital directors change frequently, or because this issue is not deemed to be as important as it should be...”
...and translation barriers:
Impact on informing decision-making
Project: Risk factors for relapse in COPD
Capacity Building
Informed decision-making
Health benefits
Overall economic benefit
Input Primary outcome
Secondary outcome
Adoption Final outcome
Relevant contribution (few
prior studies) Dissemination in
Scientific Societies
Contribution to the design care to chronic patients and
hospital management*
Cost reduction
Prevention in
excerbations
Identification of modificable
risk factors
New lines of research
Forecast of health care services to reduce
admissions
Difficulty to cultural change due to the patients (learning with the disease)l
IMPACT
PHASE
Impact found mainly in intermediate levels
Gaps or distances between potential and real impacts
Other impacts emerged implicitly and not intentionally
Impact on informing decision-making
ISOR project: Challenges
‘Ongoing project’
SIRECS
Comparability (across countries, research programmes,
disciplines, …)
Attribution
Time-lag
Interviewees (researchers, health policy or healthcare
decision-makers) interpret narrowly the terms (‘impact’)
General context: incipient culture of assessment
Thanks to Paula Adam, Marta Aymerich, Silvina Berra, Imma Guillamón, Gaietà Permanyer-Miralda, Joan MV Pons and Emilia Sánchez for their contribution to CAHIAQ strategy and the ISOR project