Post on 26-Dec-2015
transcript
CEGAA-OSI EE Partners WorkshopMarch 09
Urbanus KiokoCentre for Economic Governance and AIDS in Africa
Your Understanding of Research?
Systematic means of solving a problem
Process in which observable verifiable data is systematically
collected in order to describe, design or predict events
Systematic, objective and scientific investigation of a
phenomenon targeting specific aspect for the purposes of
discovering, interpreting facts, principles and theories
The search for knowledge through objective and systematic
methods of generalisation and theory formulation
To inform decision making by producing evidence
Evidence-Based Advocacy Framework
Project definition/over
all goal
What is your desired long-term impact, the outcome of your efforts, the rights you wish to protect or promote, constitutional and legislative obligations, international/regional commitments. What are you hoping to achieve?
Advocacy objectives
SMART objectives to bring about the change needed to achieve your GOAL. (SMART: Specific, Measurable, Achievable, Realistic, Time-bound), (Note that the advocacy strategy should begin to be developed now, but will be finalized when the preliminary research findings are available).
Problem statement
Clearly state what the problem is that you are going to address. Many sources to find topics or issues that can lead to research questions: Personal experience, Articles in professional periodicals, Unpublished research by others. Focus your research so that it is "do-able." Be careful! Don't try to do too much in one study.
Relevance in terms of being a priority problem◦ How large or widespread is the problem◦ Who is affected◦ How severe is the problem
Avoid duplication Feasibility Political acceptability Applicability of possible results and
recommendations Urgency of the results for making a decision Ethical acceptability
Defining the specific research
question/s
What do you want to answer? What principles do you want to test? What data will the stakeholders require for the advocacy messages? Use Lit.Review (throughout process)
The scope of the research
State, private, donor, household expenditure/ allocations? Sources, agencies, providers, services? Inputs/ outputs/ outcomes/ impact? Federal/national and/or state/sub-national? Recurrent and/or development? Sectoral analysis? Programme analysis? Years to be covered? Population focus? Vulnerable group/ gendered perspective? Your framework of analysis – rights-based, sectoral, issue, equity?
Research aim and objectives
To achieve the advocacy goal, identify more realistic aims of the research required to support the advocacy campaign, and develop specific, measurable objectives of the RESEARCH.
The scope of the research-
Literature review
Survey of professional literature that is pertinent to your particular question.
Find what others have done in relation to topic of interest.
Why review◦Prevent duplication◦Assist in refining statement of problems◦Acquaint researcher with methodologies already used◦Assist in justifying a research
What potential sources of data◦Individuals, groups, and organisations◦Published information, unpublished information
Where can information be found◦Local◦National◦International e.g. internet
Key partners & participation
process. Stakeholder involvement/ ownership.
Sources of data? Collectors of data? Users of data! Benefits/ risks. Plan to involve all the stakeholders to ensure ownership and relevance of data collected and research process.
Methodology:a.Study design
Quantitative, qualitative, participatory, costing, economics – micro/macro analysis, budgetary analysis, expenditure tracking techniques, community/Citizen action research – social audits, satisfaction surveys, score cards. Donor surveys, household/ clinic/ district surveys. Focus Gp Discs.
Identification of appropriate indicators
What indicators/ information will you use to show the principles you are trying to measure?
Research design can be thought of as the structure of research -- it is the "glue" that holds all of the elements in a research project together
A design is used to structure the research, to show how all of the major parts of the research project -- the samples or groups, measures, treatments or programs, and methods of assignment -- work together to try to address the central research questions.
Primary vs Secondary
Exploratory vs Conclusive
Quantitative (BMET analysis, pre-coded closed ended questionnaires, surveys) and/or
Qualitative (open ended questions)
qualitative research attempts to capture "the big picture" and see how a multitude of variables work together in the real world.
It examines naturally occurring behavior, so the investigative methods are as non-intrusive as possible
Focus group discussions)
Continuous (panels, tracking studies) vs Ad hoc / once-off (surveys)
Primary vs Secondary
Primary Research
Data generated specifically to address
the problem/ issue
Secondary Research
Data originally generated for another
reason
c. Sampling techniques
Representative sample? Apply formula. Purposive sampling of districts/ households? Give criteria for respondents’ inclusion/ selection. How will you identify and select respondents from the group you are focusing on?
d. Tools Budget analysis, expenditure analysis, literature review, focus groups, workshops, interviews (face-to-face, telephonic/ questionnaires (self-administered?), surveys, roundtable discussions, stakeholder meetings, observation?
b. Key respondents / your sample
population
Link to earlier decision on population/ vulnerable group. Identify key ‘holders’ of information (govt, donor, service users). Regions/ states? Districts? Households? Individuals? Key officials from MoF, MoH, Parliament,, donors, programme managers, relevant civil society groups, NGOs, etc.
The listing of the accessible population from which you'll draw your sample is called the sampling frame
Sampling: process of selecting units (e.g., people, organizations) from a population of interest so that by studying the sample we may fairly generalize our results back to the population from which they were chosen
The sample is the group of people who you select to be in your study.
Random sampling-gives each of the units in the population targeted a calculable and non-zero probability of being selectedAssumption is that each unit has equal chance of being
selected SRS: the members of the population (N) of interest
are numbered and a number (n) of them are selected using random numbers without replacing them
Each sample unit can only appear once in the sample
Each pop member has an equal chance of selection
Steps you need to follow in order to achieve a systematic random sample:
1. number the units in the population from 1 to N 2. decide on the n (sample size) that you want or
need k = N/n = the interval size 3. randomly select an integer between 1 to k then take every kth unit N = the number of cases
in the sampling frame ◦ Note: ◦ n = the number of cases in the sample ◦ NCn = the number of combinations (subsets) of n from N
◦ f = n/N = the sampling fraction
Let's assume that we have a population that only has N=100 people in it and that you want to take a sample of n=20.
To use systematic sampling, the population must be listed in a random order.
The sampling fraction would be f = 20/100 = 20%. Interval size, k, is equal to N/n = 100/20 = 5. Now, select a random integer from 1 to 5. In our example, imagine that you chose 4. Now, to select the sample, start with the 4th unit in the
list and take every k-th unit (every 5th, because k=5). You would be sampling units 4, 9, 14, 19, and so on to
100 and you would wind up with 20 units in your sample.
Sometimes called proportional or quota random sampling,
Involves dividing your population into homogeneous subgroups and then taking a simple random sample in each subgroup – so that each sub-group is proportionally represented in the sample – you can then make inferences about these groups in the population.
Objective: Divide the population into non-overlapping groups (i.e., strata) N1, N2, N3, ... Ni, such that N1 + N2 + N3 + ... + Ni = N.
Then do a simple random sample of f = n/N in each strata.
In cluster sampling, we follow these steps: 1.divide population into clusters (usually along
geographic boundaries)
2.randomly sample clusters
3.measure all units within sampled clusters
The selection of clusters can be multi-stage
Involves selecting districts, the primary sampling units, within a region for the sample, and within these sample electoral wards and finally within these a sample of households.
Is a deliberate non-random method sampling, which aims to sample a group of people, with particular characteristic.
The criteria for selection must be justified and relate to the questions you are trying to answer
Can be justified with difficult access to certain populations eg. HIV+ patients attending the ARV clinic
Can be justified based on costs and time eg. Small case study
Is not generalisable!!
e. Sources of data
Budget books, budget reviews, MTEF plans, expenditure reports, Auditor General reports, ministry budgets and annual reports and costing reports, strategic plans, MoF reports, Parliamentary reports especially of budgeting committees, Public Expenditure Reports, health information systems, legislation governing public financing, Reports of the implementing agencies of the strategic plan, central bureau of statistics, household data, census, surveys (economic eg. I&E, health), previous studies, programme evaluations, impact assessments, any costing exercises (against which to measure the adequacy of budget allocations), financial management agencies, donor reports, etc
h. Review process – to ensure
validity, accuracy and
reliability of data
Stakeholder meetings to present preliminary findings, to discuss and influence the report recommendations. Country and external experts to review draft report – experts in health issue, public finance, budget systems etc. Internal review process within the team – sharing of experiences and solutions, editing each other’s reports. Using civil society networks to give critical feedback.
g. Data collection & management
process
Plan for using the tools to collect the information. Means of capturing data – software programmes, research assistants, data capturers.
f. Data collection tools
Questionnaires – closed/open ended, Questionnaires (mailed or in-person) Observations Interviews-personal: interviewer works directly with the respondent, can probe/ask follow up qns
f. Data collection tools-
questionnaire design-some
tips
Question Content1.How well does the qn address the content you are trying to get at. Is the Question Necessary/Useful? Are several questions needed? double-barreled question—need to split e.g. what is your occupation and monthly income?2.Does the qn have the necessary information for the respondent to be able to answer it?3.Specific-too general making information we obtain more difficult to interpret.E.g. How well did you like this workshop? 4.on some scale ranging from "Not At All" to "Extremely Well.“ What does it mean to say you liked the workshop very well? Biased or Loaded question: your own biases and blind-spots may affect the wording e.g. you may be against random street drug testing. What do you see as the drawbacks of random street drug testing? 5.Is the wording too direct?
too direct/disturbing for respondents. How did you feel about this workshop?
Conventions or rules-of-thumb in the survey design.
Start with easy, non-threatening questions, put more difficult, qns near end
Never start a mail questionnaire with an open-ended question
For filter or contingency questions, make a flowchart
Fully brief the team – project manager, executives
Key questions to be answered Objectives and methodologySampling/ selection detailsQuality control proceduresTimelinesCosts for each stageHuman resource requirements
TIP: Take time, keep records. Successful projects are well communicated, planned and every team member is clear about their role
Project planning& management
Select, train &brief field team
Select and train interviewers very carefully- match interviewer profile with
interviewee profile? - sensitive subjects - language/ cultural issues
Document the quality control procedures Train on field research ethics and specific
project requirementsTIP: Quality control all stages of fieldwork every day through the Supervisor network
Carry out a pilot exercise
- check flow of questions, skip patterns,
- language of questions, selection procedure,
length of interview Conduct a pilot FGD
TIP: Use experienced interviewers/ moderators for good quality feedback
Pilot
Data entry,cleaning
and checking
Develop editing rules and checks for all questionnaires - Check frequency distributions - Check against known statistics - Look out for inconsistencies TIP: Apply common sense. ‘Interesting’ data are often wrong
Data analysis
Excel, Quantitative - develop analysis specification (SPSS) - use techniques to help you understand relationships in the data Qualitative (NVIRO) - develop template for verbatim transcripts - common themes and differences TIP: Think through the analysis as you design the instruments “How will I analyse this?”
Data interpretation
Revisit objectives
Triangulate with other sources
Verify prelim findings with key
stakeholders
Compare with previous work
(lit.review)
Contextualise
Find the story
Report writing and
presentation
Plan and structure to tell the story – using the data as supporting evidence Summarise the key findings Keep it short (use appendices) Illustrate with diagrams, charts, illustrative quotationsDissemination workshopPolicy briefs, media etc
TIP: Rehearse presentations, circulate reports internally for comment. The quality of the deliverable is the basis of the policy maker’s/govt’s assessment of your work
• Research does not only inform advocacy, Advocacy also makes demands of research
• Research needs to answer key questions in order to be useful for advocacy purposes
For more information contact:Urbanus KiokoCentre for Economic Governance and AIDS in Africa Email: urbanus@cegaa.orgTel: +254-720-209-100