Post on 23-Feb-2016
description
transcript
Cross-border citizens’ network for human security
in Turkey and the Western Balkans
Belgrade meetingResearch methods and methodologies
Dr Sally Stares8 November 2013
2
Introduction This research programme is rich in both its diversity and its
shared core in project themes in methodological approaches
In this presentation, some ideas for how to strengthen the connections between the different projects, in terms of research questions methodological approaches
3
Quick sketch of methodologies proposed Desk research – e.g.
To identify key themes, and background information, e.g. statistics, legal frameworks, institutional arrangements, advocacy context
Case selection – e.g. Probability (random) sampling – crucial for generalisation,
inference Key informants, volunteers? Comparative (or single) case studies: selected for
characteristics of interest
Data collection – e.g. unstructured/semi-structured interviews, focus groups, media structured interviews, questionnaires, secondary analysis of
existing data
Data analysis – e.g. content analysis, narrative analysis, discourse analysis counting frequencies or proportions of certain behaviours or
perceptions
4
Illustrations of diversity of our research questions Questions of exploration:
What kind of threats exist to young people in virtual spaces? What are perceptions of insecurity?
Questions of distribution: What is the scale of conventional crimes committed by Roma
population in North-Northwest Bulgaria? Are legal limits/standards of working time and sick support
observed?
Questions of process: What is the mechanism of controlled voting in small
municipalities in Bulgaria? How are schools compromised as safe spaces?
Questions of causation: What are the consequences of housing reforms in Montenegro? How has the privatisation process increased vulnerability of
former workers?
5
Examples of methods for exploration Qualitative approaches are well indicated, especially where
respondents are free to raise issues, to set the agenda Interviews Focus groups
Desk research and analysis of existing data sources may reveal key insights e.g. survey results, text data
Quantitative data collection methods not so directly useful, because of their highly structured format, predetermined by the researchers Although, multivariate analyses can reveal interesting
patterns and associations
6
Examples of methods for distribution Quantitative data collection methods are the most obvious
choice e.g. surveys
Then, sampling strategy is crucial For results to be generalisable to the target population,
probability (random) sampling must be used Typically outsourced to survey agencies who have
necessary sampling frame information, field force, etc.
Desk research and analysis of existing data sources may be a key tool here, if primary data collection cannot be done
Possibility of a quantitative data collection exercise at a later point in the research programme?
7
Examples of methods for process Qualitative approaches again well indicated
Interviews Focus groups
Suggest that a narrative approach to these methods might be useful? Encouraging respondents to relay sequences of events, etc.
Desk research and analysis of existing data sources may reveal key insights, and suggests avenues of enquiry for primary qualitative data collection
Quantitative data collection methods typically not so directly useful; tend to comprise a snap-shot of components rather than information on the dynamic links between them
8
Examples of methods for causation Gold standard: randomised control trials! To make claims of a causal relation between two phenomena
(say, housing reforms and insecurity), need to fulfil three requirements:1. Demonstrate an empirical association between them2. Establish the one doing the causing happened before the
one that suffered the effect3. Rule out all other possible explanations for the association
between them
Extremely difficult to achieve claims of causality, and harder as the extent of desired generalisation increases
However…connects to approaches such as process tracing, which maps on nicely to a more general narrative approach
…and, I think, is expressed in softer form in core human security questions…
9
Core research focus Understanding the spaces and forms of insecurity in the region,
using violence as a proxy for insecurity
Key overarching questions:1. How does the particular form of violence under study happen,
how is it manifested?2. How do people respond to it?3. What do people want to change? What would they wish the
situation to be? If possible to answer: how could that be achieved?
1 and 2 speak to questions of exploration and process 3 speaks to process and tentatively to questions of causation None speak directly to questions of distribution! Maybe 1 is
indirectly linked?
10
Questions for discussion (1) Can your existing research questions be reframed in terms of
these? I have a hunch that they can, but I may be wrong! Means in many cases a slight rearrangement, e.g. CRDP wants to inquire on conditions of occupational safety in
infrastructure projects in Kosovo, and examine the institutional support available to the vulnerable workers of this industry
1. Please tell me what it’s like to work here. What’s good, what’s not so good? How about your working hours, what are they like? And holidays? If people have problems, e.g. injury, what happens then?
2. How do you manage the difficult aspects of working here? Do you have any support, e.g. from the company, some security from
contracts?3. In an ideal world, what would you like it to be like here?
In the abstract, and in practice - any thoughts on what you would like to change?
Consider effects of calling security into question; double hermeneutic?
11
Questions for discussion (2) Can we arrive at a shared strategy/elements of a shared
strategy for case selection? I think case selection is key Being explicit about the communities we are studying How far we intend the results to be generalisable
Empirical generalisation, e.g. classic case of opinion poll Analytical generalisation, mapping out and identifying themes and
issues
Suspect that formal sampling may not be possible in many instances? For qualitative research, can adopt a strategy of corpus construction,
i.e. trying to discover all the relevant themes – keep sampling respondents until you are not hearing any new themes
Whichever way, explicit documentation of respondent selection will be key to methodological rigour and quality of our research
12
Questions for discussion (3) (Broader version of second question)… Exactly which parts of the different projects are core,
shared, and which are unique or peripheral or idiosyncratic? In terms of
Substantive topic (e.g. youth violence) Social group (e.g. Roma) Methods (e.g. types of information gained from interviews, from
focus groups, etc.)
A while ago I developed a taxonomy of choices in research, which I find helpful for clarifying the scope and nature of different pieces of research Then facilitates comparisons of different studies Others have found it useful, but don’t be obliged! Here it is
anyway…
Taxonomy of choices
13
Substantive research topic
Substantive theoretical framework
Research method / tool
Generalisability
Observations and data
Levels of analysis
Ways of representing (analysing and reporting)
Relationship between researched and
researcher / research instrument
Validation
Taxonomy of choices
14
Substantive research topic
Substantive theoretical framework
Research method / tool
Generalisability
Observations and data
Levels of analysis
Ways of representing (analysing and reporting)
Relationship between researched and
researcher / research instrument
Validation
Substantive research topic:
Concrete questions; Mary’s what, when, how, who, why?
Taxonomy of choices
15
Substantive research topic
Substantive theoretical framework
Research method / tool
Generalisability
Observations and data
Levels of analysis
Ways of representing (analysing and reporting)
Relationship between researched and
researcher / research instrument
Validation
Substantive theoretical framework:
Human security
Other tacit frameworks of knowledge? Contextual
social, cultural knowledge
Theoretical framework may play a major or minor role
Taxonomy of choices
16
Substantive research topic
Substantive theoretical framework
Research method / tool
Generalisability
Observations and data
Levels of analysis
Ways of representing (analysing and reporting)
Relationship between researched and
researcher / research instrument
Validation
Research method / tool:
e.g. survey, interviews…
Broadly, How to select
participants What mode of data
collection to use Key variation in extent
of: Personal contact
with subjects (internet surveys vs. in-depth interviews)
Intervention (covert observation vs. action research)
Formality of structure (experiments vs. participant observation)
Taxonomy of choices
17
Substantive research topic
Substantive theoretical framework
Research method / tool
Generalisability
Observations and data
Levels of analysis
Ways of representing (analysing and reporting)
Relationship between researched and
researcher / research instrument
Validation
Observations and data: Observations =
information in its rawest form (e.g. tape recording of interview)
Data = information in analysis format
A set of observations can be converted into different types of data
Data not ‘given’, but involve creative choices
Sometimes observations = data (e.g. questionnaires); sometimes several steps from observations to data (e.g. text coded from interviews)
Taxonomy of choices
18
Substantive research topic
Substantive theoretical framework
Research method / tool
Generalisability
Observations and data
Levels of analysis
Ways of representing (analysing and reporting)
Relationship between researched and
researcher / research instrument
Validation
Ways of representing How to represent data
to oneself as researcher: analyse
How to represent findings to an audience Sometimes
synonymous, sometimes two distinct steps
Conceptual question: nature of representation (prose, numerical system?)
Technical questions: details of system (e.g. for statistical models)
Taxonomy of choices
19
Substantive research topic
Substantive theoretical framework
Research method / tool
Generalisability
Observations and data
Levels of analysis
Ways of representing (analysing and reporting)
Relationship between researched and
researcher / research instrument
Validation
Validation Reassuring audience
and oneself of quality of research
For quantitative approaches, many angles E.g. convergent
validity, discriminant validity, reliability
For qualitative approaches, no direct equivalents, but… Quality markers, e.g.
richness of data, ‘thick description’, openness to surprise
Key that is often lacking: how would I know if this finding were wrong? i.e. guard against verificationism
Taxonomy of choices
20
Substantive research topic
Substantive theoretical framework
Research method / tool
Generalisability
Observations and data
Levels of analysis
Ways of representing (analysing and reporting)
Relationship between researched and
researcher / research instrument
Validation
Generalisability Of substantive
findings, to broader population/setting? Requires
probability sample
Of research instrument? E.g. questions
asked in interviews; standardised questionnaire
Empirical and analytical generalisation
Taxonomy of choices
21
Substantive research topic
Substantive theoretical framework
Research method / tool
Generalisability
Observations and data
Levels of analysis
Ways of representing (analysing and reporting)
Relationship between researched and
researcher / research instrument
Validation
Levels of analysis E.g. intra-individual,
inter-individual, individual-group, societal-level, individual-societal
May employ multiple levels of analysis Structural conditions
for violence Individual agency in
the face of violence Clarity on levels of
analysis used is often lacking
Harre’s ‘distributed’ and ‘collective’ representations
Taxonomy of choices
22
Substantive research topic
Substantive theoretical framework
Research method / tool
Generalisability
Observations and data
Levels of analysis
Ways of representing (analysing and reporting)
Relationship between researched and
researcher / research instrument
Validation
Relationships between researcher, research instrument, research subjects
i.e. what happens during the research process, and how prominent this issue is
E.g. questionnaires assume respondents understand and answer questions in basically the same way; where they don’t it is a nuisance
E.g. in action/participatory research, relationship defines the project
Ethical concerns key here
Key nature of human security
23
Possible next steps? Desk research completed
To clarify substantive and theoretical frameworks; set out research questions Level and/or type of generalisability agreed Levels of analysis greed
Method(s) selected To best serve research questions, and given capacities, time frame etc.
Cases selected ‘Sampling’ procedure explicitly defined Relationship between researchers and subjects explicitly defined, especially re
‘transformative’ potential of human security as a topic; measures for managing expectations, duty of care to respondents and all affected; advocacy implications
Plan made for forms of observations and data E.g. interview notes coded in any way?
Plan for how data will be analysed and reported Different for individual projects than regional report? Can we devise a common
core? Can we devise a scheme for validation/quality assurance? E.g. a standard
reporting frame for how the respondents were selected?