WRITING RESEARCH PROPOSALS & PUBLICATIONS
CILIP IL Group and LIRG
Getting started
Dr Alison Brettle
• To help LIS practitioners get started in research or evaluation projects (and then write them up)– What is research? What is evaluation?– What are the challenges and practical
considerations?– When to use quantitative and qualitative approaches– How to plan a project
• No ..isms or ..ologies or paradigms!
Aims
• Defining – which questions are suitable?
• Designing – what methods can I use?
• Doing – how do I do it?• Describing – how do I tell
people about it?
4 D’s
10 good reasons to engage in research?
Context: why research?
• Discover causes • Understand activities• Understand behaviours• Explore perceptions• Predict future trends• ‘Blue skies’ – exploratory research• Testing hypotheses• Evaluating impact
Aims of research ...
Research Evaluation
“The systematic investigation into and study of materials and sources in order to establish facts and reach new conclusions” (Oxford Dictionaries, http://oxforddictionaries.com/definition/english/research)
“An enquiry” (Robson, 2002)
“A study with a distinctive purpose; it is not a new or a different research strategy”
“Often to assess the effects and effectiveness of something” (Robson, 2002)
Research or evaluation?
Successful Unsuccessful
Activity and involvement Expedience
Convergence Method or technique
Intuition Motivation by publication, money or funding
Theory Lack of theory
Real world value (Robson, 2002 – adapted from Campbell, 1982)
Characteristics of research
Generic categories of research purpose
• Exploratory• Descriptive• Evaluative• Predictive• Explanatory• Developmental
To find out if user needs are met
To improve a service
To assess the outcomes of the service
To find out how a service is operating
To assess the efficiency of a service
To understand why a service works (or doesn’t)
What should the focus be?
How can we make the service better?
Does the service meet its planned goals?
What happens in the service?
How do the costs compare with the benefits?
Are we reaching the right group?
What happens to users as a result?
Is the service operating as planned?
Is it more or less efficient than other services?
Are we providing what users need?
Is it worth continuing?
(Adapted from Robson, 2000)
Some purposes of evaluation
• Deciding the focus• Developing the questions• Choosing a strategy• Selecting the method(s)• Arranging the practicalities• Collecting the data• Analysing the findings• Reporting what you have found
What is involved?
What skills do I need?
Everyday life skills for research
• Reading• Listening• Watching• Choosing• Questioning
SummarisingOrganisingWritingPresentingReflecting
• Research methodology – the approach or perspective taken to do the research
• Research design – how you do the research• Research methods – the tools you use to do
the research
A note on terminology
Getting started……Defining
Why do you need a research question?
• Offers direction throughout the study
• Guides the search strategy and choice of data collection method
• Suggests the format of likely answers
Start with a topic• What are you interested in?• What problems are there at work?• What issues are topical?• What do we know little about?• What do people disagree about?• What do influential people want to
know about?• What did your mum tell you to do?
???
??
?
?
Can the topic be researched?
• What is pain?
• Why are people anxious?
• Is Coke better than Pepsi?
• Why do girls get pregnant?
• Should we return to the moon?
• Can teenagers live on only noodles and beer?
• Why do patients not take their drugs properly?
Back to your topic, then…
Is it too broad (or too narrow)?
Is it researchable?
Is it worthwhile?
Does is still grab you?
Focus of questions – and studies
• What? (What is happening?) Exploratory
• What? (What has happened?) Evaluative
• What? (What will happen?) Predictive
• What? (What caused it?) Experimental
• Why? (Why is this happening?) Explanatory
• How? (How could things be different?) AR
• How? (How many?) Survey
Defining!• Explanatory – Do doctors find
things quicker after being taught search skills?
• Experimental – What works best – face to face or online teaching?
• Exploratory – What are students’ experiences of information literacy training?
• Descriptive - What are users perceptions about information literacy training?
Components of a research question
• Concise and direct
• Understandable
• A researchable problem
• Focused on central issues
• Multiple questions?
• Sub-questions?
Activity 1 - • In small groups
• What are the key issues about information literacy?
• What questions could be asked?
• …are they what, why or how questions?
– …what sorts of answers might they prompt?
• It may help to think of a problem
• Why do a literature review?
Using the literature
Why…• Demonstrates your subject knowledge• Improves your writing skills• Contextualises your research• Helps direct your research• Helps formulate your research questions• Can’t reinvent wheels – justify your original contribution• Provides material for comparison in later discussion sections
The literature can help justify/discuss
• Whether your findings confirm those of other studies• Whether your findings extend other studies• Whether your findings break new ground• Whether your work raises issues about the methodological
choices used by other studies• Whether your work challenges existing theoretical
approaches to your subject
Your literature review should be
• A coherent synthesis of existing research which – Demonstrates the context of your work– Involves thematic lines of argument round the research question– Demonstrates trends in how the topic has been treated by other
researchers– Makes links to the themes of your study– Shows a clear gap where your study fits in
What does this involve?• Assessing the value of the literature at a number of levels
– Individual papers – eg significant material– Collections or groupings of papers
• Emphasising the limitations of existing knowledge– Identifying the gaps – promote the value of your research– Justify the contribution of your study
What doesn’t it involve?• Lists of references• Long descriptions/summaries of other studies• Inaccurate citations• Illogical flow• Overuse of quotations and just paraphrasing of other’s
work• etc
Another way of thinking about it• Literature review as a map of the
field and its debates• Purposes of a literature review:
-> identify and summarize key paradigms and arguments
in your field -> position
yourself in relation to these arguments to set up your
own argument
Can learn from (or do a) Systematic Review
A review of all the literature on a particular topic, which has been systematically identified, appraised and summarised giving a summary answer.
What is a systematic review?• An overview of primary research
studies conducted according to explicit and reproducible methodology
• A rigorous method of summarising research evidence
• Shows what we know and don’t know about a topic area
• Provides evidence of effectiveness (or not) by summarising and appraising relevant evidence
Systematic review process• Define/focus the question• Develop a protocol• Search the literature (possibly 2 stages scoping and actual
searches)• Refine the inclusion/exclusion criteria• Assess the studies (data extraction tools, objective manner)• Combine the results of the studies to produce conclusion• Place findings in context – quality and heterogeniety of studies,
applicability of findings
Your protocol
• Plan what you are going to do in the review
• Set out the background and objectives
• Outline the resources you will use• Establish inclusion/exclusion
criteria• How will data be extracted (what
will be extracted)• How will you synthesise literature
• Keeps you on track and focussed
How will you refine the inclusion/exclusion criteria?
• Tighter the criteria =– less papers to review
• BUT – will your review draw any
meaningful conclusions?– Will it cover all relevant
perspectives• Focussing the question v quality
of studies
Know what you want to find out????
• Think about your structure before you start writing
• Use a tool so that you record the same information about each of the study
• Make sure it captures the elements that you wish to write about in your final report
• Don’t just cherry pick the bits you like
• Quality of studies – what evidence are you going to include? How are you going to assess the quality
• LIS specific or adapt one from healthcare– http://
nettingtheevidence.pbworks.com/w/page/11403006/Critical%20Appraisal%20Checklists
– HCPRDU tools• http://usir.salford.ac.uk/13070/
• LIS specific systematic reviews– http://lis-systematic-reviews.wikispaces.com/Welco
me
Critical appraisal and SRs in LIS
Quantitative v Qualitative?
What is quantitative research?• Objective approach, neutral• Scientific? Experimental? Non-
experimental– Introduces a change and collects
data about effects– Specifiy design, collect data about
effects• Fixed (Robson)
– Set out what you are going to do and how you are going to do it
– Follows well established procedures• Samples, variables,
measurement, control, confounders
What is quantitative research?• …allows you to count things• …may try to prove things• Answers “what” or “how” questions
– Questionnaires (to collect numerical data),
– Usage figures– Web logs
• Understanding behaviour or perceptions or views
• Less structured • Obtaining meaning
• “Why” questions or “how things are perceived
• In reality – mixed methods are often used
What is qualitative research?
Methods: summary
What? Why? How?
• QUANTITATIVE• Counting/measuring• Large samples (often)• Questionnaires• Analysis of statistics,
weblogs
• QUALITATIVE• Understanding of
behaviour• Perceptions• Exploratory research• Interviews, focus groups• Observation
Some Worked Examples• What database is best for searching on the topic
of severe mental illness? (Counting – Quantitative, simple descriptive statistics)
• Teaching online is as good as teaching face to face (Proving – Quantitative, quasi experimental or experimental)
• Which is best? Mediated searches or teaching users to find information? (Mixed – simple descriptive statistics, inferential statistics and thematic analysis of users views confirmed with focus groups)
• Do doctors receive training in searching?• Do doctors use the methods they were taught in training?• Do doctors find relevant material?• How often do doctors need to search using online
databases?• How frequently do doctors use online databases?• What do doctors think about the training they receive?• Do doctors percieve that the training they receive equips them to
be evidence based practitioners?• Does the training enable doctors to be better evidence based
practitioners?
What are doctors experiences of searching?
Defining Tips!• Make sure your question isn’t
too big/broad/narrow• Make sure your objectives are
smaller than your aims• Think – what exactly is it that
you wish to find out. Does your question really reflect this?
• Don’t build in assumptions– What are the benefits of
teaching doctors to search?• The clearer the question – the
easier it is to find the answer!
Activity 2: Defining• Think of a question that
you could answer by a research or evaluation study
• It may help to think of a problem!
• You may want to break it down into objectives that will allow you to collect data to answer the question
• Does it lend itself to quantitative, qualitative or mixed methods
Designing• What methods are you going to use to collect the data to
answer your question?– Involves asking questions of people, systems or texts– Involves testing? A hypothesis?– Could use questionnaires or data you already collect or other
measure or test or “experiment”
• Who (what) is your population?• Sampling – random, representative, purposive,
theoretical, snowball?• Valid/trustworthiness – does it do what it says on the tin?• Reliable – does it do it consistently/accurately?• Ethics – is your approach ethical ?• Bias and confounders - can you avoid them,
or account for them?
• Need to be considered at all stages:– Formulating questions– Gathering data (sampling, informed consent)– Analysing data (anonymity, confidentiality)– Writing up (reliability, accuracy)– Dissemination
• Working with particular groups– ‘hard to reach’– Children– NHS ethics
Research ethics
• As part of the degree course nurses need to learn IL skills – to help them through assessments (obtain their degree) and to ensure competency in professional practice (evidence based)
• What are the ethical issues involved?
An experimental study of information literacy training to pre-registration nurses (testing online training v
traditional teaching)
An experimental study of information literacy training to pre-registration nurses (testing online training v
traditional teaching)• Training session(s) linked to assessed work
Two training sessions – 4 tests– Ensure that both methods cover the learning outcomes in the same
way– Ensure both groups receive the same information at the same time– Ensure that the cohort wasn’t being tested for anything else– Ensure that test didn’t affect their assessment or its results– Students were able to opt out of intervention group and/or not have
their “test” results included in the study– University ethics procedure/school procedure– Student names/numbers not used (analysed by group not individual)– Crossover design
Design – Tips!• Look for examples of similar
studies – can you use/adapt the approach
• Has someone else developed a tool you can use?
• Make it feasible and manageable
• Be pragmatic - be as rigorous as you can whilst being aware of the limitations
• Make sure it is appropriate for the question
• Write a research proposal
Activity 4: Designing• Go back to your question• How would you design
your study to answer it?– What are your aims and
objectives?– What data do you need to
collect?– How are you going to
collect it?– Who are you going to
collect it from?– What
tools/methods/approaches are you going to use?
Doing• How am I going to collect the
data/information?– Eg online or paper questionnaires,
interviews, observations, recording, transcribing
• How am I going to analyse the data I collect?– Excel, SPSS, Descriptive statistics,
Inferential statistics, content analysis
• Do I need any help? (You may want to seek this at the design stage)
• Do I have the right skills?• Do I have enough resources?
Describing• What should I write about and how should I do
it?– Need to explain what you have done and how
you did it. – Need to present your study in a way that is
meaningful for the particular audience.– Need to think about style
• Where should I write about it?– Project report– Newsletters– Journals – Evidence Based Library and
Information Practice, Library and Information Research, subject specific
– Posters – conferences or study days– Presentations – internal, conferences
• If you are not going to do anything with it – why do the research?
Describing - Tips• Think – what are the key messages for
this audience• What is the best way of presenting the
data?• Can the audience understand what I’ve
done?• Can the audience work out if it is valid?
Reliable?• Have I explained and addressed the
limitations• Don’t “hide” results• Have I “answered” my
objectives/research question?