To what extent do postgraduate students understand the principles of mixed methods

in educational research?

UKCGE 3rd International Conferenceon Professional Doctorates

Florence, 2-4 April 2012

Dr David PlowrightProgramme Director: EdD

Centre for Educational StudiesUniversity of Hull, UK

d.plowright@hull.ac.uk

What’s the question?

Interviewer: How do you reconcile what is supposed to be irreconcilable or incommensurable?

ED candidate: It’s that ontological question, isn’t it?

2

Professional doctorate focus

• A professional problem or issue• Practice-based• Using mixed methods• Epistemological positioning of a

pragmatic/Pragmatic approach

3

Methodology literature

• Lack of coherence and clarity in ‘mainstream’ methodology literature – and the ‘third way’ mixed methods publications.

• Confused understanding about research generally, and mixed methods in particular.

• Still reference to qualitative and quantitative research.

4

Definitions

‘A mixed methods study involves the collection or analysis of both quantitative and/or qualitative data in a single study in which the data are collected concurrently or sequentially, are given a priority, and involve the integration of the data at one or more stages in the process of research’ (Creswell, et al,

2003: 212).

5

Definitions

‘The incorporation of various qualitative or quantitative strategies within a single project that may have either a qualitative or quantitative theoretical drive’ (Morse, 2003: 190).

6

Definitions

• Mixed methods is … • … firmly embedded within traditional research

paradigms or perspectives.• It draws from the same concepts,

methodologies and axiologies as ‘mainstream’ approaches.

7

The ‘Q words’

• The ‘Q words’ are the root - and the route - of all evil. Once you embark in either direction, then the journey is fraught with conceptual, methodological and axiological difficulties.

8

Lack of clarity over…

• Conceptually: the meaning of the use of terms qualitative and quantitative and their associations with the epistemological structuring of the research process.

• Methodologically: whether there are quantitative methods and qualitative methods

• Axiologically: to what extent personal, social and theoretical values do and/or should inform research decisions and activities.

9

Struggles

• The journey is also fraught with political and intellectual struggles associated with the privileging of particular structures of knowledge.

• Therefore…

10

TIME TO BAN THE ‘Q WORDS’?

11

Ban the ‘Q words’?

• Despite claims that using mixed methods frees researchers from the constraints of paradigmatic thinking, it ‘can actually reinforce the binary positioning of the qualitative and quantitative paradigms’ (Symonds and Gorard, 2010: 133).

12

Integration?

• A new conceptualisation of mixed methods may now be needed.

• The next development in the area of mixed methods should be based on characteristics of integration and not just ‘mixing’ or ‘combining’ different approaches to research.

13

Understandings

• There is a need to begin to understand how best to teach students about these ideas (Earley, 2007).

• One first step in this process is to be aware of how knowledgeable students are about research methodologies in general and mixed methods in particular.

14

The empirical investigation

• Research participants: students registered on masters and doctoral programmes of study from two UK universities.

• Data collection in two stages:– Stage 1: On-line questionnaire survey using

SurveyMonkey: 118 respondents– Stage 2: Small group face-to-face discussions with

one EdD cohort in one university: 12 participants.

15

Which of the following labels would youapply to the list of research approaches?

16

Mixed views of mixed methods

• There is some evidence from the findings that post-graduate researchers find the idea of ‘mixed methods’ useful but they appear to hold mixed views about the principles associated with this kind of research.

• There is also some confusion about the research process.

• And a lack of confidence

17

Confidence

People talk about qualitative and quantitative methods as a dichotomy don’t they, but a case study could have several different methods? I was just reading Burgess, and they tend to say that case studies are narrative, and that has actually put me off using them. I don’t want to just use a narrative, I actually want to use other methods.

18

Traditional

Even if you are doing an experiment…you could still have your controls but still do some qualitative data gathering…maybe not though… because it all has to be very measurable in the experimental work.

19

Confused?

Well, where do you mix and where don’t you? It is all really confusing in what is really a mixed methods, you know. I am very confused about what mixed methods is.

20

Confused?

My observation will be qualitative, and then obviously the counting of the dots and the crossing of the t’s of the scribbles will then be my quantitative, so that’s, maybe, that’s not right! That’s wrong. I don’t know.

21

Doctoral researchers

• Doctoral researchers are more likely to view educational research as:– involving measurement– collecting empirical data– unable to achieve procedural objectivity when

collecting data

22

Overseas students and researchers

• Overseas students and researchers are more likely to view research as:– involving measurement– collecting empirical data– able to achieve procedural objectivity when

collecting data– aspiring to be scientific– equating validity with objectivity and scientific

method

23

Gendered fields of academia

You can’t just go down the qualitative route in psychology, so that’s I why I am sort of veering towards a mixed methodology because I don’t really want to do some research and have it pushed aside by people as a female type approach, because being in psychology is generally gendered, you get a lot of males who do the cognitive stuff, the hard cutting edge of science and they are very dismissive of the qualitative areas, and I want to actually say well women can do the quantitative stuff as well as the qualitative.

24

Q words again

• Plowright (2011) argues that there are no qualitative/quantitative methods or approaches or strategies…

• There are qualitative and quantitative data but…even those terms are less than helpful.

25

26

What’s the problem?

• There must be a way through the difficulties, challenges and contradictions that characterise the field of research methodologies.

• Is the central issue is about the codification of experience and knowledge?

27

Codes

• A system whereby one thing represents another through, inter alia, drawing on a set of rules that are structured and are structuring…

• …through the process of signification or semeiosis…

• …whereby a signifier becomes associated with a signified through agreement among users.

28

Common characteristics…?Morse code

Algebra

Geometry

Organic chemistry

Highway code

Chess

Musical notation

Add your own…!

Fashion

Poetry

Meals

Jazz

Riding a horse

Keeping a garden

Abstract painting

Add your own…!

29

The ‘rules’ are…relatively…

Logical Poetic

Explicit Implicit

Structured Less structured

Inflexible Flexible

Certain Less certain

Codes

• Links with Karl Maton’s (2000) Legitimation Code Theory (LCT) which draws on ideas from Pierre Bourdieu and Basil Bernstein.

• Each codification process strives to generate languages of legitimation which lay claim to ‘the truth’ and an overall strategic dominance within and between fields of study.

• Some codes or set of rules within or between particular fields are privileged over others.

30

The next steps?

• The next steps in developing a conceptual framework for an integrated methodology in social and educational research?

• Does the codification of knowledge help us understand and challenge the traditional arguments underpinning the research process?

31

Is this the answer?

Interviewer: How do you reconcile what is supposed to be irreconcilable or incommensurable?

ED candidate: It’s that ontological question, isn’t it?

Interviewer: No, it’s the one about codes and knowledge structures.

32

If you would like your doctoral candidates to participate in the research, stage 1 and/or stage 2, please contact Dr David Plowright: d.plowright@hull.ac.uk

THANK YOU

33

References

Creswell, J. W., Plano Clerk, V. L., Gutmann, M. L. & Hanson, W. E. (2003) Advanced mixed methods research designs, ch 8 in Tashakkori, A. & Teddlie, C. (Eds) Handbook of Mixed Methods in Social and Behavioural Research. London: SAGE.

Earley, M. A. (2007) Developing a syllabus for a mixed-methods research course, International Journal of Social Research Methodology, 10(2), 145-162.

Eisner, E. (1993) Objectivity in educational research, Curriculum Inquiry, 22(1), 9-15.Maton, K. (2000) Languages of legitimation: the structuring significance for

intellectual fields of strategic knowledge claims, British Journal of Sociology of Education, 21(2), 147-167.

Plowright, D. (2011) Using Mixed Methods: Frameworks for an Integrated Methodolog. London: SAGE.

Symonds, J. E. and Gorard, S. (2010) Death of mixed methods? Or the rebirth of research as a craft, Evaluation and Research in Education, 23(2), 121-136.

34