Mixed Martial Arts for Researchers

Design-based Research (DBR)

Christian Glahn & Agnieszka (Aga) Palalas

mLearn 2013

Be prepared for the truth and nothing but the truth!

Chapter 1 The Nature of the Art

4

“Design experiments” > design-based research = design research = DBR = EDR = researching innovative educational designs in their naturalistic settings interventional research >practice>research

(Brown, 1992; Collins, 1992)

5

A systematic but flexible methodology aimed to improve educational practices through iterative analysis, design, development, and implementation, based on collaboration among researchers and practitioners in real-world settings, and leading to contextually-sensitive design principles and theories.

(Wang & Hannafin, 1999, p. 7)

DBR is a qualitative multi-method approach for exploring uncharted domains while designing effective solutions for these domains.

Qualitative means that the outcomes of DBR will not be able to test hypotheses

DBR is not a qualitative method!

Exploring means that, both, practice and theory do not suggest anything meaningful.

DBR identifies working hypotheses

Uncharted means that theory (aka "prior research") does not predict the potential outcome of a project.

DBR lets theory emerge by isolating research problems.

Designing means that the project outcomes include at least one working solution supports your claims

DBR does not simply transfer a solution from one domain into another

Chapter 2 When it is Time to Fight and When to Run

DBR is a flexible and powerful approach unless you are unprepared or unaware

Informed Exploration

Enactment

Evaluation: Local Impact

Evaluation: Broader Impact

Rule No. 1

Find out what you are doing (next)

Run if you want to

• Want to build a mobile app

• Use a mobile app in your practice

Fight if

• You have identified a problem rooted in practice

• You can't solve it by combining and deducing mobile solutions and theories

Rule No. 2

Find out what others did

Run if

• There are similar (commercial) tools for the problem

• There is a lot of literature

Fight if

• Tools need more than just a redesign

• The literature does not make sense for your problem

Rule No. 3

Make sure that DBR is indeed appropriate

Run if

• Your project can be structured into logical steps

• There are (working) hypotheses that can get tested

Fight if

• A practical solution is required

• The literature does not provide sufficient information

Chapter 3 There is always more than One Way

24

(Van den Akker, 1999, p. 9)

25

(Plomp, 2009, p. 19)

What are the characteristics of an <intervention X> for the purpose/outcome Y (Y1, Y2, …, Yn) in context Z?

Multi-method means that all means necessary are allowed.

It does not mean that it will be any easier

Rule No. 1

Always start from a solid research questions to guide the design process

Rule No. 2

Always answer your questions

Rule No. 3

Identify methods you can use

Rule No. 4

Select a method that helps isolating a working hypothesis

Rule No. 5

Document the identified working hypotheses for further research by others

•  Practice ï ðresearch •  Drawing on engineering and technological research •  Focusing on design, construction, implementation and

adoption of learning solutions •  Process focused and iterative •  Interventionist: applied solutions to real educational

problems •  Contextual: real people, context, and cultural

background, in-situ investigation/evaluation

DBR

•  Applied •  Participatory •  Collaborative •  Utility oriented •  Theory driven •  Unifying theory and practice •  Evolving •  Systemic

DBR

•  Rich feedback

•  Responsive

•  Flexible •  Agile BUT o  Multileveled and multifaceted o  Complex o  Messy

DBR

•  Co-developing theory and design in-situ •  Testing in real-world learning situations

involving all actors/end users •  With people for people •  End-users as partners (active co-creators) •  Combination of motivation and efforts •  Students as researchers •  Diverse evaluation perspectives

Participatory 1/2

“We are moving away from a passive information age towards an active participation age” (Farmer & Gruba, 2006, p. 149)

•  Diverse expertise of practitioners & researchers •  Collaboration & support •  Network of colleagues

o  Flexible dialogue from whenever o  Online platform and tools (e.g., Wiggio, Collaborate)

•  Dynamic feedback from actors o  At milestones and agile

Participatory 2/2

Context is king

•  Evolution of theory

•  Evolution of practice

Benefits of DBR (m-learning)

•  Build foundational understanding of m-learning trends and needs

•  Gain insight into learner m-learning behaviours •  Generate inspiration and ideas for appropriate design •  Define what functionality is most critical •  Improve the usability of infrastructure •  Inform future design solutions •  Provide professional development •  Raise awareness and understanding of m-learning

More Benefits of DBR (m-learning)

(Palalas & Hoven, 2013)

(Palalas & Hoven, 2013)

40

§  The scope of DBR •  complexity of the system – breadth •  coordination of research and design activities •  no objective measure of learning •  amount of data •  consensus-reaching and interaction among actors •  intensity

§  The role of the researcher •  multifaceted •  conflicting roles •  threats to validity

§  Transferability of findings

DBR: Recommendations

•  Rigorous data collection/analysis procedures •  Regular communication •  Findings documentation and reporting •  Sound conceptual framework •  Clear deadlines and deliverables •  “Flexible” (academic schedules) •  Solid project management a •  Collaborative DBR research team

“If a researcher is intimately involved in the conceptualization, design, development, implementation, and researching of a pedagogical approach, then ensuring that

researchers can make credible and trustworthy assertions is a challenge” (Barab & Squire, 2004, p.10)

•  Project context •  Objectives •  Audience •  Location •  Methods •  Schedule •  Outcomes

Plan Well

43

aga@epluslearning.com LinkedIn: http://www.linkedin.com/in/apalalas Presentations: http://www.slideshare.net/agaiza Publications: http://athabascau.academia.edu/apalalas

Bannan, B. (2009). The Integrative Learning Design Framework: An illustrated example from the domain of instructional technology. In T. Plomp & N. Nieveen (Eds.), An introduction to educational design research (pp. 53-73). SLO: Netherlands Institute for Curriculum Development.

Barab, S., & Squire, K. (2004). Design-Based Research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1-14. doi:10.1207/s15327809jls1301_1

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141–178.

Collins, A. (1992). Towards a design science of education. In E. Scanlon and T. O’Shea (Eds.), New directions in educational technology (pp. 15–22). Berlin: Springer.

Dede, C. (2004). If Design-Based Research is the answer, what is the question? The Journal of the Instructional Sciences, 13 (1).

Design-Based Research Collective. (2003). Design-Based Research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8. Retrieved from http://www.designbasedresearch.org/reppubs/DBRC2003.pdf

Farmer, R., & Gruba, P. (2006). Towards model-driven end-user development in CALL. Computer Assisted Language Learning, 19(2 & 3), 149-191.

Palalas, A. (2012). Design guidelines for a Mobile-Enabled Language Learning system supporting the development of ESP listening skills (Doctoral dissertation, Athabasca University). Retrieved from http://hdl.handle.net/10791/17

Palalas, A., & Hoven, D. (2013). Implications of using DBR to investigate the iterative design of a mobile-enabled language learning system. CALICO

Plomp, T. (2009). Educational design research: An introduction. In T. Plomp & N. Nieveen (Eds.), An introduction to educational design research (pp. 9-36). SLO: Netherlands Institute for Curriculum Development.

Reeves, T. (2006). Design research from a technology perspective. In J. van den Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.), Educational design research: The design, development and evaluation of programs, processes and products (pp. 52-66). New York: Routledge.

Van den Akker, J. (1999). Principles and Methods of Development Research. In J. van den Akker, R.M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 1-14). Boston: Kluwer Academic.

Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23.