Date post: | 11-Apr-2017 |
Category: |
Education |
Upload: | tim-rogers |
View: | 83 times |
Download: | 1 times |
LEARNING ANALYTICS, ACTION SCIENCE AND
CRITICAL REALISM
DR TIM ROGERS, TEACHING INNOVATION UNIT, UNIVERSITY OF SOUTH AUSTRALIA
TOPICS FOR TODAY
OUTLINE
• Short background to my underlying philosophy, critical realism
• Description of an approach to action research, action science
• A look at some practices from action science that are, or could be, applied in educational action research
• An overview of learning analytics tools as methods for educational action research
SOME PHILOSOPHICAL UNDERLABOURING
CRITICAL REALISM
• The best explanation of the progress of the natural sciences suggests underlying essences and causal powers are central, not just observable relationships between variables
• In the social sciences concerned with the causal roles of structure and agency
• So actions may involve unacknowledged conditions, unconscious motivations, tacit skills, and have unintended consequences
• A language user may not know the role of socialisation in language acquisition (unacknowledged condition), may not be aware of reasons for concealing their thoughts (unconscious motivation), may not be able to describe the grammatical rules they evidently know in practice (tacit skills) and may be incorrect in their belief about the effect of their utterance (unintended consequences).
• A trainee teacher may not know the role of their historical experiences as a student (unacknowledged conditions), may not be aware of the reasons for being angry at a student (unconscious motivation), may not be able to describe the rules underlying their explication of a concept (tacit skills), and may be incorrect in their belief about the effect of their interventions (unintended consequences).
MEANING-MAKING AND THEORY BUILDING IN THE SOCIAL WORLD
ACTION SCIENCE
• Original researchers were Chris Argyris and Donald Schön
• A meta-theory of human action: human action is (often) goal driven and involves tacit theories for achieving these aims
• In situation X (conditions), 2) do Z (strategy), 3) achieve Y (goal)
• Two theories at work: espoused theories and theories-in-use
• We are often unaware of the gaps between these two theories
LEARNING ABOUT OUR ‘FRAMES’ FOR LEARNING
META-LEARNING IN ACTION SCIENCE
• ‘Single-loop learning’ occurs when action strategies are changed, but the rest of the theory-in-use remains constant
• ‘Double-loop’ learning involves changes in goals, frames, assumptions, values, and/or standards for performance
…OR COULD BE READILY APPLIED TO TEACHING
EXAMPLES WITHIN TEACHING…
• Professor Viviane Robinson, University of Auckland:
• Robinson, V. M., Meyer, F., Sinnema, C., & Le Fevre, D. (2016). Truth-seeking or truth-claiming? Leaders' patterns of social problem solving. Paper presented at 2016 American Educational Research Association (AERA) Annual Meeting, Washington, DC, USA. 8 April - 12 April 2016.
• Le Fevre, D. M., Robinson, V. M. J., & Sinnema, C. E. L. (2015). Genuine inquiry: Widely espoused yet rarely enacted. EDUCATIONAL MANAGEMENT ADMINISTRATION & LEADERSHIP, 43 (6), 883-899.
• Le Fevre, D., & Robinson, V. M. J. (2015). The Interpersonal Challenges of Instructional Leadership Principals’ Effectiveness in Conversations About Performance Issues. Educational Administration Quarterly, 51 (1), 58-95.
…OR COULD BE READILY APPLIED TO TEACHING
EXAMPLES WITHIN TEACHING…
• Dr Jenny Rudolph, Harvard Center for Medical Simulation:
• Rudolph, J. W., Simon, R., Rivard, P., Dufresne, R. L., & Raemer, D. B. (2007). Debriefing with good judgment: combining rigorous feedback with genuine inquiry. Anesthesiology clinics, 25(2), 361-376.
• Rudolph, J. W., Simon, R., Dufresne, R. L., & Raemer, D. B. (2006). There's no such thing as “nonjudgmental” debriefing: a theory and method for debriefing with good judgment. Simulation in Healthcare, 1(1), 49-55.
LEARNING ANALYTICSFOR EDUCATIONAL ACTION RESEARCH
DESIGN LEARNING
EXPERIENCE
COLLECT DATA
ANALYSE DATA
REFINE LEARNING THEORY/MODEL
Stanford Graduate School of Education http://analyticsdose.com/big-data-the-science-of-learning-analytics-and-transformation-of-education/
LEARNING ANALYTICS…
…is the collection, collation, analysis and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning
ASSUMPTIONS AND FOCI
• Provides actionable intelligence: ‘closes the loop’ • Can accommodate ‘big data’ (usually from the LMS e.g.
Moodle) • Visualises data for summary and exception highlighting
OR ‘BRICOLAGE’
SOURCES: A CONFLUENCE DISCIPLINE
Ed theory and practice, social network analysis, data mining, machine learning, natural language processing, data visualisation, sense-making, psychology (social, cognitive, organisational), learning sciences, and more
WHY LA, WHY NOW?
• Volume of students
• Widening participation
• Constrained budgets
• Increased scrutiny on quality
• Do more, absorb greater variety, with less money, and show it works and is improving. Or else…
PREDICTIVE ANALYTICS
• Who or what do I need to watch out for?
• Previous cohort student data used to model incoming cohort risk (of attrition or failure)
SOME ISSUES WITH PREDICTION
• Self-fulfilling prophecies and ethics • Course level is key to predictive accuracy but
global algorithms often used • Helps or harms student motivation? • Who’s to say what my dependent variable
should be? • Nevertheless, can be very helpful for timely
interventions to know who might be at risk, and when
CAUSAL ANALYTICS - EXPLANATION
• Why is what is happening, happening?
• Sensemaking in the whole learning system…
• …for both educators and students
NEW TECHNOLOGIES + ESTABLISHED THEORIES = NEW INSIGHTS
• What is missing: a focus on learning process
Bringing together:
• Self-Regulated Learning (SRL) proficiency (Gasevic; Winne)
• Natural language processing (NLP) and text mining (Rose)
• Learning design and instructional conditions (Lockyer; Gasevic)
WHY MIGHT STUDENTS NOT USE A TOOL?
And what might we do to rectify this?
• A story of new technology (CLAS) meeting new analytic techniques (NLP) and established learning theory (SRL)
Gašević, D., Dawson, S., & Mirriahi, N. (Revised & resubmitted). What is the role of teaching in adoption of a learning tool? A natural experiment of video annotation tool use.
TAKEAWAY
LA will:
• Go hand-in-glove with learning design
• Provide timely process feedback about learning, rather than outcomes
• Encourage educator and student active engagement
• Facilitate educational action research and scholarship of teaching and learning (SOTL)