Image Credit: …Time… by Darren Tunnicliff, CC by 2.0 license
It’s About Time4th International Workshop on Temporal Analyses of Learning Data
LAK’15 Learning Analytics & Knowledge ConferenceMarch 16, 2015 | Poughkeepsie, NY
Organizing TeamSimon Knight, Bodong Chen, Britte Cheng & Alyssa Wise
Schedule for the Day
• 8:30 - 9:00 Introduction to the Workshop
• 9:00 - 10:00 Group Work Session 1
– Conceptual Session: Intro to the Approach, Tools & Data
• 10:00 - 10:15 Coffee Break
• 10:15 - 11:45 Group Work Session 2
– Application Session: Hands-on putting data into the tools /critical discussion of its application to specific data sets
• 11:45 - 12:30 Group Share, Discussion & Wrap-Up
Why Temporal Analysis & Learning Analytics?
• Temporality is fundamental in understanding learning processes
• But traditional research methods have not taken advantage of temporal information embedded in data resulting in limited explanatory power
• There have also been challenges in collecting sufficient data over time to allow such analyses
• New analytic methods, expanding sources of data and learning analytics' focus on studying processes create the perfect storm to move the sophistication of temporal analyses of learning forward
Coding and Counting vs True Process Data
Start End
Progressive phases
Repeating sequences
Temporal information lost
Coding and Counting vs True Process Data
Start End
Sequence captured but rate is lost
Distribution over time is not always equal
Temporal information lost
Snapshots vs Accumulation
…
Current state indexed as all events leading up to and including the most recent one
Current state indexed as the most recent event
Temporality as a Continuous Flow of Activity
• Examines characteristics of particular kinds of events within the activity stream
– Position in time (questions asked early on set the tone of discussion)
– Duration over time (how long spent answering each question)
– Rate over time (how quickly each problem is solved)
– Acceleration / deceleration (speed of problem-solving increases)
• Each can be thought of in absolute or relative terms
• Can consider growth and decay patterns
Temporality as an Arrangement of Events
• Examines ordered relationships of multiple kinds of events within the activity stream
– Co-occurrence (high coherence of student talk when gaze is aligned)
– Re-occurring sequences (arguments often followed by rebuttals)
– Non re-occurring sequences (aka phases) (debating of different points occurs before negotiation of a synthesis)
• Again can think absolutely or relatively (adjacency)
• Can consider ‘common’ vs ‘consequential’ patterns
Some Issues to Keep in Mind
• Granularity
– Segmentation of Time Windows
– Aggregation of Unit of Analysis
• Coordinating Multiple Data Streams
– Varying units and timescales
• Beyond Process for its Own Sake
– Connecting with Inputs and Outcomes
– Where does theory fit in?
GranularityAggregation & Segmentation
Whole Time
Time Window Window
Unit of Analysis
Data Data Data Data Data Data Data Data Data Data Data Data
Unit of Analysis
Unit of Analysis
Time Window Window
Un. Un. Un. Un. Un.
Temporal Analysis
Coordinating Multiple Data Streams
• Concurrent collection of multiple types of data– Activity logs and post contents
• Multi-dimensional coding– At the same or different levels of aggregation (unit of analysis)
• Question of what are the relevant timescales on which to consider these
A Bit of History
• It’s About Time v1.0– Alpine Rendezvous 2009
• It’s About Time v2.0– ICLS 2010
• It’s About Time v3.0– Alpine Rendezvous 2013
• Process vs Practice in Learning Analytics– ICLS 2014
Explored specific temporal analyses of group learning
Focused on analyzing multiple data streams
Mapped dimensions of temporal analyses
Questioned how to make meaning of processes
Goals for Today
It’s About Time v4.0 @ LAK 2015
Explore specific opportunities and challenges of temporal analysis for learning analytics
• Examine what particular methods can tell us about different kinds of data sets
• Consider what concepts of time different methods let us investigate
• Probe questions of interpretation and use of process measures
Framing QuestionsFor each analytic approach:
• What kinds of data is it suitable for?
• What grain size of ‘time’ does it address?
• What dimensions of temporality does it deal with?
• What kinds of insights about learning does it provide?
• What new learning/learning process constructs could emerge from the analytic approach?
• (How) could an educator interact with information produced by the analytic approach/tool as part of their teaching, assessment or other practice?
• (How) could students interact with information produced by the analytic approach/tool as part of their learning process?
• What limitations, gaps or other issues are there with the approach? How could the approach be developed further?
Our Groups for the Day
• Using Sequence Analysis and Optimal Matching to Analyze Classroom-Based Video Data [Betsy McEneaney]
• Statistical Discourse Analysis applied to F2F Turn-Taking Data [Ming Chiu]
• Temporal patterns in Assessing Collaborative Learning on Wikis in Secondary and Primary Schools [Xiao Hu]
• Epistemic Network Analysis to understand Trajectories of Development [Golnaz Arastoopour & Wesley Collier]
• Hidden Markov Modeling [Britte Cheng]