Multidisciplinarity vs. Multivocalitythe case of “Learning Analytics”
Nicolas Balacheff1, Kristine Lund2, CNRS1,2, University of Grenoble1, University of Lyon2
Learning Analytics & Knowlege ConferenceApril 8-12, 2013
Leuven, Belgium
Laboratoire d’Informatique de Grenoble
2
Learning Analytics / Educational Data Mining
- Conceptually grounded, coined to respond to research needs
- Socially grounded, adopted as common conceptual flagships
Multidisciplinarity, multivocality and interdisciplinarity
To what extent can the different disciplines involved in the TEL community be integrated on methodological and theoretical levels?
Origin of concepts and methods in the TEL research area
Problématique,theoretical framework and methodology
In which way does each expression solve problems identified in the TEL research area and how specific are they? What relations do they have with other concepts in the domain?
The case of LA and EDM
3
The TEL dictionary perspective:
defining in order to stop reinventing the wheel
The evolution of TEL research is rapid and motivations are diverse
The language is often ill-definedDifferences in terminology: variations among communities or conceptual differencesDifficult to ensure that the wheel is not being reinvented
The case of LA and EDM- LA: introduced in 2009 -- first conference 2011 -- no identified endogenous precursors
but strong heterogeneous (analytics)- EDM: introduced in 2000 -- first conference 2008 – precursor workshops jointly held with
ITS, AIED, ICALT, etc. – evidence of historical roots in learner modeling
Different histories, but does that imply semantic differences?
http://www.tel-thesaurus.net/
4
(partial view of the links).Colors representthe clusters centered on the most important keywords
Data (orange)link Analytics (blue) and Educational Data (red)
Source: Stellar Grand Challenge problems
http://www.tel-thesaurus.net/maps/contexteGCP/
Keywords relating tolearning analytics
5
(partial view of the links).Colors representthe clusters centered on the most important keywords
Data (orange)link Analytics (blue) and Educational Data (red)
Source: Stellar Grand Challenge problems
http://www.tel-thesaurus.net/maps/contexteGCP/
Keywords relating tolearning analytics
Closer look at the DataTEL and the Productive Multivocality workshops at the Alpine Rendez Vous 2011 - although data is at the heart of both, there is almost no shared vocabulary, apart from cognates of “learning”
- in addition, there is a great difference in terms of scope in the two workshop’s objectives
6
DataTEL“The research on TEL recommender systems can contribute to decreasing the drop-out rate”
“customize existing recommendation algorithms for learning, employ recommender systems in real-life scenarios and develop suitable evaluation criteria for different kinds of recommender systems”.
Productive Multivocality“supportive structure for a dialogical interpretation of the data in order to make the community and stakeholders aware what results converge among the different data sets and different interpretations and in order to identify open questions”.
The TEL dictionary perspective:
defining in order to stop reinventing the wheel
Data is at the core of both communities, but in different ways- One focuses on improving algorithms to treat data (recommender systems)- The other focuses on interpretation of shared data
Sharing data is a potentially productive move for TEL research, but not an easy one
What « data » means might be the next question
a challenge illustrated in the second part of this communication
7
Initial conditions for Productive Multivocality using the pivotal moment as
a boundary object
X 5 Editors: Suthers, D., Lund, K., Rosé, C., Law, N. & Teplovs, C.
8
Multidisciplinarity, interdisciplinarity and multivocality
• Neither theoretical perspectives nor actual results from different participating disciplines are integrated during multidisciplinarity
subject approached from different angles, using different disciplinary perspectives (van den Besselaar & Heimerik, 2001)
Each research group stays within their own boundaries (Choi & Pak, 2001)
• Interdisciplinary research integrates contributing disciplines by creating its own theoretical, conceptual and methodological identity
analyzes, synthesizes and harmonizes links between disciplines into a coordinated and coherent whole (van den Besselaar & Heimerik, 2001)
• Multivocal research performing multiple analyses from different epistemological and methodological
frameworks on a shared corpus (e.g. group interactions in pedagogical contexts) Productive : analytical concepts were refined, epistemological positions were made
explicit, and the conditions under which learning occurs were characterized, but with different perspectives, thus allowing discussion about learning
9
Our argument
• The LA community is much like the CSCL community Multidisciplinary with a potential for interdisciplinarity
• (Our version of ) multivocality is closer to interdisciplinarity than to multidisciplinarity
We will use an example from the Productive Multivocality Initiative to illustrate this
Multivocality and interdisciplinarity are approaches that move research fields forward
– The communities researching “Learning Analytics” are nicely positioned to benefit from such approaches, much in the same way that CSCL has been
10
How multivocality can tend towards interdisciplinarity
• Step 1 3 researchers each designate the moment they call pivotal
– Different visions of learning are made explicit– “Moments” are of differing length (cf. unit of analysis/interaction)
1 Trausan-Matu
2 Shirouzu
3 Chiu
1 Trausan-Matu
1 Trausan-Matu
“Fold, then cut out the 3/4 of 2/3 of the origami paper”
11
Multivocality without convergence
• The comparison of two researcher’s pivotal moments lead to progress in each other’s problématiques, but not to integrating on either a theoretical or methodological level
Taking another researcher’s pivotal moments and interpreting them in one’s own framework (e.g. Chiu : breakpoints in frequency of new ideas corresponded to when and how the pedagogical designer’s intentions were actualized by students’ behavior - Shirouzu)
Neither methodological nor theoretical convergence is achieved, but a discussion has begun
12
Multivocality with convergence
• The comparison of two other researcher’s pivotal moments lead to progress in one researcher’s problématique, but also to integrating one of the researchers’ approaches on a methodological level
Trausan-Matu extended the definition of an analytical concept (e.g. Bakhtinian “voices” include gestures)
extended the domain of application (e.g. from just chat to face-to-face interactions)
Deeper theoretical integration is more difficult– We do not always aspire to that because tension can be productive
13
“Data” as a boundary object for learning analytics and educational data mining
What is shared by a teacher having to manage a lesson on ratio and proportion, and by the dean of the university having to ensure the success of the freshman class?
How far is the meaning of “learning” the same in both cases?
We join Siemens and Baker in a call for cooperation with the suggestion of an analysis of…
- the nature of data
- the problématiques driving underlying commonalities and differences
perhaps for the sake of a new theoretical, conceptual and methodological identity for both Learning Analytics and Educational Data Mining
14
LA“Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Long and Siemens, EDUCAUSE 2011).
EDM“Educational Data Mining is a term used for processes designed for the analysis of data from educational settings to better understand students and the settings which they learn in.” (Desmarais and Baker, TEL Dictionary 2012)
The TEL dictionary perspective:
defining in order to stop reinventing the wheel
- Are the Learning Analytics tools imported from analytics sufficient for relevantly analyzing learning data?
- Should all data attached to the activities of a student be considered as learning data?
- Isn’t Learning Analytics reducing successful learning to the academic success of students in their institutions, limiting de facto the problématique of TEL research?
- Compared to the classical problématique of “learner modeling”, what are the specific contributions of “Learning Analytics”?
15
AfterwordFrom…
- 42 papers from LAK12
- 24 papers from LAK11 76 docs / articles
- 10 papers from JETS12
article'ssubject
article'sbody
"learning analytics"
10 564
"educational data mining"
3 40
LA "learninganalytic“
learning (1000.0) learn (909.0) analytic
(498.0) learner (372.0) activity (276.0) research (257.0) context (230.0) social (222.0) design (218.0) provide (214.0) community
(205.0) knowledge (203.0) practice (196.0)
tool (192.0) individual (180.0) datum (175.0)
model (169.0) development (160.0) environment (159.0) EDM "educationaldatummining"
cluster (140.0) use (125.0) datum (94.0) teacher (91.0) student (85.0) user (81.0) clustering (76.0) project (72.0) mining(66.0) system (64.0)
theme (62.0) pattern (57.0) lecture (57.0) study (55.0) video (51.0) online (49.0) feature (43.0)
model (43.0) class (41.0) group (40.0) question
(39.0) final (39.0) classroom (38.0) tool (37.0) level (35.0) teaching (33.0)
instructional (32.0)
1. Latent Dirichlet Allocation on the preprocessed corpus (stopwords elimination, discarding parentheses and phrases of less than 2 words & lemmatizing) -- to enforce the search for the specific concepts in the topic model – to consider them compound words.
after this step emerges: "learninganalytic" (without s) and "educationaldatummining" (with datum instead of data)
2. Trained incremental LDA topic models starting from only the LAK corpus (1.3MB) with 20 topics (a topic contains all words, with their corresponding weights in descending order with concepts semantically related that emerge from co-occurence relations
3. adding chunks of TASA corpus results to come
Thanks! To Mihail Dascalufor bringing his expertise within
very short delay! Work in progress!