Computing Education Research
Anders BerglundInformationsteknologi
Uppsala universitetUppsala
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Computing education research at Uppsala University
How do our students understand computer science concepts?
How to teach computer science?
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Computing Education Research
This talk
What can we say about how grading in an internationally distributed project-based course?
and
Why and how to do research in computing education?
Two parallel stories
My secret agenda:Inspire to Computing Education Research
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
1. Background• A research approach?• Phenomenography
2. Results concerning grading• The setting• The teachers’ grades• The students’ peer evaluation• The experience of being graded• Results concerning grading
3. Summary• Computing Education Research• Some research results• Choosing a research approach
Computing Education Research
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
What does it mean to learn something?
Unfortunately (?)….General case:
A “meaningless” question
It all depends on “what you mean by learning” or “how you see things”
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Quantitative/Qualitative research
Quantitative research is grounded on• “… the assumption that features of the social environment
constitute an objective reality … collecting numerical data on observable variables”
Qualitative research is grounded on• “… the assumption that individuals construct a social reality
in the form of meanings and interpretations. … studying … intensively in natural settings”
Implications for the role of the researcher, the concept of evidence, interpretation etc.
(Gall, Borg & Gall, 1996)
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
A research approach/methodology/ framework
Offers a way to perform research in learning
Organizes “ways to see things” A lens with a certain focus With a specific research approach:
Some issues get clearer, others blurred Offers theoretical stand on learning,
ways to see possibilities and limitations, opens to communicate with other researchers etc.
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Research approach/methodology/framework
Phenomenography(Marton & Booth, 1997)
Takes the learners’ perspective.
Aims at analysing and describing the variation in students’ experience (understanding, learning).
A empirical, qualitative research approach
Data is often collected through interviews
Outcome: A few qualitatively different ways, in which something is understood within a student cohort
Examples: TCP, experience of being graded
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Phenomenography
Students taking a course
TCP
Researcher
The researcher studies the different ways in which the students understand TCP
The students study TCP
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Why the learners’ perspective?
A teacher’s understanding of how her students understand and learn about something (CS concepts, for example TCP) is a good tool for improving teaching.
A change that is not perceived as “good” by the students does not improve learning. Example: Grades are not the driving force for
most students who take (a certain) project course.
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
1. Background• A research approach?• Phenomenography
2. Results concerning grading• The setting• The teachers’ grades• The students’ peer evaluation• The experience of being graded• Results concerning grading
3. Summary• Computing Education Research• Some research results• Choosing a research approach
Computing Education Research
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Background: The Runestone initiativeA project course in computer systems
A project-based course in distributed systems, real-time programming and computer communication.
Third/fourth year students majoring in CS. International collaboration for students who do
not go on exchanges. Experience of collaboration over ICT tools.
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Project course in computer systemsThe Runestone project
USA Sweden
• 3 + 3 advanced CS students per team
• 16 teams in total
• No lectures
• Tutoring by e-mail and chat
Communicationby e-mail and chat
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Student project
Student project: Produce a software system to control a (modified) Brio labyrinth from any Web-browser.
The task demands computer communication solutions. The task requires collaboration within the team of 6.
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Both the process and the project are graded, in relation to the teams’ own plan
Process grade is based on weekly meetings Components of grade:
• Team performance, in relation to the team’s own plan• Individual contribution• Peer evaluation• The instructor’s decision.
Team members are graded by “their” instructor Different grading schemes in Sweden and US
• Sweden: pass/fail• US: A to E
Grading in Runestone
Problematic???
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Analysing the grading in Runestone
Teachers’ distribution of grades (quantitative)
Peer evaluation The students’ evaluation of each others’ contributions
(quantitative) Students’ experienced purpose of being graded
(qualitative)
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Grades
Grades awarded by the instructors, according to the Runestone scheme (Max = 100, Pass ≈ 60)
To all students 83,61
To Americans 81,55
To Swedes 85,05
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Peer evaluation
From Swede to Swede 22,25
From Swede to American 18,79
From American to American 20,07
From American to Swede 20,07
Each student awarded USD 120.- to his team-mates
Then, what is the driving force?
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
The experienced purpose of being graded
CatGetting a
good grade …
1.
2.
3.
… has a value on its own
… is a tool to reach other aims
… is sub-ordinated to other aims
Focus is on
The grade per se
The benefits of a good grade
Me and the team
My team and other teams
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Results on grading
Getting a good grade is not the driving force for most students in this project.
”Me in the team” or ”My team in front of other teams” is often important.
How generalizable are these results?• Research in computing education research are normally
situationally bound How can we use this in our teaching?
• Both ”hard” results and insights gained by doing the research are useful.
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
1. Background• A research approach?• Phenomenography
2. Results concerning grading• The setting• The teachers’ grades• The students’ peer evaluation• The experience of being graded• Results concerning grading
3. Summary• Computing Education Research• Some research results• Choosing a research approach
Computing Education Research
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
On Computing Education Research
Theoretically sound research in students’ learning in Computing can serve to improve teaching.
Different research approaches offer various contributions to our understanding of students’ learning.
The perspective on ”reality”, what can be studied, what can be known, what the researcher’s role is, how research is performed etc. varies.
This talk: Some examples of
qualitative research
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Example, research approach Constructivism
A family of traditions Jean Piaget, 1896 - 1980
• Reality is rejected or irrelevant• Knowledge is constructed by each individual• No firm methodology• Passive learning will fail
Extremely influential in school teaching
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Empirical results from constructivism in CS Education
Students construct rules for parameters. They are only sometimes successful. (Fleury, 1991)
Students construct their own understanding of variables. (Paz, 1996; and others)
Software visualization in itself does not help students understanding (Mulholland, 1997)
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Applications of constructivism in Computer Science Education
Think twice when using visualizations Explicitly teach the model of the computer Don’t start with abstractions Teach planning, teach to avoid “bricolage” Don’t run to the computer Organize “closed labs”
(from Ben-Ari, 2001)
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Example, research approach: A socio-cultural perspective
Thinking/Learning is not influenced by the environment
⇔Thinking/Learning is an interaction between the individual and the environment
A family of traditions in research into learning Common “source of inspiration”: Lev Vygotsky
(1896 – 1934)
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Applications of the soci-cultural tradition
Why do teams of students interpret a programming task so differently? (Holland & Reeves, 1996)
Why do our students hand in “incorrect” programs? (Ben-David Kolikant, 2005)
The example: Open source community -• Linux
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Example, research approach: Critical enquiry
Critical enquiry (research with “a mission”, often to address power imbalances)
• Feminist research Cannot be defined in ontological stand or
research methodology Instead, it questions fundamental principals and
values and aims to implement changes.
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Feminist research, an example of critical research
Are there factors within computer science itself, that preserves the currently dominating gender structure? (Björkman & Trojer, 2002)
“We consider it of vital and decisive importance that gender research is done from within computer science.”
“Such research on the core of computer science and its knowledge production would serve to enrich computer science as well as education within computer science.”
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Why phenomenography? A personal view
Appropriate for the research questions • “Complex” “answers” desirable• Statistical “answers” hard or impossible to get
“Close to” learners• Students’ perspective• A way for students to “talk” to teachers/organisers• Data stems from individuals
Computer science is in focus• Results “talk to” computer scientists• Supports deployment of results in teaching • Relevant for recognition of CSE research within CS
Competence at hand/tradition• Shirley Booth
I like it
Selecting a research approach
Read! Create a network!
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
Literature
Clancy, M., Stasko, J., Guzdial, M., Fincher, S., & Dale, N. (2001). Models and Areas for CS Education Research. Computer Science Education, 11(4), 323-341
Fincher, S., & Petre, M. (2004) Computer Science Education Research, London, UK: Taylor & Francis
Berglund, A., Daniels, M. and Pears, A. (in press). Qualitative Research Projects in Computing Education Research: An Overview. To appear in the Proceedings of the 8th Australasian Computer Science Education Conference, Hobart, Australia.
Berglund, A. 2005. Learning computer systems in a distributed project course: The what, why, how and where. Acta Upsaliensis Universitatis. Uppsala Dissertations from the Faculty of Science and Technology 62
Read!
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology
CeTUSS CeTUSS ska verka för anpassning av
teknikutbildning till samhällets och studenternas behov.
Genom att utveckla och sprida information om lärandemiljöer som är
• Personligt meningsfulla• Socialt relevanta• Gränsöverskridande• Baserad på samarbete (lokalt och internationellt)
Workshops, kurser etc.
Create a network!
Info
rmati
onst
ekn
olo
gi
Anders Berglund, Department of Information Technology