Teaching, learning and assessment

Raimonda MarkevicieneWith compliments to Dr. Helen Cameron

2

“The adoption of a learning outcomes approach represents more than simply expressing learning in terms of outcomes. It entails much more due to their significant implications for all aspects of curriculum design, delivery, expression, assessement and standards”.

Adam S, 2004

What do we mean by assessment?A range of synonyms in English:

• Examinations, Evaluations, Appraisal, Judgements, Measurement, Review, Opinion, Consideration, Estimation

Practically:

• Taken to mean any ‘formal’ review of performance or ability – exams at any time, in-course assignments, practicals etc.

The purpose of assessment

Types of Assessment?

Summative assessment

Primary purposes Assessment of learning Records achievement Informs decisions about

readiness to progress Reassures clients,

public, taxpayers, employers

Accumulative assessment

Formative assessmentPrimary purposes Assessment for

learning Promotes appropriate

learning Feedback Lifelong learning

Diagnostic assessmentContinuous assessment: A combination of summative and formative assessment. Usually involves repeated summative assessments. Marks recorded.

Simple suggestions?....

1. Clearly define the learning outcomes.2. Select teaching and learning methods that

are likely to ensure that the learning outcomes are achieved.

3. Choose a technique or techniques to assess the achievement of the learning outcomes.

4. Assess the learning outcomes and check to see how well they match with what was intended

Tuning Links Learning to Assessment

STUDENTSLearning Behaviours

INTENDEDLearning Outcomes

Achievements- INTENDED Assessment

First, needs analysis = students required performanceThis informs the intended assessment. Write LOs to tell students and staff what is intended.Use LOs to write new assessments / exams

ACHIEVEDLearning Outcomes

THE Assessment

Assessment Design Must Match Learning

Learning Outcomes

Assessmentformative

summativesampling

format setting

timing/frequencycompensation/hurdles

Adapted from John Biggs 1996

Constructive Alignment …….

Learning Behaviours

Constructive alignment• Constructive alignment is the deliberate linking within

curricula of aims, learning outcomes, learning and teaching activities and assessment.

• Learning Outcomes state what is to be achieved in fulfilment of the aims.

• Learning activities should be organised so that students will be likely to achieve those outcomes.

• Assessment must be designed such that students are able to demonstrate that they have met the learning outcomes.

• Constructive alignment is just a fancy name for “joining up the dots”.

(Morss and Murray, 2005)

10

Learning outcomesModule ED2100

Teaching and Learning Activities

Assessment 10 credit moduleMark = 200

Cognitive•Recognise and apply the basic principles of classroom management and discipline. •Identify the key characteristics of high quality science teaching.•Develop a comprehensive portfolio of lesson plans

Lectures (12) Tutorials (6) Observation of classes (6) of experienced science teacher (mentor)

End of module exam.  Portfolio of lesson plans    (100 marks)

Affective•Display a willingness to co-operate with members of teaching staff in their assigned school.•Participate successfully in Peer Assisted Learning project

Participation in mentoring feedback sessions in school (4) Participation in 3 sessions of UCC Peer Assisted Learning (PAL) Programme.  Peer group presentation

Report from school mentor   End of project report.   (50 marks)

Psychomotor •Demonstrate good classroom presentation skills•Perform laboratory practical work in a safe and efficient manner.

Teaching practice 6 weeks at 2 hours per week. Laboratory work

Supervision of Teaching Practice   Assessment of teaching skills  (50 marks) 1010

How do we join the dots???

Cognitive domain

Methods of assessment

Knowledge Oral or/and written exam; testing,; maps of concepts; citation of texts, rules, facts by heart

Comprehension Narration, presentation, essay, testing, writing of a diary

Application Practical work, testing

Analysis Essay, project work, testing, maps of cencepts, case analysis

Synthesis Bibliography or literature lists, review of information sources, portfolio methods

Evaluation Eessay, research work, projects, case analysis, protfolio method, presentations

Tools of Assessment• MCQs• SAQs theoretical / applied

knowledge• Essays

Practical exams / Lab / Recital / Clinical /Pres

Continuous assessment of performance

Continuous assessment of professionalism

Multi-source feedback (self and peer feedback)

Log books and Portfolios

13

Linking Learning Outcomes, Teaching and Learning Activities and Assessment

Learning Outcomes Teaching and Learning Activities

Assessment

Cognitive(Demonstrate:Knowledge, Comprehension, Application, Analysis,Synthesis, Evaluation)

Affective(Integration of beliefs, ideas and attitudes)

Psychomotor(Acquisition of physical skills)

Lectures

Tutorials

Discussions

Laboratory work

Clinical work

Group work

Seminar

Peer group presentation etc.

•End of module exam.•Multiple choice tests.•Essays.•Reports on lab work and research project.•Interviews/viva.•Practical assessment.•Poster display.•Fieldwork.•Clinical examination.•Presentation.•Portfolio.•Performance.•Project work.•Production of artefact etc.

1313

14

The level (quality) of Learning Outcome achievement?

Rubric: A grading tool used to describe the criteria which are used in grading the performance of students.

Rubric provides a clear guide as to how students’ work will be assessed.

A rubric consists of a set of criteria and marks or grade associated with these criteria.

Example from Music

16

Linking learning outcomes and assessment criteria

Learning outcome

Assessment criteria

Grade 1 Grade 2 : 1 Grade 2 :2 Pass Fail

On successful completion of this module, students should be able to:

Summarise evidence from the science education literature to support development of a line of argument.

Outstanding use of literature showing excellent ability to synthesise evidence in analytical way to formulate clear conclusions.

Very good use of literature showing high ability to synthesise evidence in analytical way to formulate clear conclusions.

Good use of literature showing good ability to synthesise evidence in analytical way to formulate clear conclusions

Limited use of literature showing fair ability to synthesise evidence to formulate conclusions.

Poor use of literature showing lack of ability to synthesise evidence to formulate conclusions

2011.05.12/13

Assessment strategy Weight in percents

Time of assessment

Assessment criteria

Work during seminars in the classroom

20 % Teaching weeks within semester

2 points: actively participates in discussions, answer questions, formulate problems and raises questions, gives critical comments;

1 point: participates in discussions, answers questions;

0 points: almost does not participate in discussions, missed more that 1/3 seminarsWritten assignment

(15 pages)

30 % Until December 1.

The following aspects of work are evaluated:-Structure and volume of the work: structure clear and logical, shows all necessary parts (introduction, where theme, aims, goals, methods and empirical material are introduced; narrative, where analysis of empirical information and its interpretation as well as conclusions are presented), work of necessary length (0,5 points);-Analysis and conclusions: profound analysis, conclusions based on empirical material 2 points; analysis is carried out but not deep, conclusions not always well grounded – 1 point, points are not given for poor analysis.-Research style and culture: appropriate treatment of sources and quotations; formulations and style appropriate to scientific work (0,5 points).

No written work – 0 pointsEgzaminas: test (could be planned in 2 parts: at the middle of the term and the end of the term)

50 % January Test consists of 50 open and closed questions (various level of difficulty - from understanding to evaluation), each gives 1 point. Evaluation:

5: Excellent knowledge and skills. Evaluation level. 45-50 correct answers.

4: Good knowledge and skills, minor mistakes are possible. Evaluation level 35-44 correct answers.

3: Fair knowledge and skills. There are mistakes. Evaluation level. 25-34 corrects answers.

2: Knowledge and skills are below average. Essential mistakes. Level of knowledge applicability. 15-24 correct answers.

1: Knowledge and skills meet threshold requirements. Many mistakes. Level of knowledge applicability 5-14 correct answers.

0: Minimal requirements are not met. 0-4 correct answers.

Assessment strategy and criteria

Steps in writing assessment criteria

Planning the assessment 1 - Blueprinting the Programme LOs -

Programm

e LOs PLO 1

PLO 2

PLO 3

PLO 4

PLO 5

Course 1

✓ ✓ ✓

Course 2

✓ ✓ ✓

Course 3

✓ ✓ ✓

Course 4

✓ ✓ ✓

Planning the assessment 2b - Blueprinting detailed/module LOs -

Programm

e LOs Programme LO 1

Perform Lab Work

Programme LO 2Communicate

advances

ModuleLOs

K&U Pract. Skill

Professionalism

K&U Analysis

Verbal/written skills

Course 1

✓

Course 2

✓ ✓ ✓

Course 4

✓ ✓ ✓

Planning assessment 3 Blueprinting Assessment Tools v LOsPLOs PLO 1

Perform Lab WorkPLO 2

Communicate advances

Module LOs

K&U Practical Skills

Professionalism

K&U

Analysis

Verbal/written skills

Course 1

✓

MCQ ✓Course 2

✓ ✓ ✓

SAQ ✓ ✓ ✓Oral pres ✓Course 4

✓ ✓ ✓

Practical ✓ ✓Peer feedback

✓

Assessment on module level

Module LO 1 LO 2 LO 3 LO 4 LO 5 LO 6

Task

1

Task

2

Task

3

Exam

Isn’t the fist LO assessed too much? Is there no need to asses LO2? Do we need to assess LO 1, 3, 4 5 and 6 during the exam? What LO are assessed during the task 3? Is exam a real tool of assessment? Why so many LO are assessed in in the exam when other tasks/methods are also used?

Suggestions

Avoid too many LOs per course unit. It is important when it comes to AssessmentIt is unreasonable for assessors to have to evaluate students against too many LOs in one assessmentToo many assessments per unit is inefficient

Linking LO and activities (Deusto case. Statistics)The main goal of the course is to provide the students with a set of competences for the

understanding and application of statistical concepts and techniques in engineering disciplines. These competences can be classified as general competences and specific ones:

Specific competences

CE 1. Identify situations with a random behaviour and calculate probability of these phenomena.CE 2. Know, identify and classify random variables from different sources of information.CE 3.Identify and solve problems in which the studied variable follows a known probability

distribution. To build up and validate suitable statistical models for real problems.CE 4. Know the use of estimation and inference in order to study the behaviour of a model

through a sample of the population under study.CE5. Assess the importance of statistics and its proper use in specific engineering problems.

General competences

TIME MANAGEMENT. Distribute time equally depending on priorities, taking into account personal objectives. Define, organize and plan activities.

Domain level 2: define and sort objectives and plan individual activity over the medium and long terms (from various weeks to half a year).

TEACHING-LEARNING STRATEGY (Deusto case)

Classroom activities (69 hours)- Lectures explaining the theoretical material: 40 hours - Resolution of exercises and example problems: 23 hours. - Continuous assessment: 3 hours. - Final assessment: 3 hours. Out-of-class activities (81 hours): - Individual study of lecture material: 32 hours. - Undertaking of proposed exercises and revision: 20 hours. - Undertaking of intermediate mileposts and final presentation:

11 hours - Preparation for exam: 18 hours.

ASSESMENT SYSTEM (Deusto case)

–Exercises to be handed in at the end of each subject, accounting for 15% of the final grade.

–Presentation of a course summary accounting for 10% of the final grade.

–Three continuous assessment tests consisting of medium difficulty exercises undertaken in the classroom during lecture time, accounting for 75% of the final grade.

If a grade of at least 50% is obtained with the deliverable exercises, the continuous assessment tests and the presentation, it will not be necessary to take the final exam and the grade will be that obtained up to this time.

ASSESMENT SYSTEM (Deusto case)

If the student does not obtain at least a 50% of the grades, he or she has to:–do the end-of-term examination consisting of four

or five problems of medium difficulty, accounting for 75% of the final grade.

–deliver the failed or non-given tasks.–present again the course summary.

Tuning distinguish two types of indicators to measure the quality of programmes:

• The process itself for (re)designing, developing, implementing, evaluating and enhancing degree programmes

• The outcome of the process : the minimum requirements should have been met

For both purposes Tuning has developed checklists:1. Tuning List of Key Questions for Programme Design and Programme Delivery, Maintenance and Evaluation in the Framework of the Bologna

Reform (Annex 1)2. Tuning Checklist for Curriculum Evaluation

Quality Assurance: Process and outcomes

Identify needs and

necessary resources

Profile

Identify LA In terms

of Generic and

Subject Specific

Compentences

Academic structure

and content (modules and

student workload /

ECTS credits)

Approaches to

TeachingLearning

andAssessment

Quality Enhancement

Tuning Process

Tuning Process

Tuning approach for designing study programmes

THE TUNING DYNAMIC QUALITY DEVELOPMENT CIRCLE

Definition of academic and professional profiles

Identification of resources

Programme design: definition of learning outcomes / competences

Construction of curricula: content and structure + balanced ECTS credit allocation

Evaluation and improvement (on the basis of feed back and feed forward)

Selection of types of assessment

Selection of teaching and learning approaches

Programme assesment (W. E. Deming)

Design process components

Implement the plan

Implement changes

Analyse data, report, decide

on changes

Programme and course unit assessment

INDIRECT: Alumni survey and

meetings Graduating student

survey Focus groups interviews Employer survey/

interview Parents survey/

interview

DIRECT: Assessment that

directly measures achievement of LO (exams, portfolios, test)

Analysis of the study results/marks (module and programme levels)

Drop out rates Students and teacher

opinion

Student workload – issue to consider (input: Give me time to think, U-ty of Oulu)

• Preliminary work before contact hours;• Contact hours;• Individual work after contact hours. Individual

work will depend on study methods used.

Suggestied proportions of contact and individual work hours depending on study

methodsTeaching/learning methods Contact hours Individual work (h) Traditional lecture 1 3 Passive demonstration 1 2 Active learning 1 2-3 Team work 1 2 Execution of tasks 1 3 Active demonstration 1 2 Problem based learning 1 5 Seminars 1 2-4

Time allocated for the tasks depends on the type of the activity/task

• Written assignment. Time calculation - 100 words/1 hour.• Presentation. 1 hour presentation requires min. 6 hours of

preparation.• Reading literature. Students must know whether literature is

compulsory (for passing the exam) or complimentary. The text will be well understood after third reading (three staged of reading: perusal, analitical reading with notes; repetition)

• 100 pages of easy text requires 20 hours. 100 pages of difficult text or text in foreign language requires 30 hours.

Recommendations for calculation of reading the text

Complexity of the text

Time Necessary for deep reading

Humanitarian text Technical text

Easy 100 word/min 60 word/min

Average difficulty 70 word/min 40 word/min

Difficult 40 word/min 25 word/min

Difficult mathematical equations

- 1 equation/min