Between Scylla and Charybdis…
Status quo of external ad internal evaluation in Polish education
_________________________________________________________
Ewa Kedracka, Quality Coordinator
in National In-service Teacher Training Centre,
Warsaw - Poland
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 2
Scylla and Charybdis ?
By Micha F. Lindemans
Scylla - in Greek mythology, a sea monster who lived underneath a dangerous rock at one side of the Strait of Messia, opposite the whirlpool Charybdis. She threatened passing ships and in the Odyssey ate six of Odysseus' companions.
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 3
The world of practice
The world of power
(authority)
The world of knowledge(scientists, researchers)
to behere...
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 4
The world of practice
The world of power
(authority)
The world of knowledge(scientists, researchers)
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 5
Background
Legislation in Poland
The Act on the Education System - 7 September 1991 (with further amendments)
The Teachers’ Charter 1982, 2000
The Regulation of The Minister of National Education on Pedagogical Supervision (school inspection) 13 August 1999, 23 April 2004
European Union agreements
Bologne Process
Lisbona Strategy
Copenhagen Process (CQAF !)
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 6
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 7
SCHOOL can mean:
National system of education
Regional educational system
Education in local community
School
A class
A lesson
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 8
The most important: internal quality care !
„Quality maintenance and quality assurance should not be based on quality measurement, comparative assessment or quality control by an outsider (the government or inspectorate). Quality will be assured in the first place by internal quality care, i.e. constant and structural attention to quality by the institution itself and the units in the institution.”
A.I.Vroeijenstijn “Improvement and Accounability: Navigating between Scylla and Charybdis. Guide for External Quality Assessment in Higher Education” London 1995
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 9
Key problems discused all over the world:
What does it mean „a GOOD SCHOOL”?
How GOOD is our SCHOOL?
What should we do to develop a GOOD SCHOOL ?
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 10
What is quality?
Pirsig (1974)”Zen and the Art of Motorcycle Maintenance” quoted in‘Quality and Accreditation in HE’ by Don F. Westerheijden (1995)
Quality… you know what it is, yet you don’t know what it is. But that’s self-contradictory. But some things are better than others, that is, they have more quality. But when you try to say what the quality is, apart from the things that have it, it all goes poof! There’s nothing to talk about. But if you can’t say what Quality is, how do we know what it is, or how do you know that it even exists? If no one knows what it is, then for all practical purposes it really doesn’t exist at all. But for practical purposes it really does exist. What else are the grades based on? Why else would people pay fortunes for some things and throw others in the trash pile? Obviously some things are better than others… but what’s “betterness”? So round and round you go, spinning mental wheels and nowhere finding any place to get traction. What the hell is Quality? What is it?
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 11
Quality
The traditional concept:
Quality as exceptional (excellence)
Value for money….
Quality as perfection or consistency (0 errors)
Quality as a fitness for purpose
Conforming to specifications or standards
Quality - the features of objects allowing to meet the recognised and expected needs of customers
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 12
Quality and evaluation – the neverending story…
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 13
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 14
External and internal evaluation in Poland
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 15
Evaluation in education - SWOT analysis 1/2
Streghts
legislation
some experience
opportunity to share experiences
„we know”
Weaknesses
different interpretation
weak methodological skils
no universal international solutions
„we don’t understand”
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 16
A lot of questions and doubts!
“Evaluation is a good thing, but who should do it?
We all want better evaluation, but what should be evaluated?
Evaluation is necessary, but when and how should it be carried out?
Evaluation is beneficial, but who is it for?”
John MacBeath, Archie McGlynn “Self-evaluation. What’s in it for schools?” Routledge 2002, p.5
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 17
„The devil is hidden in details…”
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 18
Evaluation - assessment, review, audit, quality measurement…
A.I.Vroeijenstijn “Improvement and Accounability: Navigating between Scylla and Charybdis. Guide for External Quality Assessment in Higher Education” London 1995:
„By quality assessment we mean every structured activity which leads to a judgement of the quality of the teaching/learning process [and/or research], whether self-assessment or assessment by external experts. There is no real difference between assessment, evaluation and review. Those terms are seen as interchangeable.”
Are they??? Is there no difference between assessment, evaluation, review? What about e.g. audit?...
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 19
Once again MacBeath and McGlynn:
p.8 Evaluation and assessment - making the distinction:
to assess - to give a grade or apply a set of given criteria
to evaluate - to take a further step back, moving outside the situation, adressing additional questions
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 20
What can we do with the quality?
Measure
Manage
Control
Improve
Maintain
Assure ...
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 21
Quality Measurement - Quality Management ?Quality Management - Quality Assurance ?
Quality Measurement
Supervision
Quality Management
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 22
Quality Control and Quality Assurance
QC approach says:„We who are In charge know best – not only what to do, but how to do it. You – the workers – will do exactly as we tell you. And we will set up a “policing force” – the QC department – which will check to make sure that you are doing the right things.”
QA approach, on the other hand, says:“Let’s agree what it is we should be doing. Then we – those with the democratic responsibility, those in authority – recognize that this is a complex and difficult task, and you – the workers – know far better than we do how to do it this effectively. Therefore, we will create the conditions that allow you to exercise your own judgment as to what needs to be done. We will provide the support for you to put the strategies in place and, in a spirit of collaboration, monitor progress to make the right things happen!”
Tool for quality assurance of education for democratic citizenship (EDC) in schools.UNESCO 2005 – European Year of Citizenship through Education
Project: Education for democratic citizenship (EDC): from policy to effective practice through quality assurance.
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 23
External and internal evaluation…
„Two sides of a coin” (MacBeath, McGlynn)
The pair of hard working horses…
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 24
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 25
The crucial problem – how navigate between the Scylla of Improvement and the Charybdis of Accountablity
„External quality assessment has two purposes: quality improvement and accountability. To fulfil these two purposes, managing the EQA-process, is like navigating between two extremes. When one aims only at improvement, the system will be shipwrecked against the ‘Scylla’ because the outside stakeholders will ask for accountability and design their own EQA system. If accountability is emphasized too much, the system will disappear in the ‘Charybdis’, because improvement will be hindered or even made impossible. The challenge is to keep on course and, by doing so, reconcile the two purposes in one system. It will be not easy, as can be seen by looking at the experience of the Netherlands.”
A.I.Vroeijenstijn “Improvement and Accounability: Navigating between Scylla and Charybdis. Guide for External Quality Assessment in Higher Education” London 1995
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 26
Is Quality the task for angels?!...
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 27
Trying to do the best…Cooperation - here with Switzerland:
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 28
Cooperation and benchmarking
„Schools must speak for themselves”
„How good is our school?”
„Create a mirror for your school”
„Bridges under bounders” !
British Council Poland helps Polish schools
to use the evaluation as a mirror of quality assurance
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 30
History
Scottish Quality Management System - pilot project (1998-1999)
FORUM ‘Quality in Education’ (1999 - 2000) organised by Ministry of Education and CODN(suspended in 2001)
British Council INE seminars on quality in education (from 1998 till present)
Launch of Quality Network Poland 2001
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 31
INE seminars on quality in educationin which future members of Quality Network took part
Raising Standards through School Improvement – How good is our school?
Edinburgh 1998, 2000, 2003
Making In-service Education Effective
Cambridge 1999
Managing the Training and Development of School Leaders, Windsor 2000
Quality Management in Language Teaching
Manchester 2001
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 32
INE seminars continued
Quality Reforms in Education: aiming for international excellence, Warwick 2001
Management of ELT Provision: Managing the Future, Durham 2001
Conference of the International Network for Quality Assurance Agencies in Higher Education Bangalore 2001
From Peril to Promise: how HE can deliver? Bath 2002
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 33
Quality Network Poland
What for?
The aim of our work in this area was to provide Polish educational institutions responsible for planning and implementing quality systems at all levels with opportunities for networking and discussion through seminars and attendance at conferences.
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 34
Quality Forum
FORUM ‘Quality in Education’ (1999 - 2000) organised by Ministry of Education and CODN - suspended in 2001
***************************************************************************
Quality Forum March 2002, Zakopane
Presentation of Polish and British experience in quality assurance in education
Quality Forum November 2002, Jachranka
Presentation of quality awards and signs in Poland and EU
EFQM workshop about Excellence Model July 2003, Kraków
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 35
Quality Forum continued
Quality Forum organised together with EFQM (European Foundation for Quality Management) February 2004, Kraków
‘Towards excellence at all levels of education system’
Quality Forum „Quality Assurance in Vocational Education and Training” April 2005, Jachranka
Next:
Self-evaluation in schools January Kraków 2007
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 36
Evaluation in schools - SWOT analysis 2/2
Opportunities
Polish school will be better and better... And our life will be better and better!
Threats
In some opinion quality is like MANTRA...
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 37
The game: In the JUNGLE of QUALITYDo you know these acronims?
QM
? TQM
SQMS ?
? EFQM
CQAF
CAF
? EQF
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 38
Quality wins!
QM Quality Management
TQM Total Quality Management
SQMS Scottish Quality Management System
EFQM European Foundation for Quality Management
CQAF Common Quality Assurance Framework
CAF Common Assessment Framework
EQF European Qualification Framework
Centralny Ośrodek Doskonalenia Nauczycieli l Al. Ujazdowskie 28, 00-478 Warszawa l tel. (+48 22) 345-37-00 l fax (+48 22) 345-37-70 Str. 39
Congratulations!
Thank you for attention!
Ewa Kędracka