The research on key competences is a study necessary to define a common language and background for the implementation of the framework and the online assessment tool: it allows to valorize and integrate previous European experiences and models and includes the gathering and analysis of the existing European standards to recognize and record the competences, and the methods, practices and tools of assessment and evaluation.
Grundtvig Project VINTAGE WP2 Deliverable 4 STATE OF THE ART OF COMPETENCES AND ASSESSMENT IN EUROPE Jaap van Lakerveld Ingrid Gussen Joost de Zoete September 2013 The research on key competences is a study necessary to define a common language and background for the implementation of the framework and the online assessment tool: it allows to valorize and integrate previous European experiences and models and will include the gathering and analysis of the existing European standards to recognize and record the competences, and the methods, practices and tools of assessment and evaluation. Contents 2 2 Introduction Key competences for lifelong learning Coordinated by PLATO the VINTAGE partners each conducted desk research in which existing European standards, practices, tools and technological devices in use to assess competences at a European level were collected, recorded and analyzed. For this purpose PLATO developed a format to allow the partners to collect relevant information. This allowed the VINTAGE partners to identify criteria and instruments to build a framework for the evaluation of the key competences, setting the evaluation levels, the descriptors and the checklists on national level. The research was carried out by all partners, each of them wrote a report about the practices for the assessment of key competences used in his Countries with the main references of local literature and online resources. Key competences in adult education 4 Assessment tools 5 12 Analytical overview of assessment tools Conclusions and ideas on assessment tools References 12 14
Transcript
GrundtvigProjectVINTAGE WP2Deliverable4
STATE OF THE ART OF COMPETENCES AND ASSESSMENT IN EUROPE Jaap van Lakerveld Ingrid Gussen Joost de Zoete
September 2013
1
The research on key competences is a study necessary to define a common language and background for the implementation of the framework and the online assessment tool: it allows to valorize and integrate previous European experiences
2
and models and will include the gathering and analysis of the existing European standards to recognize and record the competences, and the methods, practices and tools of assessment and evaluation.
Contents
2
2
Introduction
Key competences for lifelong learning
Coordinated by PLATO the VINTAGE partners each conducted desk research in which existing European standards, practices, tools and technological devices in use to assess competences at a European level were collected, recorded and analyzed. For this purpose PLATO developed a format to allow the partners to collect relevant information. This allowed the VINTAGE partners to identify criteria and instruments to build a framework for the evaluation of the key competences, setting the evaluation levels, the descriptors and the checklists on national level. The research was carried out by all partners, each of them wrote a report about the practices for the assessment of key competences used in his Countries with the main references of local literature and online resources.
Key competences in adult education
4
Assessment tools 5
12 Analytical overview of assessment tools
Conclusions and ideas on assessment tools
References
12
14
GrundtvigProjectVINTAGE
WP2Deliverable4
2
1
In this report PLATO describes the results of the desk research conducted by the partners in VINTAGE in order to specify how to describe competences and their possible levels of mastery. This research resulted in a collection of initiatives focused on the acquisition and (self) assessment of key competences in adult education in Italy, Austria, Germany, Ireland, The Netherlands and Sweden. The findings of this research will be used as a source of inspiration to develop a VINTAGE assessment framework (WP3) and an online tool (WP5). The analytical questions in relation to the key competences and assessment (tools) that PLATO
2
used for this research, are:
1. Which key competences are addressed in adult education in the participating countries? (EU key competences or otherwise?)
2. Which kinds of assessment tools / methods are described? • Is it a complete tool or is it part of a tool? • In case of a portfolio: what kinds of proof are demanded? • What is the function of the assessment tool? • Which approach is chosen?
3. Which dimension(s) are being used in the tools? • Learning context or working working context? • Product (facts) or social evidence (observations of others)?
Introduction
1
Key competences in the shape of knowledge, skills and attitudes appropriate to each context are fundamental for each individual in a knowledge-based society. They provide added value for the labour market, social cohesion and active citizenship by offering flexibility and adaptability, satisfaction and motivation. These key competences are defined in the Recommendation 2006/962/EC of the European Parliament and of the Council of 18 December 2006 on key competences for lifelong learning.
Key competences for lifelong learning
This project has been funded with support from the European Commission. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.
Grundtvig Project VINTAGE - online tool for self eValuatIoN of key competences in adulT AGE Reference: 527349-LLP-1-2012-1-IT-GRUNDTVIG-GMP
Grant Agreement n. n. 2012-4192/001-001
3
4. What are the indicators of levels?
5. What is the functions of the use of key competences?
6. What similarities or differences are there in definitions of competences?
GrundtvigProjectVINTAGE
WP2Deliverable4
3
1. Communication in the mother tongue
Communication in the mother tongue is the ability to express and interpret concepts, thoughts, feelings, facts and opinions in both oral and written form (listening, speaking, reading and writing), and to interact linguistically in an appropriate and creative way in a full range of societal and cultural contexts; in education and training, work, home and leisure.
2. Communication in foreign languages
Communication in foreign languages broadly shares the main skill dimensions of communication in the mother tongue: it is based on the ability to understand, express and interpret concepts, thoughts, feelings, facts and opinions in both oral and written form (listening, speaking, reading and writing) in an appropriate range of societal and cultural contexts (in education and training, work, home and leisure) according to one’s wants or needs.
3. Mathematical competence and basic competences in science and technology
Mathematical competence is the ability to develop and apply mathematical thinking in order to solve a range of problems in everyday situations. Competence in science refers to the ability and willingness to use the body of knowledge and methodology employed to explain the natural world, in order to identify questions and to draw evidence-based conclusions.
4. Digital competence
Digital competence involves the confident and critical use of Information Society Technology (IST) for work, leisure and communication. It is underpinned by basic skills in ICT: the use of computers to retrieve, assess, store, produce, present and exchange information, and to communicate and participate in collaborative networks via the Internet.
5. Learning to learn
Learning to learn is the ability to pursue and persist in learning, to organise one’s own learning, including through effective management of time and information, both individually and in groups.
6. Social and civic competences
These include personal, interpersonal and intercultural competence and cover all forms of behaviour that equip individuals to participate in an effective and constructive way in social and working life, and particularly in increasingly diverse societies, and to resolve conflict where necessary.
7. Sense of initiative and entrepreneurship
Sense of initiative and entrepreneurship refers to an individual’s ability to turn ideas into action. It includes creativity, innovation and risk-taking, as well as the ability to plan and manage projects in order to achieve objectives.
8. Cultural awareness and expression
Appreciation of the importance of the creative expression of ideas, experiences and emotions in a range of media, including music, performing arts, literature, and the visual arts.
The first three competences (communication in the mother tongue, literacy and mathematical competence, basic competence in science and technology and communication in foreign languages) are linked with traditional school subjects and can be integrated within the traditional national and school curricula. The competences of the second group are cross-curricular in nature and can be supported by transversal capabilities and skills such as critical thinking, creativity, sense of initiative, problem solving, risk assessment, decision-making and constructive management of feelings.
GrundtvigProjectVINTAGE
WP2Deliverable4
4
1
Several countries represented in VINTAGE have developed policy to stimulate lifelong learning like Italy, Austria, The Netherlands and Germany. In all countries initiatives can be found that address competence based adult education. In the projects found one or more of the above 8 EU key competences have the focus. Some countries adopted the key competences of the EU, some use a different, but comparable set of competences. The adult target groups range from nonnative citizens to working mothers and fathers to university students and teachers, those interested in further education and those returning to a job. Competence based learning and assessment are used for study or other forms of professional development (career planning, re-integration on the labor market, study) or to personal development not related to work. Some competences are industry specific, like the competences for medical professionals in Ireland, some more general, like the German self-assessment tool: Selbst Check Beschäftigungsfähigkeit. This checklist contains statements focusing on 6 competences valuable in work contexts:
• Communicational competences • Tolerance towards frustration and
In Italy the Key Competences for lifelong learning, highlighted by the European Commission, haven’t been homogeneously implemented: if in the field of secondary education they have been acknowledged, although in an adapted form, into educational pathways, not as much can be said about adult education. Italy has been a pioneer though both in the creation and use of portfolio of competences, with 9 typologies of portfolios for school (from kindergarten to secondary schools) validated by the Council of Europe in the period 2002-2005. This development has been accompanied by training of teachers, conferences, publications, and a plethora of publishing initiatives, which have also produced a series of not validated portfolios. The Italian Ministry of Education promoted and implemented a study to develop possible
Key competences in adult education
2
standards and their relative set of descriptors for basic adult education. These standards refer to five conceptual/functional areas:
• languages • mathematics • social and economic • technology • science.
In The Netherlands several initiatives have been taken to put the focus in (adult) education on lifelong learning through the use of key competences: in Adult and Vocational Education and in combating illiteracy. The Ministry of Education, Culture and Science and the Ministry of Social Affairs and Employment) cooperate in focusing on career development for adults in the documents Reinforcing Learning & Working; Action Programme 2005-2007 and Carrying on with Learning & Working; Plan of Approach 2008-2011. The essence of the approach is to achieve a sustainable regional infrastructure for lifelong learning (regional partnerships, training and employment helpdesks, training databases) and to put in place a total of 125,000 EVC schemes and programmes combining work and study. Lifelong learning is also an important concept the Dutch document the Strategic Agenda for Adult and Vocational Education (2008-2011).
GrundtvigProjectVINTAGE
WP2Deliverable4
5
1
It focuses on competence-based revision of the MBO qualification structure, competence-based teaching and better proficiency in Dutch (the introduction of a minimum standard for all participants and the development of continuous progression of learning for language and math). The competence model used, which was developed by the centres of expertise on vocational education, training and the labour market (KBBs), is a consistent, coherent and well-organised body of terms for use in describing (25) competences in the qualification guidelines. The content of each competence is broken down into the skills, knowledge and behaviour required for a particular occupation. Main key competences involved are communication in the mother tongue, communication in foreign languages and mathematical competence. In Germany the eight key competences as set by the EU play no particularly important role in the development of assessment instruments and methods. The Swedish government did not adopt the key competences either. With the recent economic crises in Ireland, and Europe, returning to education has become increasingly popular among adults in Ireland, and equally among adults in employment who are looking to up-skill to remain in employment and those who are currently job-seeking. While many courses aim to improve digital competences, to keep up with the demands of modern employers, there are very few targeted or good practice examples from Ireland in the field of adult education with regard to key competences. Where key competences are referred to, they tend to be industry-specific competences which are being measured and not the eight competences which were prioritised by the European Commission.
Assessment tools
2
In this chapter a selection of assessment tools as presented in the national reports will be described. In the next chapter an overview of characteristics of these tools will be presented. ProfilPass (Germany) A portfolio instrument based on a biographical approach to visualize and to document competences from all areas, focused on work life and formal education, but also on informal and non-formal areas, combines self and external evaluation. Selbst Check Beschäftigungsfähigkeit (Germany) This tool assesses also e.g. ability to learn which could be linked to Learning to learn.
• Communicational competences • Tolerance towards frustration and
The Selbst Check Beschäftigungsfähigkeit combines self-evaluation and external evaluation and is meant to offer a reliable profile of the user. It aims to offer orientation and support for future plans and actions within work and job. Specific versions are available for teachers and students. The self-check is online available and tailored for employees and people with work experiences. It is explicit for self-evaluating employability and assesses those skills, competences and qualifications that are considered to be necessary in order to stay employed and to be able to react towards changing requirements in a fast changing working environment. Competences and skills are meant to be evaluated and if needed suggested to be improved. Beurteilungsbogen zu sozialen und methodischen Kompetenz – smk72 (Germany) Online tool – based on test theory – that can be accessed without registration, but
GrundtvigProjectVINTAGE
WP2Deliverable4
6
3
only for the assessment of social competences assessing on 7 dimensions: ability to cooperate, self-sufficiency, social responsibility, leading ability, communicational ability, context appropriate behavior, ability to deal with conflict. These are reflected through 42 statements. The instrument is a nominal –actual/target-performance comparison focusing on social and methodological competence in the sector of vocational training and further education and for personal recruiting processes. Also aiming to provide a profile on possible competence improvements and developments. The instrument is based on occupational and economic educational approaches where these two competences are considered to be valuable for the professional life. Intercultural Awareness and Cultural Competency (Ireland) This course is delivered by the National Youth Council of Ireland, and is only for adult learners who volunteer or work with youth organizations or young people in an out-of-school setting. Despite this narrow target group, it is still a very worthwhile course which is designed to introduce participants to the theory and methodologies required to develop and maintain cross-cultural relationships. This course offers modules on understanding culture and identifying aspects of one’s own culture; the role of culture in communication and how different cultural and ethnic groups interact with each other; identifying different ways of working and communicating; and recognizing the different stages of cultural competency (www.intercultural.ie). This course is only available on request from a local youth worker or youth organization. Competence-based self-assessment portfolio for students (Ireland) The Health Promotion Department of the National University of Ireland, Galway has drafted a competence-based self-assessment portfolio for students undertaking the Masters in Health Promotion. This portfolio tool is a good example of what could be used in a
4
wider adult education setting in Ireland. The portfolio has been heralded as a means of student assessment as it provides students with the opportunity to reflect on their learning and to assess the course as a whole, allowing them to integrate materials across modules and assignments completed and leading to a more holistic learning experience. The advantages of this type of assessment tool can be seen in the characteristics we come to associate with adult learners, namely that the learner is self-directed; their part experiences are a rich resource for learning, they are curious and self-motivated and they have a readiness to learn which has been developed through life experience (McKenna et all, p. 2). This portfolio is a combination of assessments which leads to the learner being able to identify their strengths and weaknesses by the end of the programme. This tool could act as a useful template for the development of the self-assessment tool for VINTAGE. Further information on this tool can be found at: www.nuigalway.ie/healthpromotion/documents. Assessment in an assimilation or citizenship course (The Netherlands) An example of assessment in adult education is the assessment in an assimilation or citizenship course. The target group of the course are non-native
GrundtvigProjectVINTAGE
WP2Deliverable4
7
5
speakers who want to naturalize in the Netherlands. Participants can choose between three different types of assessment: 1. a portfolio panel conversation
(participants have to hand in a portfolio with 20 practice demonstrations),
2. an assessment (consisting of 4 assessments)
3. a combination of one portfolio and an assessment (participants have to hand in a portfolio with 10 practice demonstrations). With the certificate you can apply for a Dutch passport.
See: http://www.zadkine.nl/StuderenBijZadkine/Opleidingen/Volwassenenonderwijs/Educatie%20inburgering%20en%20basisvaardigheden/Inburgeringsexamen.aspx Social Work portfolio assessment (The Netherlands) The Windesheim Academy has provided a training ‘portfolio assessment’ to teachers that are involved in the assessment of practical learning of students of the Bachelor Social Work. The structure of the portfolio assessment is being discussed. On the basis of draft materials the students handed in, the requirements are determined of a good portfolio. These are the results of the discussion: the student must point out which qualification he or she displays. Also the approval of products by a tutor is important. Sometimes it is also useful to consider other patterns in relation to the portfolio: insecurity or missing a structure. It turned out that the teachers were all on the same level in the assessment of the products. http://www.kennisbanklevenlangleren.nl/art/uploads/bibliotheek/1311076766_WindesheimnieuwsbriefLLLnr2.pdf TellVit teacher training programme (Sweden) Part of a Multimedia kit virtual trainer (video, audio, pps with animation and text). It also contains a Handbook explaining the technical requirements and how to use the tool and a
6
Training Needs Analysis Matrix. It was available both on DVD and online. The website does not exist any longer, but there is an end report of the project available on the TELLVIT-Website. A Swedish initiative for training of teachers to measure / assess learning-to-learn competencies is also available as an online learning activity, and can illustrate a practical approach to informally provide learning services on assessing key competencies. This initiative is introduced from the following URL: http://www.bedomningforlarande.se/formativaprocesser/1/3 Only available in Swedish! EdaLab PON and SAPA PON (Italy) EdaLab PON is a diagnostic tool to verify the initial and in progress competences of adult people. SAPA PON is a systematic survey of literacy competences in specific target of adult people in the region Calabria, Sicilia and Puglia. These are 2 project started in 2007. Within these 2 projects have been implemented scientifically tested instruments to assess competences. The experimentation has analysed the literacy competences in a group of adult people aged 20-55 with specific educational needs. A
GrundtvigProjectVINTAGE
WP2Deliverable4
8
7
questionnaire has been administered to participants composed by a part with questions about the socioeconomic background, to allow contextualized the surveys, and a part with cognitive items organized as follows: • Literacy test: to understand a text, a
table, a chart, to demonstrate the mastery of lexicon;
• Problem solving test: to analyse problematic situations, set and verify the solutions.
On the basis of results 5 categories of competence have been identified, ranging from simple abilities to complex abilities. NRP: National Reference Point for the recognition of qualifications (Italy) The Italian Quality Assurance National Reference Point has established a Board of representatives from governmental institutions (Ministry of Education, Ministry of Labour, and Regions), the social partners and VET providers. In addition new decrees on the reform of technical and vocational schools include references to the EQAVET Recommendation and the need to develop indicators for self-evaluation and external evaluation. These are examples of the “top down” process. On the other hand the Quality Assurance National Reference Point works with VET providers through their
8
associations or through the school networks on quality issues. The Quality Assurance National Reference Point also promotes pilot projects and new approaches to quality assurance. In 2010 the Ministry of Labour has issued guidelines for the training, which provide for the reorganization of the national system of qualifications as a basis for the transferability of the results of non-formal and informal according to the European guidelines. In April 2012, the Ministry of Labour has presented a bill for the establishment of national rules and standards to ensure usability and transparency to the skills acquired in the workplace and in everyday life. PAEPEDA: PErsonalizzare l’APprendimento in ambito EDA: To personalize Adult Learning Pathways (Italy) Is an Italian Pilot project developed within the ESF OB. 3/C4 – concluded in 2005 (PEAPEDA 2005), which carried out a research about the development of meanings and approaches of learning personalization in the Adult Education in Italy, highlighting, just in the 2005, the lack of models and of practices for adult training. The research, carried out by a group of researchers and experts of educational processes and adult education, shows: • a historical analysis of the development
of the concept of personalization in education in Italy and in Europe;
• the explication and definition of the meaning of term “personalization” chosen and adopted in the PEAPEDA approach.
• an exam of the theoretical approach about adult learning and personalization approach in education;
• the design of the map of competences of the innovative figure of learning personalization trainer;
• a prototype of a web based system of self evaluation for trainers dealing with
GrundtvigProjectVINTAGE
WP2Deliverable4
9
9
adult learners, based on a map of competences to compare themselves with.
@ of self-evaluation. A collective transactional process of self-evaluation (Italy) The self-evaluation is a model designed by prof. Marco Guspini, experienced by a group of researchers and experts of educational processes and adult education, in the university training of trainers. It has been also recently experienced and tested during the sessions of training of trainers within a Leonardo da Vinci Project Transfer of Innovation ITUBE - Innovation Transfer in continUous education of an integrated model Based on pErsonalization and digital portfolio (Reference n.: N. 7DB846DCEAB0604E). The of self-evaluation is based on an inductive, not didactic nor directive approach. The role of the trainer is to scaffold and encourage peer tutoring and collaborative interactions. During the whole life cycle of the process the trainer with his/hers assistants (one or two) go through the groups, gives suggestions, answers the questions, encourages participants who are less involved, makes questions, etc. The process is mainly based on a collaborative (rather than cooperative) approach. The model focuses however on the results reached by the group before those reached by a single individual. The goal is therefore to assess, more precisely self-assess, the work group’s efforts. The self-assessment of the group focuses on the communicative, emotional and social areas as well as its contents. Steps in the self-evaluation process are:
1. Focus brainstorming; 2. Introduction of representative
biographies; 3. Sharing the self-assessment
philosophy; 4. Skill manifestation in designing;
10
5. Transforming skills into competences; 6. Integration; 7. Project or design; 8. Outside learning or benchmarking; 9. Good practice identification; 10.Focus group.
The process of self-evaluation focuses on: • Task; • Process development; • Duration; • Organization. LABOBS (Italy) Self-evaluation approach during and after training: portfolio, personal development plans, open interviews, a blog. LABOBS – Learning Abilities Observatories is a Grundtvig Multilateral Project (134430-CP-1-2007-1-IT-GMP, (www.labobs.eu). The objective of LabObs project is the promotion of the empowerment of all the adults who wish to further their education along all their life. To pursue this aim the project examines the strategies applied in order to recognize the different educational experiences, thus increasing the number of people involved in non-formal education and promoting specific tools, like for example the Study circles that are an experimental approach experienced in Genova Region in Italy. the attention focused on: • analysis of the 8 key competences as a
base of every learning experience; • informal learning deriving from the
participation of adults in the Study
GrundtvigProjectVINTAGE
WP2Deliverable4
10
11
Circles; • a self-evaluation approach adopted in
order to identify, record and make visible the changes incurred during and thanks to the learning experience: a personal development plan leads the adult to reflect on the expected progress of the learning experiences, an open interview guides the adult to describe the perceived changes, a blog is used in order to track the changes as a sort of portfolio.
CARE talents (Italy) EVC tool. Validation and certification of competences acquired on the job by operators dealing with elderly and disabled people. The adopted approach explicated as follows: true/false items assess theoretical knowledge; exercises with open answers and “fill in the blanks” assess practical knowledge; case history assess the competences as enacted behaviour. Action Research project (Italy) Research project about the design of a strategy and a toolkit for evaluation and self evaluation of key competences in the VET for trainers and students. The focus is on key competences, the research and developed strategy and toolkit were limited to the key competence Mother Tongue. The experience and the realised
12
products are available on the website http://www.agrariosereni.it/as/pagine/progetti/valutazione/valuta.php http://cambiolavoro.com/testgram.htm (Italy, ASSOCIAZIONE FORMAZIONE 80) This is a self-assessment test about management of information, management of risks, management of creativity, management of emotions, management of time, etc. The website and test were developed by a group of graduates from Università Bocconi in Milan, after some experiences in the marketing sector. http://cambiolavoro.com is a website providing services for matching supply and demand on the labour market. www.competenzestrategiche.it (Italy, ASSOCIAZIONE FORMAZIONE 80) This platform is the result of a research about planning and implementation of a guide and digital device for self-assessment and for development of strategic competences in study and work) commissioned by the CNOSFAP of Rome and realized in 2010-2011. The development of the platform and guidelines was appointed to a group of researchers and expert from important academic institutions (Università Pontificia Salesiana di Roma, Università degli Studi Roma Tre), coordinated by Michele Pellerey.
GrundtvigProjectVINTAGE
WP2Deliverable4
11
1
TOOLS APPROACH LEVELS FUNCTIONS ARRANGEMENTS USERS It 1 Competence
profile to compare
yourself with;
Diagnostic
Not able, able, expert
Guidance
Self assessment
Learners
Group self evaluation
Organisational
learning; action learning
Levels of impact
Professional development
Focusgroup
Benchmarking
Groups
Self assessment
questionnaire/ Grid
Inventory
Yes/no Does/ doesn’t
Validation
Self assessment and external assessment
Teachers Students
Selfevaluation in study circles
Formative
Validation
Open interviews
Study groups
APL tool/open and closed
items
Evidencing on the job
Validation/ certification
Assessors
assess disabled workers
It 2 Management skills test
Test
Formative
Demand and supply of labour
Employees
Digital self assessment of stategic skills
Study and work
improvement
guidance Work/school setting
Students/ Employees
G Portfolio
Biographical Visualisation
of competences
Validation
Adults
Kompetenz
bilanz(2)
Biographical Self reflection
checklist
Rating 1-5
Empowerment
Adults
Computer based test
test
Use of
resources Linked to activity
Contextualized Social
situation etc.
Employablity
Returners to
work
Kompetenz check
Orientation and formative
Qualitative indicators
Work orientation/
employability
Au Citizenship Self reflective empowerment Political, Civil, community
context
Citizens
Analytical overview of assessment tools
GrundtvigProjectVINTAGE
WP2Deliverable4
12
2
IRL Certificate programme
Educative
Ratings on identified
dimensions
Certification
Education context
Adult learners
Self assessment
Portfolio
Self diagnostic Self direction Higher education Students
SWE Teachers’ assessment in teaching the
KC’s Self
assessment and peer support
Educative
Networking Peer
assessment
Certification
Benchmarking
(In service) Teacher
education
Web community
Teachers
Adults
Social interactive online tool
Virtual Community dialogue; Survey;
Gaps analysis
Competences hierarchy
Validation, certification
and EU mobility
Virtual community platform
Vocational learners
NL Acknowledgement of prior learning and experience
A 6 – 12 weeks
trajectory of introduction
portfolio building,
assessment and advice
European framework
based system for NL, NLQF.
Validation Certification (learning)
career planning
Trajectory in a certified APL organization
Adult learners/ Workers
Competence assessment
School based assessment +
audit and STAR
interviews
NLQF
Summative assessment
School based assessment, employers
feedback and STAR interviews
Vocational students
Portfolio
Traineeship + log and
documentation
Formative and summative assessment
School setting
Vocational learners
360 degrees feedback
Appraisal and assessment
Professional development
Medical setting
Learning professionals
In service education
Inservice trajectory including training,
coaching and assessment
NLQF
Formative assessment
Training setting
Learning professionals
Portfolio on
language skills
Demonstration
Typology of language settings
Entitlement to
a passport
20 cases to
demonstrate your language abilities
Nonnative citizens/ learners/ Workers
Appreciative
approach
Discovery, dreaming, designing,
destiny
Talent
development
Evidencing
Building your
own case
Portfolio building
Documentation
and monthly interviews
Social work
trainees
NB. In the first column the national reports are mentioned. Note that the order does not correspond with the previous section. This however does not affect the conclusions presented in the following section.
GrundtvigProjectVINTAGE
WP2Deliverable4
13
1
The VINTAGE partners found several assessment tools that might be interesting for the VINTAGE assessment tool to be. In this section we give a summary of the analysis done on the matrix presented above. We analysed the matrix by going through it horizontally first, thus getting a picture of the examples given. After that we summarised the columns of the matrix vertically, so we could compose lists of ideas and features of what the VINTAGE tool could be like. Thus we managed to create a kind empirically based tool typology, that may serve as a source in our further process of developing the VINTAGE online tool. The typology consist of summaries of the columns include in the matrix: • Tools • Approaches • Functions • Levels • Arrangements • Contexts • Users.
Tools The tools found are self-assessment tools, group assessment tools (like 360 degrees feedback, benchmark activity in a group, individual or group interviews) or portfolio’s containing several kinds of assessment instruments/activities. Some instruments can be used online, some are paper documents. The list of tools actually provides a rich harvest
2
of potential tools of various levels of sophistication and of social involvement. On the one hand we find simple kind of tests, checklists, questionnaires; on the other hand we see tools that involve group work and social interaction either face to face or on line. An interesting other dimension is the dimension of tools that focus on self-assessment, peer assessment, involvement of external observers, or external assessors.
Approaches Several approaches to assessment can be detected in the instruments: • Biographical • Self-diagnostic • Diagnostic • Orientation • (professional)Learning • Evidencing • Guidance/coaching • Validation/certification (by an assessor) • Networking/community • In-service learning • Appreciative approach • Career development /CV building
Levels Though little is explained in the national reports on how the levels of mastery of competences are defined, operationalized and measured, the harvest here still is interesting. In most tools there is no information available
Conclusions and ideas on assessment tools
GrundtvigProjectVINTAGE
WP2Deliverable4
14
3
on levels of mastery, but in the cases in which there are, we see references to the EQF and national derivates of that. We see also other attempts to create hierarchies of learning outcomes and ways to rate them. In an interesting German case we see an attempt to define competence as the ability to act, to use resources in a particular context and social situation. This approach comes very close to the approach elaborated in the Vintage background article.
Functions The assessments described or mentioned in the national reports, show a variety of functions, or purposes. The functions closely relate to the approaches described above. They may be summarised in the following way: • Raising awareness of one’s potential • Empowerment • Support in one’s
learning/development process (formative assessment)
• Providing evidence of competences and achievements
• Summative assessment to identify learning outcomes
• Validation • Certifications • Employability/mobility • Self-direction • Access to a country • Talent development • Career building • Evaluative assessment to provide
educators with curricular feedback
4
• Educative assessment Integrated within learning activities, educative assessment builds student’s (and faculty’s) insight in, and understandings their own learning and teaching. Assessment thus is a form of learning.
Arrangements (settings and target groups) The assessment arrangements may be either • individual computer based tests, or • arrangements involving peers • assessments involving assessors in
the educational setting • assessors on the job • closed or open items of assessment • in educative, work, HRD, and
community settings • at various levels of education • single events or trajectories • involving various intensities of
evidence (from answers to questions to in depth interviews or serious case studies etc.)
Users The following target groups may be recognized in the national reports: • Pupils/young adults in transfer from
school to vocational training and/or from training into professional life/job.
• Students in the course of study or in transfer period towards work
• Professionals in companies/schools/universities
• Adult learners • Trainees • Returners to work • New citizens / migrants (civic
integration programmes) • Teachers Based on the Vintage background article and this empirical overview of potential tools, the partners will have to identify the specifics of the eventual Vintage online tool.
GrundtvigProjectVINTAGE
WP2Deliverable4
15
The Grundtvig Project Vintage
Vintage is a European project funded through the Grundtvig action of the Lifelong Learning Programme. The acronym means “online tool for self eValuatIoN of key competences in adulT AGE”: in fact, the project aims to develop and test an online tool for the self-evaluation of key competences, to be used by adult learners involved in non vocational adult education learning pathways.
The project has started in January 2013 and will last two years. It brings together partners from 6 European countries: Italy, Austria, Germany, Ireland, Sweden and The Netherlands.
European Commission (2004), Key competences for lifelong learning. A European reference framework, Bruxelles.
European Commission (2006), The Key Competences for Lifelong Learning – A European Framework. Annex of a Recommendation of the European Parliament and of the Council of
European Commission (2009), Assessment of key competences. Joint seminar of the Peer Learning Clusters ‘Key Competences’, ‘Teachers and Trainers’, ‘Maths, Science and Technology’, and ‘Learning Outcomes’, Bruxelles.
European Commission (2010), Draft Background Paper for the Belgian Presidency meeting for Directors-General for school education, 8 July 2010. Assessment of key competences. Bruxelles.
European Commission (2012), Assessment of Key Competences in initial education and training: Policy Guidance, Strasbourg.
European Commission (2006), Recommendation of the European Parliament and of the Council of 18 December 2006 on key competences for lifelong learning (2006/962/EC), Bruxelles.
European Commission (2012), Education and Training 2020 Work programme. Thematic Working Group 'Assessment of Key Competences'. Literature review, Glossary and examples, Bruxelles.
References
GrundtvigProjectVINTAGE
WP2Deliverable4
16
2
2
Gordon, J. et al. (2009), Key competences in Europe: Opening doors for lifelong learners across the school curriculum and teacher education. CASE-Center for Social and Economic Research on behalf of CASE Network, Warsaw.
Eurydice (2002), Key Competencies. A developing concept in general compulsory education, Bruxelles.
OECD (2005), The definition and selection of key competencies. Executive Summary, Paris.
Takayama K. (2013), OECD, ‘Key competencies’ and the new challenges of educational inequality, “Journal of Curriculum Studies”, 45:1, 67-80.
18 December 2006 on key competences for lifelong learning, Bruxelles.
VINTAGE Partners LEARNING COMMUNITY - IT www.learningcom.it [email protected] WISAMAR Bildungsgesellschaft gemeinnützige GmbH – DE www.wisamar.de DIE BERATER Unternehmensberatungsgesellschaft Mbh – AT www.dieberater.com STPKCSwedishTelePedagogicKnowledgeCentreAB–SEwww.pedagogic.comMEATHCOMMUNITYRuralandSocialDevelopmentPartnershipLimited–IEwww.meathpartnership.iePLATO‐PlatformOpleiding,OnderwijsenOrganisatie–NLwww.socialsciences.leiden.edu/plato/Formazione80‐ITwww.formazione80.it
