17
www.aqu.cat Yes, there is impact. But is it positive, negative or “none of the above”? The case of Faculty of Sciencies and Faculty of Biosciences of the University Autonoma de Barcelona
www.aqu.cat
Yes, there is impact. But is it positive, negative or “none of the above”?
The case of Faculty of Sciencies and Faculty of Biosciencesof the University Autonoma de Barcelona
2
Context: Verifica Process
Assessement
Proposal structure:1. The name and description of the degree.2. The justification for the degree.3. The competences.4. Student access and admission.5. The programme content.6. The academic staff.7. The physical resources and services.8. The intended outcomes.9. The internal QA system.10. The timetable for introduction.
Degree proposal
Degree Committe
Institutional support (IQAS)
Provisional reportAppeals
(Amended proposall) 2nd assessementFinal report
Royal Decree 1393/2007
Degree Committe
Institutional support (IQAS)
Disciplinary Review Panels (include, at least, one student
and one professional)
Disciplinary Review Panels (include, at least, one student
and one professional)
3
Yes, there is impact:
Standard 5.1 The set of modules or subjects that make up the curriculum are consistent with the competency profile title.
Provisional Report Final Report
The Master is structured in 6 ECTS compulsory, 24 elective (choice of 36), 15 for internships and 15 Final Master Thesis. The Commission appreciates 6 ECTS as insufficient to ensure a common basis for all graduates of the master. In addition, this subject is structured as a series of sessions on various aspects of genetics (2-3 hours) that are neither advanced nor offer the possibility of a common academic body to master. Ultimately these 6 ECTS, taught in this way, can not be the body of doctrine of the master.
The Master is organized into 18 ECTS mandatory, 12 optional (choice of 36), 15 and 15 internships Final Master Work. The set of modules is consistent with the profile of training.
AQU (Catalan University Quality Assurance Agency )
Specific committee for Sciences 5 academics:
Prof. Francisco Marques (president, UPC)
Prof. Carmen Cámara (Universidad Complutense de Madrid)
Prof. Joan Fibla (Universitat de Lleida)
Prof. Juan J. Nuño (Universitat de València)
Manuel Sánchez (Universidad de Granada)
1 professional: Dr. Lluís Bellsolell
1 student: Ms. Rosa del Hoyo (UAB)
1 secretary of AQU
4
Stakeholders I
In 2012 and 2013, the Specific committee, in the individual assessment phase, involved 23 academics outside the Catalan University System (Oceanography, Advanced genetics, Photonics…).
FacultiesSciences
Biosciences
Teaching Quality Office
Susana Segura Núria Marzo
5
Stakeholders II
Provisional reports about 12 proposals for master’s programmes
assessed by the Commission of Sciencies of AQU Catalonia
Questionnaires for academics, deans and technical unit (UTQ)
List of the master’s programmes analysed
6
Methodology I
7
Methodology II: questionnaires
A) ASSESSMENT OF MODIFICATIONS REQUIRED
Rate how much the modifications required in the interim report are required and have helped to improve the proposal. We have classified theseimprovements in terms of how they affect the design of the training profile , access , resources or other aspects of the proposal. Required amodification involves a change that can positively or negatively . This modification may be relevant (appropriate) or not.
Modifications relating to design profile
Review the name of the title (B2) Add competencies to specify the main type technology tools that use titled know ( x programming , managing and bases , etc.). (B2) Correct CE1 writing, eliminating the term " demonstrate " (B1) Define the skills and / or learning outcomes resulting from content related to programming in the module "Programming in Bioinformatics”
(B3) Provide a model agreement placements (A1) Add the information requested on the Final Master (A4)
1. Rate the relevance and the extent to which these changes improve or not the proposal:
2. Overall, rate the extent to which these changes improve or not the proposal:
Not applicable No improvement 1 2 3 4 Improvement
8
Results and discussion I
The institution "has learnt" from errors or omissions made
9
Results and discussion II
Type of modifications ans recommendations
TYPE A Technical-administrative aspects: regulations, credits, access, justification of the proposal and requests for further information
TYPE B Academic aspects: skills, proposal's approach, the coherence of the academic planning in relation to the educational profile and methodologies and evaluation systems
10
Results and discussion III
7 out of 10 academics say the provisional report requirements have improved the proposal
My assessment is positive. This report leads to the
conclusion that the commission analysed the proposal in
detail and "detected" both specific technical-type
deficiencies and more important defects whose correction
substantially improved the final proposal. The
suggestions and requests were very well-expressed and
therefore have proved to be useful. (Respondent 1)
11
But, there is a wide range of opinions…
It is an opaque, very bureaucratic process. The emphasis
was always on formal aspects with no practical importance,
instead of aspects that (at least in my opinion) have a
real effect on teaching quality. (Respondent 2)
ACCES TEACHINGDESIGN
ACADEMICRESOURCES
GLOBAL PROCESS
AQU
OQD
ACADEMICS
12
Qualitative results
Specific committee for Sciences: in 9 cases the Committee ask for changes in the student profile specifications for the master in order to guarantee the attainment of the programme profile.
Academics: 5 of 9 consider that they do not improve the proposal. One believes that creating a committee for assessing student entrance profile will be bureaucratic.
Teaching quality office: considers that having a collegiate body analysing the students previous studies will give more guarantees to students.
13
Disagreements: 1st example
A contextual remark: In Catalunya, in 2013 there are a total of 485 masters degrees, and they must have at least 20 students enrolled per year. Competition is fierce.
Specific committee for Sciences: in 10 cases the Committee requires further information regarding the institution resources.
Academics: 7 of 9 consider that this does not improve the proposal. It’s unrealistic and difficult to know in advance the individual lecturers that will be involved.Teaching quality office: it does not improve the proposal, only adds more information.
14
Disagreements: 2nd example
A contextual remark: The standards apply to all institutions (public and private). This point is of doubtful necessity in large centres with a long tradition, where teaching staff are appointed through externally-regulated mechanisms.
15
Conclusions
Both administrative and technical requirements and recommendations have diminishedfrom one year to another -> TQO good practice .
Programme approval has impact over the design of the degrees.
Different stakeholders -> Different viewpoints!!! Not unanimity among the different stakeholders neither about the “goodnes” nor about the “wrongness” of the process
Areas of improvement detected: foster interaction between external assessors and coordinators, enhancing communication about the assessment aims and objectives, etc.
On the whole….
success in bringing degrees close to the EHEA
But not all the objectives of the reform have been achieved (e.g. benchmarking of syllabuses)
16
Impact and typologies of external assessments (audit, institutional assessment, program approval): which are the ones that have more impact, which are the most useful, and why.
Educative reforms and the role of external assessment: it is enough with “external controls” or should other mechanism be in place to steer change?
When the change is mandatory: risks, benefits and precautions
Different views between different stakeholders: which should prevail?
Questions for discussion
