1
Reconceptualising Competency-based
Education and Training: with particular reference to education for occupations in Australia.
Andrew Gonczi
Submitted for the Degree of Doctor of Philosophy (by publication).
University of Technology, Sydney
August 1996
2
CERTIFICATE
I certify that this thesis has not already been submitted for any degree and is not being submitted as part of
candidature for any other degree.
I also certify that the thesis has been written by me and that any help that I have received in preparing this thesis,
and all sources used, have been acknowledged in this thesis.
3
CONTENTS
Linking Essay
Publication (a)- Gonczi, A., Hager, P. and Oliver, L. (1990). Establishing Competency-based Standards in the Professions. National Office of Overseas Skills Recognition. Canberra: Australian Government Publishing Service (pp 70).
Publication (b)- Hager, P. and Gonczi, A. (1991). Competency based standards: A boon for continuing professional education? Studies in Continuing Education, 13,1, 24-40.
Publication (c) - Gonczi, A., Hager, P. and Athanasou, J. (1993). The Development of Competencybased Assessment Strategies for the Professions. Canberra: DEET, Australian Government Publishing Service (pp 117).
Publication (d)- Scheeres, H., Gonczi, A., Hager, P. and Morley Warner, T. (1993). The Adult Basic Education Profession and Competence: Promoting Best Practice, Sydney: University of Technology, Sydney (pp 80).
Publication (e)- Hager, P. and Gonczi, A. (1993). Attributes and competence. Australian & New Zealand Journal of Vocational Education Research, 1, 1, 36-46.
Publication (f)- Hager, P., Gonczi, A. and Athanasou, J. (1994). General issues about the assessment of competence. Assessment and Evaluation in Higher Education, 19, 1, 3-16.
Publication (g) - Gonczi, A. (1994) Competency Based Assessment in the Professions in Australia. Assessment in Education - principles, policy, practice, 1, 1, 27-44. Publication (h)- Hager, P., Athanasou, J. and Gonczi, A. (1994). Assessment: Technical Manual. Canberra:
DEET, Australian Government Printing Service (pp 158).
Publication (i)- Gonczi, A. and Hager, P. (1994). The Distinction between Skills Based and Qualifications Based Procedures for Recognising Migrants Professional Skills. International Migration, XXXIII, 1, 127-144.
Publication (j)- Gonczi, A. (1996). Future Directions for Vocational Education in Australian Secondary Schools. Australian & New Zealand Journal of Vocational Education Research (forthcoming). This is an adaptation of the paper below. Gonczi, A. (1995). Future directions for post compulsory education: integrated
competency based approaches to vocational education in NSW. Paper presented at the Vocational Education in Schools conference. May. Coffs Harbour (pp33).
Publication (k)- Gonczi, A. (1996). Competence Based Assessment by Alison Wolf. Review Symposium, Assessment in Education, 3, 2, 241-247.
Appendix (a)- Ash, S., Gonczi, A. and Hager, P. (1992). Combining Research Methodologies to Develop Competency-Based Standards for Dietitians: A Case Study for the Professions. National Office of Overseas Skills Recognition Research Paper No. 6, DEET. Canberra: Australian Government Publishing Service.
Appendix (b) - Gonczi, A. Hager, P. and Palmer, C. (1994). Performance Based Assessment and the NSW Law Specialist Accreditation Program. The Journal of Professional Legal Education, 12, 2, 135-149.
Appendix (c)- Gonczi, A., Curtain, R., Hager, P., Hallard, A. and Harrison, J. (1995). Key Competencies in on-the-job Training. Sydney: UTS-DIRETFE.
4
LINKING ESSAY
5
CONTENTS
The thesis P 1
Overview of the publications P 2-3
The aims of the linking essay P 4
The policy context of competency-based education P 4-16
Education policy in Australia and the link with international developments.
The relationship between the competency debate and wider policy issues
The theoretical context of competency-based education P 16-32
The concept of competence
Key Competencies and their conceptualisation
Arguments against the competency approach to education
Implications of the integrated approach for competency
standards development P 32-33
Implications of the integrated competency approach
for curriculum and teaching P 33-36
Other uses of competency standards P 36-37
Competency-based assessment P 37-49
Introduction
Contribution of the publications: competency based assessment and validity
Further questions in competencybased assessment
Developing competency-based assessment systems P 49-50
Conclusion P 50-51
- 1 -
Reconceptualising Competency-based Education and Training: with particular reference to education for occupations in Australia.
Andrew Gonczi June 1996
An essay written in partial fulfilment of the requirements for the degree of Doctor of Philosophy (by
publication).
The thesis
The thesis that emerges from the publications nominated for examination, is that a holistic or integrated competency based approach to vocational education and training (VET) and professional education (both initial and continuing) has many advantages over traditional approaches: • It provides a curriculum framework which links practice to theory in more coherent
ways than currently exist; • It potentially provides a way of breaking the old dichotomy between "knowing that"
and "knowing how" which has characterised Anglo-Saxon education and which has resulted in the belief that education which is practical is both different from and inferior to that which is abstract;
• It provides the basis for teaching and learning approaches which could enhance students' adaptability and flexibility over their lives;
• It has the potential for developing in occupational education more valid assessment strategies than those traditionally used and also for reducing the deleterious effects on learning of measurement-based assessment approaches.
In summary, it is argued that the integrated approach to competency-based education provides a conceptual base for the competency movement and a promising direction for educational reform for all levels of occupational education.
It is further argued that competency standards developed through an integrated approach can facilitate the implementation of a number of other areas of social
2
and economic policy, such as the recognition of qualifications of overseas professionals in Australia, and the internationalisation of professional services. Overview of the publications The publications span a six-year period from 1990 to 1996. The first of them was written at a time when there was very little literature in the area (and virtually none in Australia) and when there was a great deal of confusion about the nature of competency, how to develop competency standards and the implications of the competency approach for education and training. What literature did exist, was mostly twenty years old and was largely a reaction against educational curricula which, it was felt, had failed to adequately prepare students for occupations or for life more generally. In place of a curriculum based on the acquisition of knowledge most of the critics suggested that curriculum should be based on an analysis of what people needed to do. Conceptually, as Wolf (1995) and others have pointed out, it was based on a niave reductionism arising out of behaviourist approaches to education. This approach was quite powerful for a brief period in the 1970s in teacher education programs in the United States. However the challenge to behaviourism from cognitive and humanist approaches to learning seemed to undermine the conceptual basis of the competency movement and very little was written about competency approaches until the late 1980s. As Raven (1996) has recently pointed out, the literature on competency-based education which has appeared recently is also a reaction against "something that is sensed to be wrong" (p.74). But what this is, what needs to be achieved and how this could be done is not clear. He suggests that the contemporary competency literature lacks a conceptual and analytical base and that there is little recognition of the need for a research program which develops a better understanding of the nature of competence, how it might be developed in individuals, how it might be assessed and what impact this would have on individuals, organisations and society generally. It is these issues that the publications submitted for examination have addressed. They have attempted to provide a conceptual base for competency-based education and a framework for how competency might be developed and assessed. Much of the recent literature in Australia has built on the approach which the publications originated.
3
The publications can be divided into those dealing with the nature of competency, particularly the integrated model, (a, b, d, e) those dealing with curriculum and teaching issues (b, j) and those dealing with assessment of competence (c, f, h, i, k). The theme which unites them, is the integrated approach to competency and its capacity to provide a coherent framework for improved educational practices in all occupational education.
Another possible way of categorising the publications would be by educational level. For reasons associated with the traditional division of labour in our workforces we tend to think about the differences between educational levels rather than the similarities. It is usual to think about higher education for example, even when it prepares people for occupations, as substantially different from other occupational education. This is underlined by the fact that there is no term, in common usage, to encompass both what is currently referred to as middle level or vocational education, and education for the professions. Despite its specific nature, professional education is often identified with academic and general education, while vocational education is identified with practical education and is assumed to be devoid of substantial theoretical content. In fact much of higher education for the professions is practical and much vocational education is grounded in theory, even if it is not always made explicit.
A conclusion which I believe can be drawn from these publications as a whole is that the difference between higher education for the professions and vocational education for middle level occupations is one of degree rather than one of kind. Obviously most professional work is more complex than work at, say, trades level. But it is better to conceptualise these levels on a continuum rather than to see them as essentially different. There will be many instances when professionals need to do things which are routine where simple competencies are used. Conversely many tradespeople will need to use complex combinations of competencies to solve challenging problems. Hence, it is not useful to divide the publications into those dealing specifically with the professions (of which there are six- a, b, c, d, g, i) and those dealing with issues relevant to all sectors of education (of which there are five- e, f, h, j, k). What the publications have to say about the nature of competency, how to develop competency through curricula and teaching and how to assess it, is broadly applicable to all occupational education irrespective of the context in which it is discussed.
4
The aims of the linking essay
This essay has three major aims: to demonstrate how the publications that follow link and form a coherent thesis; to provide a commentary on the publications, showing how each contributes to the overall thesis, and to develop further a number of the nascent ideas in the publications.
To achieve the first and second aims, this essay places the publications within the context of international policy developments in vocational education and training (including education for the professions), and the specific policy framework of the beginnings of the National Training Reform Agenda in Australia. This is necessary because the publications are in some measure a reaction to a particular educational context and policy-the introduction of competency-based education in Australia over the last seven or so years. The publications have attempted to reconceptualise the nature of competency-based education, in order to influence these education policies in Australia. There has been no attempt in the publications, however, to contribute explicitly to an understanding of the policy process itself.
More importantly this essay places the publications within a theoretical context which is not always made explicit in the publications themselves. The contribution to educational thinking made by the publications is arguably a result of being able to clarify practical educational issues through research and development of theory.
I have continued my research on some of the themes contained within the nominated publications. This is particularly the case with regard to the implementation and conceptualisation of the Mayer "Key Competencies" in vocational education and training (VET) settings. Hence, to achieve the third aim, I have included a brief account of my recent research on key competencies and tried to show the link between this research and the theoretical concepts informing the earlier work contained in the publications.
The policy context of competency-based education
Education policy in Australia and the link with international developments. Over recent years there has been an increased interest internationally in the relationship between education and the workplace. Countries in almost every part of the world including Scotland, England and Wales, Canada, the United States, Mexico, many South American countries, Australia, New Zealand, Sweden, France, Kuwait, Indonesia, Korea and Thailand, have all, undertaken or are about to
5
undertake substantial reforms of their vocational education and training systems. A number of them are also undertaking substantial changes to higher education and to their school systems. It is in the VET area where the changes have been most significant. These reforms include: attempts to link traditional vocational education more closely to industry requirements; attempts to increase student numbers in VET; introduction or expansion of vocational options in schools; attempts to develop partnerships between vocational and other educational sectors and to raise the status of vocational education; the development, by industry bodies with government support or participation, of occupational and employment-related competency standards which influence the content of the curriculum in both general and vocational education; schemes to facilitate student transfer from one level of education to another, and to facilitate the recognition of prior and experiential learning; blurring of the distinction between vocational and university education; attempts to increase the quantity and quality of industry training and to develop alternatives to what is perceived as a government monopoly in the delivery of VET.
In addition to these systemic reforms, virtually all the countries mentioned above are committed to a particular approach to VET-competency based education/training-though what is meant by this differs substantially between countries. What the reforms have in common is that the content of VET courses is based on occupational competency standards, that it is expressed in terms of outcomes to be achieved by students, and that the assessment of students is (at least partly) based on the criteria expressed in the competency standards. They also have in common the fact that many people outside the education profession have been involved in the formulation of the aims and content of education. This includes industrialists, governments, unionists, lobbyists and bureaucrats outside the education field. What is different between countries is how these competency standards are conceptualised, how and by whom they are developed and the degree to which the standards shape the curriculum and assessment of vocational education and training.
While developments in education for the professions have been less dramatic than those in middle level occupational education, a number of countries have attempted to link professional education more closely to middle level education and have encouraged the development of competency standards in the professions. This has been the case in Australia and New Zealand. In thesecountries the debate about competency-based higher education has been fierce and has been important in shaping the debate in the VET sector.
In the school sector, many countries have introduced reforms include greater access to vocational education subjects and which seek to ensure that the general curriculum
6
includes competencies focused on the transition from school to work. [see publication (j)]
The international interest in VET, in more "relevant" professional education and in employment-related competencies in general education in schools, is partly a product of a complex range of factors associated with the national and international economies. Technological change, industry restructuring due to changes in global trade arrangements, changes in the capital and labour markets and so on, have contributed to general unease and uncertainty about the aims of education. The reforms in higher education, the upgrading of vocational education and training, and the introduction of employment competencies in the school sector, have all been justified by their capacity to increase the levels of skills and flexibility in the economy, assumed to be a prerequisite to increased international competitiveness in the global economy.
In Australia the first explicit statement which argued for a closer link between education and the economy came from the Australian Council of Trade Unions (ACTU). In 1987 the ACTU in it's Australia Reconstructed argued the need for reform of the existing award structure and the need for a greater training effort from industry. They also praised the European system of developing occupational standards and assessing students and trainees against these standards. Some of these ideas were developed in the federal government's landmark paper, Skills for Australia (1987). This paper was the first official statement of government policy on skills formation and its role in structural change in Australian industry. It marked a significant change in government thinking about the role of education and its relationship to national goals. It revealed the government's determination to harness the education system to fight Australia's major economic problems. Not only did the paper suggest the need for greater overall participation in education and training and for more private investment in training, but also that the balance and emphasis of the education effort needed to change so that it better met the long term needs of the economy and the labour market. But it went even further, suggesting that the content of education, particularly in the TAFE sector, was inappropriate to the challenges that Australia faced. It suggested that TAFE was not responsive to
7
industry needs, that it was inflexible in its delivery strategies, that its structures were outmoded, that there was no national consistency, and that it needed to rethink its mission.
In 1988 the federal government sought to initiate discussion about informal training in its Industry Training: the need for change. This document suggested that training was most uneven across enterprises and that the overall industrial environment contained major disincentives to training. Some of these were industrial in nature and its recommendations linked training reform to wider micro economic and industrial reform. This general thrust was supported by the unions, who saw in the reforms a way of overcoming industrial rigidities and of potentially providing career structures for their members. This paper was followed quickly by another, Improving Australia’s Training System (1989), in which the government stated its desire to encourage the development of competency-based training as part of a set of wider reforms to improve the quantity and quality of training. In the same year a special meeting of Ministers of the Commonwealth and States endorsed this policy and set up a national body, the National Training Board, to oversee the development of industry competency standards. After a government overseas mission to investigate training returned in 1990, they stated in their report, (COSTAC 1990), that competency-based education and training based on industry standards would help to tackle many of the problems of vocational education and training. Later that year guidelines for the implementation of the system were published.
The sudden adoption of a competency-based approach to vocational education and training can be explained by its appeal to the key groups interested in reform of vocational education and training and in wider micro economic reform. Industry, as represented by the peak bodies, perceived that such an approach would make public education more responsive to industry's needs, a feeling shared by government and the unions. Government saw the potential of competency-based education to raise the quality of training and to underpin the creation of a national and consistent qualifications framework as part of its micro economic reform strategy. For the unions the reform of the award structure was paramount. They saw the potential of a competency-based structure to assist in that task by facilitating the upgrading of workers with competencies learnt on the job and unrecognised by the qualification system. There was little or no debate with educators and clearly, in retrospect, little understanding either of the conceptual issues or the practical problems involved in the implementation of a competency-based approach. Certainly the first attempts of the National Training
8
Board to explain what they meant by competency standards was mechanistic and relied heavily on their understanding of the approach adopted in England. This approach is dealt with in a variety of the publications.
The suggested changes to VET were paralleled by initiatives in higher education. In the white paper Higher Education: A policy Statement (1988), the federal government sought to develop a balance between the traditional independence of the system and the perceived need to link it more closely to the skills needs of the economy. The subsequent systemic developments in higher education are outside the scope of this thesis. However, the White paper and government policy subsequent to it, led to an intensified debate about the role of the higher education sector in society and about the nature of the content of a university education. The encouragement by government of the development of competency standards in the professions from the end of 1990
[see publications (a) and (c)] and the Labor government's thinking on educational quality first enunciated in the Higher Education Council's (HEC) paper, The quality of Higher Education (1992) ensured that this debate has continued, with a particular focus on the content of education. The argument in that paper, that quality of higher education should be judged by outcomes in the form of attributes possessed by graduates, has already had an impact on the content of some higher education courses. Most institutions have developed profiles of the graduate that their courses seek to develop, though the extent to which their courses explicitly seek to develop these attributes varies. In addition, as has been pointed out by Hager and Gonczi (Edwards et al 1996), a number of courses in a range of universities have sought to link their courses explicitly to the competency standards in their professions.
In the schools sector the “Finn” report, Young People’s Participation in Post Compulsory Education and Training (1991), and "Mayer" report, They Key Competencies: Putting General Education to Work (1993), have attempted, amongst other things, to develop and incorporate employment-related competencies into the general education curriculum.
The aim of the Finn and Mayer committees was to explore ways to develop in young people competencies which would make them more effective participants in work, more flexible and adaptable, and more able to move into higher education. These can be referred to, as they are in the Mayer report, as employment-related competencies. The work of both committees arose out of industry's view that schooling has failed to develop in students, skills that are
9
'needed' for the high-skills workplace of the 1990's. The Finn report argued that Australia had inherited a tradition of education which accepted the divisions between general and vocational education and that the two needed to be brought together both for the sake of individuals and for the economy generally. It argued that schools needed to be concerned with employability and that the vocational sector needed to be concerned with more general, broader vocational education. It recommended that schools needed to be concerned with competency areas that all young people needed for employment: language and communication; mathematical and technological understanding; cultural understanding; interpersonal; problem solving. In 1991 a new committee which has become known as the Mayer committee was given the task of developing the areas outlined in the Finn report into a number of more specific competencies. In the Mayer report, Putting General Education to Work (1993), seven key competencies were developed: Collecting, Analysing and Organising Information; Communicating Ideas and Information; Planning and Organising Activities; Working with Others and in Teams; Using Mathematical Ideas and Techniques; Solving Problems; Using Technology. Later another was added Displaying Cultural Understanding.
One reading of these arguments is that they are part of a conservative agenda to refashion education in the interests of "Capital". Another, however, is that introducing employment related competencies in the curriculum is essential to equity policy-the desire to increase life chances of individuals (most often those from working class backgrounds) failed by the traditional education system.
Most academic educators have opposed these reforms on the grounds that:
• they give too much weight to industry needs at the expense of the needs of students-that the reforms are too economically driven and are overly instrumental;
• they attempt to control teachers'/ trainers' work through national curriculum and assessment;
• the outcomes of education should be far wider than vocational outcomes and must include values and orientations related to justice, aesthetics, democracy life skills and so on.
They also claim that these reforms are inimical to the interests of students. Any vocational emphasis in education, it is argued, is technocratic, specific, practical, managerial. It supports a view of the social function of education; that it is concerned with the transmission of exploitable knowledge and participation in the market through the development of skills which possess an exchange value (Feinberg 1983).
10
General academic education on the other hand is democratic, egalitarian, critical, collaborative (Wellington, 1993).
Interestingly, similar arguments are used by at least some conservative politicians. They claim that such developments are merely 'trendy', that they deemphasise the traditional "canon" and thus lead to a reduction in educational rigour. In the United Kingdom this has led to demands from some strands of the conservative heirachy to strengthen academic education (the so called "Gold standard") at the expense of education designed to appeal to a wider cohort of students.
However, it can be argued that these criticisms of the education reforms in schools (which include criticisms of the role of key competencies in general education settings) are ill founded. They rest on a view of vocational education which is both elitist and conceptually impoverished as is pointed out in publication (j). Furthermore they ignore the potential of key competencies to liberalise vocational education as well as to liberalise, even transform, the curriculum and pedagogy of general education. Incorporating the key competencies into vocational education and training will ensure that VET becomes more liberal, supporting Dewey's concept of education through rather than specifically for occupations.
In addition, incorporating the key competencies into general education may provide the impetus for the reform of traditional subjects and traditional modes of teaching (see Lohrey 1995). A key competency such as 'problem solving', for example, can be developed through traditional academic subjects but lends itself better to holistic, interdisciplinary knowledge and its application and the use, for example, of problem based teaching methods.
Taken as a whole these reports, which cover all the educational sectors in Australia, mark a significant change in educational policy. For the first time, education has been openly debated in instrumental terms and is being judged, at least partly, by the extent to which it is succeeding in advancing "national "goals related to economic restructuring and competitiveness.
The relationship between the competency debate and wider policy issues While there are always a large number of people in various roles and at many levels involved in the development and implementation of educational policy,
most of them are influenced by the dominant ideas of the day. In the last decade in Australia and elsewhere, these have been dominated by economics as Pusey (1991),
11
Marginson (1993), Harris et al (1995) and many others have pointed out. In their recent book Harris et al (1995) suggest "economic factors are increasingly becoming the rationale for educational policy decisions and the means of measuring their success" (p 11). Yet as Walker (1993) has argued, it is not just the economic significance of education which has been the driving force behind educational policy, but a more general cultural anxiety which has focused on the education system as both the cause of and the possible solution to the national uncertainty. The result is that over the last decade, governments have become much more directly involved in educational decision-making than in the past. Rather than providing general directions for education, they have sought to influence the actual content of the curriculum and also the assessment of it. Some of these issues are taken up in publication (j) which examines the development of vocational education in schools in Australia and analyses the arguments for and against its introduction.
While it is no doubt true that the nexus between educational policy and the economy is being more openly expressed in many countries than at any time in the last 50 years, the link itself is not new. Since world war two successive governments in most developed countries have justified educational spending and the enormous expansion of provision largely on the basis of its economic returns- both for the individual and society as a whole. This justification became known, in the 1960s, as human capital theory. As the critics of this approach to education have long argued (e.g. Blaug 1972) there is little evidence that supports a causal link between educational provision on a macro level and economic growth and productivity. However, in the three decades after 1945 the very fact of economic growth was enough to ensure that a detailed examination of the rationale for education spending was not undertaken. The relative cultural stability of this period also helps explains the lack of desire to examine the funding and directions of the education system. In Australia in the two decades after the war, there was a far greater confidence in the directions in which the country was moving and in the nature of its cultural identity than there has been over recent decades. In retrospect, it easy to characterise this confidence as complacency and to discern the beginnings of cultural anxiety as the Protestantanglo- saxon hegemony came under pressure and the structural weaknesses of the economy were exposed. This uncertainty has certainly intensified during the globalization of the 1980s and 1990s and is at the root of much of the educational policy of the last decade. It is certainly at the heart of the currently perceived need
12
to develop skills formation policies and to enlist the education system at all levels in this process. But it also helps explain the desire of a variety of groups to influence and even determine the content of educational provision. This fact is at the core of the differences between traditional human capital theory and the new skills formation theories currently championed by the OECD and adopted by most developed countries. While the traditional theory argued the need to build education systems and improve educational attainment in a macro sense without much consideration for the content of education, the new skills formation policies have concentrated on the balance between the various sectors and on the content of education within them. There have been increasing calls by industry leaders, governments and unionists, for education that is 'relevant' to industry's 'needs 'and which meets the needs of the consumer of education rather than its suppliers. (See for example the recent Allen report into VET [1994] the HEC's paper on educational quality and the Finn report into school education). While these calls for educational 'relevance' are not new, the determination of groups outside the education profession to actually intervene and play a substantial role in influencing the content of curriculum has been a recent development in Australia.
Competency based education provides a framework for those anxious about the directions of the society to intervene in a direct way in the content of education. Clearly it was (and is) the expectation of some of the proponents of a competency approach that a process of defining the tasks that need to be performed in industry (and the "key" competencies needed for effective working life) would lead to an education system that was relevant to the needs of industry.
It is important to point out, however, that the proponents of competency based education come not only from amongst industry leaders and conservative governments, but also from many educational reformers, and union leaders. While this group would obviously pursue different ends from conservatives, they too see the potential of competency-based education to address their own anxieties about the directions of contemporary society. These points are discussed in more detail below. Nevertheless, it is true that concurrent with the recent calls for education to be tied more closely to the needs of the economy, and the introduction of competency based education as a means of realising this aim, there has emerged the development of new theories about the role of the public sector. Boston (1994), has referred to these ideas as "the new contracturalism". The origins of many of these ideas can be traced to neo-classical economics and radical libertarianism. The policy implications of these ideas were regarded by
13
most governments as being too radical to implement in the 1970s and 80s. However, they have evolved over recent years, and are now being expressed in ways that seem to be attractive to both governments and at least some sections of the community. In fact there has been some support for these policies from radicals seeking to make state institutions more accountable.
These ideas provide a rationale for reforming the public sector with the intention of increasing its effectiveness. The policies based on these ideas manifest themselves in the privatisation of State activities, cuts in government programs, modelling the public sector on private enterprise lines, contracting out of services and so on. The intellectual underpinning of these reforms: rational choice theory, agency theory, transaction cost analysis and the new managerialism, have existed as independent (and not particularly influential) theories for some time. What is different today is that the social and economic conditions which have resulted in the need for economic restructuring and the widespread cultural anxiety discussed above, have provided the circumstances within which these ideas can converge. As a whole, they seem to provide not only a powerful critique of traditional public policy theory, but solutions to a range of problems, including many in the area of education.
The essential argument of rational choice theory is that all individuals are dominated by self interest. In the public sector it is argued that this results in the desire to maximise budgets within all levels of the bureaucracy irrespective of the impact this has on the public. Thus, the theory argues that there should be a separation of the policy and delivery arms of government and that, at the very least, there should be a plurality of providers of services which opens public services to competition. More extreme versions of this theory suggest that there should be the complete privatisation of services which would result in greater consumer choice and an increase in the efficiency of services offered to society.
Alongside this plank of the new contracturalism is agency theory. This argues that social and political life should be seen as a series of contracts in which two parties enter into an exchange relationship. An agent undertakes work for a principal for reward. The usual reason for such an agreement will be to increase efficiency (the agent can do it cheaper/more effectively than the principal). However, the principal needs ways of ensuring that the agent performs the task adequately. These include the need to guard against adverse selection and to monitor performance. It is argued that the public sector does both these less efficiently than the private sector, though it is usually conceded that where
14
13
output is difficult to measure there is an argument for continued use of the public sector.
When combined with the new managerialism which dominates public sector thinking, the implications for education of the totality of these theories is substantial. There is a certain logic in separating the creators of education policy from those who provide it, in sanctioning a range of educational providers, and, in an extreme application of the theory, of actually contracting or completely privatising all educational delivery. Competency-based education, which uses industry standards as the basis for curriculum, potentially takes the monopoly of the curriculum away from education authorities and lends itself to the possibility of a range of providers reaching agreements with a range of students about how and where they can develop the skills which relate to these standards. This is not to suggest that the competency movement is merely a part of the new contractualism. It has an autonomous existence related to the need for increased competitiveness. However the conjunction of these agendas has produced propitious circumstances for the implementation of competency based education. The combination of concern about the content of education and the new thinking about the public sector may explain the attraction of competency-based education for radical conservatives in a range of countries.
At the same time, however, there has been considerable support for competency --based education from progressive educational reformers. Some suggest that it provides a counter to the dominance of educational curricula by disciplinary boundaries (see eg Lohrey 1995). Others see it as a way of bringing together general and vocational education (eg Barker 1995), and of reducing the gap between practice and theory in occupational curricula, (Hodgkinson 1992). Yet others see it as a way of breaking down the dominance of institutional learning (and credentialling) and of democratising education. Even those who oppose some aspects of the competency-based approach to education acknowledge its progressive elements. An interesting article by Magnusson and Osborne (1990) for example, presents what they call a 'deconstructionist' analysis of competency based education. They argue, amongst other things, that the recent rise in competency-based approaches is a result of the populist demand for education reform resulting from the perceived failure of humanistic education of the 1960s and 70s to transform society. The failure has led both left and right wing critics to suggest educational reforms. They conclude that while competency-based education tends to reinforce the status quo, (what goes into these standards is currently determined largely by employers who thus are assured of a supply of
15
individuals who are trained in their philosophies) there is the possibility of removing the control of what counts as knowledge from dominant groups and placing it 'in the hands of the people'. Such possibilities will come to fruition in better economic circumstances, they suggest.
Others, like Barker (1995), stress the progressive elements of competency approaches and suggest that they are a way of tackling serious educational problems which have been discussed for decades by progressives, but for which they have offered few solutions. In a clear delineation of the social and political factors which led to the standards and qualifications framework of the New Zealand Qualifications Authority, he points out that the value ascribed to various qualifications by society has largely been a result of factors to do with social and class divisions rather than the intrinsic value of the qualifications themselves. Effectively those qualifications which stressed the abstract and general have been given higher status than those which stressed the practical. This general /vocational dichotomy he suggests have led to:
a binary division of qualifications, binary sectors of learning, binary division of social and financial rewards. This polarity has created extremes which haunt our social, economic and educational structures. We need whole citizens, integrated knowledge and skills but we have a fundamental division rooted in our thinking and behaviour which prevents this (p. 19)
He suggests that students employers and governments want and need a system which is based on outcomes, which abandons the "dubious morality" of norm referenced assessment and replaces it with performance assessment based on the holistic notion of integrating skills and knowledge. Similar points are made in publication (j), where the arguments for vocational education in schools are examined.
In summary the competency movement in general and the key competency movement in particular, exists within a complex policy environment. While their opponents would claim that they are a manifestation of a conservative political agenda, their proponents see in them a potential weapon to challenge such an agenda. The competency debate has made the question of purposes and outcomes of education explicit and therefore open to political debate in ways that have not occurred previously. The debate about competency-based education is also a case study of the complexity of public policy in the contemporary world. The fact that there is support for the competency agenda from groups which would usually be in conflict, is a particular instance of the more general
16
disintegration of conventional political and social alliances in developed countries.
It is hardly surprising, given the politically charged atmosphere in which it was first advocated, that there has been little real analysis of the nature of such education and the concepts on which it is based.
The theoretical context of competency-based education
The concept of competence
Competence is a difficult concept which can be explained and interpreted in a variety of ways. As Stevenson (1995) has argued, 'constructions' of competence alter in different contexts. The meaning given to competence in everyday life, in VET settings and in other academic settings, are quite different. What is more, they are likely to change over time within each of these contexts and also to alter according to different value positions within them. That is, competency-based standards and the education based on them, have a normative component.
It is in explaining the nature of competency that the publications which are included in this thesis have made their major contribution. The conceptualisation advanced in these publications has had an effect on the metamorphosis of the concept in VET and in professional education. The current understanding of the concept in VET has changed considerably over the last six years and is much closer today to the ideas contained in these publications than it was in 1990. In the university sector, in those instances where competency approaches have been used as the basis of curriculum and assessment they are based on the integrated model outlined in the publications.
The competency of individuals derives from their possessing a set of attributes (such as knowledge, values, skills and attitudes) which they use in various combinations to undertake occupational tasks. Thus, the definition of a competent person is one who possesses the attributes necessary for job performance to the appropriate standard. Some tasks in some contexts will be quite specific and will require simple specific combinations of attributes. In other contexts, similar tasks will require more complex combinations of attributes because they have, say, to be completed more quickly or in more difficult circumstances. All occupations contain fairly general tasks eg planning an activity, which will require different combinations of attributes. This
17
conceptualisation of competence has become know as the integrated approach to competency and is discussed in more detail in publications (a) and (b) and (e).
What is suggested in the publications is that the nature of the concept is relational-linking attributes and tasks, within one conceptual framework. In doing this, this concept of competency goes beyond traditional conceptualisations which concentrate only on the tasks that need to be performed or on the generic attributes or capacities that are said to underpin competency, irrespective of the contexts in which these need to be applied. These two traditional approaches, the behaviourist and the generic are discussed below.
Another way to think about the integrated approach to competency is in terms of the distinction between holism and reductionism. Competency standards and the education based on them are holistic in the sense that they bring together a multitude of factors in explaining and developing successful occupational performance. There are a number of ways in which the integrated approach to competency can be said to be holistic: First there is the point made above about the need to link (combinations of) attributes with tasks. Second is the way we think about tasks themselves. The usual way of doing this is within a behaviourist framework. Here the approach is to break down (or reduce) tasks into their sub-components in the hope of arriving as their essence or at 'complete transparency' as Wolf (1993) has described it. The holistic approach to competency argues the need to think about tasks at a more general level, as for example when a basic education teacher: " modifies student's programs as a result of continual monitoring" [see publication(h)] or when a family lawyer "obtains relevant information from sources other than the client" [see publication(g)]. Third is the need to see that tasks, even when thought of in this more general way, are rarely isolated from other tasks in the real world of occupations. So that in the case of lawyers, while they are gathering facts from clients, they are at the same time developing a relationship with them. Similarly, in the case of teachers, while they are communicating effectively with students in the classroom, they will also often be managing time and monitoring learning. The nature of this holism is detailed in publication (e) and discussed later in this essay in relation to the attacks on the competency movement.
Another important component in the conceptualisation of competence is their normative nature. As practitioners engage in work they increase their understanding of the culture of their occupation and of their workplace. As their participation moves from being peripheral to becoming more central, they are
18
increasingly capable of meshing this cultural understanding with their technical knowledge and their skills and attitudes. This combination of attributes enables them to make increasingly informed individual judgements about how they should act in the situations in which they find themselves. Such judgements clearly have a normative aspect. They are answering the question 'how ought I to act in this situation'? Effectively, practitioners are clarifying the nature of competency in their occupations every time such decisions are made. Thus, rather than being a predescribed and predetermined set of behaviours, competency will be an ever evolving reality which allows for critique and improvement of currently accepted ways of acting. This highlights the distinction between the potential richness of the notion of 'competent performance' and the tendency towards behaviourist reading of 'competence as performance'.
Individual judgements, of course, will be guided, at a general level, by the set of competency standards which has been developed for the occupation. These standards represent the best attempts of a representative group of stakeholders to state what are the attributes needed to perform the major tasks in the occupation at a particular point in time. They are, however, necessarily quite general and do not claim to exhaust the possible contexts in which these attributes will be employed. What actually constitutes competence in an occupation will be constantly evolving as new contexts are encountered and dealt with.
The integrated approach to competency incorporates all the elements discussed above and overcomes all the objections to the competency movement that have been identified in the literature. This approach to competency incorporates ethics and values, the need for practitioners to engage in reflective practice and to come to terms with context/ culture. Finally it allows for the fact there will almost always be more than one way of practising competently.
While his approach is different, Carr (1993) reaches similar conclusions. He makes the distinction between competency as conceived of in the sense of it being a "capacity" and in the sense of it being a "disposition". In the first sense, he is referring to a more holistic view of competence, of a person being, say, a good lawyer. In the second, he refers to particular abilities which are used in particular contexts (a more atomistic view) as, for example, when lawyer x was able to communicate competently with his client. The capacity sense of competence entails "the use of judgement in the light of rational knowledge and understanding" (p.257). On the other hand, dispositions are inherent abilities which (with training) enable a person to perform effectively. The former is about
19
rational action based on principle, the latter about routine and effectiveness. Both in Carr's view, are important to a notion of competence and become interwoven in our every day discussions of the concept. However, he makes the point that the starting point from which an overall concept of competence can be derived is the capacity sense of competence not the dispositional sense. Moving from dispositions as the starting point with the hope of building up a complete picture requires a reductive behaviourist approach which is a caricature of competence.
Carr's definition of capacity is at odds with the generally accepted definition which is usually cast in terms of the capability to learn and understand. In fact this dictionary definition is more akin to his idea of a disposition. But so far as the argument here is concerned, Carr's idea of capacity, involving as it does the ability to act, based on knowledge and principle, is very like the integrated conception of competence developed in the publications. It is concerned with the ability to make judgments and to act intelligently and ethically in a variety of situations. This is exactly the concept of competence has been argued for in publications (a) and (b) and (e).
Publication (a) and (e) deal with the conceptualisation of competence from a broadly (though not explicitly) philosophical perspective. Publication (g) examines the various construction from a more overtly psychological perspective, drawing on recent studies in cognitive psychology in particular. These studies add weight to the claims that there is a need for a holistic approach to competency and that contextual /cultural awareness is a vital component of the construct. This perspective is developed in publication (j) and in a conference paper, The False War: Competency based Education and its Critics [Gonczi and Tennant (1994)]. This examines the implications of a competency approach to curriculum and teaching and is dealt with later in this essay (see p. 34-36).
Research into learning in the workplace, though still in its infancy, suggests that workers/ trainees can learn general competencies without explicit instruction. Gonczi and Tennant (1994) examine some of the key research, especially the work of Scribner (1984) in her study of informal learning in a dairy. This work demonstrates that workers use the environment in which they work to develop skills which help them in their work. That is, skilled practical thinking draws aspects of the given environment, be they people, things or information, into the problem solving system. The physical environment does not however, of itself, determine the problem-solving process, but rather it is drawn into the process through worker initiative.
20
Counting routines, for example, were found to be precisely adapted to the shape of things to be counted: stacks of crates five high prompted counting by fives; six high, counting by sixes. Further, experienced drivers modified their arithmetic operations and problem formulation depending on whether they had on hand pocket calculators or pencil and paper. Workers even made mental "tools" of the environment - a stack of cases becomes a counting unit, for example.
Workers also attempt to adopt the most parsimonious solution to work problems. Scribner sees effort saving as a higher-order thinking strategy, which entails the psychological reorganisation of work tasks to reduce or simplify the number of physical or mental steps required for the completion of the assigned tasks.
In the workplace, least effort strategies were acknowledged as "cultural norms". Individuals often explicitly described their active search for short cuts or easier ways to do a job. Product assemblers reformulated orders to save physical moves; inventory staff constructed mental representations that enabled substitution of short-cutting arithmetic procedures for lengthy processes of enumeration:
the hallmark of expert problem-solving lay in the fact that the experienced worker was able to use specific [workplace] and job related knowledge to generate flexible and economical solution procedures. Expert problem solving procedures were content-infused, not content free. (Scribner 1984a:39)
Scribner also found that workers were adept at problem formation. By problem formation, Scribner refers to the process of formulating not simply solving problems. Formal problem solving models, she suggests, see problems as "given", the intellectual work consisting of selecting and executing a series of steps which will lead to a solution (Scribner 1985). In contrast, practical thinking actually reformulates or redefines problems. Problems were recast in the workplace: unit price problems into case price problems; 'take away' problems to 'add on' problems; inventory men squared off irregular areas to transform counting problems into multiplication problems.
The capacity to undertake these practical problem solving strategies (effectively learning strategies) is also related to the time on the job. While experience is not a guarantee of success in developing competencies it does seem to be necessary.
21
Thus the question of how competencies can be developed needs also to take into account the question of experience. Since most work-sites entail working with or at least alongside others, is it also concerned with how novice workers (or trainees) learn to interact in increasingly knowledgeable ways with people who do something competently or expertly (Lave 1988).
This conceptualisation of learning-as the development of practical thinking in a specific context-has implications for strategies to develop competencies in industry and professional settings. It challenges the tendency to mirror, uncritically, traditional formal learning in classrooms. The best way to develop competencies may be to provide situations in which learners experience real problems and in which their practical thinking can be tested against the thinking of other more expert thinkers.
The mainstream literature on the nature of expertise provides an interesting conceptualisation of what competence means in terms of cognitive structures. This literature has been widely summarised [see for example, Chi (1988), Stevenson(1995)] so there is no need to repeat it here in detail. However, what it suggests is that the capacity to act expertly depends on having an organised base of knowledge (a structure) which can be accessed quickly and which enables the expert to anticipate problems and devise strategies to overcome them. It is quite possible to equate expertise with competence as Glaser has done (Stevenson, 1995). If this is accepted, what we gather from the literature is that competence depends on a rich knowledge base which allows for metacognition, problem solving and adaptability amongst other things. Thus greater weight (and value) seems to be being given to propositional knowledge representing various schemata and able to be expressed in symbolic form, than to procedural knowledge. As Stevenson (1995) points out, however, psychologically based conceptions of expertise have changed considerably over the last 15 or so years and these conceptions reflect different values, despite the assumption of psychology being a value free science. When expertise was conceptualised as "automatic fluid action" (p.12) procedural knowledge was accorded higher value. Partly as a result of this recognition of the values implicit in the mainstream research, other aspects of how knowledge is acquired such as the role of context, emotions, culture, are also on the research agenda.
Recent research in the area of cognitive psychology [see for example Brown et al (1989), Lave (1988), Prawat (1993) Raizen (1994)] in addition to the work of Sternberg (1985) on practical intelligence and Scribner (1984) on the development
22
of expertise through learning in the workplace, gives support to some of the ideas of competency based education. Essentially this work suggests that much learning is "situated," that is it occurs within a specific context/ culture. The implication of this for those concerned to develop competent practitioners is that there is a need to mesh subject knowledge with the demands of practice. This process incorporates procedural and propositional knowledge along with what (Sternberg, 1990) refers to as "meta control". This research provides a theoretical basis for the emphasis in competency-based education on learning in the workplace.
In summary, recent scholarship in learning theory is increasingly undermining the generally accepted view that there is a separation between knowing and doing. This axiom had been the basis of the teaching of abstract, decontextualised concepts in secondary education curriculum for a very long time. It has also been the basis of the separation of theory and practice in vocational curricula. That is, it has been almost universally accepted that in order to be able to understand, students need to be taught abstract concepts before and distinct from the context in which these concepts might be applied. However evidence has been building up which suggests that learning and cognition are fundamentally contextual or 'situated'. That is, understanding develops through students engaging in the social and physical context, that performance and understanding were one and the same thing. Nevertheless there is little hope in the short term for a comprehensive theory of knowledge acquisition and its relationship to action.
While the cognitive psychology literature as a whole does not provide a comprehensive theory about the nature of competence or its acquisition, there is a good deal of the more recent literature which could be seen to cohere with the integrated notion of competence outlined in the publications and this essay. The capacity to bring together knowledge, values, attitudes and skills in the actual practice of an occupation is the kernel of the integrated concept. As much of the recent research in cognitive psychology suggests, an understanding of the culture of the workplace and of the occupation are an important ingredient in how these attributes are able to brought together. Experience seems to be a necessary though not sufficient part of the process of becoming competent, a process which also seems to incorporate an affective dimension. Clearly there is a good deal of research to be done, but there is at least the promise of a concept of competence and its acquisition which is able to unite philosophical and psychological
23
approaches and clarify both the relationship between thought and action and the normative issues involved. Key Competencies and their conceptualization
The policy context in which the Key Competencies have been developed in Australia (and internationally) has been discussed above (p. 5-14). This section of the essay concentrates on conceptual issues related to the nature of the key competencies and their significance in occupational education and argues the need to conceptualise key competencies within the integrated competency framework outlined above. The reason for the discussion being here rather than in the publications themselves is that much of the research was undertaken after the publications had been nominated for examination.
A major question which needs to be asked about the key competencies is how coherent was the thinking of the Mayer committee. The committee made certain assumptions in coming up with the key competencies: that there were such things as generic capacities which could be transferred to a variety of contexts; that it was possible to classify them into various levels; that they were discrete, distinct competencies in their own right (though they did acknowledge the likelihood of their coming together); that they were of the same order of complexity; that it was possible to develop them in students and trainees; and that they could be assessed in ways that were nationally consistent and reliable and which facilitated students undertaking the transition to work or to higher education.
As Young (1995) has argued, this was bold and probably necessary to make a break with the past, but it was not based on careful consideration of evidence about the validity of the assumptions. In fact the assumptions of transferability, foundational knowledge and the notion of generic capacities are all contradicted by most of the evidence, as has been pointed out above. Equally the assumptions about their discrete nature is contradicted by new evidence outlined below (Gonczi et al, 1995 and Hager et a1,1996).
A better way to think about the key competencies is in terms of the integrated conception of competence outlined above and dealt with in more detail in the various publications. The key competencies should be seen as being relational. They bring together attributes possessed by individuals, and the contexts in which, through performance, these attributes are demonstrated. They are at a higher level of complexity than other more simple competencies, i.e. they bring
24
together more attributes, [see Publication (a)] but they do not exist without a context. So 'Problem Solving' or 'Collecting Analysing and Organising Information', for example, should not be thought of as discrete competencies to other competencies or as somehow underpinning other competencies. There is no such thing as the generic competency of problem solving - only individuals bringing together the appropriate attributes in a particular context to solve the specific problem that confronts them. Thus the key competencies are really no more than "complex" competencies as defined in the integrated model of competence. They will almost always be employed in combination with other simple competencies where single attributes (such as recalling some aspect of knowledge) are necessary but not sufficient to complete a task. In effect the key competencies will never stand alone.
There are likely to be some similarities between the combinations of attributes required to solve a problem in similar contexts. So that in solving problems in social work with troubled juveniles, for example, different social workers will use many of the same combinations of attributes. Even in this restricted arena, however, every problem will be unique and different combinations of attributes will often be needed to solve what seem, ostensibly, to be similar problems. But, in social work with the aged, substantially different combinations of attributes will be needed to solve problems, though it is possible that some will still be common to social work with juveniles. The attributes needed, however, will have to be rethought and recontextualised rather then simply transferred.
It is also dangerous to think of the key competencies as discrete stand-alone competencies. In real-world situations many tasks are complex. The more complex the work (i.e. the tasks that have to be performed) the more combinations of the key competencies are required to perform successfully. The assumption that these key competencies are discrete and that they can be divided into levels is another instance of the tendency for atomistic or reductionist thinking discussed earlier in this essay.
The key competencies represent, at a high level of abstraction, the most usual processes needed for successful functioning in particular contexts. Another danger, however, is to imagine that the processes identified by Mayer exhaust the possibilities. It is likely that other combinations of attributes that are required for successful performance will be developed in the future, eg aesthetic competence or design competence.
25
Despite the fragility of some of the assumptions of the Mayer committee, the attempt to link knowledge, skills, attitudes and values to performance of various kinds is a useful synthesis. It challenges schoolteachers to reconsider the desirability of curricula which emphasise knowledge to the exclusion of its application. It challenges TAFE teachers and industry trainers to think about ways of developing general capacities in what have typically been specific, narrow curricula. The current attempts in universities in Australia to develop graduate profiles (effectively "key competencies" required by graduates) have the potential to pose similar challenges to university academics.
There are currently a large number of pilot projects throughout Australia experimenting with the implementation of these key competencies in schools and VET settings. There are also a number of projects which are summarising the results of them. Hence, it is too early to suggest whether the aims of the Finn and Mayer committees are being realised. However, some recent work carried out in the VET sector by Gonczi et al 1995 (see Appendix c) and Hager et al 1996, has produced some interesting results which provide empirical support for the utility of the integrated notion of competence. It appears from this research that work contexts both integrate specific skills and key competencies and integrate clusters of the key competencies themselves. There were many cases of this occurring in both the observation of work trainees undertaken in the first study and the trialing of approaches designed to develop key competencies in trainees of the second research project. For example, in the first report it was observed that the setting up of a banquet in a hotel required the trainees to use a range of key competencies: teamwork, communications, collecting and analysing information, problem solving, using mathematical techniques. In the second report the critical incident scenarios developed in Hairdressing for example all centred on some significant workplace incident in which a competent response integrates both a range of specific skills and various key competencies.
This work challenges the view that key competencies are discrete competencies that exist independent of context and which can be described and taught in isolation. As these research projects found, specific skills are deployed in contexts which typically changes somewhat from case to case. The requirement that skilled work take into account changing context is usually enough to bring the key competencies into play. There is the suggestion too that the capacity to use the key competencies and adapt them to new contexts rests as much on affective as on cognitive capacitie
26
Dispositions such as daring and confidence seem to be important in recontextualising the key competencies.
The results of these projects support the premises underpinning the integrated model of competence. The implications for training in industry settings suggest the need to train in whole job functions rather than in specific atomistic skills. There will be a need, too, to ensure that trainers and trainees are aware of the contexts in which these competencies are being employed. For trainers there is a need to identify workplace jobs which are whole enough for the key competencies to emerge. Setting up a banquet, dealing with a customer complaint, ordering some complex equipment from suppliers etc, all have the potential to incorporate clusters of key competencies. For the trainees the awareness of their own use of the key competencies (through reflection on and in practice) can be a powerful tool in helping them to recontextualise their use of these more general competencies. The capacity to do this appears to occur through the growth of confidence ie it is affective as much as cognitive. The implication is that care needs to be taken by trainers to be sensitive to the need to develop affective as well as cognitive attributes in trainees and that this could be the key to flexibility in the workplace and, more importantly, the basis of lifelong learning. Clearly more research needs to be undertaken into the role of the affective domain in learning and acquiring competence.
Arguments against the competency approach to education
There are a number of arguments against competency based approaches to education which have emerged in the literature over the last few years. These can be classified into four categories, though they are not discrete:
1. Arguments against behaviourist approaches to CBE. Such arguments are based on the assumption that this is the only approach to competency based education.
2. A more sophisticated version of this anti-behaviourist argument, is the claim that all competency approaches are behaviourist. The proponents of these arguments, unlike those in the first category, are aware that there are other approaches, but they claim that any attempt to set predetermined standards or outcomes for education cannot avoid being behaviourist.
3. Arguments against the generic approach to competence. The nature of the generic approach is outlined in publication (a) and (b).
4. Arguments against any competency approach on the basis of its normative assumptions about the nature of "the good". These arguments are often also
27
about the aims of vocational education -the extent to which it should meet the needs of industry/ professions (as opposed to the individual).
Publications (a) (b) and (e) support the first and third of these categories of argument against competency approaches. They suggests that most objections to competency based approaches are an attack on a particular construction of competence. They make the further point that many of the objections to the narrow behaviourist approaches to competency-based education are justified as are the warnings that the competency agenda, in the wrong hands, could see the development of a kind of educational Taylorism.
The publications argue also against the generic approach to competency, the third of the arguments above. The psychological literature which suggests that there is no evidence for claims of generic competencies is reviewed briefly in publications (g) and (j). Essentially this literature, and also that in the area of critical thinking, demonstrates that people cannot transfer expertise across domains of knowledge. Put another way, it demonstrates empirically that the so called higher order competencies (e.g. problem solving) cannot be transferred from one context to another, without at least some relearning. Though these points are touched on, the arguments against the generic approach to competency are not fully developed in the publications and there are a number of additional points which need to be raised in this essay.
Barrow (1991) makes the point that much of our thinking about general competencies springs from a view of psychology, popular in the late nineteenth century, that the mind consisted of various specific faculties (such as memory imagination etc) which could be treated as self contained capacities and developed by teaching and training. While recent research on the brain discredits such assumptions (for example, work which demonstrates that different parts of the brain can take over functions damaged through trauma), Barrow argues that such thinking still dominates our thinking, as in when we say that X is a creative person or Y is a good communicator. The problem with this reasoning is the assumption that these are self contained abilities which can somehow be developed as a single simple thing. In fact, there may be persons who are creative in many situations but if this is the case, it will be because the person possesses a range of characteristics some of them values, some abilities, some dispositions and so on. It is unlikely that the combinations of these things would enable a person to be "creative" or "intelligent" in all contexts.
28
Barrow gives the particular example of critical thinking, and points out that it is not possible to be critical without a context - one has to be critical about something. Nor is it a matter of simply adding knowledge of a field to a general capacity to think critically. "The formal generic capacity has to be instantiated in a particular form... to be logical in discussion of art is not a matter of combining logical ability with information about art. It is a matter of understanding the logic of art, of being on the inside of aesthetic concepts and aesthetic theory." (p.12)
Carr (1993) too makes the point that in philosophy there has always been suspicion of the nature of generic abilities. He refers to his distinction between the "capacity" view of competence and the "dispositional" view outlined above, and suggests that the "generic" is often assumed to be a disposition (i.e. an ability) whereas in fact it should be thought of in terms of a capacity, i.e. the practical dimension or expression of sophisticated knowledge and understanding, which, by definition, occurs in specific contexts and consists of the working through of the normative dimensions of the particular situation.
The second and fourth arguments against competency approaches are a direct challenge to the thesis presented in this essay and the publications. Both, however, can be refuted.
The argument that all approaches are behaviourist is addressed indirectly in the publications but is more comprehensively refuted in a recent paper by Hager and Beckett (1995). They identify numerous criticisms of the integrated conception of competence. What they call the "closet behaviourist" argument, the "utopian preconditions fallacy" and the "technology of testing problem", are the most substantial. The first of these criticisms suggests, in essence, that since concepts of competence are demonstrated by performance against a predetermined standard and all performance is limited to the demonstration of behaviours, all concepts of competence are behaviourist.
The fallacy of this view is the assumption that all evidence gained from performance is behaviourist. It is in fact possible to infer knowledge, values and attitudes from performance if samples are carefully chosen. Further, the inferences made about such values, knowledge and attitudes, are no different from any other inferences about knowledge, scholarship, etc in an academic test or practical class in a formal institution. Almost all assessment uses the specific to make inferences about the general. There is also the essential point (made above when explaining the third element of the integrated approach) that an
29
understanding of culture and context is essential to competence and that coming to terms with this means that predetermined standards are no more than a guide to what Walker (1993) has called "situational understanding".
The utopian preconditions fallacy is a common mistake of reasoning where conditions are set up which cannot be met but which are different to the conditions claimed by the proponents (of a policy or concept etc). In this case the claim, for example, that competency assessment cannot anticipate all situations, ignores the fact than nobody would claim this and that in any case no other method of assessment could accomplish this either.
This issue and assessment issues generally are taken up in a number of the publications on assessment [see (c) (f) (g) (h)] and in the later section on assessment in this essay.
The fourth of the arguments against competency approaches has been advanced by writers such as Stevenson (1994, 1995) Field (1991), Magnusson and Osborne (1990), Newman (1994), Feinberg (1984) and Wellington (1993). While their arguments differ in some ways, they are all critical of the process which has led, in their view, to the aims of education being circumscribed by, or even becoming identical to, the needs of industry. They variously attack the role of employers in developing competency standards (that is, of deciding what is valuable), the role competency-based education plays in shoring up the selection function of education, and the role competency-based education plays in limiting teachers' opportunities to empower, emancipate and broaden individuals. However, what becomes dear when their arguments are examined closely is that they are not so much arguments against competency-based education in particular, as against any curriculum which does not have as its basic aim the desire to broaden and empower. These arguments are largely an ideological objection to the legitimacy of economic and industrial considerations in any curriculum.
One example of this argument has already been discussed in this essay in relation to the development of the key competencies and the implications for schools (p.9-10). There are, however, some other wider arguments against the competency approach in general which need to be examined.
As Stevenson (1995) argues "[ under a competency-based system] a preemptive good is assigned to education, viz the achievement of more efficient industrial practice .... What is problematic is the nature of the norms that are adopted ... the
30
ethical questions about what is good for the community has been preempted." (p.14)
On another occasion Stevenson (quoted in Quirk 1994) suggests what this good might be: "There is no room [in the current vocational curriculum based on competencies] to overcome meaninglessness barbarism or oppression... no room for such goals as wholeness ... ability to improve and transform society ... no consideration of such values as acceptance of others reason and freedom" (p.64)
The suggestion that vocational curriculum should deal with such issues is outlined in another article (Stevenson 1994) which cites inter alia work on Australian "desirable futures" concerned with moral responsibility, caring, provision of supportive networks and so on.
A belief in the desirability of a moral, caring society in which oppression and barbarism are absent need not, however, lead to the conclusion that VET curricula, or curriculum in professional faculties in universities, should deal with such issues. In fact, few of these aims are dealt with in general education in schools or universities except in a few subjects or courses. Is the suggestion that medical degrees, say, should seek to overcome oppression, or that mathematics courses in schools should seek to overcome barbarism or meaninglessness? Wherever such degrees and courses have attempted to introduce general studies subjects which have such aims they have been notoriously unsuccessful. More successful have been the integration of courses in ecology for engineers or sociology of medicine for doctors, which have more limited aims such as developing an understanding of the impact of their occupation on society. Competency-based courses in TAFE or universities have the potential to contribute to these more limited but important moral issues. Consideration of the need to accept others, the impact of an individual's occupational decisions on the environment, the ability to improve society through the carrying out of one's occupation, are all compatible with, indeed they are a component of competent practice in most occupations and increasingly (though not universally) recognised by standards developers to be so. But it is going too far to suggest that the need to tackle barbarism, oppression and meaninglessness is appropriate to such courses.
In summary, it is not necessary to draw a dichotomy between the needs of industry and the needs of the individual. While it is clear that some subjects in
31
some educational courses will concentrate on the broadening of individuals, it is not necessarily the case that a subject or course which has other ends cannot be a broadening experience. It is certainly not the case that, by its nature, a competency-based approach will prevent a course from meeting needs of both industry and of individuals. There is too a hidden assumption, discussed in publication (j), that preparation for a vocation is somehow illegitimate or certainly less important than general education.
Field's argument that competency based education is a way of shoring up the selection function of education is examined in publication (b). The argument in that publication is that this view is simplistic and ideological and that the United Kingdom experience is too limited a context from which to draw general conclusions about the value of a competency approach.
Newman presents a curious analysis of competency-based approaches in a chapter of his book Defining the Enemy (1994). He scrupulously outlines the desirable qualities of a competency-based system (recognition of skills already possessed by individuals, breaking the monopoly of certain institutions that certify, breaking the regime of time based courses, concentration on general rather than narrow skills) but then he goes on:
But this is the problem. The competency movement has resulted in educators who in other contexts might be concerned with mystery, discovery, emancipation, making lists ...(p.122).... The response of educators and trainers to these calls [ for competency based education] has been to develop a form of analysis that involves the fragmentation of work into jobs then into competencies then into elements of competencies- that is, to draw up lists (p. 123)
There are two related issues raised by this particular analysis. First there is the issue of atomisation versus holism in analysis of an occupation, and second there is the issue of predetermined performance standards in that occupation (versus mystery and discovery).
As is pointed out in publication (e), atomism and holism are relative terms. While there is a clear case for the rejection of a competency approach which fragments an occupation into a myriad tasks, an analysis which can say nothing about the parts that make up the whole will be unsatisfactory. As Hager and Beckett (1995) put it, this is the equivalent of saying that we know what good
32
teaching is but we can't say what it consists of. Clearly there is a need in any analysis of an occupation for some atomisation which can become the basis of a synthesis where the whole becomes more than the sum of its parts.
It is quite compatible with the competency approach advocated in these publications, to provide the opportunity, inter alia for learners to become emancipated and to pursue discovery learning. It is also a hallmark of the integrated competency approach that there is rarely one way of demonstrating competence in any context. Certainly this is the case as contexts become more complex. That is, the approach allows for the development of the unknown and the novel. However there is an assumption that underpins the approach- that individual progress in learning and performance is enhanced by building on what is known (the standards) and that learners who know what is expected of them will be more likely to learn effectively that those who do not.
Implications of the integrated approach for competency standards development
Since the competency standards form the backdrop to curriculum, teaching and assessment issues, it is obviously important to have coherent occupational competency standards which reflect the integrated approach outlined above. Such standards need to integrate attributes with key tasks, to be sufficiently holistic to identify tasks at an appropriate level of generality, and allow for individual judgement and more than one right way of performing occupational tasks.
Two of the publications tackle the issue of how to develop such standards. Publication (a) details the methodologies that can be used for the development of standards as well as explaining the strengths and weaknesses of each of the methodologies. It also suggests the practical problems that need to be considered in research of this kind. Publication (d) Takes the general methodology suggested in (a) and applies it to the case of a branch of the teaching profession - Adult Basic Education teachers. The report: The Adult Basic Profession and competence. Promoting Best Practice, is included in the publications to illustrate the application of this methodology and the actual competency standards which were developed using the range of research methods (see particularly chapters 3, 4, 5). Similar standards have been developed in the Dietetics profession using similar methods. The process and standards are detailed in Ash Gonczi and Hager (1992). (See Appendix A). Effectively, the development of competency standards for an occupation is a piece of
empirical research which needs to fulfil the usual criteria for social science research. This is not the place for an account of research methods in the social sciences. However it can be inferred from publication (d) that these are understood and have
33
been applied in a rigorous fashion. The use of a range of techniques allows for triangulation, a key feature of good social science research.
It also allows for the combination of qualitative and quantitative methods, an emerging paradigm for such research.
The standards which were developed using the methodologies outlined in publication (d) exhibit the features of an integrated sets of competency standards. They take account of the attributes that underpin competent performance and link these with the tasks that the teacher undertakes. Throughout the standards there is reference to a specific knowledge base, how this meshes with dispositions underpinned by values and how these combine in the undertaking of particular tasks. For example, in the first Unit, the performance criteria for the task dealing with 'use of a variety of learning and teaching strategies' (1.4), include the need to understand theories of language and mathematical learning, to actually design activities and strategies based on this knowledge, to respond to students with patience and humour, and to be flexible enough to change strategies if the circumstances dictate.
This same example includes the other features of an integrated set of standards i.e. it is sufficiently general to encompass real world tasks but not so general that the nature of that task is unclear, and it allows for (indeed suggests) the possibility that there is more than one way of undertaking this task. The set of cues or examples includes a range of possible actions that might be used as evidence for competent performance of this task, but these will vary according to the context of the task, the nature of the students and of the workplace. Similar examples are to be found throughout the adult basic education competencies.
Implications of the integrated competency approach for curriculum and teaching
The nature of curriculum and teaching practice based on the integrated conception of competence are examined in some detail in publication (j). This publication deals with future developments in vocational education in schools, but in the course of doing so, it examines the curriculum and teaching in professional education and in VET. It suggests that problem-based curricula and the more recently suggested "ideas based" curricula (Prawat 1993) is an approach
which is compatible with the integrated conception of competence. This approach is also congruent with the psychological literature, both in relation to the recent work on situated learning and the literature on expertise. The implications of the latter
34
literature for curriculum is not dealt with in publication (j) so a few points are included here.
As Gonczi and Tennant (1994) suggest, in addition to developing curriculum which attempts to solve problems in real situations, it may be possible to construct curriculum which is based on expert models of cognition, though only that version of expertise which is compatible with the integrated notion of competence (see earlier p. 22-22). Similar points are made in different ways by Evans and Butler (1992) and Gott (1995).
Should the way experts operate dictate the nature of education and training programs? That is, should we simply mimic the processes underlying expert functioning? Obviously we need to have a notion of what constitutes expertise in a particular domain, but we also need to know something about the development of expertise. At the core of this process is experience, but it is at this point that the literature on expertise reveals some gaps. The question of the acquisition of expertise is not addressed sufficiently in the literature. Instead of continuing to compare 'experts' (considerable experience) with novices (little experience), it may be instructive to compare experts with non-experts, nonexperts being those who do not seem to have profited from extensive experience. In this way we can begin to map the way in which experts, as opposed to nonexperts, utilise their experiences for learning. Thus from the perspective of curriculum, while it is essential to provide some experience (or integrate real work with the off the job component of courses), mere provision of experience is not sufficient.
It is crucial to distinguish between expertise as an outcome and the acquisition of expertise as a process. For example in Chi et al's (1988) summary of the generic qualities of expertise they note that experts are faster and more economical, partly because they do not conduct an extensive search of the data or information available. This does not imply that novices should be warned against conducting extensive searches of the data or urged to take short cuts. Quite the contrary. Extensive searches of the data are presumably important at the novice stage, and in this sense expertise is built upon the experience of being a novice. As educators, however, we are concerned with how experience is used to become 'expert'. Should the curriculum be 'staged' from novice to expert and based upon
an analysis of the levels of competence at each stage? Or should some pedagogical method, like problem-based learning, be employed which mimics the workplace learning process, and therefore contains within it the seeds for the development of
35
competence and expertise? The latter would be the most compatible with the integrated notion of competence but would never be able to replicate certain aspects of the real workplace such as the workplace culture and the affective experience of being a beginning worker. Obviously a curriculum which offered a practical or co-operative program would be highly desirable and probably necessary to produce a truly competent worker. However for a variety of practical reasons this may not be possible. At the least we should be encouraging curriculum developers to experiment with some of these approaches. As Quirk has pointed out, the use of problem based education in universities is now widespread and deserves to be evaluated. As Gonczi in publication (j) has pointed out, in the absence of co-operative programs (and even with them in place) problem-based approaches are appropriate competency approaches as they seek to combine knowledge, skills and attitudes in authentic or near authentic situations.
Any program which is designed to facilitate the development of expertise in a particular domain should take into account the way in which experts in that domain were able to use their experiences for learning. Teachers need to understand the conditions under which experience can lead to expertise and the teaching methods which will facilitate any generic skills.
Evans and Butler (1992) attempt to construct a model of expertise in welding, a complex skills in which, as they put it: "there are motor perceptual and conceptual factors involved" and to tease out the implications for curriculum and teaching. They conclude that the model has significant implications for curriculum and for the teaching of this skill. First, the teaching of the skill usually ignores "task feedback clues", which their model shows are an essential part of expert welding. Second, they claim there is a need to allow in curriculum for overall planning, for mental rehearsal, for monitoring and regulating the process, and observing and interpreting the outcome. All these things are part of their model of expert performance which they take to be synonymous with competence. Of these a number, but particularly the monitoring and regulation, are under-represented in the welding curriculum and in teaching/ training.
Gott (1995) in her work with the US Air Force has attempted to determine the components of skilled performance in real world tasks, to ascertain what cognitive models can to inform training and how these can be put together in curricula to produce skill acquisition. She argues that as skills needed in the workplace become more cognitive, under the influence of advanced technology, they become more difficult to observe and separate into discrete components. Hence the need for cognitive models based on real world expertise rather than the acquisition of individual skills as is typical in much vocational education and training. Such models,
36
she claims, have demonstrated that experts engage in: "adaptive opportunistic reasoning that involves the co-ordination of... procedural, device (or system) and strategic control knowledge" (p. 64). She further argues that skill acquisition requires "successive approximations of targeted expertise". The implications for curricula and teaching are that there is a need for learning which is situated, sequenced and supported. While she goes on to suggest that this can be done through the use of a computer tutor system, it has implications for teaching and learning generally. Simulations, coaching with immediate feedback, and observations with explanation by experts, are methods which suggest themselves.
Other uses of competency standards
Much of the debate about competency-based approaches has concentrated on their implications for educational programs in schools, TAFE and universities. However, as is suggested in publications (a) and (b), competency standards have many uses outside formal education. These include such things as facilitating the accreditation and reaccreditation of professionals (including the recognition of migrants' professional or other occupational competencies) providing the basis for continuing professional education courses provided by professional associations; providing information for the public on a profession's roles and responsibilities; facilitating articulation between paraprofessions and professions; assisting employers to evaluate performance and recruit staff (Hager and Gonczi in Edwards et al, 1996).
The uses of competency standards for such purposes are examined in publications (b) and (i). The former deals with the possibility of using competency standards for both the design of courses and for facilitating choices made by professionals about how to fulfil mandatory continuing professional education (CPE) requirements. The latter publication argues the superiority of a competency-based approach over a qualifications-based approach for the recognition of migrants for registration as professionals in Australia, though it
37
points out that there are serious problems to overcome before such an approach can be implemented. Gonczi Palmer and Hager (1994) also describe how a competency-based approach has been used for the accreditation of legal specialists in NSW (See appendix b). This scheme has now been operating for three years in a variety of areas of legal practice. Some aspects of the assessment used in one of these areas of speciality are outlined in publication (g). Both these publications demonstrate that a competency based approach, even in the absence of an educational program designed to develop the competencies, can be of benefit to the public. In this case it helps the public to identify solicitors appropriate to their needs and enables them to judge the quality of the services they receive. It also has the potential effect of encouraging an improvement in the quality, speed and cost of legal services and of providing practitioners with an incentive to increase their competence in a chosen field of practice. Competency-based assessment Introduction
Competency-based assessment is at the heart of a competency-based approach to education. Jessup (1991), the leading English proponent of competency based education, suggests that not only does a competency based approach to education require new forms of assessment but: "assessment takes on a more significant role, becoming an integral part of the learning process as well as a means of evaluating it" (p. 46).
The notion that assessment affects learning is not new. Neither is the idea that assessment can be used as a tool for classroom learning, as Scriven's (1967) work on formative evaluation demonstrates. However, in the competency-based approach to education which has emerged in the 1990s, assessment reform has been seen as a vehicle with which to address a wide range of educational issues which extend beyond the classroom. For some there is the potential to improve classroom practices by reforming assessment. This includes reducing the power of standardised assessment to determine what is taught, demystifying assessment by providing students with a clear picture of what needs to be learnt, and breaking down the dichotomy between knowing and doing (otherwise known as propositional and procedural knowledge). However, there is also the possibility of reducing the power of educational institutions generally, by recognising skills and knowledge learnt informally.
38
Some conservatives, on the other hand, have seen competency based assessment as a way of ensuring that institutions produce the sorts of students that industry wants. Many industry groups, for example, have argued that to gain middle level qualifications, students should be competent to industry standards and be assessed on the job undertaking real tasks.
If the arguments in favour of the integrated approach to competency are correct, there is a need to tease out the implications of it for assessment. A number of the publications (c) (f) (g) (h) have attempted to do this. They argue for a holistic performance-based approach to assessment which is criterion referenced, which is direct, which relies on multiple sources of evidence and which is judgement based. If this is achieved, it is claimed that competency-based assessment will be more valid than traditional assessments [see(g)]. The publications also point out that in the Australian and British contexts, most of the arguments against competency-based assessment are arguments against one version of it, based on the task-based concept of competence. These arguments are detailed in publication(f).
Contribution of the publications: competency-based assessment and validity The contribution that the publications have made to the nature of competency-based assessment in Australia is clarified by placing them within the more general assessment literature particularly that which has emerged over the past decade and even more specifically over the past three or four years. The genesis of this literature is unrelated to the rise of the competency-based approach to education, but it has much to say of relevance to it.
There are various themes in the general assessment literature that need to be considered: • that to do with criterion referencing which predates the competency movement; • that do with authentic and direct performance assessment in schools which has
emerged largely out of the dissatisfaction with what is perceived to be the deleterious effects of standardised testing on teaching and learning (Mehrens, 1992) and the dubious assumptions of trait theory (see Taylor, 1994);
• that to do with performance assessment in the professions, i.e. assessment which concentrates on performance of individuals in a prospective occupational role in society.
39
Competency-based assessment has been concerned largely with vocational education (including education of the professions in universities) and competency in actual occupational roles. However since 1994 in Australia the interest has been expanded to assessment of "key" competencies which are assumed to be generic, i.e. unrelated to particular occupational roles. This has added complexity to an already difficult assessment debate and has made it even more important to examine the literature on performance assessment in the school sector. While these literatures have emerged in response to different educational problems in different contexts, it seems they are beginning to merge and some theoretical consensus is beginning to develop (e.g. see Messick 1994). What this will mean in practice is still to be seen. However, the emergence of performance-based holistic assessment in the professions and increasingly in VET in Australia, suggests the direction in which assessment will move. This is towards standards/criterion-referenced assessment based on teacher judgement and using multiple sources of evidence [see publication (g)].
As Tuck (1994) and Wolf (1995) have pointed out, criterion referencing is not new. Tuck identifies a 1922 paper by Washburne on criterion referenced assessment before the well known work of Ebel (1972) Glaser (1963) and Popham (1978). Competency-based assessment can be seen to be a subset of criterion referenced assessment, similar in its desire to specify objectives and criteria which are public and to assess against these standards (criteria), but different in that the context is occupational competence rather than achievement of school curriculum objectives. The advantages of criterion referenced assessment are clear. If it is possible to specify learning outcomes which cover the aims of the curriculum, then these could be taught and assessed in unambiguous ways overcoming problems of validity reliability and fairness. The central problem with criterion referencing, of course, is the problem with the task based approach to the conceptualisation of competence: the potential for reductionism and the overemphasis on observed behaviours from which knowledge and understanding of the appropriate constructs cannot necessarily be inferred. Of course this need not be the case as Griffin (1995), using the framework of Rasch's (1980) latent trait theory, points out. He suggests a criterion referenced approach which interprets competency in terms of a progression of tasks or stages. Here, elements of competency and activities link into cohesive groups. As he puts it "These complex procedures ...... are often and naturally assessed in terms of the relative levels defined by ordered performances [thus]... a person's performance and the task are both interpreted by their relative position in a continuum" (p. 42). The problem, as he points out, is that this requires the development of
40
statistically validated progressions or scales of increasing competence. This would be a complex activity requiring extensive psychometric analysis and considerable cost. Neverthless it provides at least the theoretical possibility of marrying measurement and judgemental/evidence-based approaches to competency assessment [see publication (h) which incorporates this possibility].
Criterion referenced and competency-based assessment are almost always contrasted with norm referenced assessment, though a number of writers have pointed out it is not possible to escape from norms, since all assessors have ideas about what is reasonable performance in domains with which they are familiar. Nevertheless, there are differences between the assumptions of a measurement model and a standards based model (i.e. based on some sort of standard such as a competency standard) which lead to quite different assessment practices.
Taylor (1994) suggests that the main assumption of the measurement model is that of trait theory. That is: that humans differ from each other on various traits; that it is possible to measure these differences relative to others (i.e. on national or state norms); and that we can do this reliably. The aim of such a system is to rank individuals, usually using standardised tests and usually a single score obtained through psychometric analysis and manipulation. (The word 'usually' is used because there is some evidence that those who hold these assumptions are turning to performance assessments-this is discussed below.)
There is, of course, value in ranking, for example where selection for places in courses with limited places is required or where prizes need to be awarded to the best performers in an activity. However, when we are interested in a multifaceted concept like occupational competence where we need to judge whether individuals can do things, with understanding, and in a variety of contexts, single measures are not likely to be very useful. This leads to the point that different assessment systems are needed for different purposes, and the less obvious point that one assessment regime cannot fulfil all the purposes of assessment. There is a strong temptation to try to use one assessment for both comparing students (and institutions) on the one hand, and asking whether they achieve complex learning standards on the other. Norm referenced tests have been used for criterion referenced purposes and where this has occurred the evidence suggests this has had unfortunate long term results, such as decline in standards (Taylor, 1994). The danger is that the new criterion referenced tests, which are being championed, will ignore the dangers of using such assessment for both comparing schools/colleges/students and assessing individuals' progress
41
against standards. Because the assumptions underlying each purpose are so different, attempting to use such tests for both purposes could undermine attempts to increase overall standards for a wider number of students.
The assumptions of the standards model are quite different from those of the measurement model. This model claims that it is possible to set public standards; that most students can achieve them; that different performances can reflect the standards; that assessors can internalise the standards and that they can judge different performances consistently. Such assumptions can present difficulties, as discussed below and in publication (f), but they have the advantage of suggesting that there is the need for a multifaceted approach to assessment based on what an individual can do. Furthermore, what an individual needs to do is anchored in publicly defined standards, criteria and targets.
Recently, there has been a sustained attack on the assessment practices associated with the measurement model. The kernel of the criticism has been the damaging effect of standardised testing (particularly multiple choice questionnaires) on teaching and learning. A smaller number of critics have attacked the basic assumptions that underpin the approach. The impact of this criticism can be evidenced by the fact that some measurement experts have sought to develop new performance based assessment methods. However, the problem is that many of these experts appear to have done so without altering their basic assumptions. This is particularly the case with the recent interest in performance assessment and what has been called 'authentic' and 'direct' assessment. The testing "establishment" has given such assessment methods tentative approval but only as adjuncts to traditional testing regimes. In other words they seem to have tried to incorporate them in their approach without altering their own assumptions (e.g. see Mehrens 1992, Messick 1989, 1994).
The case for performance assessment is put most forcibly and clearly by Wiggins (1989), Frederickson and Collins (1989), Linn (1993), and in Australia by Bailey (1993, 1995). They argue that performance assessment designed to test performance standards (or competency standards) can motivate students better than norm referenced tests, that they are able to fulfil validity criteria which will be acceptable to the testing profession and that they will be more valid than traditional tests.
As major validity criteria, Linn (1993) proposes consequences (i.e. what impact does such testing have on student performance and motivation, on
42
disadvantaged groups etc and on teachers' assessment procedures?), fairness (does it provide opportunities for all to learn, will all institutions be able to deliver teaching and assessment equally well?), efficiency and economy (can we afford it and are we able to implement it?), and generalizability (how generalizable and transferable are the performances from one task to another?).
Fredericks and Collins (1989) propose directness/ authenticity reliability and transparency in addition to content coverage. By 'directness', they mean a test which evaluates a cognitive skill through a performance which is as close as possible to the real word situations in which this skill will need to be demonstrated. By 'transparent' they mean that the criteria for judgement should be clear to the learners.
Wiggins (1989) adds the need for content coverage through multiple performances.
How convincing are these special criteria for validation of performance assessment? Messick, a recognised leader of the measurement establishment (1994), suggests that there is a need to meet more general validity criteria, particularly those related to construct validity. He specifies, minimal construct underrepresentation and minimal construct irrelevant variance (essentially that the assessment should not be too narrow or too broad to achieve its purpose) as the main validity problems. How well do performance tests that are direct and authentic counter these two problems? Messick claims that the specialised criteria proposed do not do this adequately. The major reason for his conclusion is the alleged failure to adequately consider two sets of distinctions: first, that between assessing performance per se and the constructs which the performance represents and second, that between assessing breadth and depth in the chosen domain. One way of clarifying the distinctions is to examine the differences between product and process (or performance) assessments. Where product is the focus of assessment, Messick claims it is not possible to infer attributes that underlie the product. Where the focus is the construct and the performance/ process is the vehicle through which the construct is assessed, issues of generalizability need to be considered. The question is, to what extent can one infer from the performance, attributes which enable performance in other contexts to occur? Often these two are conflated.
This latter issue and most of the others raised in the general assessment literature are taken up in publications (c) (f) (g) and (k).
43
Essentially they argue the need, in an integrated holistic model of assessment, to focus assessment on constructs rather than tasks and that this can be best done through the use of multiple assessments where tasks serve as the vehicles for assessing the construct. In other words, there is a need to choose appropriate tasks at a sufficient level of generality from which the possession of appropriate attributes (and hence competency) can be inferred. This means using a range of assessment events to ensure that adequate domain coverage occurs. Using the integrated model of competence as it applies to competence in occupational roles, the relationships between the tasks and attributes which underpin them are already established in a good set of competency standards and the issue of adequate coverage is not especially problematic. Thus, the need to assess constructs is already established in the arguments about the very nature of competency and the rejection of both the behaviourist task-based approaches and the generic approaches. In this regard the construct of competency is arguably easier to identify for occupational competence than it is for more general cognitive and affective capacities which are the focus of general education. This is a point also made by Bailey (1993). It is simply easier to elucidate what a carpenter or a family lawyer should and can do, than what a competent student critic of poetry should be able to do.
As is pointed out in publication (k), recent work in performance assessment in the professions also argues for the assessment of constructs rather than tasks. The major reason argued in the publications is that the performance of tasks, of themselves, cannot be generalised. The constructs, which guide the choice of the tasks which are assessed can be generalised. The questions in developing an assessment system are: what knowledge, skills, attitudes dispositions and values are needed by a person to demonstrate competency and what tasks will we choose which will enable us to infer that these are present in the person being assessed.
The need for directness in the assessment of competence is also argued in the publications. The potential advantage of this for validity is obvious, but as Linn (1993) argues there is still a need to provide empirical evidence of construct validity and particularly consequential validity. Evidence of this kind in building in Australia. This evidence is not discussed in the publications so there is a need for some discussion here. Some of the most interesting evidence comes from recent work in the legal profession [see publication (g) for an outline of the Family Law assessment scheme, and also Gonczi, Hager and Palmer 1994 appendix b].
44
In a recent review of the Specialist Accreditation Program of the New South Wales Law Society (Armitage Roper and Vignaendra, 1996) found that the program was largely meeting its objectives. The aims of the program are to benefit the public and the professions by:
offering the public and the professions a reliable means of identifying solicitors as having special
competency in an area of practice;
encouraging improvement in the quality, speed, and cost of legal services; providing practitioners with an incentive and opportunity to improve their competency (p. 219).
The program is based on an assessment strategy which seeks to determine whether solicitors meet the criteria specified in sets of competency-based standards. The reviewers used focus groups and a survey to assess the perceptions of clients. While they caution readers to be careful in interpreting their results, they conclude that there has been an increase in expectations of clients as a result of the scheme and that there are high levels of satisfaction among clients:
Overall the data reveal an increasingly discerning clientele whose expectations of service have consistently risen and have almost universally been satisfied or exceeded by their specialists. These findings reflect the ultimate performance indicator [ their italics] for the accreditation program which is client satisfaction with specialists' service: and more specifically they demonstrate the overall effectiveness of the assessment process in providing a mechanism for the identification of solicitors as having special competence in an area of practice (p. 17).
When asked whether the assessment had influenced their practices, 62% of all the successful candidates reported that it had. In the specialist area of Wills and Estates the proportion was as high as 79%. They identified a range of areas such as: review of office procedures; increased awareness of precedents; heightened awareness of time limits in practice; increased knowledge of law and legal principles.
This is strong evidence of the consequential validity of this particular competency-based assessment. Similar results need to be gathered from other professions and occupations, however, before we can talk about the validity of such assessment in any general way.
45
In addition to the other arguments, the publications also argue for an evidence based approach to assessment of competence (see (c) (f) (g) (h) (k)]. That is, they argue that validity is enhanced by undertaking the collection of a wide range of evidence on which to base an assessment judgement. This approach has been advocated in Australia by Bailey (1993, 1995). He argues that there is a need to adopt an evidence-based assessment strategy, akin to the way the legal system makes judgements in the courts. There, as much evidence is collected as is needed to make a safe judgement, either "beyond reasonable doubt" or "on the balance of probabilities", depending on the seriousness of the matter. Similarly, sufficient evidence is needed to make a safe judgement about occupational competence. Judgements are made, as they are in social science research, on the basis of evidence which produces triangulation. That is, assessment of this type is compatible with the epistemology which underpins general social science research. Of course there are other approaches to competency assessment to be found in the Australian literature. These include the use of item response theory (Masters 1993), and latent trait theory (Griffin 1995) which are also compatible with the holistic approach, and task based (Francis 1993) and measurement based approaches (Catts 1993) which on the whole are not. As Docking (1996) has pointed out, most of the Australian studies are conceptual in nature and there is a need for empirical studies to test some of the ideas which have been advanced. There is a suggestion in Docking's work that without the evidence that these studies would produce, all the approaches should be seen as equally valid. The claim being made for the integrated/ holistic method of assessment of occupational competence on the other hand, is that it is based on a clearer, more coherent conceptualisation of the nature of competency and that this makes it easier to identify the constructs that we wish to assess than do other approaches.
While they do not use the terms, the publications, through their development of the integrated approach to assessment, provide some answers to the problems of minimal construct underrepresentation and minimal construct irrelevance variance. The former problem has been dealt with above. The latter problem manifests itself in assessees having to undertake irrelevant tasks which might prevent them demonstrating adequately the competencies that should be the focus of the assessment. Having to assess irrelevant material will distort assessors' judgements. The integrated approach to assessment overcomes this problem by focussing on the relevant constructs. Also the grounding of the constructs in tasks (albeit quite general ones) prevents the development of marking schemes which are too generic.
46
Further questions in competency-based assessment
There are still a number of questions in competency based assessment (using a performance assessment approach) which remain unanswered. These are the degree to which contextualised assessment, such as is recommended in the publications, need to be supplemented by decontextualised assessments; the question of whether to assess complex versus disaggregated skills; and, in the particular case of high stakes assessment, the credibility and feasibility of performance assessment (Mehrens 1992, Baratz Snowden 1990).
The first problem would seem to be of greater concern to general education than to vocational education and assessment for occupational certification since, in the latter cases, domains should be quite clearly specified in competency standards. Hence contextualised assessments should be able to assess the appropriate attributes. The need to use decontextualised assessment in general education is outside the scope of this thesis. However, it does appear to be the case that the less clear the constructs being assessed, the more there is the need for tests of decontextualised knowledge.
There is no ready answer to the question of whether to assess complex or disaggregated skills. The first point to be made is that there is a relationship between parts of a skill and the whole complex skill. This relates to the earlier discussion about the concept of competence and the debate about holism and atomism within this concept. The point was made in answering some of the critics of competency approaches (p25) that monistic holism is as invalid as mindless reductionism. In order to understand molecules we need to understand the properties of the atoms that make them up. The second point is that often the capacity of students to synthesis and integrate parts of a skill depends on their being able to do the constituent parts. Perhaps it is wise then to attempt assessment of both complex/ holistic and disaggregated skills. The danger of this approach, of course, is the possible descent into behaviourism, the very thing that the holistic performance assessment system sought to redress. The extent to which one needs to assess the disaggregated skills is really an empirical question on which there has been little research to date.
The questions of credibility and feasibility are also important empirical questions. Happily there is evidence accumulating that competency-based assessment has
47
the confidence of the professions and other occupations as well as governments and unions and that it is relatively cost effective. In the specialist accreditation of the NSW Law Society for example, there has been an evaluation every year of the extent of the acceptability of the assessment methods. Each year solicitors who are intending specialists are asked to comment on the various assessment methods they have undertaken. What these evaluations show is that those methods which concentrate on the performance of 'real' tasks meet with strong satisfaction while traditional written tests meet with relatively low approval. In 1994 for example, in the Personal Injury speciality, the mock file (a task which asks solicitors to undertake research, draft pleadings for court, organise a brief for barristers etc) was rated by 53% as likely "to a great extent" to give them an opportunity to demonstrate their ability. This was the top ( the fifth point) of a 5 point scale. Taking the fourth and fifth points together, the satisfaction rates were 91% for the mock file. The same level of satisfaction for the written test of knowledge was 13% for the fifth point of the scale and 41% for the fourth and fifth points combined. In the same year in the Family Law speciality, satisfaction rates were 19% and 1.6% for the top of the scale for the mock file and written knowledge test respectively. For the fourth and fifth points of the scale, satisfaction levels were 81% for the mock file and 37% for the written test of knowledge. Similar results are available for the other specialities. One hundred percent of candidates felt that the assessment was "educational" in the sense that it gave them the opportunity to exchange ideas with colleagues, revise and get up to date. In the professions generally , a recent survey by the Australian National Office for Overseas Skills Recognition (see Hager and Gonczi in Edwards et al 1996), revealed that over a third of those professions that had developed competency standards used them for assessment of various kinds and they all appeared to be satisfied that these were working adequately. Feasibility, in terms of time and money, has not been a major issue in the Law specialties but the evidence from the wider educational systems is still being gathered. Certainly there is some reason to suggest that there are feasibility problems in the assessment of key competencies in the school sector (McCurry 1996). Here attempts to use portfolios to assess and report on the possession of the key competencies have come up against the potential problem of the time involved in producing reliable assessment and in writing up reports on students. There are, however, suggestions that teacher judgements within a framework developed by central authorities (eg State Boards) but without excessive
48
moderation can be reliable. Plans to trial such methods are under way, as are experiments with computer generated descriptors designed to save teachers time in reporting.
Much of this discussion of competency-based assessment has so far concentrated on validity issues. The final issue relates to questions of reliability of performance assessment and competency-based assessment.
As is well known, the major threats to reliability are: sampling issues (how many observations of performance are sufficient); subjectivity of scoring; and the question of how consistently we can generalise from the specific performance to the larger domain.
The fact is that there has been relatively little research on reliability issues in competency based /performance assessments. What there is comes from the professions (mostly the medical profession) and while it does not answer all the questions above, it suggests that the judgement of experts is quite consistent. There is also some work on the number of performances that need to be assessed to obtain consistent judgements (Newble 1988). These issues are discussed in publication (f).
In situations where there are few examiners, discussions of actual performance, especially when it is videotaped, seems to be a good way of ensuring consistency. This issue is taken up (g) and in the article by Gonczi Hager and Palmer (1994) in discussion of assessments in the legal profession. Clearly more experimentation needs to be undertaken with different types of scoring charts and different types of occupations to try to overcome problems of this type.
The questions of sampling and generalisability are also taken up in the publications and have been discussed earlier in the arguments about validity. It is true that there are questions about the degree of generalisability of scores across contexts, as Swanson, Norman and Linn (1995) have pointed out. However, this does not necessarily lead to the conclusion that performance assessment is not capable of being reliable. Rather it suggests that a variety of assessment methods need to be used in a competency-based assessment system. This means, in effect, that performance tests are not a panacea for competency-based assessment. This is exactly what the publications have been arguing.
49
In conclusion, although there are still questions about the extent to which performance based and competency-based assessment can be sufficiently valid and reliable in occupational education and licensure to satisfy psychometricians, the work reported in the publications has made a substantial contribution to our understanding of these issues. Furthermore the evidence of construct and consequential validity is accumulating as is that of both the acceptability and feasibility of this approach to assessment.
Developing competency-based assessment systems
The development and implementation of a competency-based assessment system depends on a sound conceptualisation of the nature of competence and an understanding of the general assessment literature. However such knowledge and understanding will not be sufficient. Issues of management, feasibility acceptability will often involve "political" questions rather than technical ones. Designers of assessment strategies will always need to take into account the needs of the various stakeholders a process which involves negotiation and compromise. In addition they will need to consider the cost effectiveness of their strategies.
The publications attempt to demonstrate how such a system can be developed and implemented. Publication (g) describes the process of actually developing an assessment system in the Dietetics profession and in branches of the legal and medical professions. Publications (c) and (h) explain the process in more detail. They examine the alternative frameworks for such a system, how to carry out the assessment, how to make judgements on the basis of evidence collected and how to manage such a system.
In addition, publication (h) canvasses the option of grading rather than the traditional binary decision of competent/not competent. It argues the possibility, amongst other ways of addressing this issue, of using a scale of increasing competence within a criterion referenced framework. This would be an innovative way of combining traditional psychometric and judgemental assessment paradigms. However as has been pointed out by Griffin (1995) the development of these scales is a complex and costly exercise for which there seems to be no support currently.
These implementation issues are not technical matters alone, though decisions on frameworks and how to judge performance depend on an understanding of
49
50
the technical/ theoretical concepts. But there are policy issues as well as practical questions to consider. A major contribution of the publications is to delineate the actual decisions that need to be made to made to implement a competency-based assessment system and show how they can be based on sound theoretical principles. Conclusion The publications and the subsequent research contained in this overview essay have made a substantial contribution to the debate about competency-based education. Their major contribution has been to clarify the nature of competency and to explore the implications of this conceptualisation for educational practices. Competency-based approaches to education have generally arisen out of a desire of governments, industry and the professions to make educational institutions responsive to their perceived needs. By developing competency standards for occupations and expecting that curricula be based on them, these groups have increased their influence on the content of education. Some educators would suggest that this is undesirable and that it is a sufficient reason to resist competency-based approaches. However, as the publications have shown, competency-based education based on the integrated model can also act as a focus for reform for educators dissatisfied with some long-standing educational problems. Important amongst these is the long-standing dichotomy between propositional and procedural knowledge. At the level of educational practice this dichotomy is manifested in the dominance of abstract concepts in educational curricula and teaching, and the devaluing of capacity to "do" things. It is represented at the institutional level in the gulf between vocational and general education and between middle level and professional education, a polarity which has helped to perpetuate the inequalities in our social and economic system. Competency-based education based on the integrated approach can make a contribution to overcoming these dichotomies by demonstrating that knowing and doing are inextricably interwoven. Another long standing problem which competency-based education potentially addresses is the deleterious effect on learning of assessment based on psychometrically derived measurement principles. The publications suggest that competency-based assessments which are integrated, direct, evidence based and which rely on professional judgement, are capable of surmounting this problem, without substantial sacrifice of rigour in the assessment process.
51
There is, too, the potential in competency-based education to challenge the power of educational institutions by rewarding people for skills and knowledge developed in informal settings.
These add up to a substantial cluster of educational reforms. Many of them are yet to be realised but the publications have opened up the possibility of pursuing these reforms and have developed for those who wish to do so, a sound theoretical and practical base on which to build.
52
References Allen Consulting Group (1994). Successful reform. Report to The Australian NationTraining Authority. Melbourne: Allen Consulting Group. Armytage, L., Roper, C. and Vognaemdra, S. (1995). A Review of Aspects of the Specialist Accreditation Program of the Law Society of New South Wales. Sydney: Centre for Legal Education. Ash, S., Gonczi, A. and Hager, P. (1992). Combining research methodologies to develop competency-based standards for dietitians: A case study for the professions. National Office of Overseas Skills Recognition Research Paper No. 6, DEFT. Canberra: Australian Government Publishing Service. Australian Council of Trade Unions (1987). Australia reconstructed: ACTU mission to Western Europe. Canberra: AGPS. Bailey, M. (1993). Theoretical models and the assessment of competency. Paper
presented at the VEETAC National Research Forum. Bailey, M. (1995). Competency assessment, evidence and the discourse of judgement. Australian and New Zealand Journal of Vocational Education Research, 3, 1,1-13. Baratz Snowden, J. (1990). RFP-National Board for Professional Teaching Standards. Washington DC: NBPTS. Barker, A. (1995). Standards based assessment: The vision and broader factors, in Peddle, R. and Tuck, B. Setting the Standards. Palmerston North: The Dunmore Press. Barrow, R. (1991). The generic fallacy, Educational Philosophy and Theory, Special issue on critical thinking, 23, 1, 7-18. Barrows, H.S. (1971). Simulated patients (programmed patients): The development and use of a new technique in medical education. Springfield, IL: Charles C. Thomas. Becker, G.S. (1964). Human capital: A theoretical and empirical analysis with special
reference to education. New York: National Bureau of Educational Research. Bengtsson, J. (1993). Labour markets of the future: the challenge to education policy makers. European Journal of Education, 28, 2, 135-158. Blaug, M. (1972). An introduction to the economics of education.
Harmondsworth: Penguin. Boston, J. (1994). The theoretical underpinnings of public sector restructuring in New
Zealand, in Boston, J. (ed). (1991). Reshaping the State. Oxford: Oxford University Press. Brown, J. S., Collins, A. and Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18, 32-42. Carr, D. (1993). Questions of competence. British Journal of Education Studies, 41,
3,253-271. Catts, R (1993). Reliability of workplace assessment. Paper presented at the Testing
Times Conference. Sydney: National Centre for Vocational Education Research. Chi, M. T. H., Glaser, R. and Farr, M. J. (Eds.) (1988). The nature of expertise.
Hillsdale, N.J.: Erlbaum.
53
COSTAC (1990). A strategic framework for the implementation of a competency based training system. Canberra: AGPS. Department of Employment Education and Training (1987). Skills for Australia.
Canberra: AGPS. Department of Employment Education and Training (1988). Higher education: A policy
statement. Canberra: AGPS. Department of Employment Education and Training (1989). Improving
Australia's Training System. Canberra: AGPS. Docking, R. (1996). The assessment of learning. Paper presented at the Research
Directions for Workplace Learning and Assessment Seminar. UTS Research Centre in Vocational Education and Training.
Ebel, R (1972). Essentials of educational measurement. Englewood Hills, N.J.: Prentice Hall.
Edwards, R., Hanson, A. and Raggatt, P. (1995). boundaries of adult learning. London and New York: Routledge and Open University.
Evans, G. and Butler, J. (1992). Thinking and enhanced performance in the workplace. Paper presented at the Fifth International Conference on Thinking, Townsville.
Feinberg, W. (1983). Understanding education: Towards a reconstruction of educational inquiry. Cambridge: Cambridge University Press.
Field, J. (1991). Competency and the Pedagogy of Labour. Studies in the Education of Adults, 23 (1), 41-52.
Finn, B. (1991). Young people's participation in post compulsory eduation and training. Canberra: AGPS. (Australian Education Council).
Francis, J. (1993). Cost effective competency based assessment. Paper presented at the Testing Times Conference. Sydney: NCVER
Frederiksen, J. R. and Collins, A. (1989). A systems approach to educational testing. Educational Researcher, 18, 9, 27-32.
Glaser, R (1963). Instructional technology and the measurement of learning outcomes. American Psychologist, 18, 519-521.
Gonczi, A. (1994). International perspectives on competency based education. Paper presented at the CBI International Conference, Holland College, Prince Edward Island.
Gonczi, A. and Tennant, M. (1994). The false war: Competency-based education and its critics. Proceedings NSW TAFE Curriculum Conference, October. Gonczi, A.,Hager, P. and Palmer, C. (1994). Performance-based assessment and the NSW Law specialist accreditation program. The Journal of Professional Legal Education, 12, 2, 135-149. Gonczi, A., Curtain, R, Hager, P., Hallard, A. and Harrison, J. (1995). Key
competencies in on-the-job training. Sydney: UTS-DIRETFE. Gott, S. (1995). Rediscovering learning: Acquiring expertise in real world problem
solving tasks. Australian and New Zealand Journal of Vocational Education Research Vol. 3, 1, 30-69.
Griffin, P. (1995). Competency assessment: Avoiding the pitfalls of the past. Australian and New Zealand Journal of Vocational Education Research Vol 3, 2,34-60.
Hager, P. and Beckett, D. (1995). Philosophical underpinnings of the integrated conception of competence. Educational Philosophy and Theory, 27, 1, 1-24.
54
Hager, P. and Gonczi, A. (1996). Professions and competencies, in Edwards,
R,Hanson, A. and Raggat, P. boundaries of adult learning. London and New York: Routledge.
Hager, P., McIntyre, J., Moy, J., Comyn, P., Stone, J., Schwenke, C. and Gonczi, A. (1996). Workplace keys. Sydney: NSW Dept of Training and Education Co-ordination.
Harris, R., Guthrie, H., Hobart, B. and Lundberg, D. (1995). Competency-based education and training. Melbourne: Macmillan.
Higher Education Council (1992). The quality of higher education. Canberra: National Board of Employment, Education and Training.
Hodgkinson, P. (1992). Alternative models of competence in vocational education and training. Journal of Further and Higher Education, 16, 2. Jessup, G. (1991). Outcomes. NVQ and the emerging model of education and training. London: The Falmer Press.
Karpin, D.S. (1995). Enterprising nation. Report of the Industry Task Force on Leadership and Management Skills. Canberra: AGPS.
Lave, J. (1988). Cognition in practice. Cambridge: Cambridge University Press. Lave, J. and Wenger, E. (1988). Situated learning. Cambridge: Cambridge University Press. Linn, R. L. (1993). Educational assessment: expanded expectations and challenges.
Educational Evaluation and Policy Analysis, 15, 1-16. Linn, R L., Baker, E. L. and Dunbar, S. B. (1991). Complex performance based
assessment: Expectations and validation criteria. Educational Researcher, 20, 8, 15-21. Lohrey, A. (1995). Transferability and the key competencies. A report of the Centre for
Workplace Communication and Culture. Sydney: UTS. Magnusson, K. and Osborne, J. (1990). The rise of competency-based education: A Deconstructionist analysis. The Journal of Educational Thought, 24, 1, April. Marginson, S. (1993). Educational research and education policy. In Walker, J. (ed). Educational Policy Development and Implementation, Review of Australian Research in Education No 2. Canberra: AARE. Marsick, V. and Watkins, K.(1990). Informal and incidental learning in the
workplace. London: Routledge. Masters, G. N. (1993). Certainty and probability in assessment of competence.
Paper presented at the National Assessment Research Forum. Sydney. Mayer, E. (1993). The key competencies: Putting general education to work. Canberra: Australian Education Council and Ministers for Vocational Education, Employment and Training. McCurry, D. (1995). Approaches to assessing the key competencies. Draft paper,
Australian Council of Educational Research. Mehrens, W. A. (1992). Using performance assessment for accountability purposes.
Educational Measurement: Issues and Practice, 11, 1, 3-10. Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23, 3, 13-23.
National Board of Employment Education and Training (1989). Industry training: The need for change. Canberra: AGPS.
Newble, D.I. and Swanson, D.B. (1988). Psychometric characteristics of the objective structured clinical examination. Medical Education, 22, 325-34.
Newman, M. (1994). Defining the Enemy. Sydney: Victor Stuart Publishing. 54
55
Popham, W. J. (1978). Criterion referenced measurement. Englewood Hills NJ: Prentice Hall.
Prawat, R.S. (1993). The value of ideas: Problems versus possibilities in learning. Educational Researcher, 20 Aug/Sept, 5-16.
Pusey, M. (1991). Economic rationalism in Canberra: A nation building state changes its mind. Melbourne: Cambridge University Press. Quirk, R (1994). Educational issues in the design of courses based on competency standards Sydney: NSW TAFE/UTS. Raizen, S. (1994). Learning and work: The research base. In Vocational Education and Training for Youth: Towards Coherent Policy and Practice. Paris: OECD. Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press. Raven, J. (1996). Researching competence. Higher Education Review, 28, 74-89. Resnick, L. (1987). Education and learning to think. Washington DC: National Academy Press. Sadler, D.R. (1987). Specifying and promulgating achievement standards. Oxford Review of Education, 13, 191-209. Scribner, S. (1984). Studying working intelligence. In B. Rogoff and J. Lave Everyday Cognition: Its Development in Social Context. Cambridge, UK: Cambridge University Press. Scribner, S. (1985). Thinking in action: some characteristics of practical thought. In R J. Sternberg and R Wagner (Eds.) Practical Intelligence: Nature and Origins of Competence. Cambridge: Cambridge University Press. Scriven, M. (1967). The methodology of evaluation. In Perspectives of Curriculum
Evaluation. Ralph W. Tyler, Robert M. Gagne, Michael Scriven. Chicago: Rand McNally.
Sternberg, R. J. (1990). Metaphors of mind: Conceptions of the nature of intelligence. Cambridge: Cambridge University Press.
Stevenson, J. C. (1994). Cognition at work: the development of vocational expertise. Adelaide: NCVER.
Stevenson, J.C. (1995). The metamorphosis of the construction of competence. Inaugral professorial lecture. Griffiths University, Brisbane.
Swanson, D. B., Norman, G. R and Linn, R. L. (1995). Performance-based assessment: Lessons from the health professions. Educational Researcher, 24, 5-11.
Taylor, C. (1994). Assessment for measurement or standards: The peril and promise of large scale assessment reform. American Educational Research journal, 31, 2, 231-262. Taylor, R (1996). Review of the ANTA Agreement. Canberra: AGPS. Tuck, B. (1994). Issues of objectivity in assessment: A plea for moderation. In Peddie, R. and Tuck, B. Setting the Standards. Palmerston North: The Dunmore Press. Walker, J. (1992). The value of competency based education. Paper presented at the
conference: The effects of competency based education on universities. University of Western Australia, June, 1992.
Walker, J. (1993). A Focus on Policy, Review of Australian Research in Education number 2, 3-15. Canberra: Australian Association for Research in Education.
56
Walsh, M. (1995). Technology's part in growth. Sydney Morning Herald, 31 October p 25
(from Maddison, A. Monitoring the World Economy 1820-1992. OECD 1994). Wellington, J.J. (1993). The work related curriculum. London: Kogan Page. Wiggins, G. (1989). Teaching to the (authentic) test. Educational Leadership, April 41-47. Wolf, A. (1993). Assessment issues and problems in a criterion-based system. London:
Further Education Unit. Wolf, A. (1995). Competence-based assessment. Buckingham: Open University Press. Young, R. (1995). Report on key competencies program. Private paper in the hands of
author, University of Sydney, Faculty of Education.
