© Curriculum Foundation 1

Part 2

Assessing our wider aspirations

Unit 2 looked at the ‘building blocks’ of the curriculum:

Knowledge

Skills

Understanding

Possession of information

Ability to perform mental or physical operation

Development of a concept: putting knowledge in a framework of meaning

Knowledge

Skills

Understanding

State, name, label, draw, identify, describe

Carry out, perform, find, investigate, explore

Explain, justify, analyse, give reasons for

If you recall, each of these levels of expectation had some related verbs that made them explicit. These help us unlock what is in the brain.

These are the verbs that also help us develop the assessments.

© Curriculum Foundation 4

Do you remember this model from Unit 4? We were suggesting that the criterion for selecting the subject content was whether it was necessary for developing the key concept. The animated film was the way of bringing this together. So if the key concept is the important bit, then when we come to assessment, it is the key concept that we need to look at – not the subject content.

And if we are also interested in the competency skill (communication in this case), then we would want to know how well are pupils are doing in terms of this too.

© Curriculum Foundation 5

Many of the aims were were looking at in Unit 1 were not even elements of knowledge, skills and understanding. They were attitudes and values. Some fell into the category of ‘21st century competencies’

These things are what we value – but do we need to assess them? If so how? We shall come back to this in Part 4 – first a look at the

different parts of the model.

© Curriculum Foundation 6

The subject content is on the whole the easy bit to assess because it consists mainly of knowledge. We can be both valid and reliable in our assessment of knowledge. Not only that, we can assess whole classes at a time because we can do it through written papers. This is probably why we are so keen on doing it.

The key concepts are much harder to assess because they involve a student’s understanding. This is where we really need to get inside the brain. We are looking for ‘proxy measures’ such as the ability to explain or give reasons. These will give us some idea about the extent of understanding. This is best done through oral questioning of individual students in a series of different situations over time. Much more expensive – and much less popular. It also runs into issues of reliability.

© Curriculum Foundation 7

To assess skills we need to see them being performed. We could ask someone, “Can you play the violin?” or “How well can you play the violin?”, but this would be neither valid nor reliable. So we are left with putting people into practical situations where they have to deploy the skills being assessed. This happens in practical examinations, MFL orals and in the specialist examinations for things like music and dance.

These have high validity and to make them more reliable we can develop ‘rubrics’ or mark schemes. These can add structure to our subjective judgments and can go some way to ensuring that similar standards are being applied to different students and different situations. In well established procedures, such as the grade examinations for music, a high degree of reliability can be established – but it is expensive and still not perfect. We have to remember our limitations here and not write students off because our assessment methods are less than perfect.

However, there are several approaches that can help.

© Curriculum Foundation 8

Part 3

Approaches to assessment

© Curriculum Foundation 9

Do you remember Bloom’s Taxonomy from Unit 2? There was an ascending scale of learning from knowledge through understanding to application.

© Curriculum Foundation 10

This was an early (1969) attempt to put some structure into the progression of learning that made it more than an increasingly long list of things to be learned. It also gives a structure for assessment. There is some criticism of it today because of the difficulty in establishing the boundaries between one level and another. However, this is to misunderstand Bloom’s intention which was not to establish a rubric for assessment, but to begin the process of identifying different levels of demand in learning.

The problem is in the distinction between application and then analysis (separating things into their component parts) and synthesis (putting separate ideas together into new whole). Many people suggest that these are not strictly hierarchical, and so they do not always occur in this order, but depend upon the context. In some contexts, for example, it might be easier to analyse than to apply.

However, the taxonomy is recognised around the word and still underpins our understanding of intellectual progression.

© Curriculum Foundation 11

* Biggs, J. B. and Collis, K. (1982) Evaluating the Quality of Learning: the SOLO taxonomy. New York, Academic Press

A different taxonomy that is very popular today was put forward by Biggs and Collis* in 1982. (The book cover does not come out very clearly here! You’ll just have to buy it!)

This is the ‘Structure of Observed Learning Outcomes’ (SOLO) that puts forward five levels of understanding.

© Curriculum Foundation 12

The five levels are:

Pre-structural - The task is not attacked appropriately; the student hasn’t really understood the point and uses too simple a way of going about it.

Uni-structural - The student's response only focuses on one relevant aspect.

Multi-structural - The student's response focuses on several relevant aspects but they are treated independently and additively.

Relational - The different aspects have become integrated into a coherent whole. This level is what is normally meant by an adequate understanding of some topic.

Extended abstract - The previous integrated whole may be conceptualised at a higher level of abstraction and generalised to a new topic or area..

© Curriculum Foundation 13

© Curriculum Foundation 14

SOLO level Verbs

Uni-structural Define, identify, name, draw, find, label, match, follow a simple procedure

Multi-structural Describe, list, outline, complete, continue, combine

Relational Sequence, classify, compare and contrast, explain (cause and effect), analyse, form an analogy, organise, distinguish, question, relate, apply

Extended abstract Generalise, predict, evaluate, reflect, hypothesise, theorise, create, prove, justify, argue, compose, prioritise, design, construct, perform

There’s more at: http://uq.edu.au/tediteach/assessment/docs/biggs-SOLO.pdf

This also suggests the verbs that will help with assessment:

© Curriculum Foundation 15

The drawback of SOLO is that we get to Level 4 before we arrive at ‘adequate understanding’. This seems to leave a great deal of ground – all of what Bloom sees as application, analysis, synthesis and creative application – squashed into one level.

Even if one were to ignore Bloom, it would be helpful to be able to distinguish between various levels of application and the increasing skill levels involved in problem solving and critical thinking. SOLO does not easily offer these distinctions.

However, more recent work has developed these ideas.

© Curriculum Foundation 16

A more recent approach was put forward by Prof Norman Webb of Wisconsin University in 1997. This saw four levels of ‘Depth of Knowledge’ (DOK).

Knowledge is used here in a wider sense that encompasses understanding and the ability to process and apply that knowledge. ‘Knowing how to ..’ and ‘Knowing about’ as well as ‘Knowing that..’.

Webb’s DOK has become the basis of the entrance exams for universities in the USA – and well as for a wide range of assessment of deeper understanding and application.

There’s more at:http://www.aps.edu/rda/documents/resources/Webbs_DOK_Guide.pdf

© Curriculum Foundation 17

Level 1Recall and reproductionRecall of a fact, information or procedure

Level 2

Application of skills and conceptsUse of information or conceptual knowledge – two or more steps

Level 3

Strategic thinkingRequires reasoning, developing plan or a sequence of steps, some complexity, more than one possible answer

Level 4

Extended thinkingRequires an investigation, time to think and process multiple conditions of the problem.

Norman Webb’s ‘Depth of Knowledge’

© Curriculum Foundation 18

Webb suggests that we should be helping our students to embrace complexity – not just making things more difficult for them and seeing this as progression.

He distinguished between things that are difficult and things that are complex. For example:

Who is the President of the USA?

Who was the 19th President of the USA?

© Curriculum Foundation 19

You probably got the first one right. It is seen as an easy question because almost everyone knows the answer.

But you probably didn’t get the second one. It’s seen as difficult because very few people know the answer.

If you are interested, it was Rutherford B Hayes.

Of course, both questions are at the same level of complexity. All they require is the remembering of someone’s name. And what we are aiming for in deep learning is complexity – not mere difficulty.

© Curriculum Foundation 20

Webb’s DOK analysis fits well with our model of the tree.

Recall

Reproduction

Application of skills

Application of concepts

Strategic thinking

Extended thinking

© Curriculum Foundation 21

The point of looking at three different approaches is not to say that one is right and the others are wrong (although you will notice that much of the literature about SOLO is directed at rubbishing Bloom!). The point is that they all give us a way at looking at learning in terms of its increasing depth or complexity. As we said earlier, the brain is an extraordinarily complex organ, and no simple taxonomy of levels will describe what’s really going on.

However, approaches such as these help us to plan learning in terms of greater depth, and also to find out how well our students are doing in these terms.

It does not matter which one you use, or whether you find some blend that suits you best. What is important is to think about how the intellectual level is being increased, and so what needs to be assessed.

Three Approaches