Progress in English 7–11 DigitalGuidance and Information for Teachers

2

Published by GL AssessmentThe Chiswick Centre, 414 Chiswick High Road, LondonW4 5TF, UK

GL Assessment is a division of Granada Learning Limited

First published 2008© GL Assessment 2008. All rights reserved.

Digital Tests from GL Assessment

Progress in English 7-11 (PIE 7-11) is part of a selection of standardised testsavailable in both paper and digital editions from GL Assessment. Thedevelopment of digital editions of major series such as the Cognitive AbilitiesTest and Group Reading Test, as well as PIE 7-11, is a response to the need forschools to test large numbers of pupils at regular intervals and to make thatprocess as efficient as possible by automating the scoring, analysis andreporting. At the same time, by developing digital editions of established tests,teachers and pupils can be assured of the robustness of these tests.

PIE 7-11 Digital is often used as an end-of-year test and if it is used year on yearcan offer important evidence of pupil progress and value-added. As such it is animportant test and one on which decisions about, for example, setting orremediation, may be made in conjunction with teacher assessment and anevaluation of pupil performance throughout the year.

PIE 7-11 Digital must be administered in a formal test environment with pupilsmade aware that they are taking a test and that the usual expectations ofbehaviour and constraints of a test session will be in place. Pupils’ experienceof working at a computer may lead to the impression that taking a test using aPC is not as important as the more familiar test session in the school hall orrearranged classroom.They may expect to spend time in the computer suite onless formal activities, engaging in learning that is presented in a highly visual oreven game-like way.While GL Assessment digital tests do engage pupils and,in the case of PIE 7-11 Digital, have the benefit of full colour illustrations, theyare tests and must be approached in the same way as the more familiar papertest process.

For fully comprehensiveinformation about using digitaltests from GLAssessment, pleasedownload the testwise usermanuals from the help sectionof the websitewww.testingforschools.com

NBTestwise is an online assessmentsystem that delivers tests to over500,000 pupils a year.

3

Introduction to Progress in English 7-11 Digital

The digital versions of Progress in English comprise the short form of the testsfor 7 to 11 year olds. The short form exercises are:

Exercise 1: SpellingPupils are asked to spell the words missing in a passage. The words have beenselected to exemplify spelling patterns that are particularly important for pupilsat different ages.

In Progress in English 7, 8 and 9, pupils use an onscreen alphabet to give theirresponses to the items in Exercise 1. They click on a letter and it appears in theanswer box. An onscreen ‘eraser’ can be clicked to remove a letter or lettersand the student can then re-select letters until the word has been completed.

Exercise 2: GrammarThe exercise assesses the ability to correct mistakes in a passage. The itemscover a variety of grammatical rules, appropriate to the age of the pupils takingthe test.

In PIE 7, 8 and 9, this exercise is multiple-choice with pupils presented withfour answer options.

Exercise 3: Reading comprehension (narrative)In this exercise, pupils read a story and answer questions by selecting one offour multiple-choice options. This format makes it possible to assess literal andinferential comprehension, together with some understanding of authorialtechniques, in an accessible way.

Exercise 4: Reading comprehension (non-narrative)This exercise complements Exercise 3 by presenting an information text linkedto the story. Once again, multiple-choice questions are used to assess pupils’understanding of what they have read.

Exercises 1 and 2 are presented together as the first part of the test andexercises 3 and 4 are presented together to complete the test.

All exercises are untimed.

The development of these versions of GL Assessment’s major Englishassessment series was undertaken by the National Foundation for Educational

4

Research to provide teachers and pupils with an alternative format to the papertests with the added benefits of immediate scoring and reporting.

The content of these digital tests is the same as their paper equivalents.However, methods of answering questions in exercises one and two aredifferent (see above).

An equating study was carried out during autumn of 2005 to ensure that thedigital tests were equivalent to the paper tests.

Use of Progress in English tests

Progress in English is particularly suitable for use during the second half of theacademic year. It can also be used as a start-of-year test, in which case it isrecommended that pupils be given the test intended for the year below, as setout below:

Administration

When adding a student to a Progress in English register, all four exercises areallocated to the student.

The four exercises may be taken over two test sessions.

TheTestwise url is:

http://www.testingforschools.com

Pupils should click on EnterTestwise after selecting their region from theglobal map on the screen, and then select their school from the displayed list.

Once the school has been selected successfully and pupils have been added tothe register by the administrator, they then click the Student icon. They willthen enter the register ID.

Year group Autumn term Spring/summer term

Year 2 in England &Wales Paper version of Progress in English 7Year 3 in Northern Ireland Progress in English 6Primary 3 in Scotland

Year 3 in England &Wales Progress in English 7 Progress in English 8Year 4 in Northern IrelandPrimary 4 in Scotland

Year 4 in England &Wales Progress in English 8 Progress in English 9Year 5 in Northern IrelandPrimary 5 in Scotland

Year 5 in England &Wales Progress in English 9 Progress in English 10Year 6 in Northern IrelandPrimary 6 in Scotland

Year 6 in England &Wales Progress in English 10 Progress in English 11Year 7 in Northern IrelandPrimary 7 in Scotland

5

If the children are very young, the teacher may prefer to log on for each student.

Pupils will then see the following screen and they should select their name andenter their password.

Pupils can then click on TakeTest.

6

Please note that after an exercise has been completed it may remain on the listof assessments available to a student until results have been stored. The latterprocess can take some time at particularly busy test periods.

A holding screen similar to the one below will appear. This enables pupils tostart each pair of exercises at the same time.

Each exercise starts with an explanation of what is required of pupils and somepractice questions. The content of the latter has been kept very simple: theitems are there to enable pupils to become familiar with ways of answeringdifferent questions. These include clicking in an answer box and typing with thekeyboard, selecting an answer from a series of options, and selecting lettersfrom an onscreen alphabet.

All of the instructions are given via the headphones to reduce the amount ofreading required.

Practice item for Progress in English 7, Exercise 1: Spelling

The practice section has been constructed to enable pupils to go through theseitems at their own pace. At the end of the practice section, the followingscreen acts as a holding screen. It is possible, but not necessary, to wait untilall pupils have reached this point so that they start the exercise simultaneously.

7

Once pupils have completed the first pair of exercises and clicked on the “EndTest” button, they will be returned to the following screen, at which point theycan select the second pair of exercises or log off.

Pupils with Special Assessment Needs

Having read the description of the content of the test, you should carefullyconsider whether the test is appropriate for pupils with severe readingdifficulties, or for those whose English language fluency is very limited. Thesepupils will obviously be at a disadvantage in any test such as this, andcompensation cannot be made by simply allowing extra time for the test or byreading out the test material. Therefore, the test should be used withdiscretion, if at all. If it is necessary to give pupils extra help, it is recommendedthat they be tested in a separate group so as to minimise disruption to the restof the class.

TheTest Environment

PIE 7-11 Digital is an untimed test. However, Exercises 1 and 2 should takeapproximately 10 minutes each; Exercise 3 should take 20 minutes; andExercise 4 should take 10-15 minutes. Therefore, Exercises 1 and 2 may begiven in a single lesson period but if all four Exercises are to be taken in asingle session, a double lesson period will be required.

8

Each pupil will need a computer, headphones and mouse and all equipmentneeds to be in good working order. Pupils should be told that they are going totake an English test and explained the purpose of the test: ‘to find out whatyou can do or where you may need help’ or ‘to let your teacher next year knowwhat you can do’. Pupils should be told that they must work in silence but thatif they have a query they should raise their hand and wait for the teacher toapproach them. Answer any questions at this stage and explain that you cannothelp with any of the test questions but that they should try to do their best andat the end go back and check their work.

While pupils are taking the test the teacher should walk round the computersuite to check that they are progressing appropriately, that they are not havingdifficulty with the methods of answering questions and, importantly with digitaltests, that they have not rushed through any part of the test without attemptingto answer some questions.

Unexpected incidents during a test session

As with the paper test, should anything unexpected occur during the testsession, the incident should be recorded and appended to the group report forthis specific group of pupils. This will allow the incident to be taken into accountwhen scores are being considered.

If there is a failure in your computer system while pupils are taking an exerciseit will not be possible to re-enter the test at the point at which the failureoccurred. In this instance, pupils will need to re-take the complete exercise. Ifpupils complete an exercise and results are stored (i.e. they have clicked the“end test” button) and then the system fails, it will be possible to retrieveresults, and therefore reports, from the GL Assessment back-up server.

Should this happen, please contact the GL Assessment Customer SupportTeam on 0845 602 1937 and you will be connected to aTestwise adviser.

Testwise Progress in English Report

Only pupils who have completed both pairs of Progress in Englishexercises will be included in the analysis presented in the group report.

Such pupils will be highlighted in Section A with a note of whichexercises they have completed and their raw score.They will not beincluded in the CSV report.

A sample Progress in English group report may be viewed at:http://www.gl-assessment.co.uk/pie

The report is in three sections:

Section AThis section summarises each student’s attainment on the test as a whole,giving their age at the time of test, their raw score, standardised score, stanine,percentile rank and group rank (based on standardised score).

The standardised score is particularly useful as it shows the student’sattainment in relation to a nationally representative sample of pupils of thesame age. The national average standardised score is 100, and two thirds ofpupils will score between 85 and 115. The student’s standard score is alsoshown as a vertical line with a horizontal line showing the 90% confidenceband. It is recognised that any test score represents a performance on a

9

particular day, and the score should therefore be placed within such aconfidence band. If the test were taken again, nine times out of ten one wouldexpect the score to fall within this range.

The stanine score shows the standard nine score the student achieves incomparison with the national sample, with 9 being the highest score and 1being the lowest.

The national percentile shows the percentage of pupils in the national samplewhose scores were lower than the student.

Final columns of this section give scores against curriculum categories:spelling, grammar, reading comprehension (narrative) and readingcomprehension (non-narrative).

Section BThe first table in this section shows the mean score for the class/group bygender against the national average. This allows the group’s attainment to beevaluated as below, at, or above the national average. The information displayedon this table is illustrated by the graph beneath the table, which as well asdisplaying the group and national means, shows the 90% confidence bands.Descriptive comments are also given to explain whether the average and therange of scores for the class/group are significantly different from the nationalaverage.

A second table shows the distribution of pupils across nine score bands(stanines) compared to the national distribution. This data is presented in agraph to show the distribution for the class/group separately for boys, girls, allpupils and the national sample.

There can be many reasons why the attainment of pupils might be significantlyhigher or lower than the national average. Factors might relate to the pupils’motivation, to levels of support at home, to the quality of their previousexperience of education, etc.Whatever the reasons, the first step to improvingpupils’ attainment is to know accurately where they are now. This report helpsby giving an overview of the current performance of the group as a wholeagainst national standards.

The final table in Section B shows results analysed by Process Area.

All the test questions in the short form of Progress in English series can begrouped into one or more of the following categories:

Curriculum Content Category Process CategoryWriting Writing• Spelling • Spelling• Grammar • Punctuation

• Consistency

Reading Reading• Reading comprehension narrative • Simple inference• Reading comprehension • Complex inferencenon-narrative • Retrieval

• Authorial techniques

The bar charts show the percentage success rates for each of the categoriesfor the class/group against the national average. In some cases, the profile forthe class may be above the national average, or indeed below the national

10

average, in all process areas. In other cases the results may reveal strengths inone particular process area, but a relative weakness in another.

The results presented are based on raw test scores that have not beenadjusted for age. If the average age of the group is more than 3 monthsolder/younger than the national standardisation sample, then their mean rawscores may be slightly higher/lower respectively than the national averagebecause of this. The purpose of this part of the report is to focus on the relativestrengths and weaknesses in different areas and on different questions, ratherthan a simple comparison to the national norms.

Section CThere is a wealth of information available from a detailed analysis of pupils’performance on each of the questions within the tests. This data can also becompared with the difficulty of the questions as established during the nationalstandardisation.

– Question by Question GraphThis graph gives a quick overview of the success rates for each question for theclass/group (bars) compared to the national average (thick dark line). Thequestions are sorted from left to right according to their difficulty, as indicatedby the percentage of pupils answering the question correctly at the time of thenational standardisation. For example, the question on the extreme left of thegraph is the easiest question in the test (usually answered correctly by over90% of the national sample), and the question on the extreme right is the mostdifficult question in the test.

– Question by Question ListingThis data is also presented in the form of a table that includes a briefdescription of each question alongside the question number. The questions arelisted in the order they are presented in the Question by Question Graph, thatis, from left to right across the graph.

– Question Listing by School-National DifferenceThis table presents the Question by Question Listing in a slightly differentorder. The difference between the group/class and the national success rate iscalculated and shown in the ‘group national difference’ column. The table issorted so that the questions where the class is most ahead of the nationalaverage are listed at the top, and the questions where the class is most behindthe national average at the bottom.

Some of the questions that can be answered using the Question by Questionreports are:

• Which questions did the class find most difficult, and which relatively easy?

• Are there any common elements in the questions the class found mosteasy/difficult? For example, do they all relate to a particular aspect of thecurriculum?

• Are there any questions where no student in the class answered correctly?What implications might this have for teaching? Have these areas of thecurriculum been covered? Do such areas need to be reviewed?

11

Individual Scores

An individual’s scores can be viewed by selecting ‘CSV report’ in Testwise.This will produce an Excel spreadsheet with one or more pupils’ raw score andstandardised score, item by item, with process categories for writing (spelling,punctuation and consistency) and reading (simple inference, complex inference,authorial techniques and retrieval).

The Chiswick Centre

414 Chiswick High Road

LondonW4 5TF

Tel: 0845 602 1937

Email: info@gl-assessment.co.uk

www.gl-assessment.co.uk