Student academic language proficiency: an investigation into a non-first language

developmental intervention

by

Dr Cay van der Merwe, Central University of Technology, Bloemfontein

and

Prof Driekie Hay, University of the Free State, Bloemfontein

SAAIR, Port Elizabeth

September 2009

2UNIVERSITEIT VAN DIE VRYSTAAT • UNIVERSITY OF THE FREE STATE • YUNIVESITHI YA FREISTATA Tel (051) 401 3971 • E-mail: hayd.rd@ufs.ac.za • www.ufs.ac.za

Introduction

• English is the mother tongue of only 8,6% of South African students

• As from Grade 4 for the majority of learners : no mother tongue instruction

• Academic language proficiency level of 85% below Grade/K 12

3UNIVERSITEIT VAN DIE VRYSTAAT • UNIVERSITY OF THE FREE STATE • YUNIVESITHI YA FREISTATA Tel (051) 401 3971 • E-mail: hayd.rd@ufs.ac.za • www.ufs.ac.za

Implications for higher education

• Students coming from a variety of educational backgrounds

• Rote learning

• Problems with note taking, synthesising, analysing, critical thinking, reasoning as vocabulary is lacking

• Poor reading development levels

4UNIVERSITEIT VAN DIE VRYSTAAT • UNIVERSITY OF THE FREE STATE • YUNIVESITHI YA FREISTATA Tel (051) 401 3971 • E-mail: hayd.rd@ufs.ac.za • www.ufs.ac.za

Theoretical points of departure

• educational viewpoint: collapse of the secondary school system

• Moral / transformational viewpoint: social responsibility

• Economical stance: costs of failure / shortages in the economy

5UNIVERSITEIT VAN DIE VRYSTAAT • UNIVERSITY OF THE FREE STATE • YUNIVESITHI YA FREISTATA Tel (051) 401 3971 • E-mail: hayd.rd@ufs.ac.za • www.ufs.ac.za

Role of academic language proficiency testing

• Most SA universities : UCT developed PTEEP

• Tests among others: comprehension

• Less than 50% : placed in an academic language proficiency course

• Purpose of our paper is to share findings of an investigation into a non-first language development intervention

6

METHODOLOGY

Analyse UFS data of students from 2006/7, on

• School-leaving outcomes

• English proficiency test outcomes

• English development course outcomes

• End of first year results over all courses

7

3 TYPES OF DATA ANALYSES

• Pre-test vs Post-test Means

Students selected on basis of “vulnerability” by PTEEP (English Proficiency Test), were directed to English Development Course. After about 8 months, underwent similar English Proficiency test; outcomes compared by Student t–test for dependant samples - accounts for intracorrelation.

Result Significant (α=0.5) increase in means of pre-test to post test

But Means were 37% and 47%, whereas maximum scored by anyone in 2006/7 in PTEEP was 90%

And Comparisons of mean differences for particular constraints (e.g. vocabulary) weren’t available – could prove interesting.

8

3 TYPES OF DATA ANALYSES (cont..)

• CORRELATIONSCorrelation matrix of 5 variables (not age) with 1st and 2nd Eng. Proficiency test scores of this group of about 200 students yielded……- No significant correlation between 1st test

score and end-of-year results, but - Significant correlation between 2nd test score and end-of-year results.- Bigger and significant correlation between

2nd test score and score in English Development Course

9

Comparisons among 2006-2007 first-year intakes

The variables are:• PTEEP = Test in English for Educational Purposes (UCT)

• Mark = Mark attained in an English development/basic Language course

• Age = Age in years at enrolment (February)

• Mscore = An aggregated/weighted total of school-leaving results

• Enrol = Number of courses (subjects) for which a student enrolled

• Passed = Number of courses passed during the first year.

• Subj. Pass Ratio = Number of courses passed divided by number of courses enrolled.

• Avg Percent = Average percentage attained over all courses at end of first year.

• Race = African (Black) / White (Caucasian) / Coloured / Asian.

10

Comparisons among 2006-2007 first-year intakes (cont…)

• Summarised statistics for these variables:

Variables Valid N Mean Median Minimum Maximum Std.Dev.

PTEEPmark 1537 52.98 53.00 8 90 14.15

Mark 1343 55.24 56.00 1 86 11.72

Age 3565 20.92 21.00 18 24 1.28

Mscore 3447 28.45 29.00 2 56 8.40

Enrol 3565 9.47 10.00 1 34 4.31

Passed 3315 6.78 6.00 1 34 4.23

SbjPassRatio 3315 0.70 0.75 0.05 1.00 0.28

AvgPercent 3521 51.08 52.00 1.00 94.00 15.25

11

Race as a factor

• One-way ANOVA’s on PTEEP; Mark (EngDevCourse), SubjPassRatio and AvgPercent.

• All means differed significantly (α=0.05) over races.

• Followed-up with post-hoc tests to see which differ.

• Used Scheffe (conservative test) to see which races differ with regard to variables above.

.

12

Race as a factor Example: Scheffe post-hoc table, variable PTEEP score (Marked

differences are significant at p<.05000 )

African and White means differed significantly from each other and from Coloured and Asian groups (significantly higher mean scores) in PTEEP tests.

With end of year outcomes, the White students’ results differed significantly from the other 3 groups, but the paired comparisons for the latter means did not differ that much.

Race {1} {2} {3} {4}

M=47.658 M=66.116 M=58.581 M=57.560

African {1}   0.0000 0.0000 0.0000

White {2} 0.0000   0.0000 0.0000

Coloured {3} 0.0000 0.0000 0.9700

Asian {4} 0.0000 0.0000 0.9700

13

PTEEP scores cut-off points

• Choice of cut-off level • ≤49% - must do English Development Course

• ≥50% - don’t do English Development Course

• Four categories: 0-44%; 45-49%; 50-54%; ≥55%

• ANOVA’s on MARK (in English Development Course); SubjPassRatio; and AvgPercent showed highly significant (α=0.0001) mean differences. Post-hoc tests showed :

14

Test group versus control group

• Compared outcomes for groups who tested below 50% in PTEEP, between those who did an English academic proficiency course and those who should have attended but (voluntarily) did not.

• End-of schools results (composite) not applied in these analyses

15

Test group versus control group (cont’d…)

Distribution of two groups, in AvgPercent, for example.

Box & W hisker Plo t: AvgPercen t

Mean Mean±SD Mean±1.96*SD ENPro f NO NE

LangCrse

10

20

30

40

50

60

70

80

Avg

Pe

rce

nt

16

Test group versus control group cont’d…

Means of average percentage obtained differed significantly (*) between the two groups.

Variable Mean ENProf

Mean NONE

t-value df p Valid N ENProf

Valid N NONE

SbjPass Ratio

0.6729 0.6481 0.8216 721 0.4116 633 90

Avg Percent

50.6867 46.3404 3.0043 740 0.0028* 648 94

17

Essays

• Voluntary essays on students’ own perceptions of benefits of development in academic English proficiency were very positive.

• Extend qualitative investigations to systemic analysis, with random sample of essays and interviewsas well.

18

CONCLUSION

Thank You

HE obliged to invest resources in English Academic Proficiency improvement.

Trade-off between burden of shouldering secondary education’s level shortcomings, taking decade at least,withStudents’ success ….less attrition, efficient throughput, economic gain, less wastage and trauma.

Student academic language proficiency: an investigation into a non-first language developmental

intervention

by

Dr Cay van der Merwe, Central University of Technology, Bloemfontein

Prof Driekie Hay, University of the Free State, Bloemfontein

SAAIR Port Elizabeth

September 2009