Edmund Cannon

Electronic and traditional learning on an econometrics unitSeptember 2009

2 Introduction

Which students use Blackboard (BB) and how do

they use it?

Handouts (Word/pdf documents) versus Power-

point slides.

Possible consequences.

3Estimating treatment effects

Even with a perfectly-designed controlled

experiment, effects would need to be large for

them to appear statistically significant (sample

size 145 or fewer).

I have not performed a controlled experiment so

any correlations will not identify treatment effects.

But we can still learn something...

Nb in “idiot” regressions, the best explanators for

exam performance are lecture attendance and 1st-

year exam mark.

4Powerpoint etc.

Chen and Lin (2008, IREE) argue PP slides

improves student performance: other papers more

cautious.

Students like PP.

Need to think about how they use it.

5What are print-outs of PP slides for?

Probably helps dyslexic students.

May reduce the number of mistakes on some of the

pieces of paper in a student’s file.

May make students more passive (substitutes visual

for kinesthetic learning).

May change a student’s lecture attendance.

May substitute/complement other paper

resources.

i ii

X eé ùê úê úë ûåΕ

6What are (Word doc) handouts for?

Nb assuming a technical unit like

econometrics

Supplementary information to the text book.

Provides material to discuss in the lecture/class.

Lecturer too lazy / arrogant to use text book

properly.

7 QM3 Econometrics

Compulsory second-year unit for most Economics

students

Excluding Erasmus, etc, I analyse 145 students

(whole sample) or 115 students (omit EconMath).

All students have AAB+ at A-level, Maths A-level

grade A/B and have “passed” QM1 (Maths) and

QM2 (Statistics) in the first year.

Most students have a further compulsory

econometrics unit in the third year (not

EconMath).

8 QM3 Econometrics, cont'd

Textbook-based unit testing technical skills

(Murray: Econometrics: A Critical Introduction).

First term overlaps QM2 material - OLS (chapters

1-10).

Second term all new – heteroskedasticity, auto-

correlation, consistency, IV/2SLS, Demand and

Supply (chapters 11-14).

Exam: answer three questions on algebra, stata,

economic interpretation.

Statistics for total BB useage9

QM3 QM2 / Stat

Ever Term 2 Vac Term 3 Ever Term 2 Vac Term 3

I 47.0 8.1 2.5 4.9 47.7 9.6 2.1 4.9

II.i 44.3 8.8 2.0 5.2 41.0 7.6 1.8 4.6

II.ii 43.7 7.7 2.0 4.6 39.6 6.8 1.4 4.5

III 39.6 7.6 1.2 4.2 36.7 5.7 1.5 3.3

F 37.2 6.5 1.2 3.4 33.5 5.3 0.8 3.7

Ordered probit relating class of mark to BB use

is significant [p = 0.018].

10 What did I put on Blackboard?

Exercises (one electronic, the rest paper).

Handouts (Word/pdf documents).

Power-point files for lectures.

Lecture given

PP slides put on BB

Word doc put on BB

Lecture 15 30 Jan 7 Feb

Lecture 16 6 Feb 22 Feb

Lecture 17 13 Feb 22 Feb

Intro to IV 16 Feb

Sargan 16 Feb

Number of times students accessed teaching material

11

Tentative conclusion: students use BB as a

“mobile reference”

Number of times accessed this doc on BB12

QM3 Power-point slides (L15) Handout (Intro IV)

Ever Term 2 Vac Term 3 Ever Term 2 Vac Term 3

I 7.7 4.7 1.5 0.8 2.5 0.9 0.4 1.3

II.i 8.2 5.4 1.2 1.0 2.5 1.0 0.3 1.2

II.ii 7.1 4.5 1.2 0.7 2.2 0.9 0.4 0.8

III 6.7 4.5 0.7 1.0 1.8 0.8 0.1 0.8

F 6.4 4.0 0.7 1.0 1.7 0.8 0.2 0.8

Nb strong correlation between downloading

handout and downloading PP

Number of times accessed13

QM3 Power-point slides (L17) Handout (Sargan)

Ever Term 2 Vac Term 3 Ever Term 2 Vac Term 3

I 1.5 1.1 0.2 0.3 1.1 0.3 0.1 0.7

II.i 1.8 1.3 0.2 0.3 1.1 0.3 0.2 0.6

II.ii 1.3 1.0 0.2 0.1 0.9 0.2 0.2 0.5

III 1.4 1.1 0.1 0.2 1.0 0.4 0.0 0.4

F 1.6 1.0 0.3 0.4 0.7 0.4 0.0 0.2

14First time students accessed PP slides

First time students accessed handout

15

Proportion of students accessed on BB for the first time

16

QM3 Power-point slides (L17) Handout (Sargan)

Ever Term 2 Vac Term 3 Ever Term 2 Vac Term 3

I 0.80 0.70 0.05 0.05 0.70 0.30 0.10 0.30

II.i 0.81 0.68 0.08 0.05 0.68 0.30 0.14 0.24

II.ii 0.69 0.54 0.12 0.04 0.58 0.19 0.04 0.35

III 0.70 0.57 0.08 0.05 0.62 0.30 0.00 0.32

F 0.64 0.44 0.08 0.12 0.52 0.28 0.00 0.24

Conclusion: many students did not access material

that was needed to supplement the textbook.

17 Correlation with exam performance

QM3 mark QM3 mark Chen-Lin

Attend L15 21.7(6.5)

18.6(9.6)

0.064[0.029]

PP L15 26.9(8.5)

17.4(10.8)

0.0228[0.0149]

Attend * PP -19.8(8.8)

-16.5(10.7)

-0.0194[0.0316]

Lecture att 2.3(1.2)

1st year mark

4.9(1.3)

N 145 115

R-squared 0.16 0.31 Panel

18 Conclusions

Positive correlation between using BB and exam

mark – tells us nothing new;

Students access material on BB repeatedly

(reference);

Both good and bad students access material after

course is over (consistent with cramming);

What are students using material for?

19Estimating treatment effects

Consider a perfectly-designed controlled

experiment, where the treatment effect is a

binary variable and the explained variable is exam

mark.

{ }0,1 ; ,

i i i i

i i i i

Y d e

d d e

g= + +

Î ^

X b

X

Suppose the cross-section regression has an R2 of

0.15 (which would be considered reasonable),

that the standard deviation of exam mark is 15%

and the treatment effect raises performance by

5%.

Degrees of freedom must be about 150 for

statistical significance.