Student self-assessment and independent learning in a 1st year mathematics module: identifying and responding to needs and challenges through technology
Peter Morris & Shane Dowdall
Dept. Computing & Mathematics,
Dundalk Institute of Technology
Background
Where the project started…
Background
Where the project started…
Image credit: http://www.promathtutoring.com/regents.html
Background
Where the project started…
SIF 2 ERA ‘Repositioning Learner Assessment Project’ Research
Grants 2009
Educational Context
B.Sc. in Computing
Level 7
Common 1st Year
Approx 100 students
Mathematics for Computing 1
Semester 1 module
Lectures & Tutorials
No Computers!
Student Cohort 67% from Leaving Certificate
1% > 350 points
11% > 300 points
7% < 200 points
Student Cohort 16% Mature Students
7% International
7% Further Education
Issues
Minimum entry requirements (Leaving Certificate Maths):
B2 grade at Foundation Level
D3 at Ordinary level
E at Higher Level.
Mature students not studied maths for many years
Students entering from further education have no maths requirements
Minimum requirements vs. lecturer’s requirements
AimThe aim of this initiative was to
improve mathematical fundamentals,
for a diverse 1st year student cohort,
through the introduction of software
that incorporates
self-directed online tutorials and
assessments for learning.
Key Objectivesa. Inform students of their initial maths competency.
b. Promote independent learning.
c. Offer flexible timing of assessments.
d. Enable students to be assessed quickly.
c. Build students’ confidence in mathematics.
d. Improve engagement in mathematics.
e. Analyse the effect of initiative on students.
Literature Assessment for Learning Petty(2006)
“the most powerful single moderator that enhances achievement is feedback” Hattie (1999)
Benefit of using computers in teaching mathematics
for students who “lack foundation in basic mathematics” Bennett, F. (1999)
Multimedia Learning
Multiple Representation Principle
Contiguity Principle Mayer & Moreno (1998)
Tutorials & Tests Alessi & Trollip (2001)
Literature Variables influencing mathematics
achievement Pajeres (1996)
Cognitive ability, socio-economic status, prior mathematics achievement…
Mathematical self-efficacy the student’s self-belief in
their ability to solve mathematical problems
Ayotola & Adedeji (2009), Hackett (1985), Lent & Hackett (1987)
Mathematical Anxiety occurs when maths “is performed
under timed, high-stakes conditions”, Ashcraft & Moore (2009)
Hoffman (2010), Aiken (1970), Ashcraft (2002, 2005)
What did we do?
Introduced mathematics software
Understanding Mathematics
Visual Tutorials & Cumulative Assessments
Algebra
Fractions
Powers
2-hour weekly supervised computer sessions
20% of assessment mark
Diagnostic Test 1 & Survey 1
Inform students of their initial competency
Surveyed them on their response (40 responses)
About half were “surprised” with their result
About ¼ expected to do BETTER
About ¼ expected to do WORSE
Students did not have an accurate perception of their own self-efficacy
62.5% result acted “as an incentive” to engage with initiative
0% were not incentivised by their result
Test 1 – Sample Questions
Test 1 – Sample Questions
Supervised Sessions
Wednesday 3pm-5pm
Initially two computer labs (about 40 PCs)
Introduce students to software
Students complete tutorials & assessments
Formal assessments:
Students indicate they want to take assessment
Supervised exam setting
Marks are recorded by supervisor
The Software
The Software
The Software
The Software
The Software
The Software
The Software
Evaluation: Diagnostic Tests
Diagnostic Test 1: before initiative
Diagnostic Test 2: after initiative
Both tests were similar:
36 Questions
Paper based
Topics covered by software
No attempt mark
48 students completed both Tests
Evaluation: Diagnostic Tests
For 48 students who sat both tests
Average mark changed from 20.1 to 24.52 marks
12% increase
Evaluation: TestsGroup Test 1 Test 2 Improvem
ent
A – 22 students
20+ 20+ Small
B – 16 students
<=20 20+ Big
C – 6 students
Less than 20
Less than 20
Big
D – 4 students
Less than 20
Less than 20
None
Evaluation: Survey 2
Last 3 weeks of initiative
Gauge perceived changes in
Mathematical Self-Efficacy
Mathematical Anxiety
Mathematical Competency
Get students’ response to software
Evaluation: Survey 2
Are you less fearful of maths?
63% Yes
Has your confidence
in doing maths improved?
70% Yes
Survey 2: Comments
Helping build confidence:
“I as a mature student who had been out of school over 25 years found it very good.It allowed me to practice and gain in confidence.”
“i would advice people to use as it gives you a confidence boost and reminds you of the maths you may have forgotten from school.”
“…good for reminding old stuff from the school”
“i thought it was very useful because i hadn't done maths in appox 8 years…”
Survey 2: Comments
Mixed comments for “weak” students:
“its a good system to cherry pick the parts of each subject you are weak on and brush up on them.”
“it should not be relied upon to teach students who are already weak at maths”
“…favors those already good with maths as they fly through it and takes up a good lot of free time for others not so good.”
Survey 2: Comments
On setting 20% of module’s assessment mark for initiative:
“Don't force students to sit this subject by marking them on it. It is unfair if they have no choice but to sit the classes. They are extra classes meant to help those who don't know how or want to get better at maths. Forcing those who are already competent enough makes things [annoying].”
“i dont like those classes doin them jst because of 20% [otherwise] wouldn't be there”
Reflections & Findings
Decoupling of fundamentals from module content
Positive effect on programme in general
Particularly strong effect on mature students
Time consuming for supervisors
Recording of assessments
Providing guidance in using software
Reflections & Findings
Poor attendance
Delay in installation
Timetabling issues
Supervised session not on timetable
Competing with clubs & societies
Long gaps in timetable
Split of group into two 1-hour slots
Only 40% of students had undertaken any assessments by week 9 (of 12)
Some did all 3 assessments in one session
Issues & Recommendations
Compatibility of software
Issue with IE6
Use of Virtual Machine
Early Installation
Prepare introductory material
Build confidence from start
Spend more time profiling students
Accessibility of Software
Limited access to the labs
No access at home
Must be considered!
Issues & Recommendations
Consider time allocation for supervisor
Video tutorials for introduction to software
Automating capture of results
Identify students who’ll benefit
Is the initiative worth running for all students?
Can you quickly assess students already competent?
Other benefits, e.g. independent learning
Future Considerations
Use of IT Learning Centre ONLY for the delivery of the software
Changing class contact hours for module to include practical lab classes
Other degree programs, e.g. engineering
Access to 3rd level courses
Life-long learning & Schools
Institute-wide induction programmes
Pre-entry skills tests (literacy & numeracy)
Questions
12% increase for group as a whole
38.5% to 66.5% - Group B’s improvement
Mature students gained in confidence
Improvements in mathematical self-efficacy & anxiety
Thank you for listening!
Evaluation: Diagnostic Tests
0
5
10
15
20
25
30
35
40
0 5 10 15 20 25 30 35 40
Test
2
Test 1
ScatterPlot: test 2 vs. test 1
4 no improvement and not proficient (Group D)
Over 61.5% of those who initially <=20
6 improved but still not proficient (Group C)
16 proficient (Group B)
22 proficient (Group A)
