Mathema’cs curriculum, assessment and teaching for living in the digital world: Computa’onal tools in high stakes assessment Kaye Stacey University of Melbourne [email protected]Paper presented at Third Interna?onal Mathema?cs Curriculum Conference under the auspices of the Center for the Study of Mathema?cs Curriculum (CSMC), on the theme “Mathema?cs Curriculum Development, Delivery, and Enactment in a Digital World”, November 79, 2014, University of Chicago.
Transcript
Mathema'cs curriculum, assessment and teaching for living in the digital world:
Paper presented at Third Interna?onal Mathema?cs Curriculum Conference under the auspices of the Center for the Study of Mathema?cs Curriculum
(CSMC), on the theme “Mathema?cs Curriculum Development, Delivery, and Enactment in a Digital World”, November 7-‐9, 2014, University of Chicago.
Computational infrastructure for doing mathematics
Sketch of computa'on changes
End of school and university entrance Year 12 examina'ons, Victoria, Australia
Smooth shiD to graphics calculators 1995 • Advantages
– Up-‐to-‐date – Easy access to dynamic graphs in class and for student use – Quick numerical solving of equa'ons possible from tables and graphs, which led to a flourishing of numerical approaches
– BeNer sta's'cs calcula'ons and data handling – Mul'ple representa'ons
• Barriers – Cost to students and hence concern with equity – Need for teacher professional development to learn to use technology and to teach with it
– Need to change a few examina'on ques'ons – Some concern about students losing pen-‐and-‐paper capability e.g. to plot and sketch graphs
Changing examina'on ques'ons BEFORE Which of the following four graphs below shows the deriva've func'on of f(x) = x(x-‐1)(x+1)(x-‐2)? AFTER Which of the following four graphs shows the deriva've of the func'on in this sketch?
func'on
correct
CHOICES
Changing examina'on ques'ons – exact rather than numerical answers, more parameters
BEFORE Find turning points of f(x) = x(x-‐1)(x+1)(x-‐2) (Use calculus) AFTER Find exact turning points of f(x) = x(x-‐1)(x+1)(x-‐2) x = (sqrt(5)+1)/2, x = (-‐sqrt(5)+1)/2, x = ½
f(x) = x(x-‐1)(x+1)(x-‐2)
Zooming in on a turning point
Handheld CAS by late 1990s
• Compelling reasons to use it – Up to date – Could do more mathema'cs – Pedagogical opportuni'es
Transi'on to symbolic algebra has bigger impact on: • Curriculum
– Challenges core of course content: algebraic manipula'ons, differen'a'on and integra'on, trigonometry
• Teaching – harder for teachers and students to learn to use – poten'ally more pedagogical opportuni'es, related to symbolic work in addi'on to graphing and other capabili'es
Changing examina'on ques'ons – exact rather than numerical, more parameters
BEFORE Find turning points of f(x) = x(x-‐1)(x+1)(x-‐2) But these can be found from the graph. AFTER Find exact turning points of f(x) = x(x-‐1)(x+1)(x-‐2) x = (sqrt(5)+1)/2, x = (-‐sqrt(5)+1)/2, x = ½
symbolic algebra with calculus Zooming in on a turning point
Introduc'on of MM(CAS) “Mathema'cal Methods (CAS)” • Research project 2000 – 2002 CAS-‐CAT
– (Stacey, McCrae, Ball, Flynn, Leigh-‐Lancaster et al) • Studied curriculum, assessment and teaching • Parallel content to MM, with a few expansions
– Same func'ons and calculus core with sta's'cs etc – Fewer restric'ons on testable func'ons (e.g. absolute value func'on)
– Transi'on matrices added – Con'nuous probability added
• First Year 12 examina'ons in 2002 • 3 volunteer schools 2000 – 2002, then expanding pilot • 3 CAS brands, expanding range but s'll limited • KS – Chicago CSMC 2008 and USACAS – experiences in
the project
Graphing Calculator Capabili1es for AP Calculus AB (2015) • Examiners assume student access to four calculator capabili'es:
– Plot the graph of a func'on within an arbitrary viewing window – Find the zeros of func'ons (solve equa'ons numerically) – Numerically calculate the deriva've of a func'on – Numerically calculate the value of a definite integral
• When using one of the four capabili'es above, students – write the setup (e.g., the equa'on being solved, or the deriva've or definite integral being
evaluated) that leads to the solu'on, – along with the result produced by the calculator.
• For solu'ons obtained using other calculator capability, students – must also show the mathema'cal steps that lead to the answer; – a calculator result is not sufficient.
• Exam ques'ons do not favor students who use graphing calculators with more features.
Difference: MM(CAS) allows all calculator func'ons to be used
Changes since pilot • MM(CAS) and MM ran in parallel 2002-‐ 2010 • Two subjects merged (using CAS) in 2010 • Technology-‐free examina'on (Exam 1) introduced in 2006 (same items for MM and MMCAS)
• CAS permiNed with Specialist Maths (higher level) from 2010, also with technology-‐free examina'on
• Substan'al professional development for teachers • Teachers interested to use CAS from Year 9 • LiNle use of other computa'onal technology in senior years (e.g. dynamic geometry)
Sketch of computa'on changes
CAS Graphics Scien'fic Log tables
Comparing student performance with and without CAS – studies mainly by examiners and examina'on authority • Evans, M., Norton, P., & Leigh-‐Lancaster, D. (2005). Mathema'cal Methods Computer Algebra System (CAS) 2004
Pilot Examina'ons and Links to a Broader Research Agenda. Proceedings of 28th conference of the Mathema'cs Educa'on Research Group of Australasia. hNp://www.merga.net.au/documents/RP342005.pdf
• Forgasz, H., & Tan, H. (2010). Does CAS use disadvantage girls in VCE Mathema'cs? Australian Senior Mathema?cs Journal, 24(1), 25–36.
• Leigh-‐Lancaster, D. (2010). The case of technology in senior secondary mathema'cs: Curriculum and assessment congruence? Proceedings of 2010 ACER research conference. (pp.43 – 46). hNp://research.acer.edu.au/cgi/viewcontent.cgi?ar'cle=1094&context=research_conference
• Leigh-‐Lancaster, D., Les, M., & Evans, M. (2010) Examina'ons in the Final Year of Transi'on to Mathema'cal Methods Computer Algebra System (CAS) Proceedings of 33rd conference of the Mathema?cs Educa?on Research Group of Australasia. hNp://www.merga.net.au/documents/MERGA33_Leigh-‐Lancaster&Les&Evans.pdf
• Leigh-‐Lancaster, D., Norton, P., Jones, P., Les, M., Evans M., & Wu, M. (2008). The 2007 Common Technology Free Examina'on for Victorian Cer'ficate of Educa'on (VCE) Mathema'cal Methods and Mathema'cal Methods Computer Algebra System (CAS) Proceedings of 31st conference of the Mathema?cs Educa?on Research Group of Australasia. hNp://www.merga.net.au/documents/RP382008.pdf
• Norton, P., Leigh-‐Lancaster, D., Jones, P., & Evans, M., (2007). Mathema'cal Methods and Mathema'cal Methods Computer Algebra System (CAS) 2006 -‐ Concurrent Implementa'on with a Common Technology Free Examina'on Proceedings of 30th conference of the Mathema?cs Educa?on Research Group of Australasia. hNp://www.merga.net.au/documents/RP492007.pdf
• Zoanex, N., Les, M., & David Leigh-‐Lancaster, D. (2014). Comparing the Score Distribu'on of a Trial Computer-‐Based Examina'on Cohort with that of the Standard Paper-‐Based Examina'on Cohort. Proceedings of 37th conference of the Mathema?cs Educa?on Research Group of Australasia. hNp://www.merga.net.au/documents/merga37_zoanex.pdf
Do students with CAS lose by-‐hand skills? • Data from the no-‐technology exam 2006 – 2009 comparing MMCAS with MM students
• Slightly beNer results for MMCAS students each year – Similar percent of students get high scores – Fewer MMCAS students get very low scores – Mean of middle MMCAS students is slightly higher
• Effect persists when controlled by – general ability test score (math/science/tech component) – Overall score in all Year 12 examina'ons
• 2009: 7189 MMCAS and 8887 MM students
How does using CAS rather than GC affect performance? • Studied technology-‐permiNed exams 2006 – 2009 • Items classified as
– technology independent • e.g finding the maximal domain of log|x-‐b|
– technology of assistance but neutral with respect to graphics calculators or CAS
• e.g. finding the numerical probability of 8 or more heads in tossing a coin 10 'mes
– use of CAS likely to be advantageous • e.g. solving |2k+1| = k + 1
• Compared common items (about half items of exam 2)
How does using CAS rather than GC affect performance? • CAS students slightly beNer in every year 2006 -‐ 2009 • 2006 had 35 common ques'ons
– total score on 35 ques'ons: MMCAS > MM – MM > MMCAS on 2 ques'ons – MMCAS > MM on 12 ques'ons, including items which are technology independent, neutral or CAS affected.
• Examiners/researchers reported use of CAS avoided simple algebraic errors (oDen used for checking) and so assists students to engage with further parts of ques'ons.
How does using CAS rather than GC affect performance? (controlled) • 2009 tech-‐permiNed exams, taking scores on common
items (excluding CAS affected) – 17 of 22 mul'ple choice ques'ons – 21 of 32 extended answer ques'ons
• Rasch regression model, with score on no-‐tech exam as measure of ability
• Outcomes: – No difference at the top end – MMCAS > MM for all other abili'es – Peak difference for mul'ple choice items for below average students (-‐1 logits)
– Peak difference for extended response items for students slightly beNer than average (+0.5 logits)
Should by-‐hand skills be the covariate, or the target? That reverses the interpreta'on!
No technology
With
techno
logy
Why were MMCAS students slightly beNer? • Sampling bias? Probably not, especially by 2009.
– Schools transi'oning first may be adventurous ??? – But very high-‐scoring schools tended to be slow – Effects persist even when controlled for ability in three ways
• Evidence from examina'on scripts – CAS students avoid careless errors and so get further into items – CAS students become more accurate in CAS/GC entry
• 2004 solve numerically ln(x+1) = 1 – x to two dec. places • MMCAS 90% correct, MM 80% correct • CAS entry not easier than GC– more aNen'on to detail?
• Possibili'es of beNer learning as research predicts
Gender Equity – does using CAS instead of GC disadvantage girls? • Some reports e.g. Forgasz & Tan 2010 • In all examina'ons
– More girls get good but not excellent grades – More boys than girls always score very badly – Biggest gender gaps are in the A and A+ range and at the boNom
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
MM MMCAS
Excess % of boys over girls receiving A and A+ grades (2009)
Exam 1 -‐ no tech
Exam 2 -‐ gc or cas
Gender Equity – does using CAS instead of GC disadvantage girls? • Puzzle with unanswered ques'ons:
– why would using CAS affect girls more than using GC? – why would girls learn by-‐hand algebra worse in CAS environment?
• Gender differences in axtudes – boys like using technology for maths; – girls value it for what it can do (Pierce, Stacey, Barkatsas 2007)
• Australian gexng a gender gap – significant in recent TIMSS Gr 4 and 8 and PISA 15yr olds
0 0.5 1
1.5 2
2.5 3
3.5 4
4.5
MM MMCAS
Excess % of boys over girls receiving A and A+ grades (2009)
Exam 1 -‐ no tech
Exam 2 -‐ gc or cas
Comparing performance with calculator or computer CAS • Pressures
– Schools and parents do not want to buy dedicated devices for maths – Administra've benefits by moving from paper examina'ons – Some schools already using Mathema'ca
• 2013 first trial – 62 volunteer students from 5 schools – Examina'on 1 same – Examina'on 2 delivered and answered in Mathema'ca notebook
• Results – No significant mode effect in predic'on of Exam 2 score from Exam 1
score, with and without general ability score in regression. – Only one item showed sta's'cally different DIF (iden'fying a variable
normally distributed and finding an associated probability). No apparent reason for difference.
How is CAS used at school? • Pierce and Ball – large teacher survey 2009
– teachers generally op'mis'c about effects on teaching and learning – 25% strongly concerned about effect on by-‐hand algebra skills – 25% feel need to learn to teach CAS use takes too much 'me from
effec've mathema'cs instruc'on – these groups of dissen'ng voices overlap and many mid-‐career teachers
• Pierce & Stacey (2013) – studied ‘early majority’ adopters with strong school support – hard to learn technology – transi'on in teaching prac'ces extremely slow – tendency to use CAS like a ‘calculator’ – just step by step assistance with
by-‐hand solu'on – and advances in technology means they need to keep learning – students ask more ques'ons (but about technology rather than maths –
less ‘shame’?)
Many teachers use CAS in Years 9/10
Marina’s Fish Shop Finding the fish sign with the minimum area – an inves'ga'on with quadra'c func'ons
How is CAS used at school?
• Pierce and Bardini – 2013/2014 survey – First year Uni Melbourne maths students – Generally very good students – 334 of 2000 answered survey and used CAS in Yr 12 exam
• Students asked about their own use of CAS and their percep'ons of teachers’ use of CAS in class – Learn by hand skills – Use real data – Explore regularity and varia'on – Simulate real situa'ons – Link representa'ons
FUNC T IONAL OPPORTUNIT IE SPrimary purpose and strength
Execute algorithms quickly and accurately
PE DAGOG IC ALOPPORTUNIT IE S
Exploit contrast of ideal & machine
mathematics
Build metacognitionand overview
Explore regularity
and variation
Learn pen-and-
paper skills
Link representations
Change classroom
didactic contract
Change classroom
social dynamics
Pedagog ic al map for
mathematic s analys is s oftware
Use real data
Re-balance emphasis on skills,
concepts, applications
C URR IC UL UM C HANGE
AS S E S SMENT C HANGE
Simulate real
situations Tas
ks(Impr
oved
spe
ed,
acce
ss, a
ccur
acy)
Class
room
(Impr
oved
display
, per
sona
l autho
rity)
Sub
ject
(Re-‐as
sess
ed
goals & m
etho
ds)
Students’ percep'ons of teachers’ use
• Teachers’ in-‐class demonstrated uses – Graphing clearly main use (over 50% oDen) – Then matrices, solving equa'ons, applica'ons (26% oDen) – Not oDen for tables or hard algebra, not much to support teaching by-‐hand skills
• PaNern of use indicates diffusion of innova'on may have stalled – a few teachers use CAS oDen in most topics – a small percent use CAS oDen in some topics and occasionally in most
– the majority is skewed towards liNle use – a few have almost no use
Students’ use more than teachers’: consistent finding across all studies
Pierce and Bardini, in press
One teachers’ reflec'ons (12 years) on self, colleagues & beyond (Sue Garner) • Common teaching approaches
– Some colleagues show liNle change • use CAS algebra only as answer checker • avoid ‘explosion of methods’
– Self and others have established different norms • turned teaching around, to start with purpose and applica'ons, and then look at techniques in detail
• TENDSS: “Teaching the ends and sides of a topic with CAS” • Celebrate ‘explosion of methods’ with more student input
• Sue’s students’ reac'ons – “[CAS] is a friend that is useful some'mes, annoying some'mes and plainly a waste of 'me at other 'mes. The task is to find when! “ (Student C and typical of successful students)
– “I s'll want to marry my CAS.” (Student M)
Garner and Pierce (in press) CAS: More than ten years on
Examiners’ play “beat the CAS”
Also played with mul'ple choice distractors
Consequences for students
• Garner’s categories of students – “Stayers”: steadily achieve in all environments – “Resenters”: CAS devalues their skills – “Flyers”: find and enjoy new solu'on methods, rela'onships and paNerns with CAS (“explosion of methods”)
– “Enabled”: CAS compensates for unreliable by-‐hand algebra
• Early CAS classes (without no-‐tech exam) had small but significant numbers of “enabled”. Sue says these students have gone.
Reflec'ons • Successful innova'on
– accepted, fair, con'nuing (even with new Australian curriculum) – many teachers and students finding it very rewarding – teachers adapt the innova'on to their personal preferences and values for teaching mathema'cs
– teachers finding some uses as early as Year 9 • Hard for many to learn to use and teach with CAS
– also need to regularly update and to learn associated technology e.g. computer presenta'on
– aggravated by equipment problems even in well equipped schools
– but great progress overall • “No-‐tech exam” has mollified fears, and has emerged as
standard model around the world – “Beat the CAS” items keep focus on by-‐hand algebra – Students can maintain comparable by-‐hand skills with CAS
Reflec'ons • No-‐tech exam has stalled thinking about impact on curriculum and
purpose of mathema'cs – some ques'ons are harder, but not doing more in a purposeful way – no more real world applica'ons than before – not much shiD towards analy'c rather than GC numerical solu'ons taking
advantage of symbolic algebra – examiners need new skills , but harder ques'ons mean students fail (Brown 2010) – gradual decline in student numbers
• happening around Australia so not caused by use of CAS • could it be arrested by a curriculum which deals with real, real world problems
– s'll not using technology fully as an amplifier rather than to compensate for inadequate algebraic skills
• Technology environment con'nues to change – BYOD, special purpose apps with ‘wizards’, internet CAS (e.g. Wolfram Alpha),
Geogebra and other free tools, internet sta's'cal analysis, …. – Mul'-‐page open tools with built-‐in computa'on
• Over 30 years since mμmath! – philosophical challenges unresolved in teachers’ minds – prac'cal challenges and opportuni'es keep changing
Specialist Mathema'cs 2013 (with CAS)
Ques'on 3 from Examina'on 2, Specialist Mathema'cs 2013, Victorian Curriculum and Assesssment Authority.
