Towards an Open Assessment Ecosystem

Adam Blumadam@opened.io

@openedio

OpenEd Jam Conference 2014San AntonioJuly 26, 2014

What’s Going On In Assessments

• Open educational resources are thriving• OERs such as OpenEd, Gooru, WatchKnowLearn

make quality K-12 videos and games available for free

• But not formative assessments crucial for mastery-based learning implied by CCSS• Virtually no free formative assessment content

• SBAC and states recognize this but have been slow to fill the gap

• Most districts only have one item bank

• Assessment item types have evolved via CCSS/Smarter Balanced/ PARCC

What Exists Today

• standards for interoperable content– QTI, SCORM, SIF

• some older free tools–Hot Potatoes, eXe

• some expensive current tools–QuestionMark, Respondus– paid LMSes – expensive in costs AND complexity

• expensive item banks–NWEA, McGraw Hill, HMH Riverside, Pearson– not available to individual teachers

QTI 2.1 Conformant Authoring Tools

•Onyx editor– 278 euros/seat, 3998 euros/institution

• Mocah QTI editor– http://qti-work.lip6.fr/soumission/IMS-QTI-editeur/README.pdf

–Single Response, Multiple Response, Free Response

• Respondus? –Not listed. Doesn’t load QTI

• Canvas– Failed to load QTI samples– No Technology Enhanced Items– Otherwise pretty good – but embedded only in larger LMS

SCORM Quiz Creators

• Articulate Quizmaker– $699

• WonderShare QuizCreator– $129

•iSpring Quizmaker– $397

• eXeLearning– open source, no QTI

But SCORM doesn’t really standardize item content

What Needs To Exist

• free “modern” assessment items• free authoring tools • free grading/analytics• web-based (no client install) for easy access

• available hosted• all open source• exchangeable, loadable content

Choose Resources

Make/TakeCourses

Assess

Analyze

What OpenEd.io Does

OpenEdCatalog

Teach, Assess and Analyze with the Largest K-12 Resource Catalog

Next Gen OER

Flipped Class LMS

Quizzing Tools

Mastery Charts

OpenEd.io’s Content Mission• Largest K-12 educational repository

– resources aligned to all standards– infuse all teaching with use of resources– free for teachers and users– paid for fees from paid content providers

• But where are the assessment items?– If they are out there OpenEd would link to them – free or paid

– But the free items aren’t out there– Even paid items aren’t easily accessible by individuals

OpenEd.io’s Assessment Tool

• free and open source• web-based - hosted at www.opened.io• modern item types

– Select Response, Multiple Response, Free Response

• loads QTI content• offering multiple item banks from partners

–With teacher approachable payment model

• “resource-backed”–Resources to students based on the questions they miss

Check out the Hangout demo: https://plus.google.com/events/cp6jdts1vfjcino3c3ivgqjqo44

What Educators Can Do To Help

•Create assessment item content– free or paid, but easily available– Creative Commons when possible– Exchangeable and loadable in LMSes (QTI – But hosted somewhere in some form

• Fullfledged assessments for all standards

– mix of DOK, Difficulty– “bitesized” appropriate for formative (5-10 items)

•Guidelines for new items OPENED PROVIDES CREDIT FOR SUBSCRIPTION TOCONTRIBUTORS

What Developers Can Provide

• new Technology Enhanced Item types–Not necessarily just the SBAC/PARCC ones

–Richer Technology Enhanced Items for specific disciplines: math, physics.

• better interop formats–QTI is not good XML design

– SIF has virtually no adoption for assessments

– only 2 participants on IMS program groups

– assessment providers are engineering own formats

– how about JSON-based?

• hosted grading service– broker assessment response to graders, software assistance for grading

OPENED WILL PROVIDE GRANTS FOR DEVELOPERS THATWANT TO TACKLE ANY OF THESE OPEN SOURCE

Tell Us What You’ve Done

• adam@opened.io• @openedio• @adamblum

Thank You

Assessment Interop Requirements

• assessment item content• assessment assets• item accommodations• item metadata

–Standards alignment, perf statistics, usage/exposure data, relationship to other items

• test forms (complete assessments)• scoring/rubric• reporting/results

Assessment Interop References

• http://www.imsglobal.org/cc/statuschart.cfm

• http://specification.sifinfo.org/Implementation/NA/3.1/

• http://images.pearsonassessments.com/images/tmrs/tmrs_rg/AssessmentInteroperabilityStandards_FINAL_111710.pdf?WT.mc_id=TMRS_Next_Generation_Assessment_Interoperability_Standards