Assessment Portfolios: Answering Users’needs in a Sakai OAE Type

Environment

Jacques Raynauld, Faculty, HEC Montréal and MATI Montréal

Overview

• Short history – link to the community• Sakai OAE vs US• Portfolios and OSP : the needs • Short Demo of a proof of concept (activity)• Building blocks• Competency Gradebook• Road-map

2

Context of the session

• MATI Montréal is a research center for HEC Montréal, École polytechnique and University of Montréal

• Started a strategic initiative on guided- assessment portfolios in 2008 – many schools involved

3

12th Sakai Conference – Los Angeles, California – June 14-16 4

Context of the session

• Actively participated to the Sakai portfolio community

• Presentation at the Sakai Denver Conference : Learning portfolios : a structured approach

• Choose a Sakai OAE type framework for development

• Complementary …knowing that OAE team is working on free form portfolios (NYU)

12th Sakai Conference – Los Angeles, California – June 14-16 5

Sakai OAE vs US

• jQuery Javascript + widgets

• TinyMCE free form editor

• Sling REST API• noSQL back-end

• Google WebToolKit javascript widgets

• Model-based structural editor

• Sling REST API• Jack-Rabbit• Slight interface differences• Use of external open

source tools (ORBEON, EMaEval)

12th Sakai Conference – Los Angeles, California – June 14-16 6

Portfolios and OSP

• Portfolios answer many needs: assessment and presentation (free-form)

• Discussion over the year: personas, sce-narios,use cases mini-specs revolve around • Competency/learning outcomes• Artefact construction by students• Reflection/Feedback• Rubric/Evaluation

12th Sakai Conference – Los Angeles, California – June 14-16 7

Portfolios scenarios

• Although the needs are similar, our experience so far is that all portfolios are slightly different

• Need a very flexible tool!

DEMO

8

Simple demo – proof of concept

• Playing the portfolio• Student• Tutor

• Reports• Creating forms using ORBEON• Creating the portfolio structure• Competency gradebook EMaEval

9

12th Sakai Conference – Los Angeles, California – June 14-16 10

Demo

• MATI portfolio Editor• Orbeon Form Server• Sling

• EMaEval – Competency Gradebook

• Front-end GWT • To create and fill forms• Back-end – data

persistence and workflow

• Manage competency framework, competency related tasks for students, tutors,

Set of Web services th

at

link the different tools

10th Sakai Conference - Boston, MA, U.S.A. 12July 2009

All these entries are activities to be executed sequentially by the actors

10th Sakai Conference - Boston, MA, U.S.A. 13July 2009

Sakai OSP cell interface

KEY BUILDING BLOCKS – A MODEL-BASED APPROACH TO PORTFOLIOS

14

Key building blocks

1. Resources? Types? 2. Assembling the resources: creating activities3. Views (matrices)4. Roles? Instructor, tutor, student5. Workflow 6. Export7. Reports and the competency gradebook

1. Resources

1. Resources• Text• Files (Power Point, PDF, Doc, videos, etc.)• URL• Comments• Forms

16

1. Resources

17

• Learning outcomes/competencies1. Civic knowledge & engagement (local &

global)2. Creative thinking3. Critical thinking4. Ethical reasoning & action5. Foundations & skills for lifelong learning6. Information literacy7. Inquiry & analysis8. Integrative learning9. Intercultural knowledge & competence10. Oral communication11. Problem solving12. Quantitative literacy13. Teamwork14. Written communication

Association of American Colleges and Universities

http://www.aacu.org/

1. Resources

18

• Rubrics (forms)

Association of American Colleges and Universitieshttp://www.aacu.org/

2. Assembling the resources

19

2. Organizing the resources according to a model approach• Structure• Unit• Context• Resources

Root

Structure

Unit

Context

Resource

2. Assembling the resources - OSP

20

Root

Structure

Unit

Context

Resource

Competency 1

InstructionsText file

Submit formForm to submit the artefact (student)

EvaluationGrading rubric form (tutor)

Porfolio

Year1 Competency 1 Competency 2

Year 2 Competency 1 Competency 2

3 - Matrice view

21

Competency 1 Competency 2

Year 1 Link

Year 2

For us, matrices are a view (different presentation) of the information already available elsewhere. Competency 1

InstructionsText fileSubmit formForm to submit the artefact (student)EvaluationGrading rubric form (tutor)

12th Sakai Conference – Los Angeles, California – June 14-16 22

4. Role

• Very important - in any learning activity, the student an be an editor, an evaluator, write comments, etc.

12th Sakai Conference – Los Angeles, California – June 14-16 23

5 - Workflow

• Our workflow are coded manually using Sling but played dynamically played

• Next step: use a workflow standard like XPDL and a workflow engine (Bonita)

12th Sakai Conference – Los Angeles, California – June 14-16 24

6- Export

• Vous have a project to export in the LEAP2A standard

12th Sakai Conference – Los Angeles, California – June 14-16 25

7. Reports and competency gradebook

• EMaEval is an open source environment that is specialized for competency evaluation (Pentila – Savoie - used extensively in France) • Competency framework• List of competency related activities with actors

involved• Tracks and saves the results• Can generate reports and certificates

(accreditation)

Highly flexible model based

approach – can tackle very

difficults scenarios

12th Sakai Conference – Los Angeles, California – June 14-16 26

6. Reports and competency gradebook

• We are going to write specialed Web services to leverage all the EMaEval possibilities

12th Sakai Conference – Los Angeles, California – June 14-16 27

Road-map

• Proof of concept is working fine – we understand the whole chain

• Small Pilot with the engineering school in September (faculty of education?)

• Numerous smaller pilots with our mock-up engine

• More stable and robust application next year

12th Sakai Conference – Los Angeles, California – June 14-16 28

Road-map

• We should report on the various experiments

• Any questions? Comments? Ideas?