Learner Enhanced TechnologyUnderstanding engagement as a measurable processJames Ballard

jameslballard

JamesBallard2

@jameslballard

My Research QuestionsIs engagement

something we can measure?

Should we be measuring it?

Should it constitute formal assessment of teaching?

“As we start a new school year, Mr. Smith, I just want you to know that I’m an Abstract-Sequential learner and trust that you’ll conduct yourself accordingly!”

OverviewIntroduction

• Icebreaker activity• Why is engagement important?• Where does the metric idea originate?

Activity 1 – What should we look for?• Introduce an engagement matrix• Each group to identify and discuss engagement patterns

Activity 2 – How can we measure this?• Introduce an engagement framework metric• Each group to explore a data driven case study

Conclusion/Discussion

4

Icebreaker ActivityShould engagement constitute a formal assessment of teaching?5 mins – introduce yourselves and give initial reactions to this question

5

Introduction 1Why is engagement important?

Improved Learning Outcomes

General abilities and critical thinking

Practical competence and skills transferability

Cognitive development

Self-esteem, psychosocial development, productive racial and gender identity formation

Trowler and Trowler (2010)

Multicultural Classrooms

UNESCO (2014): Global Flow of Tertiary Students

Learner Voice

Teacher

Learner

Spontaneous

Concepts

Creative Investigativ

e

Learner’s World

Scientific Concepts

Creative Directive

Ballard & Butler (2015)

9

Introduction 2Where does measuring engagement come from?

Engagement Surveys

Economic or social inequality

Issues of race, ethnicity, or nationality

Religious or philosophical differences

Different political viewpoints

Issues of gender or sexual orientation

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

During the current school year, about how often have you had discussions about the following?

Never Sometimes Often Very often

NSSE 2014: Experiences with different perspectives

Involvement

Interaction Intimacy Influence

Social Marketing: Engagement as Metric

What to track

• Site visits• Time spent• Pages

viewed• Search

keywords• Navigation

paths• Site logins

• Contributed comments to blogs

• Quantity / frequency of written reviews, blog comments, forum posts

• Sentiment tracking on 3rd party sites

• Sentiment tracking of internal sites

• Opinions expressed in customer calls

• Net Promoter (NP) score

• Service satisfaction ratings

• Brand affinity

• Content forwarded to friends How to track

• Web Analytics

• eCommerce• Social

Media

• Surveys• Brand

monitoring• Service

calls

• Surveys• Brand

monitoring

Haven & Vittel (2008)

Haven (2007)

Social Marketing

Haven (2007)

13

Activity 1What would we want to measure?

ENGAGEMENT TYPESFREDRICKS, BLUMENFELD AND PARIS (2004)

ENGAGEMENT POLESTROWLER AND TROWLER (2010)

• Tends to result in expected learning behaviours

Positive Engageme

nt

• A passive rejection of such behaviours – alienation or inertia

Non-engageme

nt

• An active and hostile rejection of expectations

Negative Engageme

nt

Dimensions of Engagement

• Participation in academic and social activities

Behavioural

• Feelings towards staff and peersEmotional

• Willingness to invest effort in complex tasks and mastery of skills

Cognitive

POSTIVE ENGAGEMENT NON-ENGAGEMENT NEGATIVE ENGAGEMENT

BEHAVIOURAL

Attends lectures, participates with enthusiasm

Skips lectures without excuse

Boycotts, pickets or disrupts lectures

EMOTIONAL Interest Boredom Rejection

COGNITIVE Meets or exceeds assignment requirements

Assignments late, rushed or absent

Redefines parameters for assignments

5 mins - Feedback an example to the wider audience

15 mins – Discuss: “how do we recognise when students are engaged or not?”

Discussion Activity: Small Groups

Examples and prompts:

16

Activity 2How can we measure this?

Involvement

Interaction Intimacy Influence

Appropriating Engagement Metrics

What to track

• Site visits• Time spent• Pages

viewed• Search

keywords• Navigation

paths• Site logins

• Contributions to class

• Quantity and frequency of content

• Type of content accessed

• Learning dispositions

• Discourse tracking Social network analysis

• Evaluation surveys

• Links to outside world

• Extra-curricular contributions

• Practice based research and reflection

How to track

• System Logs

• Web Analytics

• System Logs

• Learning Design

• Surveys• Content

analysis

• Student Development Model

ACTIVITY CASE STUDY AND STUDENT PROFILE

Discussion Activity: Small Groups

Provides real student data

Make notes in the student profile

Mark each component from low to high

10 mins - Feedback for the whole group

30 mins – Discuss the case study and complete the 2 student profiles in the worksheet provided

INVOLVEMENT

INTERACTION INTIMACY INFLUENCE

• Average activity profile

• Regular access throughout year

• Accessed different support areas

• Interacts with peers via messaging

• Some portfolio building

• Few course based activities

• Enthused by university visit and tour – not sustained

• Potential surface learning strategy

• Limited to small social group - connected to hub

• Use of 1-to-1 messaging

• No shared links

Medium Low Medium Low

Student Profile 1: Jo March

INVOLVEMENT

INTERACTION INTIMACY INFLUENCE

• High activity profile

• Regular access across 2 years

• Interacts predominantly with course materials

• Several assignment submissions

• Potential strategic learning strategy

• Peer interactions not present

• No shared links

High Medium Low Low

Student Profile 2: Oliver Twist

Further Research

Final thoughts

Is engagement something we can measure?

Should we be measuring it?

Should it constitute formal assessment of teaching?

Learner Enhanced Technology Understanding engagement as a measurable process

Thank You

James Ballard

jameslballard@outlook.com

JamesBallard2

@jameslballard

jameslballard