The Problem with Reflection: Lessons Learned

Alfredo Gaitan (alfredo.gaitan@beds.ac.uk_University of Bedfordshire

Cathy Buyarski (cbuyarsk@iupui.edu) Indiana University-Purdue University Indianapolis

Presentation delivered at the AAEEBL meeting 25-28 July 2011.

Session Overview• Introduction of presenters and portfolios

• Three sections:– Using reflection – institutional contexts and overview of portfolios

– Assessing reflection – how, when, how well

– What we learnt about reflection – in our projects and from each other

• We will aim to share two different projects/ perspectives, e.g.– Campus versus department

– Freshman versus across enrolment

– Summative assessment versus formative assessment

– Insights on the students’ experiences of constructing eportfolios

Institutional contexts of PDPIUPUI

Personal development planning is a process which will enable first year students at IUPUI to understand, implement, and mark progress toward a degree and career goal by creating and following a personalized plan that is open to revision and reevaluation every semester in collaboration with an academic advisor, faculty member, or other mentor.

RISE Each undergraduate student is challenged to include at least two of the four RISE experiences - research, international, service learning, and experiential learning - into their degree programs.

Principles of Undergraduate Learning

UoB

• E-portfolios fit with the PDP model of learning, support the personalisation agenda and can also serve to structure learning for students studying remotely from the University. “We will further develop the use of e-portfolios as a framework to enable all students to record and review their progress and e-PDP as a means for structured and supported development.”

• CRe8. The ‘realistic learning’ dimension emphasises a learning experience which is meaningful, active, reflective, collaborative and

challenging.

Overview of ePortfoliosIUPUI

• Implemented in first-year seminar course across the entire campus (342 students in fall 2010).

• Reflection (with very specific guided prompts) in seven areas:

• About Me• Educational Goals and Plan• Career Goals• Academic Showcase• Campus and Community

Connections• My College Achievements• Resume

UoB

• Implemented at levels 1, 2, 3 of the Psychology curriculum with 225 students In Year 1 and 342 in Year 2.

• Reflection (with prompts) in a optional structure:

– Background, – My Learning Experiences – My Learner Development– My Employability (includes a

resume)– Showcasing my performance

Portfolio Examples

Example

Examples

IUPUI • UoB

Assessment of ReflectionIUPUI

• Assessment to date has been formative and designed to improve prompts and rubrics.

• Main assessment of the First Year Seminar (1 credit): 100%.

• Assessed at the level of each prompt.

• Rubrics based Bloom’s Taxonomy.

• Feedback provided to students on each section of portfolio.

• Conducted through content analysis process.

UoB• Assessment was summative of

students’ planning, experience, reflection, prospective view.

• Part of the assessments in each of three units (30-credit modules): 5%, 20%, 10%.

• Assessed at the level of sections or whole portfolio.

• Rubric loosely based on experiential learning (e.g. Kolb).

• Feedback: comments written on each page/section and the portfolio.

• Moderation in some units.

Lessons LearnedIUPUI

N Avg. H.S. GPA

Avg. Fall GPA

 % Fall GPA

below a 2.0

Fall DFW Rate

Fall –Spring

Retention Rate

ePDP346 3.32 2.95 13% 12.10% 91%

Not e-PDP1936 3.30 2.78 18% 17.23% 89%

Overall 2282 3.30 2.81 18% 16.45% 89%

Note 1: Missing cases were excluded from analyses. Note 2: Students who Withdrew or who were Administratively Withdrawn from Seminars were excluded (N=66 students). Note 3: Bolded items are significantly different based on independent samples t-test or chi-square results (p < 05). 

Table 1: 2010 e-PDP Compared to Not E-PDP First-Year Seminar Sections: Student Characteristics and Academic Success Indicators.

Lessons Learning University of Bedfordshire

Table 2. Submissions of e-portfolios over the two years of the project.

Year 1 (2008-9) Year 2 (2009-10)Level 1 49 (50%)

(n=96)

134 (75%)

(n=184)Level 2 64 (72%)

(n=89)

66 (91.7%)

(n=72)Level 3 8 (11%)

(n=70)

74 (86%)

(n= 86)Total 121 (53.8%)

N=225

274 (80.1 %)

N=342

Lessons LearnedIUPUI

• Assessing reflection in developmentally appropriate ways

• Scaffolding the use of reflection• Faculty development

• Rubrics need to include cognitive development AND critical thinking

• Grading versus assessment

Lessons Learning University of Bedfordshire

An emergent ‘grounded theory’ 1 of the students’ experiences: interconnected themes

(1) Glasser & Strauss, 1967; Pigeon, 1996; Pigeon & Henwood, 1997)

What We Learnt from Collaborating

Cathy• Moving the PDP to embed

in academic programs is an important next step

• Faculty buy-in and development will be key factors in our success

• Use of themes from UoB will inform our thinking about student participation

Alfredo• The eporfolios at UoB could be

more directly linked to the coursework.

• Our portfolios are snapshots of a year, and only the Year 3 takes a long view (3 portfolios rather than one).

• The strong ethos and framework (e.g. RISE) may provide an environment favorable to e-portfolios.

• What can eportfolios do for your institution?

Let’s talk!

• Any questions?• What have you learnt from your own

experiences that you can offer us?

Thank you!