Bazley Developing And Evaluating Online Resources

Martin Bazley www.ICT4Learning.com

Developing and evaluating online resources

Bits 2 Blogs conferenceMiddlesbrough

5 Feb 2009

Martin BazleyMartin Bazley & Associates

www.martinbazley.com


Intro: Martin Bazley• Consultancy / websites / training /

user testing ICT4Learning.com• Chair of E-Learning Group for

Museums, Libraries and Archives link

Previously:

• E-Learning Officer, MLA South East (3yrs)

• Science Museum, London, Internet Projects


View slides online

www.slideshare.net/martinbazley

About producing material for the web:

- why it is different

- why this matters

Why do I need to learn how to write for the web?

I can write perfectly well already, thank you very much…

The web is different

People use the web differently…

… from the way they use books, object labels, magazines, newspapers,

information screens, etc

For most people the web is a

predominantly visual

medium

We are all different and some people like to

read all the text on a web page before

deciding what to do next, even though a lot of

it might be pretty redundant but most

people – or at least most regular users of

the web – scan (as opposed to reading through them in detail) the web pages they are using, or at least the

ones where they are still trying to work out

where to go next

eyetracking study, recorded 232 users, 000’s of Web pages. Users' main reading behaviour was fairly consistent dominant reading pattern looks like an F :

http://www.useit.com/eyetracking/

http://www.useit.com/eyetracking/

Some exampleshttp://www.nationalarchives.gov.uk/gettingstarted/

in_depth_guides.htmhttp://www.nationalarchives.gov.uk/gettingstarted/in_depth_guides.htm

http://www.manchester.gov.uk/site/scripts/documents.php?categoryID=448

http://www.manchestergalleries.org/the-collections/highlights-of-the-collection/narrativeobject.php?irn=876

http://www.nottingham.ac.uk/mss/

http://www.nhm.ac.uk/nature-online/british-natural-history/index.html

Exercise: Make this web page Exercise: Make this web page betterbetter

Oregon History ProjectOregon History Project activity 1• Edit/redesign the home page

=> flip chart• Edit/redesign the

How to use this site: Teachers and Students page=> flip chart

www.ohs.org/education/oregonhistory

Oregon History ProjectOregon History Project activity 2• Review structure of

Subtopic : Introduction: Themes for an Urban History

About website structure, About website structure, ways people use the web and ways people use the web and implications for writing for the implications for writing for the

webweb

Certain types of websites impose linear user journeys:

TheTrainline.comCinema ticket bookings

Self assessment tax return online

In most websites, although there are some linear elements …

… people like to have other pathways available to them…

… and most journeys are very non-linear

Also, most people reach your website via Google

Only 5% arrive at your website on the home page

Most of the other 95% may not have had your site in mind when searching

30% of them go to home page to ‘try and work out what this site is about’

So each page on the website should: (a)engage users (b)give sense of what site is about –

otherwise most will leave

‘Writing for the web’ is not just about text…

… but also choosing the right images… layouts

… graphical look and feel…website structure

etc etc

Key point of paragraph/

section

Image clearly related to text

Broken into short paras

Print vs web• Reading on screen much slower +

people scan more than read• On web, connections between pieces

almost as important as words themselves – also juxtaposition with images, graphic layout etc

• Most people find things via Google, so keywords important, and ‘instant context’ essential

Watch ‘TV’ while reading

Immediate links to related

stories

Juxtaposition of stories puts

particular slant on this

one

Other areas also available - and the rest

of the Internet!

Other context-sensitive links

encourage browsing

Why evaluate websites?

How will you ever know?Very few people will ever voluntarily provide

critical feedback on your website – if you have an Email us link or feedback form, how many responses do you get, compared to the number of web visits?

How will you ever know?Even when asked people are generally polite

A lot of time and money is spent developing websites that very few people will ever use (in any meaningful way)


Why evaluate websites?Why do evaluation and user testing?

Isn’t it really expensive and time consuming?

1. Save money – avoid substantial, hurried redevelopment later in project

2. Audience feedback improves resource in various ways – new activity ideas, etc

3. Demonstrate involvement of key stakeholders throughout project


Making websites effective3 key success factors

• Understanding audience

• Learning experience and learning outcomes – right for audience and clearly stated

• Evaluation – esp in classroom or home (observe in ‘natural habitat’ wherever possible…)


Who for what for ...• Who for? (audience)

– Need to be clear from starte.g. ‘ for teachers of yr5/6 in local area with whiteboards’

• What ‘real-world’ outcomes? (learning outcomes)– What will they learn or do as a result?

e.g. plan a visit to museum, learn that Romans wore funny clothes, discover that they enjoy using a digital camera…

• How will they use it? (learning experiences)– What do they actually do with the site?

e.g. work online or need to print it? - in pairs or alone? - with or without teacher help?

• Where, when and why will they use it?– context is important

How to evaluate websites

When to test or evaluate, and why


When to evaluate or test and why

• Before funding approval – project planning

• Post-funding - project development

• Post-project – summative evaluation


Testing websites• Need to think ahead a bit:

– what are you trying to test?

– how do you intend to test it?

– why? what will do you do as a result?

The Why?Why? should drive this process


Test early

Testing one user early on in the project…

…is better than testing 50 near the end


Testing is an iterative process

Testing isn’t something you do once.

You make something, test it, fix it, and test it again.


Before funding – project planning• *Evaluation of other websites

– Who for? What for? How use it? etc– awareness raising: issues, opportunities– contributes to market research– possible elements, graphic feel etc

• *Concept testing – check idea makes sense with audience– reshape project based on user feedback

Focus group

Research



Post-funding - project development• *Concept testing

– refine project outcomes based on feedback from intended users

• Refine website structure– does it work for users?

• *Evaluate initial look and feel – graphics,navigation etc

Focus group

Focus group

One-to-one tasks

MAG Ford Madox Brown website



Post-funding - project development 2

• *Full evaluation of a draft working version – usability AND content: do activities work, how

engaging is it, what else could be offered, etc

Observation of actual use of website

by intended users,

using it for intended purpose,

in intended context – classroom, workplace, library, home, etc








• Video clip Moving Here key ideas not lesson plans



Post-funding - project development 3

• Acceptance testing of ‘finished’ website– last minute check, minor corrections only– often offered by web developers

• Summative evaluation– report for funders, etc– learn lessons at project level for next time


User testing – who should do it?• The worst person to conduct (or interpret)

user testing of your own site is…– you!you!

• Can be done cheaply – tips on how to do it available (MLA SE guide): www.ICT4Learning.net/onlineguide

• Useful to have an external viewpoint• First 5mins in a genuine setting tells you

80% of what’s wrong with the site, etc• Info and guidance from Martin Bazley

http://www.ict4learning.net/onlineguide



Two usability testing techniques

“Get it” testing- do they understand the purpose, how it

works, etc

Key task testing- ask the user to do something, watch how

well they do

Ideally, do a bit of each, in that order



Who for what for ...• Who for? (audience)

– Need to be clear from starte.g. ‘ for teachers of yr5/6 in local area with whiteboards’

• What ‘real-world’ outcomes? (learning outcomes)– What will they learn or do as a result?

e.g. plan a visit to museum, learn that Romans wore funny clothes, discover that they enjoy using a digital camera…

• How will they use it? (learning experiences)– What do they actually do with the site?

e.g. work online or need to print it? - in pairs or alone? - with or without teacher help?


• How can you ensure you do get these right?– Build questions into the planning process – Evaluate/test regularly– Get informal feedback whenever possible –

and act on it

• Who is it for?• What are the real world outcomes?• How will they use it?• Also When, Where, Why?

Who for what for ...


Making websites effective3 key success factors

• Understanding audience

• Website experience and outcomes – right for audience and clearly stated

• Evaluation – esp in classroom or home (observe in ‘natural habitat’ wherever possible…)

Simple, low cost web enhancements for websites to help give users what they

want

Simple, low cost enhancements

Freebies (or cheapies) for museum websites: •Google map sign up [FREE]

•Google AdWords •WordPress CMS [FREE] for small websites

•Blogs, podcasts, etc [FREE]

Understand your users:–SurveyMonkey – online questionnaires–Google Analytics, Piwik (open source)

Google Trends, Benchmarking [FREE]

http://www.flying-museum.org.uk/

https://www.google.com/local/add/login?hl=en-GB&gl=GB&utm_medium=et&utm_source=bizsols&utm_campaign=mapsF

http://adwords.google.com/



http://www.elearninggroup.org.uk/

http://www.elearninggroup.org.uk/

http://www.surveymonkey.com/

http://www.google.com/analytics/en-GB/

WordPress CMS example:

E-Learning Group for Museums, Libraries and Archives

IronCurtainSites

Free Google / Yahoo map• embed a map panel in the website, or • provide link to customised map

• User can pan around on the map, • zoom in and out, • get more information on key points

nearby, and • get directions to or from the

destination and print off

Schematic mapDepending on area and transport

connections, a schematic map (specially drawn for this purpose) can still be useful:

highlight relevant features e.g. motorways or train stations, or other museums, attractions or accommodation

– although all of this can also be implemented via the Google mapping tools: www.google.com/local/add

http://www.google.com/local/add

Increasing visits to your website

Findability via Google etc• Search Engine Optimisation (SEO) has

become a significant industry in itself…

• …aimed mainly at commercial companies looking to differentiate themselves within narrow, highly competitive markets – ensure a particular company is the highest on the list for people searching for printer ink supplies, for example.


For cultural sector the situation is rather better:

strong USPs (Unique Selling Points) and authority


But to be found by Google a website needs to be ‘transparent’ or

visible to the automated archiving of the web by search ‘bots’ (robots).


Also, the more other websites that link to it, the further up the search results the museum website will come.

This is one advantage of getting involved in blogs and social networking sites such as Facebook, MySpace, YouTube and Flickr: links from there back to the museum site increase traffic directly, and also improve the rating in Google.


• More information on improving Google rankings are available on the Google site itself. A recent and increasingly popular development used by a number of museum webmasters is Google Analytics: www.google.com/analytics

Search Engine Optimisation- article

http://www.google.com/analytics

Understanding online audiences – a coordinated

approach

Geffrye Museum web stats linked with events etc

Q11 What age range are you in? (compared to national averages 2006)

-25%

-20%

-15%

-10%

-5%

0%

5%

10%

15%

20%

25%

0-18 18-24 25-34 35-44 45-54 55-64 65+ Prefer not tosay

Horniman

Geffrye

LTMus

MOL

MiD

All 5

Age of visitors to the museum (relative to population averages)

-25%

-20%

-15%

-10%

-5%

0%

5%

10%

15%

20%

0-18 18-24 25-34 35-44 45-54 55-64 65+

Horniman

Geffrye

LT Mus

MOL

All 4

Q6 Ideally, what else would you like to see on the website?

0% 10% 20% 30% 40% 50% 60%

More information on what the galleries are like

More detailed information on the museum's objects

More detailed information about collections in store

More information about how to physically access collectionsin the museum store

More details on events, exhibitions

More videos

More opportunities for me to comment or upload my owncontent

More entertaining things to do online - games, etc

More things for people unable to visit the museum

More opportunities to buy things online

Other

Horniman

Geffrye

LTMus

MOL

MiD

Most commonly requested web content relates to

visiting• The top 4 overall are: - information on what the galleries are like- information on the museum’s objects- details of events and exhibitions- information about collections in store• what most online users want is more

information about the museum visit• Is that how money is being spent…?

Data gathering techniques - strengths and

weaknesses

Online questionnaires(+) once set up they continue to gather

numerical and qualitative data with no further effort and so given time can build up large datasets

(+) the datasets can be easily exported and manipulated, can be sampled at various times, and structured queries can yield useful results

(–) respondents are self-selected and this will skew results – best to compare with similar data from other sources, like visitor surveys

(–) the number and nature of responses may depend on how the online questionnaire is displayed and promoted on the website

Focus groups

(+) can explore specific issues in more depth, yielding rich feedback

(+) possible to control participant composition to ensure representative

(–) comparatively time-consuming (expensive) to organise and analyse

(–) yield qualitative data only - small numbers mean numerical comparisons are unreliable

Visitor surveys

(+) possible to control participant composition to ensure representative

(–) comparatively time-consuming (expensive) to organise and analyse

(–) responses can be affected by various factors including interviewer, weather on the day, day of the week, etc, reducing validity of numerical comparisons between museums

Web stats(+) Easy to gather data – can decide what to

do with it later(+) Person-independent data generated - it is

the interpretation, rather than the data themselves, which is subjective. This means others can review the same data and verify or amend initial conclusions reached

(–) Different systems generate different data for the same web activity – for example no of unique visits measured via Google Analytics is generally lower than that derived via server log files

Web stats(–) Metrics are complicated and require

specialist knowledge to appreciate them fully

(–) As the amount of off-website web activity increases (e.g. Web 2.0 style interactions) the validity of web stats decreases, especially for reporting purposes, but also for diagnostics

(–) Agreeing a common format for presentation of data and analysis requires collaborative working to be meaningful

What is Web 2.0?

Web 2.0 …

Web 2.0 is a vague labelrefers to recent trends in

attitudes and ways of using the web, not specific technical developments.

Web 2.0 is …• A continually evolving spectrum of

approaches, embodying– User focus– Agile development– ‘always Beta’ – Social trust– User generated content ‘ by and for

users of a particular group’

Web 2.0 is …• A continually evolving raft of

technologies (distributed services)– Blogs, podcasts, RSS events etc– Wikis – collaborative workspaces– Mash ups – live data, filtered/combined– Social networking (Facebook, Bebo,

Linked In etc)– Social bookmarking (del.icio.us)– social tagging

Web 2.0 is …• A continually evolving set of

companies including– Google– Yahoo– Flickr– YouTube– del.icio.us– etc

Web 2.0 examples• Google – various apps• TripAdvisor – wisdom of crowds / social

reviewingFlickr example

• YouTube example

•

Events – wouldn’t it be great if you could just note your events or collections description information once, and it would appear anywhere on the web where people might want it? => RSS and aggregated databases e.g. Culture 24 (24 Hour Museum)

More info from:

Martin Bazley0780 3580 727

www.ICT4Learning.com

or martinbazley.com

Date post:	18-Nov-2014
Category:	Education
Upload:	martin-bazley
View:	793 times
Download:	0 times

Bazley Developing And Evaluating Online Resources

Education