What is Crowdsourcing?

Nicola Osborne nicola.osborne@ed.ac.uk

Digital Education & Service Manager, EDINAUniversity of Edinburgh Crowd Sourcing & Citizen

Science Network

“collaborative production of knowledge and change” (CSCS 2016)

citizen science n. scientific work undertaken by members of the general public, often in collaboration with or under the direction of professional

scientists and scientific institutions. (OED 2014)

• Generally consists of large specialist tasks broken into smaller non-specialist tasks that anyone can do – especially in “Citizen Science”.

• Usually managed via online and/or mobile platforms but… • Crowd sourcing is not a new concept…

• The first edition of the Oxford English Dictionary was crowdsourced (Winchester 1998).

• RSPB Big Garden Bird Watch was conducted on paper for years before online crowdsourcing emerged.

Crowdsourcing is…

• Citizen-led initiatives to gather data not otherwise available/collected – e.g. OpenStreetMap, FixMyStreet.• Research or analysis at scale – such as facilitated annotation,

analysis, pattern spotting etc. of large data sets, e.g. Old Weather. • Research-led initiatives where crowd sourced data and/or analysis

has a clear research purpose, e.g. GalaxyZoo, UK Lamprey Watch.• Collaborative creative projects, e.g. Wikipedia. • Paid participation on bit-work basis, e.g. CrowdFlower, Amazon

Mechanical Turk• Mining and analysis of available crowd sourced data sets–

Twitter, Flickr etc. e.g. WeSenseIt, Reading the London Riots• Games where research tasks are abstracted, e.g. FoldIt• On the ground reporting of in-the-moment events, e.g.

COBWEB Flooding, Guardian Witness

Crowdsourcing & Citizen Science can take many forms

Why Crowdsource?

• Cost – enables collection of data at scale and (usually) without payment.

• Access – citizens on the ground are well placed for incidents, unexpected events, or to regularly access data about their location.

• Engagement – makes your presence visible; enables development of relationships with communities; route to engaging citizen in research and policy making.

• Scale – some tasks are not realistic at large scale for expert contributors.

• Serendipity – the crowd bring a fresh eye and can find anomalies and make unexpected connections

Challenges of crowdsourcing• Time/Cost – to engage with a community takes authentic commitment,

substantial communication channels, significant time, appropriate resourcing.

• Data Quality – isn’t guaranteed, addressed through training/guidance in platform, simple interfaces, QA processes, comparison to authoritative data, sound research design.

• Expectation Management – the crowd need to be motivated and rewarded for participation, how will you do that but manage expectations of change resulting from their participation?

• Participation Levels and Locations – to work crowd sourcing projects have to attract a significantly large group of participants, in the right places, and sustain their attention.

• Lack of Control – the crowd will not always work as you expect and may develop its own priorities and etiquette.

• Legal Risk/Liability, Safety – especially in emergency situations. T&Cs and warnings may be required.

• Data Visibility and Unexpected Implications – such as changes in insurance premiums due to better understanding of flood area extent/risk; exposure of location of at-risk species, etc.

Project Design and Engagement Matter• Solid research design matters - hard to change approach part-way

through. • Identifying audience(s)/community(ies) and level of engagement

desired helps you to tailor engagement.• Communication is critical: feedback and reward mechanisms are

essential to ensure citizens remain motived.• Technical considerations need to include; what is possible; what can

citizens access/use.• Balance priorities: question/problem and data capture/analysis needs

to be useful for project team, but accessible and realistic for non-specialists.

• Communities/participants require support and guidance.• Quality assurance processes need to be established, whether through

metadata capture, comparison, manual checking, etc.• Contingency planning – in case of poor participation, less successful

outcomes, disapointment/expectation management.• Plan/mechanism to conclude, exit or handover project.

Technology and Tool Selection

• Impacts accuracy of data collected (e.g. volunteered vs. GPS vs. IP based location).

• Interface design, usability, familiarity and language - can ease or increase complexity of participation.

• Home computer based tasks can exclude participation by e.g. lower income, older, less able, or more remotely located communities (see RSE 2014).

• Mobile devices limit to those with appropriate devices, often also limited to those with iOS/Android/etc. Also require wifi or 3G signals, or apps/interfaces which can function offline.

• App and data upload size may mean volunteers incur costs when participating.

Quality AssuranceTrust in Citizen Science by researchers and policy makers varies, quality impacts re/use.• Quality Assurance method/process :

– Manual/moderation (e.g. Conker Tree Science);

– Repetition and redundancy (e.g. Galaxy Zoo);

– Targeted comparison with trusted data (e.g. sensors);

– Technical measures (e.g. location of submission);

– Trust/expertise level (e.g. based on previous submissions).

• Raises social challenges of managing potential rejection of volunteered data/effort.

• See Chapman 2015.

Questions/Comments?

References & Resources• Chapman, C. 2015. COBWEB D.2.2 Value Adding to Crowdsourced Data for Decision

Making. COBWEB Project. Available from: https://cobwebproject.eu/news/publications/deliverables

• Dunn, S. and Hedges, M., 2012. Connected Communities: Crowdsourcing in the humanities, a scoping study. AHRC and Kings College London. Available from: http://www.ahrc.ac.uk/documents/project-reports-and-reviews/connected-communities/crowd-sourcing-in-the-humanities/

• Graham, G.G., Cox, J., Simmons, B., Lintott, C., Masters, K., Greenhill, A. and Holmes, K, 2015. How is success defined and measured in online citizen science: a case study of Zooniverse projects. Available from: http://eprints.whiterose.ac.uk/86535/

• Roy, H. et al, 2012. Understanding Citizen Science for Environmental Monitoring. CEH. Available from: http://www.ceh.ac.uk/news/news_archive/documents/understandingcitizenscienceenvironmentalmonitoring_report_final.pdf

• Royal Society of Edinburgh, 2014. Spreading the Benefits of Digital Participation in Scotland Final Report. Royal Society of Edinburgh. Available from: http://www.royalsoced.org.uk/1058_SpreadingtheBenefitsofDigitalParticipation.html

• Winchester, S., 1998. The Professor and the Madman: A tale of murder, insanity and the making of the Oxford English Dictionary. HarperCollins Publishers.

Crowd Sourcing & Citizen Science Resources

• European Citizen Science Association: http://ecsa.biodiv.naturkundemuseum-berlin.de/

• UK-EOF’s Understanding Citizen Science and Environmental Monitoring:http://www.ceh.ac.uk/products/publications/understanding-citizen-science.html

• SWE4Citizen Science – OGC Standards for Citizen Science Projects: https://cobwebproject.eu/about/swe4citizenscience-github

• University of Edinburgh Crowd Sourcing & Citizen Science Network: http://citsci.ed.ac.uk/

• COBWEB: Citizen Observatory Web Project – an EU-standards compliant citizen science infrastructure, including flooding demonstrators and QA work: https://cobwebproject.eu/

• Crowd Flower: https://www.crowdflower.com/ • Edinburgh CityScope – infrastructure for open creation and sharing of

data, including crowd sourcing projects: http://www.edinburghcityscope.org/

Citizen Science Projects and Tools Mentioned Here

• Amazon Mechanical Turk: https://www.mturk.com/mturk/welcome • Click to Cure: http://www.clicktocure.net/• FieldTrip GB – mobile app for small scale crowdsourcing:

http://fieldtripgb.blogs.edina.ac.uk/ • FoldIt: https://fold.it/portal/ • GalaxyZoo: http://www.galaxyzoo.org/• Guardian Witness: http://www.theguardian.com/community/series/

guardianwitness • Leafwatch: Conker Tree Science: http://naturelocator.org/leafwatch.html • Old Weather: https://www.oldweather.org/#/ • OPAL: http://www.opalexplorenature.org/• OpenStreetMap: http://www.openstreetmap.org/ • RSPB Big Garden Bird Watch: http://www.rspb.org.uk/birdwatch/• UK Lamprey Watch: https://envscot-csportal.org.uk/lampreywatch/• WeSenseIt: http://wesenseit.eu/ • Wikipedia: https://en.wikipedia.org/wiki/Main_Page