Date post: | 30-Nov-2014 |
Category: |
Internet |
Upload: | andrew-prescott |
View: | 189 times |
Download: | 0 times |
Professor Andrew Prescott, Theme Leader Fellow
AHRC Digital Transformations Strategic Theme
Big Data: Some Initial Reflections
• The Met Office currently generates about 20TB of data each day
• ‘The problems which confront the meteorologist today will be faced by the humanities scholar within ten years’
• Large Hadron Collider: 600 million ‘collision events’ per second
• One million jobs run by servers each day, with over 10 GB of data per second transferred at peak times
• Approx. 20 petabytes of data produced annually• Over 70 universities involved in processing the data
http://www.flickr.com/photos/ibm_research_zurich/6777192080/in/set-72157629212636619
Whole brain imaging of neurone activity in a zebra fish, made using light sheet microscopy by Misha Ahrens and neuroscientists at the Howard Hughes Medical Institute. Each image comprises over 1
terabyte of data.
Link: http://www.youtube.com/watch?feature=player_embedded&v=KE9mVEimQVU
• Some working definitions of big data• Big data exceeds the capacity of existing
desktop machines and networks: you need help to deal with it
• Data that is so large that existing methods of analysis simply don’t work: you have to change your methodology (probably to something quantitative)
• Gartner definition: “Big data” is high-volume, -velocity and –variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.
Examples of everyday big data of research value
• Retail data generated by supermarkets• Online retail data: Amazon• Transport information: Oyster card• Hospital data• Data from utility companies• Social media
Visualisation of languages used in tweets in London in Summer 2012: Centre for Advanced Spatial Analysis, UCL:
http://mappinglondon.co.uk/2012/londons-twitter-tongues/
Wolphram Alpha analytics of my Facebook friends
Analytic of my friend network
Does Big Data Yet Exist for the Humanities?
Letter of Gladstone to Disraeli, 1878: British Library, Add. MS. 44457, f. 166
The political and literary papers of Gladstone preserved in the British Library comprise 762 volumes containing approx. 160,000 documents.
George W. Bush Presidential Library:200 million e-mails
4 million photographs
A Thousand Words: Advanced Visualisation in the Humanities
Texas Advanced Computing Center
Link: http://www.youtube.com/watch?v=kvOuJ2RwBTA
‘Big data’ has already been an issue for linguists for many years
Another familiar example of big data in the humanities: censuses
Moving images and sound present some of the most challenging big data issue for arts and humanities
Archives and library catalogues as big data: Visible Archive browser: visiblearchive.blogspot.com
Visualisation by Jon Orwant of Google of Library of Congress subject categorisations of books published
between 1600 and 2010: winedarksea.org
Commons Explorer: experimental interface to allow exploration of large quantities of images in Flickr
Commons: http://mtchl.net/cex/
The Anglo-American Legal Tradition: web site holding seven million images of medieval
legal records in the National Archives: www.aalt.law.uh.edu
Fabio Lattanzi Antinori,The Obelisk (2012): Open Data Institute: http://www.theodi.org/culture/obelisk-2012
Asia Trend Map: predicting popularity of games, manga and anime: www.asiatrendmap.jp
Some Big Data Issues
• Research has historically been hypothesis-driven; is a more data-driven research required?
• How valid are predictive and probabilistic techniques in arts and humanities research?
• Data quality issues: do we lose a sense of the context and stratigraphy of the data?
• Danger of thinking that data=truth
Digital Transformation theme and Big Data
• Theme seeks to promote new research methods: using digital tools and materials to develop completely new type of scholarship
• Additional funding of £4m has been allocated to work on big data
• Following this workshop, call for big data projects will be issued
• Smaller projects (up to £100k)• Larger projects (up to £600k)