WhatisBigDataandWhyShouldWeCare?
GianlucaDemar-niLecturerinDataScienceUniversityofSheffieldgianlucademar-ni.net
Dataishuge
• Banks,citycouncils,governments,shops,etc.
• Facebookprocesses750TB/dayofdata– 48kiPhoneseveryday– 7PBofphotostorage/month
• Thisrequirescomputers(alotofthem!)
2
FacebookDataCenter(Sweden)
3
Data
4
Machines
• Googlehasaround900,000servers(260millionwaUs)0.01%globalenergy
• CERNLargeHadronCollider180MW• Theflux-capacitorneeds1.21GW(just5xGoogle)
DatafromJuly2013 5
BigData
• DefinedasVs– Volume:Justaboutsize,Giga,Tera,Petabytes– Variety:Formats,text,databases,pictures,excel– Velocity:Speed,10000tweetspersecond,2000picturesonInstagrampersecond
6
BigDataResearchinSheffield
• Lookingatdataintegra-onacrosssources
7
InstagramMarkZuckerberg
Shouldwecare?
• Thisdataisaboutus!• Data:GMail,Facebook,debitcards,shoppingfidelitycards,transport,mobilephones,…
• Usage:Mortgageapplica-on,healthinsurance,carinsurance
8
Algorithmsruletheworld
• Somedatamustnotbeprocessedbypeople!– GMailcontentisprocessedbycomputerstodecidewhichadver-sementyouseeontheWeb
9
Algorithmsruletheworld
• Uberpricesaredecidedbyasoiwareprograms– ThebossofUberdriversisacomputer– Itdecideshowtheyworkandhowmuchmoneytheymake
• Computersknowalotaboutpeoplebutnottheotherwayaround
10
Isitallbad?
• Duolingo:Data-drivenforeignlanguagelearning– Whatisthebestwaytolearnalanguagedependsonyourna-velanguage
• Languagetransla-on
11
DataScience• “DataScien-st:TheSexiestJobofthe21stCentury”,inHarvardBusinessReview
• Companieswantdata-drivendecisions
• GraduatesfromtheMScDataScienceinSheffieldgoworkin:– Telecommunica-ondataanalysis– Cancerresearch– Housingmarket
12
Isyourdataready?
• Decisionsaboutyouandyourlifewillbedrivenbydata!
• Areyouincontrolofwhichdataisbeingcollected?
• Areyouawareofhowthedataisstoredandprocessed?
• Doyouknowhowdecisionsaretaken?Doyouwanttoknow?
13