ConceptsandTechnologiesforDistributedSystemsand
BigDataProcessingGuidoSalvaneschi
1
People
GuidoSalvaneschi
PascalWeisenburgerMatthiasEichholz
[…]2
Motivation
• Distributedsystemsareubiquitous• Emails,WWW,…
• Areaisquicklychanging- sorryforthebuzzwords:)• Bigdataanalytics• Complexeventprocessing/streamprocessing• Cloudcomputing• Newprogrammingmodels(streams,actors)• ...
• Yet,manyconceptsremainthesame• Abstractionoverlow-leveldetails• Faulttolerance• Performance:throughput,latency• Asynchonouscommunication• ...
3
ConceptsandTechnologiesforDistributedSystemsandBigDataProcessing
• ProvideanoverviewofrecentdevelopmentindistributedsystemsandBigDataprocessing
• Focusonconcepts,notontechnology.• Technology wasdifferent5yearsago,willbedifferent in5years• Wewilllookattechnologyonlyasawaytobettergrasptheconcepts
• Selectedtopics• Differentspeakersforeachtopic+guestlectures• Dr.GuidoSalvaneschi,Dr.MichaelEichberg,Prof.PatrickEugster,Dr.AlessandroMargara,Prof.Philipp Haller,…
4
Aboutthiscourse
WhatthiscourseIS• Anintroduction onselected topics• …ondistributedsystemsandbigdataprocessing• Acourseaboutconcepts…andabitabouttechnologies
WhatitisNOT• Itisnot(only)acourseaboutrecenttrendsinBigData.• Itisnotasystematic courseondistributed systems
• DistributedSystems:Principles andParadigmsAndrewS.Tanenbaum,MaartenvanSteen
• Itisnotatutorial onhowtoprogrambigdatasystems
5
April12- Intro,motivationApril19- Introtobigdata,mapreduceApril26- hadoop,hdfsMay4- ComplexeventprocessingMay10 ComplexeventprocessingMay17– ConcurrentprogrammingMay24- Futures,actors,streamsMay31- Component-based architecturesJune7- RESTarchitecturesJune14- Sparkstreaming/Spark?June21- GeodistributionJune28- Security,ResourcemanagementJuly5– DBGuestlecture?July12– Guestlecture/Exampreparation
July26- Exam
Tentativecourseschedule
Dateandroomchange!11:40-13:20CETroomS101/A03
Bigdataandcomplexeventprocessing
Architecturesandprogrammingmodelsfordistr.systems
Cloudcomputingandadvancedtopicsinresourcemanagement
Subjecttochange!Seecoursewebsiteforupdates 6
Exercises
• Nogradedexercises• Exerciseswillbeprovidedaftersomelectures.• Solutionsdiscussedinthenextlectureifneeded
• Why• Getmoreconfidenton thetopicsofthelectures• Getafeeling thetypeofquestions thatcancomeup intheexam
• Examples• Readapaper• Answerquestionsbasedonthecontentofthelecture• Discussacasestudy• Smallcodingexercises
7
RegistrationandGrading
• RegisterinTucan forthecourse!
• Writtenexam– July26• 90mins
• Questionsaboutthetopicscoveredinthelectures• Simple“programming”tasks• Understandacodesnippet thatisrelevantforoneofthecoveredtopics• Completeacodesnippet
8
Resources
• Website(slides,exercises)• http://stg-tud.github.io/ctbd/• Thisistheofficialplaceslides,exercises,updatedschedule,etc.
• Forum• https://www.fachschaft.informatik.tu-darmstadt.de/forum/viewforum.php?f=580
• Pleaseusetheforumforyourquestions.Answerswillremainasareferenceforotherpeople.
• Pleaseaskassistantsforquestionsthatcannotbepostedontheforum• Butthink ifthisisreallythecase:)
9