April 22, 2023

1

Doing an MSc Thesis in the GroupParallel and Distributed Systems

D.H.J. Epema Parallel and Distributed Systems Group (PDS)

Department of Software TechnologyFaculty EEMCS

April 22, 2023 2

Contents

• The PDS group• Research topics in the PDS group• Procedure for doing an MSc in the PDS group

April 22, 2023 3

Parallel and Distributed Systems (1)• Faculty:• prof.dr.ir. H.J. Sips (full professor, chair)

• ir.dr. D.H.J. Epema (associate professor)

• dr. K.L. Langendoen (associate professor)

• dr.ir. J.A. Pouwelse (assistant professor)

April 22, 2023 4

Parallel and Distributed Systems (2)• PhD students and PostDocs

• 10 PhD students, 4 PostDoc, 4 programmers• MSc students

• 12 MSc students (Computer Science, Computer Engineering, Embedded Systems)

• Location: 9th floor

April 22, 2023 5

What is PDS about?• We make resources work:

• CPUs, memories, power, networks• resource management, performance analysis

• Internet applications: peer-to-peer systems (P2P)• Middleware: grids• Networked systems: wireless sensor networks

(WSN)• HPC systems: parallel systems

Common theme of our research: new concepts => design => implementation => analysis

April 22, 2023 6

Research topic 1: Peer-to-peer systems• In most DSs, some nodes have more authority or

functionality than others (e.g., in a client-server system)• In peer-to-peer systems, all nodes have the same authority

and functionality, i.e., these systems are decentralized• Most P2P systems used for file sharing • Topics in P2P:

• Searching for content• Downloading • Dynamics• Scalability• Freeriding

P2P systems account for more than 50% of the Internet traffic

April 22, 2023 7

P2P: TV distribution over the P2P: TV distribution over the InternetInternet• Distributing TV is the killer P2P application in the internet in the next decade • recorded: millions of PVRs form one huge repository

(how to find things)• live: low-cost entry for content distributors

(how to stream things)

• P2P-TV forms a foundation for sharing with your friends (creating virtual communities)• content (you can have what I have)• interest profiles (you may like what I like)

• P2P-TV is a viable and innovation-driving alternative to (server-client) IP-TV

April 22, 2023 8

P2P: Research in DelftTribler• Is based on the Bittorrent P2P file-sharing system• Looks at the peers as really representing actual

users rather than as anonymous computer programs

• Adds social-based functionality• De-anonymizes peers:

• peers have a quasi-unique public permanent identifier, which

• can be used to challenge a peer for its identity• Does recommendations on content to users• Has been released on 17 march 2006• V4.0 released on may 23, 2007 • Has been downloaded about 150,000 times

April 22, 2023 9

P2P: Research in Delft People (in PDS):Johan PouwelseHenk SipsDick EpemaPawel GarbackiMichel MeulpolderJan David MolJie YangMaarten ten BrinkeFreek ZindelJelle Roozenburg Jacco Taal

More information:• www.cs.vu.nl/ishare• www.tribler.org• www.ewi.pds.tudelft.nl (publications database)

Tribler overview paper to appear in C&C-P&E,available on www.pds.ewi.tudelft.nl/~epema

Projects:• I-Share• TLP2PS• P2P-Fusion• P2P-Next

April 22, 2023 10

Research topic 2: grids

• Grids are large wide-area collections of computer resources (supercomputers, clusters, networks, storage, etc)

• Usually used for scientific research (simulations etc.)• Problems in grids:

1. Scheduling and resource management2. Security (authentication, authorization, etc)3. Programming models4. …

April 22, 2023 11

Research topic 2: grid scheduling Problems in grid scheduling:

1. Grid schedulers usually do not own resources themselves

2. Grid schedulers have to interface to different local schedulers

3. The set of grid resources is heterogeneous and dynamic

4. Workloads are heterogeneous and dynamic

5. Structure of applications

LS

local jobs

global job

GS

LS LS

load sharing and

co-allocation

non-local job

local schedulers

April 22, 2023 12

Grids: Research in Delft• The design and implementation of the KOALA co-

allocating grid scheduler• Main goals:

1. processor co-allocation: (un)ordered/flexible jobs2. data co-allocation: move large input files to the locations

where the job components will run prior to execution3. load sharing: in the absence of co-allocation4. run alongside local schedulers

• KOALA• is written in Java• uses Globus components (e.g., RSL and GridFTP) • for launching jobs uses its own mechanisms or Globus

DUROC• has been deployed on the DAS2 in september 2005

April 22, 2023 13

Grids: Research in Delft• People:

• Dick Epema• Alexandru Iosup (grid interoperation)• Hashim Mohamed (design of KOALA)• Ozan Sonmez (scheduling)

• Publications• see PDS publication database at www.pds.ewi.tudelft.nl

• Web sites: • www.st.ewi.tudelft.nl/koala• grenchmark.st.ewi.tudelft.nl• gwa.ewi.tudelft.nl

• Projects:• VL-e: www.vl-e.nl• GUARD-G, CoreGRID

April 22, 2023 14

Research topic 3: Wireless sensor networksMoore’s law will bring us:• ultra low-power devices, with• small form factor, at • very low cost

fostering a new range of networked embedded systems applications

Smart Dust (1998): Autonomous sensing and communication in a cubic millimeter

April 22, 2023 15

WSN: The first steps …• Develop COTS hardware• Develop software (TinyOS)• Run experiments• Prototype applications

Tmote SkyImote

BTnode rev3Mica2

TNOdes

ATmega128L CPU (8-bit, 8 MHz)• 128 KB FLASH (program)• 4 KB DRAM (data memory)

Chipcon CC1000 radio (868 MHz)• modulation: FSK 76.8 kBaud• output power: -20 to 10 dBm

Delft/Wageningen

LOFAR-agro

April 22, 2023 16

Topics:• self-configuration• node localization • low-bitrate communication• ad-hoc routing• in-network data processing• time synchronization

WSN: Research

[ACM SenSys, IEEE SECON, EWSN, ACM Trans. on Sensor Networks, …]

47 43 50 70 226 447946

1960

3400

4400

0500

100015002000250030003500400045005000

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Constraints:• robustness• limited resources• energy efficiency

#publications

April 22, 2023 17

People:• Koen Langendoen• Gertjan Halkes (MAC) • Aline Baggio (Localization)• Muneeb Ali (Routing)• Tom Parker (Data aggr)

WSN: Research in Delft

Projects:• LOFAR-agro (precision agriculture)• Relate (search & rescue) • Smart Surroundings (context awareness)• Consensus (networking fundamentals) Facilities:

• 24 node testbed• real-time power tracing

IN4181: WSN seminar (4e kw)• 2 introduction lectures• 5 seminars (students presenting)• hands-on lab

April 22, 2023 18

Research topic 4: parallel systems

handhelds: video streaming

home cinema: set-top boxes

high performanceembedded systems

April 22, 2023 19

Research topic 4: Sensors and HPC

processing hugestreams of data

LOFAR

April 22, 2023 20

Parallel Systems: Research in Delft

People:• Henk Sips• Alexander van Amesfoort• Rob van Nieuwpoort• Ana Varbanescu

Projects: • Astrostream • Scalp

Web sites:• astrostream.ewi.tudelft.nl• scalp.ewi.tudelft.nl

April 22, 2023 21

PDS Lab facilities: the DAS-3

There is no end to the computing capacity required by many applications!!

• five clusters (one in Delft)• about 550 CPUs• 10-Gb lightpaths

www.cs.vu.nl/das3

April 22, 2023 22

PDS MSc Procedure

• Make an appointment with the MSc coordinator:D.H.J. Epema (tel. 83853, d.h.j.epema@tudelft.nl

• Fill out form with list of courses (on EEMCS web site)• PDS colloquium (every 2 months)• More information on www.pds.ewi.tudelft.nl:

• “Research projects”• “MSc projects” (also previous MSc theses)