+ All Categories
Home > Documents > Master's Project Report HVNS RC3

Master's Project Report HVNS RC3

Date post: 30-May-2018
Category:
Upload: alex-maskovyak
View: 223 times
Download: 0 times
Share this document with a friend

of 77

Transcript
  • 8/9/2019 Master's Project Report HVNS RC3

    1/77

    Simulating the Performance of Data

    Distribution and Retrieval Algorithms

    withHardware Varying Network Simulator

    Masters Project

    Alexander G. Maskovyak

    Rochester Institute of Technology

    Golisano College of Computing & Information Sciences

    Department of Computer Science

    102 Lomb Memorial Drive

    Rochester, NY 146235608

    [email protected]

    May 2010

    Chair Hans-Peter Bischof

    Reader Axel Schreiner

    Observer TBA

  • 8/9/2019 Master's Project Report HVNS RC3

    2/77

  • 8/9/2019 Master's Project Report HVNS RC3

    3/77

    P a g e | 3

    Contents

    Abstract .................................................................................................................................................. 7

    1. Introduction .......................... .......................... ......................... .......................... ......................... .... 8

    1.1. Motivation ......................... .......................... .......................... ......................... ......................... 8

    1.2. Distributed Data .......................... ......................... ........................... ......................... ................ 8

    1.3. Simulation .......................... .......................... .......................... ......................... ......................... 8

    1.4. Current Simulators ........................... ......................... ......................... .......................... ............ 9

    2. Project Description ......................... ......................... .......................... ......................... ................... 10

    2.1. Terminology ....................... .......................... .......................... ......................... ....................... 10

    2.2. Architecture ....................... .......................... .......................... ......................... ....................... 11

    2.3. Report Structure ......................... ......................... ........................... ......................... .............. 12

    3. Simulation ......................... ......................... .......................... ......................... ........................... ..... 14

    3.1. Static versus Dynamic Simulation .......................... .......................... ......................... .............. 14

    3.2. Deterministic versus Stochastic Simulation ......................... ......................... .......................... . 14

    3.3. Continuous versus Discrete Simulation .......................... .......................... .......................... ..... 14

    3.4. Analytical versus Agent-Based Simulation .......................... ......................... .......................... . 15

    4. Simulation Implementation ........................ .......................... ......................... ........................... ..... 16

    4.1. Simulator ........................... .......................... .......................... ......................... ....................... 16

    4.2. Simulatables........................................ .......................... .......................... .......................... ..... 18

    4.3. Operation Bound Simulatables .......................... ......................... .......................... .................. 20

    5. Network Model ......................... .......................... .......................... ......................... ....................... 23

    5.1. Node .......................... ......................... .......................... ......................... .......................... ...... 23

    5.2. Connection Adaptor ........................ .......................... ......................... .......................... .......... 24

    5.3. Connection Medium............................ .......................... ......................... ........................... ..... 25

    5.4. Protocol Stack ......................... ......................... .......................... ......................... ................... 26

    6. Hardware Model ........................ .......................... .......................... ......................... ....................... 28

    6.1. Hardware Computer Node ........................... .......................... ......................... ....................... 29

    6.2. Harddrive ........................... .......................... .......................... ......................... ....................... 29

    6.3. Cache ......................... ......................... .......................... ......................... ........................... ..... 29

    6.4. Connection Adaptor ........................ .......................... ......................... .......................... .......... 29

    7. Distribution and Retrieval Algorithms ........................... .......................... ......................... .............. 31

    7.1. Operation Model ......................... ......................... ........................... ......................... .............. 31

  • 8/9/2019 Master's Project Report HVNS RC3

    4/77

    P a g e | 4

    7.2. Implementation .......................... ......................... ........................... ......................... .............. 32

    7.3. Client-Managed Distribution Algorithm ......................... .......................... .......................... ..... 33

    7.4. Server-Managed Distribution Algorithm ........................ ......................... ........................... ..... 37

    8. Configuration ......................... .......................... ......................... .......................... .......................... . 42

    8.1. Java API .......................... .......................... ......................... .......................... ......................... .. 43

    8.2. HVNSLanguage ........................ ......................... .......................... ......................... ................... 46

    8.3. HVNSL Example Configuration ........................... ......................... .......................... .................. 47

    8.4. Configuration Directory Structure ........................ ........................... ......................... .............. 51

    8.5. HVNSL Grammar ......................... ......................... ........................... ......................... .............. 53

    9. Algorithm Benchmarking ........................ .......................... ......................... .......................... .......... 55

    9.1. Metrics............................... .......................... .......................... ......................... ....................... 55

    9.2. Logging Implementation .......................... .......................... ......................... ......................... .. 56

    9.3. Log File Format ........................... ......................... ........................... ......................... .............. 56

    9.4. Generating Logs .......................... ......................... ........................... ......................... .............. 57

    10. Simulation Expectations .......................... .......................... ......................... ........................... ..... 58

    10.1. Varying Adaptor Speed ......................... .......................... ......................... ......................... .. 58

    10.2. Varying Cache Size........................... .......................... ......................... ........................... ..... 58

    10.3. Varying Cache Speed ....................... .......................... ......................... ........................... ..... 59

    10.4. Varying Server Quantity ........................... .......................... ......................... ....................... 59

    10.5. Varying Redundancy ........................ .......................... ......................... ........................... ..... 59

    11. Simulation Results .......................... ......................... ........................... ......................... .............. 60

    11.1. Varying Adaptor Speed ......................... .......................... ......................... ......................... .. 61

    11.2. Varying Cache Size........................... .......................... ......................... ........................... ..... 62

    11.3. Varying Cache Speed ....................... .......................... ......................... ........................... ..... 64

    11.4. Varying Server Quantity ........................... .......................... ......................... ....................... 65

    11.5. Varying Redundancy ........................ .......................... ......................... ........................... ..... 66

    12. Simulator Comparison..................................... ........................... ......................... ....................... 67

    12.1. ns-2 ........................ ......................... .......................... ......................... ........................... ..... 67

    12.2. JiST ......................... ......................... .......................... ......................... ........................... ..... 69

    12.3. OMNeT++ ....................... .......................... .......................... ......................... ....................... 70

    13. Conclusions ........................ .......................... ......................... .......................... ........................... 71

    14. Future Work .......................... .......................... .......................... ......................... ....................... 73

  • 8/9/2019 Master's Project Report HVNS RC3

    5/77

    P a g e | 5

    14.1. Architecture ........................ ......................... .......................... ......................... ................... 73

    14.2. HVNSL ........................ .......................... ......................... .......................... ........................... 73

    14.3. Algorithm Design .......................... ......................... ......................... .......................... .......... 74

    14.4. Benchmarks ........................ ......................... .......................... ......................... ................... 74

    References ............................................................................................................................................ 75

  • 8/9/2019 Master's Project Report HVNS RC3

    6/77

    P a g e | 6

  • 8/9/2019 Master's Project Report HVNS RC3

    7/77

    P a g e | 7

    Computer Science is a science of abstractioncreating the right model for a problem and

    devising the appropriate mechanizable techniques to solve it.

    - A. Aho and J. Ullman

    Abstract

    Softwarebased network simulators are often an essential resource in networking research. Network

    simulation affords researchers the ability to test communication protocols and topological design where

    this would otherwise be economically or physically infeasible. A heretofore unexplored avenue for

    simulation which has not been explored is the interplay between the computer hardware of devices on

    a network and the impact this has on the performance of data distribution algorithms. This is line of

    inquiry is becoming increasingly important as the amount of data produced begins to exceed the

    capacity of local storage and must instead be stored on remote servers and later retrieved.

    Hardware Varying Network Simulator (HVNS) was designed to fulfill this niche. HVNS models how

    hardware attributes like disk-read speed, communication link bandwidth, and cache size affect the

    performance of one data distribution and retrieval algorithm over another. HVNS can demonstrate

    when adding functionality like cache utilization or a faster connection adaptor results in no performance

    gain because of hardware properties of the network. HVNS can be used to test purely theoretical

    hardware and its effects. This can be used to discriminate between algorithms which may not currently

    be viable but may become viable should certain hardware capabilities become available in the future.

    A cursory glance over network-related primary literature demonstrates that Network Simulator version

    2 (ns-2) [1] is the de-facto standard in discrete event-driven network simulation. It is widely used in

    transport, network, and multicast protocol testing for both wired and wireless networks. However, it

    and similar simulators model the network at an inappropriate level of abstraction for this domain.

    These simulators are slower, harder to learn, and more difficult to extend. HVNS was designed to be

    easy to use, extensible, easily configurable, and perform well for this domain. This project examines the

    design of HVNS, its configuration language, and provides insight into the affects that hardware can have

    on data distribution and retrieval.

  • 8/9/2019 Master's Project Report HVNS RC3

    8/77

    P a g e | 8

    Data expands to fill the space available for storage.

    - Parkinsons Law of Data

    1. IntroductionData storage and access requirements in the present and nearterm are going to necessitate the

    deployment of distributed hardware solutions and the use of data distribution and retrieval algorithms

    for a wide variety of fields, namely scientific research. There is active ongoing research into the

    development of these algorithms. Network simulators exist to test network protocols but are not

    specifically designed to test the interaction between I/O algorithms and the computer hardware of the

    devices in that network topology.

    1.1. MotivationThe amount of data generated worldwide is increasing at an exponential rate [2]. Scientific research

    using high performance computers produces a considerable amount of data. The experiments run on

    CERN's Large Hadron Collider are expected to generate 2 gigabytes of data every second, with a total

    yearly production of 10 to 20 petabytes of information [3][4]. The Wide Field Infrared Camera (WFCAM)

    of Cambridges Astronomical Survey Unit captures approximately 2200 images of the night's sky which

    take up 230 gigabytes of space [5]. It is physically and economically infeasible with current technology

    to store and backup this data on a single localized storage medium [6]. Instead, these data are stored

    on multiple networked machines which are retrieved in intelligent ways to mitigate the costs associated

    with nonlocal access.

    1.2. Distributed DataDistributed file systems allow these data to be stored and accessed from across multiple file servers as

    though they were local resources. Distributing data incurs performance penalties. Additional time is

    spent on I/O over the network connection for sending the data to be stored initially and retrieving it

    later for an application's/client's later use. A variety of techniques are being explored in research to

    increase the performance of file distribution and retrieval algorithms which include the use of caching,

    data duplication, and data compression [7].

    1.3. SimulationSoftware

    based network simulators are often an essential resource in networking research. Network

    simulation affords researchers the ability to test communication protocols and topological design where

    this would otherwise be economically or physically infeasible. Simulation can similarly be applied to the

    study of data distribution and access algorithms where hardware properties can greatly impact the

    performance of one algorithm over another. Simulation also provides an additional benefit aside from

    circumventing resource limitations: theoretical hardware models can be constructed to test the impact

    of technology that is on the horizon or indevelopment [8].

  • 8/9/2019 Master's Project Report HVNS RC3

    9/77

    P a g e | 9

    1.4. Current SimulatorsA wide variety of simulators have been deployed in professional and academic settings. Some popular

    simulators include Ns-2, JiST, and OMNeT++. Network Simulator version 2 is a discreteevent network

    simulation environment and the defacto standard used in the field of network research to test, design,

    and benchmark network protocols in wired environments. Ns2 offers support for simulating a full

    network protocol stack, multicast protocols, and routing. The ns2 configuration file requires

    knowledge of OTcl, a scripting language designed to be embedded in applications [7]. Java in Simulation

    Time (JiST) is a prototype for virtual machine-based simulation where a Java applications code is

    intercepted and transformed into bytecode which contains simulation time-semantics [9]. It is efficient

    and allows a developer to avoid the use of a domain-specific simulation language to define a simulation.

    OMNet is a discreteevent, network simulator [10]. It is an opensource project which provides model

    frameworks which can be used to develop domainspecific projects. It is designed for extensibility.

  • 8/9/2019 Master's Project Report HVNS RC3

    10/77

    P a g e | 10

    The hardest part of the software task is arriving at a complete and consistent specification, and

    much of the essence of building a program is in fact the debugging of the specification.

    - F. Brooks

    2. Project DescriptionThis project examines the interplay between remote storage-and-retrieval algorithms and how they are

    affected by aspects of the hardware and network topology on which they are deployed. Specifically the

    question is under what circumstances remote read requests can outperform local read requests. One

    way to answer this question is via simulation. The approach is to design a custom simulator, model the

    applicable network and computational domains, implement several distribution algorithms, and

    measure their performance on a variety of network/computational configurations.

    2.1. TerminologyThis project deals with several computer science domains: simulation, networking, and hardware.

    Computer simulationexplores how an abstract models state changes over the course of time. The

    modelis a representation of an object or grouping of objects as well as their behavior in response to a

    variety of events. A simulatoris responsible for performing a simulation and is generally responsible for

    managing the flow of time and monitoring system state changes. A discrete scheduled event simulator

    manages time as a series of events in an event queue [11]. The simulator handles events scheduled to

    occur at some point in time which can trigger the scheduling of additional events further into the future.

    Events are created by the model as a part of its behavior and cause the model to undergo state changes.An agent-based simulation explicitly models individual entities as opposed to an analytical model which

    operates with underlying mathematical equations or otherwise abstracts out entities as numeric

    properties. Agents respond to events based upon a set of internal rules or logic [12]. These responses

    can include an alteration to their state and/or a scheduling of future events within the simulator. The

    behavior of a state-based agent can be modeled by a Finite State Automata. Finite State Automata (also

    known as a Finite State Machines) are graphical models which describe the states of an entity, the

    events an entity receives, the behavior of an entity in response to events, and how an entitys state

    changes in response to events when it occupies any particular state [13].

    A simulation run is the result of a simulator performing these functions on a model with some initialstate conditions. Typically, the simulation run will produce some form of diagnostic data regarding the

    behavior of the system. An interested party will run several simulations with varying initial conditions

    and model parameters in order to observe patterns, trends, or to benchmark performance against

    some variety of metrics.

    Computer networking deals with computational devices which are connected via media to facilitate

    communication and the sharing of computational resources like hardware, software, or data. Networks

  • 8/9/2019 Master's Project Report HVNS RC3

    11/77

    P a g e | 11

    range in size from the small local area network (LAN) of a family-unit with a handful of devices

    connected to a wireless access point to the internet which consists of hundreds of thousands of

    networks of computers connected across the globe.

    Software applications typically have two roles on a network: client or server. A clientrole indicates that

    a piece of software is performing a request on a piece of software in the serverrole which is respondingto / servicing the request. Software communicates across the network via discrete messages which are

    known as datagrams, packets, segments, or messages depending upon the level of abstraction being

    used. Aprotocolis an agreed-upon dialogue that two pieces/instances of software program use that

    governs that piece of the programs behavior/interactions in response to messages. Here too, an FSM

    can be used to describe this interaction. Theprotocol stackis a layered series of modular protocols

    which implements a suite of computer networking protocols. Each layer of the stack handles one aspect

    of communication so that the layer above it does not have tothey are said to provide a service to

    the layer above [14]. The protocol stack has 3 basic layers (though there are commonly more and

    groupings can differ) with one protocol each handling media access, message transportation, and

    applications. The media access layerhandles transformation of information into a form that can betransported by the underlying medium to which a device is connected. The transportationlayer

    determines how messages find their way to their intended destination as they pass through a series of

    devices. The application layeris where most end-user centric, desktop applications reside and is the

    layer of the distribution algorithms developed for this project. The distributed algorithms above make

    use of virtual hardware to store data and send messages to one another.

    The hardware model includes both data storage and communication hardware. A harddrive has large,

    long-term storage but generally has slow access times. A cache is much smaller, short-term storage but

    has comparatively much faster access times. Caches are used to speed up data retrieval. Caches store

    frequently accessed information so that the fast speed of the cache can be leveraged and the slowaccess of the harddrive avoided. When requests are uniform or predictable, the cache can further be

    used to store information that has not yet been requested but which may be requested at some point in

    the future. Due to the small size of a cache, it is important to make intelligent decisions about what is

    stored there; otherwise no speedup over harddrive access can be realized. A connection adaptor is the

    gateway to the medium connecting computational devices to one another. Network infrastructure

    speeds are rapidly increasing, and in some cases, are much faster than the access time for local storage

    [15].

    The last observation is the impetus behind this project.

    2.2. ArchitectureHardware Varying Network Simulator or HVNS is the result of the approach described in the description.

    HVNS is an agent-based, discrete-event simulator. The simulator runs in its own thread with a thread-

    safe, event priority-queue. Simulatables are entities which schedule events with the simulator and

    handle messages contained within an event from a sender.

  • 8/9/2019 Master's Project Report HVNS RC3

    12/77

    P a g e | 12

    Events are containers that hold a message, intended receiver, and a time of delivery. Events are held in

    the simulator queue first in time-order, and second by a priority order flag. The simulator pops events

    from the top of its queue, updates its time to match the events time, and then delivers the message to

    the intended simulatable recipient. The simulatable then proceeds to perform some operations in

    response to this event which may result in the recipient scheduling additional events with the simulator.

    Events can be scheduled by any simulatable for any simulatable. Messages from a simulatable to itself

    are given the highest priority and are considered control messages that have no transit cost. They are

    intended to allow a simulatable to schedule the alteration of its state for a future time and bootstrap

    itself for future operations/work.

    The network model consists of nodes directly connected with some set of neighbors via Ethernet-like

    connection medium. Nodes exchange messages via packets over this medium via next-hop routing. The

    computational model views nodes as computers, each of which contains a number of components

    which can store and retrieve varying amounts of data at varying rates.

    Distribution algorithms are the primary agents in this model and make use of the computational

    facilities to request data to be distributed, data to be stored, and data to be retrieved. One algorithm on

    one node is designated as the client. This client is told to execute which causes it to schedule a

    bootstrapping event for itself in the queue. The client will then proceed to request a specified amount

    of data from its harddrive and send this information out to select devices on the network for storage.

    The interval of time between the first harddrive request and the last harddrive response is considered

    the baseline local read-time. Server nodes will begin to store, replicate, and propagate this information

    across algorithmically selected devices. Once distributed storage is complete a message is sent back to

    the client node. The client node can then begin requesting the data back from the server nodes. The

    interval of time between the first remote request and the reception of the last requested piece of data

    is considered the experimental remote read-time. Smaller time intervals indicate greater/betterperformance. Ideally, the confluence of the algorithms design and the attributes of the available

    hardware will allow the algorithms remote read-time outperform the local read-time.

    The simulator environment and the model are configured through a domain-specific language called

    HVNSLanguage (HVNSL). This configuration language has relatively few keywords and has a fairly

    predictable, uniform syntax across the entire configuration space. The configuration language is

    domain-specific which allows it to focus on simplifying the configuration of hardware-components as

    well as the creation of network topologies.

    2.3. Report StructureThis report begins with an examination of simulation theory with an emphasis placed on the techniques

    employed by HVNS. It explores the architecture and interfaces of HVNS, the design decisions involved,

    and the benefits and drawbacks of this approach.

    The models under simulation are then explored in the same way. The design and abstractions employed

    for the network, its protocol stack, and routing infrastructure is discussed. This is followed by an

  • 8/9/2019 Master's Project Report HVNS RC3

    13/77

    P a g e | 13

    examination of the architecture and abstraction behind the computer model, the hardware employed,

    and the hooks included for the distribution algorithms. The report next explores the distribution

    algorithms themselves, their operation, and the design intent.

    HVNSLanguage (HVNSL), the simulator and model configuration language, is then discussed including its

    syntax and its semantics.h

    The benchmarking apparatus, measurements, and expectations are detailed. The results of the

    measurements are then described. The report analyzes the results in comparison to expectations.

    Further analysis is given comparing the actual simulator architecture itself to competing simulators

    employed in the field. This includes a discussion on the benefits and drawbacks of the various

    approaches described in relation to HVNS.

    The report concludes with an evaluation of the project as a whole. Here the success of HVNS, the

    algorithms, and benchmarking are discussed. Suggestions for future work are next discussed which

    includes potential improvements to the simulator, model, configuration language, and testing

    apparatus. The lessons learned from the approach taken to this projects development are also included

    as guidance for future work.

  • 8/9/2019 Master's Project Report HVNS RC3

    14/77

    P a g e | 14

    To dissimulate is to feign not to have what one has. To simulate is to feign to have what one

    hasnt. One implies a presence, the other an absence.

    - Jean Baudrillard

    3. SimulationIn the broadest sense, simulation is the fabrication or imitation of something which is real. An

    imitation reproduces some subset of the defining characteristics of the real item. These characteristics

    may be behavioral, physical, or intangible. In Computer Science, the preceding abstraction is known as a

    model and the imitated properties are considered the models state. Computer simulation is generally

    concerned with utilizing inputs in tandem with this model to affect change in the models state over

    time (the cardinality of the change set can be one as well). How this occurs depends upon the

    classification and specific implementation of the model.

    Computer simulations can be classified into several types based upon aspects of the simulator on which

    or model upon which they are being performed.

    3.1. Static versus Dynamic SimulationSimulators possess either static or dynamic models. Static models produce a single solution for a

    simulation run. Dynamic systems have models which can assume several states over the course of a

    simulation run. Static models are useful for analyzing the relationships between sets of input with

    output variables. Dynamic systems are useful for analyzing how a system gets from some arbitrary start

    condition to an arbitrary ending condition inside the span of time simulated [16].

    3.2. Deterministic versus Stochastic SimulationSimulators are either deterministic or stochastic. Deterministic simulators will evolve systems in an

    identical way across all simulation runs when given the same input conditions. Stochastic simulators will

    evolve systems with some degree of variance across all simulation runs when given the same input

    conditions. Stochastic simulators rely upon pseudo-random number generators to introduce

    randomness into a run [17].

    3.3. Continuous versus Discrete SimulationSimulators are either continuous or discrete. Continuous simulators have models with explicit state

    variables whose values are governed by differential-algebraic equations or differential equations.

    Periodically (i.e. at some fixed time interval), the simulator will alter its state by solving the equations to

    produce values for state-assignment. Discrete simulators can alter their variables at only a fixed number

    of points in time. Discrete event simulators are an important subset of discrete simulators. Discrete

    event simulators operate on a succession of events whose occurrence moves time forward [17].

  • 8/9/2019 Master's Project Report HVNS RC3

    15/77

    P a g e | 15

    3.4. Analytical versus Agent-Based SimulationSimulators have either analytical or agent-based models. Analytical models rely upon a collection of

    rules, equations, or functions to determine how the state of the system as a whole is advanced. In

    contrast, an agent-based system involves some number of autonomous rule-based entities whose

    actions and interactions affect the system. These agents respond to events that occur in the system,

    which may alter their internal state and cause them to schedule additional events. The state of the

    system is comprised of the aggregation of the states of all agents and any state variables external to

    them [17].

  • 8/9/2019 Master's Project Report HVNS RC3

    16/77

    P a g e | 16

    A good simulation, be it a religious myth or scientific theory, gives us a sense of mastery over

    experience. To present something symbolically, as we do when we speak or write, is somehow to

    capture it, thus making it ones own. But with this appropriation comes the realization that we

    have denied the immediacy of reality and that in creating a substitute we have but spun another

    thread in the web of our grand illusion.

    - Rudolph Heinz Pagel

    4. Simulation ImplementationHVNS runs on the JVM and uses Java as its implementation language for the simulator and all

    simulatable constructs [18][19]. Java has strong OOP properties and an expansive standard library of

    classes which implement many design patterns which have been leveraged (e.g. interfaces, observers,

    threads) for HVNS. HVNS makes extensive use of design patterns and objectoriented principles to

    allow it to be extensible and modular.The architecture is designed around interfaces, abstract base

    classes, instantiations which are external from constructors, and factory patterns.

    4.1. SimulatorHVNS is a dynamic, deterministic, discrete event-scheduling, agent-based simulator. HVNS is a dynamic

    simulator with a model that experiences a range of states before the simulation concludes. HVNS is a

    deterministic simulator in that it does not introduce randomness in any aspect of its event scheduling or

    delivery. This means that the simulator itself neither creates random events nor alters the scheduling of

    events by simulatables. HVNS is a discrete event-scheduling simulator which maintains an event queue

    and execution threads. HVNS is an agent-based simulator as it relies upon the goal-based interactions of

    its registered simulatables to cause the evolution of its system state.

    The simulator and simulation environment are distinct from the network/computational model

    employed. This means that the operation of HVNS can be made to emulate aspects of the alternative

    simulation classes described in Chapter 3 through alterations of the model (HVNS itself can also be sub-

    classed to produce these effects). As an example, a discrete simulator can be modeled as a single

    simulatable which schedules do work message for itself at a fixed time interval. An analytical

    approach can be created by having the aforementioned simulatable maintain the equations and state

    variables which it recalculates with every do work message. As another example, non-determinism

    can be introduced into the simulation via the model being simulated. Simulatables can employ random

    number generators in a way that affects if and when they are going to schedule an event, or the way inwhich they determine the recipient of their events message. Alternatively, simulatables could be

    wrapped inside a parent simulatable which can be made to intercept events, modify when and/if they

    are to be delivered, and schedule a series of events to occur at random points in the future.

  • 8/9/2019 Master's Project Report HVNS RC3

    17/77

    P a g e | 17

    The simulator runs in its own thread and possesses a thread-safe priority-queue for events. There are

    three major portions of the simulator that are discussed here: simulator startup, scheduling events, and

    the main event loop.

    The startup procedure for HVNS operation and for discrete event simulators in general is depicted in

    Figure 4-1:

    Simulator Startup

    Set ENDING_CONDITION to FALSEInitialize event queueInitialize state variables / register agentsStart event thread

    Schedule bootstrap event(s)

    FIGURE 4-1. PSEUDO-CODE FOR THE SIMULATOR START-UP SEQUENCE.

    Simulator startup consists of a few tasks. The ending condition is set to false for the simulator. This is

    an important step otherwise the main event loop would never execute. The ending condition is

    implementation specific and largely depends upon the model under consideration. The ending

    condition may be to stop at some specific time, when a particular state variable or a derivative of that

    variable reaches a value, or when a certain number of events have been processed. The ending

    condition must be able to be set to false, otherwise simulation may proceed indefinitely. The ending

    condition for HVNS is dependent upon a stopped variable. HVNS depends upon the models

    configuration to tell it when to stop, as otherwise if will wait indefinitely for new events to be

    introduced into the queue. The event queue is next initialized so that it is ready to handle the

    scheduling of events. The models state is set, which can include the initialization of state values and/or

    the creation and registration of agents (i.e. simulatables). The main event thread is then initialized

    which begins the main event loop where event execution and message delivery occurs. Bootstrap

    events are then scheduled, allowing the model to evolve.

    Events and their execution are the main concern of the simulator. Events move the simulations time

    forward. The chronological sequence of events represents the evolution of models state over a

    simulation run. A simulator without scheduled events cannot evolve because the agents it contains are

    passive entities who can only act upon the execution of an event/reception of a message. The bootstrap

    events kick-off the simulatables to begin reacting to events.

    Events are the primary form of communication between a simulatable and the simulator. Discrete

    event simulators use events to cause alterations to the model at a specific time. Agent-based simulatorslike HVNS use scheduled events to pass messages between agents. The basic scheduling algorithm for

    events is shown in Figure 4-2.

  • 8/9/2019 Master's Project Report HVNS RC3

    18/77

    P a g e | 18

    Event Scheduling Algorithm

    if( event time < simulator time )Add event to queueSignal scheduled event

    FIGURE 4-2. PSEUDO-CODE FOR EVENT SCHEDULING.

    Simulatables call the scheduling function with their event. The simulator will add the event to the queue

    so long as the event occurs in the future and not the past, otherwise non-causal events can occur (i.e.

    events which depend not just upon past events but also future events). The event queue is a priority

    queue which sorts events based upon an event comparator. Generally, events are sorted first in

    ascending time-order, and second in descending priority-order. This ensures that events occur in a

    chronologically ascending sequence allowing causality to be maintained. The priority-order allows

    simulatables to send control messages to themselves that will always be received prior to external

    messages. The main event thread is then signaled if it is waiting on an empty queue.

    Main Event Loop

    while( ENDING_CONDITION is FALSE )

    while( queue is empty ) { wait for scheduled event }

    Remove event from queueUpdate simulator time to event time

    Execute event (deliver message to recipient)

    FIGURE 4-3. PSEUDO-CODE FOR THE SIMULATOR'S MAIN EVENT HANDLING LOOP.

    The main event loop handles the execution scheduled events and the delivery of messages. It is

    depicted in Figure 4-3. It runs until some ending condition is met as discussed above. The event loop

    avoids busy waiting by waiting on a condition variable when the queue is empty. The event loop wakes

    upon being signaled by the scheduling function that a new event was added to the queue.

    An event communicates three important pieces of information to HVNS: the time of delivery, the

    message, and the intended recipient of the message. HVNS removes the head element from the queue,

    updates its time to match the events time, and then executes the event by delivering the events

    message to the specified simulatable recipient.

    4.2. SimulatablesSimulatables, in aggregate, represent the simulators model. They are stateful entities (even if only

    possessing a single state) which are capable of handling messages delivered to them as the result of an

    event occurring. Simulatables are also agents which implement entity-specific logic which dictates how

    they respond to messages. This response can include a state-change, the scheduling of additional

  • 8/9/2019 Master's Project Report HVNS RC3

    19/77

    P a g e | 19

    events (and messages to other simulatables), the generation and registration of additional simulatables,

    etcetera.

    Simulatables are passive entities that live inside the main event loops thread. They can only ever act if

    an event with a message for them is scheduled and subsequently executed. Simulatables use events to

    communicate with the simulator. Events are containers for messages. A message would otherwise be amethod call between one simulatable and another. Events allow such a call be to be affected by

    simulation imposed limits on operations and the passage of time.

    Simulatables receive messages which are labeled as implementing the IMessage interface. The content

    of a message is completely dependent upon the implementation and expectations of the particular

    simulatable subclass which is receiving the message. Messages are expected to contain information that

    allows a simulatable to act in an event-appropriate fashion.

    It is useful to use the abstraction that messages are time-delayed method calls. As such, the message

    type flag or the message class itself can be used to indicate the method which is to be called. The fields

    and/or message accessors provide the required parameters for this method call. If a message contains a

    type flags, then it is generally heavyweight as it must provide parameters (even if null or empty) for

    every method type. If the class of the message itself indicates the method that is to be called then the

    message need only provide the parameters appropriate for that single method. This comes at the cost

    of an expensive class type check. Systems with heterogeneous simulatables may implement different

    interfaces. To keep messages compact and easy to understand, it is recommended that different

    message types be employed for each of these interfaces.

    Simulatables, as passive entities, must receive messages in order to act. The bootstrap events

    scheduled at the start of a simulation run allow one or more simulatables to perform operations at

    simulation onset. Typically, but ultimately dependant on the model, this will start a cascade of eventscheduling which moves the model between states and forward in time. Simulatables, too, can

    bootstrap themselves or other simulatables. A simulatables accomplishes this by scheduling a control

    message to itself to be received at some point in the future. The event containing this message has its

    priority field set to INTERNAL which has a higher priority than the default EXTERNAL priority which

    ensures that it is delivered prior to any external events.

    Figure 4-4. depicts an example of this mechanism with a simulatable that wishes to accomplish some w

    amount of work in work-state[n] followed by some w` amount of work in work-state[n+1]. The control

    messages allow this simulatable to do work in a state until all work is exhausted, it can then proceed to

    schedule a state change and schedule additional work messages to be completed in the next state.

  • 8/9/2019 Master's Project Report HVNS RC3

    20/77

    P a g e | 20

    Work[2]Work[1]

    [receive bootstrap doWork[1]

    schedule doWork[1]]

    [receive doWork[1] && hasWork[1]

    doWork[1]send doWork[1]]

    [receive doWork[1] && !hasWork[1]send doWork[2]]

    [receive doWork[2] && hasWork[2]

    doWork[2]send doWork[2]]

    [receive doWork[2] && !hasWork[2]send doWork[n-1]]

    Work[n-1]

    [receive doWork[n-1] && hasWork[n-1]

    doWork[n-1]send doWork[n-1]]

    Work[n]

    [receive doWork[n] && hasWork[n]

    doWork[n]send doWork[n]]

    [receive doWork_n && !hasWork_nsend doWork_n+1]

    Bootstrapping Future Work

    FIGURE 4-4. FSM OF A SIMULATABLE BOOTSTRAPPING ITS WORK AND STATE CHANGES.

    4.3. Operation Bound SimulatablesAn operation bound or performance restricted simulatable is a stateful entity with limitations placed on

    its ability to respond to, or schedule events during an interval of time. An Operation Bound

    Simulatables operation is represented by the FSM depicted in Figure 4-5.

    Fully-Awake

    [Get RequestDelegate Response

    ops--refresh_sent_t = t;

    Send(Renewal, t+refresh_sent_t, internal)]

    Partially-AwakeBlocked

    [Get RefreshOps = max_ops]

    [Get RefreshOps = max_ops]

    [Get RequestSend Response

    ops--ops = 0

    ]

    [Get RequestDelegate Response

    ops--ops > 0]

    [Get RefreshOps = max_ops]

    [Get Request

    Send(request, t+refresh_sent_t, external)]

    Operation Bound Simulatable FSM

    FIGURE 4-5. OPERATION BOUND SIMULATABLE FSM DEMONSTRATES A NODE THAT HAS A LIMITED CAPACITY TO ACT AND SO PUTS OFF

    ADDITIONAL WORK FOR A FUTURE TIME WHEN IT WILL HAVE THE CAPACITY TO ACT AGAIN.

  • 8/9/2019 Master's Project Report HVNS RC3

    21/77

    P a g e | 21

    Operation bound simulatables possess three basic states which govern their ability to act. These states

    include:

    Fully-Awake the simulatable is able to perform up its maximum number of operations during atime interval.

    Partially-Awake the simulatable is able to perform some reduced number of operations duringthe current time interval.

    Blocked the simulatable has exhausted its ability to perform any additional operations duringthe current time interval.

    Operation bound simulatables begin in the Fully-Awake state. Fielding a request and/or sending a

    response in this state causes the simulatables allowed operations to be decreased by some amount, a

    refresh message to be scheduled, and a transition to the Partially-Awake state.

    The simulatable may continue to perform operations while in the Partially-Awake state, with each

    operation performed resulting in additional decreases to the simulatables ability to field further

    requests. Once all operational ability has been exhausted, the simulatable goes into a blocked state.

    The Blocked state indicates that no operations can be performed. Any requests received in this state

    are rescheduled by the simulatable to reoccur during the start of the next time interval. This ensures

    that the events can be handled once the simulatable is again able to do so.

    The refresh message is contained in an event which is scheduled by and for the simulatable. It is

    scheduled to occur at the beginning of the next operation interval which is governed by the

    simulatable's refresh_time variable. The refresh message is sent with the highest priority possible

    ensuring that it is received before any other events can be received by the simulatable. Reception of the

    refresh message indicates the start of the next operation interval and indicates that the simulatable mayreturn to the Fully-Awake state with the ability to perform the maximum number of allowed operations.

    If the simulatable has been in the Blocked state and has rescheduled events to occur for this time

    interval, these events will be redelivered by the simulator and can be handled normally.

    The operation bound simulatable abstracts out performance restriction for subclasses by implementing

    a function hook which is called during the Fully-Awake and Partially-Awake states. Subclasses override

    this hook method so that they can be the delegate of model-specific functionality. The parent class

    takes care of determining if and when delegation can occur. An FSM representing this is depicted in

    Figure 4-6.

  • 8/9/2019 Master's Project Report HVNS RC3

    22/77

    P a g e | 22

    Fully-Awake

    [Get Requestops--

    refresh_sent_t = t;Send(Renewal, t+refresh_sent_t, internal)]

    Partially-AwakeBlocked

    [Get Refresh

    Ops = max_ops]

    [Get RefreshOps = max_ops]

    [Get RequestSend Response

    ops--ops = 0

    ]

    [Get Requestops--

    ops > 0

    ]

    [Get Refresh

    Ops = max_ops]

    [Get RequestSend(request, t+refresh_sent_t, external)]

    Delegated FSM

    FIGURE 4-6. SUBCLASSED OPERATION BOUND SIMULATABLE OVERRIDING HOOK METHOD.

    An alternative approach that was explored, involved the use of a separate message queue on all

    simulatables. Simulatables would control their ability to poll the message queue by sending a refresh

    message at the start of their polling operations. A simulatable would return to a blocked state once

    they handled as many messages as allowed by their configuration. The simulatable would subsequently

    wake up upon reception of the refresh message and start the entire process over again. This approach

    was abandoned since it requires the duplication of existing functionalitythe simulator already

    possesses a priority queue for events.

  • 8/9/2019 Master's Project Report HVNS RC3

    23/77

    P a g e | 23

    An idea is always a generalization, and generalization is a property of thinking. To generalize

    means to think.

    - Georg Hegel

    5. Network ModelThe network model represents communication devices which can generate, receive, and propagate

    collections of data in a container known as a packet. It is the model on top of which the hardware

    model resides. A network is composed of three basic entities: Nodes, Connection Media, and

    Connection Adaptors. All network entities use operation bound simulatables as their base which allows

    their performance to be altered.

    The network model uses a simplified single addressing scheme whereby a node has a single address

    which is shared by all of its connection adaptors. This is as opposed to having separate MAC and

    network addresses for each adaptor as would be the case in a real-world network.

    It is important to note that the network model entities are all simulatables, specifically operation bound

    simulatables. This means that these entities do not directly communicate via method calls during a

    simulation. Instead, all communication between network entities occurs through the use of events

    containing messages. The message passed to a network entity during an event is roughly equivalent to a

    method call on its interface. As an example, when a connection medium is said to propagate a packet to

    the connection adaptors to which it is attached, it is not actually calling each adaptors receive method

    and providing it with the packet. Instead, the connection medium is actually scheduling an event for

    each of these connection adaptors. This event will contain a message of the type

    ConnectionAdaptorReceiveMessage and will contain the packet which the connection adaptor is toreceive and inspect. This ensures that all events are affected by the temporal mechanics of the

    simulation environment.

    5.1. NodeA node represents a packet generating/receiving device. Nodes are roughly equivalent to everyday

    systems such as personal computers, file servers, phones, etcetera. Nodes are logically connected to

    one another through a connection medium. Internally, a node may have several connection adaptors

    which physically connect it to several connection mediums. A node sends messages by making use of

    its protocol stack.

  • 8/9/2019 Master's Project Report HVNS RC3

    24/77

    P a g e | 24

    Node

    ConnectionAdaptor

    Node

    ConnectionAdaptor

    Medium

    Two Nodes Connected via Medium

    FIGURE 5-1. DEPICTION OF THE PHYSICAL CONNECTION BETWEEN TWO NODES AND THE RELATIONSHIP OF THE NETWORK ENTITIES TO ONE

    ANOTHER.

    Nodes in the network model can be logically connected to several other nodes. They also possess a

    protocol handler, known as the NetworkProtocolHandler, which can perform routing services to

    determine the next hop destination for a packet. These characteristics allow nodes to also emulate the

    functionality provided by switches or routers. As such, the network model has been simplified to

    exclude explicit implementation of these network objects.

    Nodes also provide the transport API which allows algorithms (or any application layer item) to make

    use of the protocol stack. They also fulfill the unreliable transport layer role by wrapping application

    layer messages into datagrams which are then provided to the network layer for routing.

    5.2. Connection AdaptorA connection adaptor is a packet propagator and sender. It represents any physical layer device which

    interfaces with the medium itself. Examples of such devices include network interface cards and

    wireless antennas. One or more connection adaptors can be contained within a node.

    A connection adaptor must deal with two events, packets that are outgoing and packets that are

    incoming (both of which are in respect to the attached node). A connection adaptor that receives a

    packet from its own node (specifically from a protocol handler above it in the stack) sends that packet

    out across the connection medium. A connection adaptor that receives a packet from a connection

    medium must inspect that packet to determine its intended destination. If the packets destination

    address does not match the connection adaptors address, then the connection adaptor was not the

    intended recipient and the packet is dropped. However, if the addresses match then this connection

    adaptor is considered the next hop and the packet must be handled by a protocol handler further up the

    protocol stack.

    NetworkProtocolHandleris the protocol handler one further up the stack. It is a network layer protocol

    handler that provides routing services. It is responsible for determining the address of the next-hop

    node along the path of nodes to the destination.

  • 8/9/2019 Master's Project Report HVNS RC3

    25/77

    P a g e | 25

    Node B

    Connection

    Adaptor

    Connection

    Adaptor

    Network Protocol Handler

    Node A

    Connection

    Adaptor

    Node C

    Connection

    Adaptor

    Medium

    NetworkPH NetworkPH

    Medium

    Network of Three Nodes

    FIGURE 5-2. THIS NETWORK OF THREE NODES DEPICTS THE INTERNAL MAKEUP OF A NETWORK OF NODES THREE NODES CONNECTED IN

    SERIES. A NODE POSSESSES A CONNECTION ADAPTOR FOR EVERY MEDIUM TO WHICH IT IS CONNECTED. A SINGLE CONNECTION NETWORK

    PROTOCOL HANDLER HANDLES ROUTING RESPONSIBILITIES FOR ALL CONNECTION ADAPTORS AND ALGORITHMS INSTALLED ON A NODE.

    When NetworkProtocolHandler receives a datagram from a higher level protocol, it interrogates its

    routing table to determine the address of the next closest node on the path from the current node to

    the destination node. The datagram is encapsulated into a packet and delivered to the connection

    adaptor to send out. In some cases, this next hop node may be identical to the destination node. In

    other cases, there may be several intermediate nodes that must also receive and route the packet.

    NetworkProtocolHandler also handles datagrams received as the payload of a packet from the lower

    level protocol. The destination address of the datagram is inspected. If the address matches the

    address of the node, then the payload data is removed from the datagram and delivered to the protocol

    handler that matches the protocol of the datagram (i.e. the TransportProtocolHandler itself). If the

    address does not match the address of the node, then the connection adaptor proceeds to act as if the

    datagram had been received from the higher level protocol. It determines the next hop address via its

    routing table, packages the datagram into a packet with this next hop address, and gives the packet to

    the connection adaptor to send across the medium.

    Node B

    ConnectionAdaptor

    ConnectionAdaptor

    NetworkProtocolHandler

    Node A

    ConnectionAdaptor

    Node C

    ConnectionAdaptor

    Medium

    NetworkPH NetworkPH

    Medium

    Network of Three Nodes Routing a Packet

    FIGURE 5-3. NODE A IS SENDING A PACKET WITH NODE C AS ITS DESTINATION. THE NEXT HOP NODE ALONG THE PATH IS B. WHEN B

    RECEIVES THIS PACKET, IT INSPECTS THE DESTINATION ADDRESS OF THE DATAGRAM, DETERMINES THE NEXT HOP ADDRESS, REPACKAGES

    THE DATAGRAM INTO A NEW PACKET ADDRESSED TO C, AND FINALLY SENDS IT TO NODE C.

    5.3.

    Connection Medium

    Connection medium is a packet duplicating/propagating device. It represents the physical layer /

    medium of an actual network which may be a coaxial cable, twisted pair Ethernet, airwaves, or

    otherwise. Depending upon implementation, a connection medium can logically connect multiple nodes

    to one another to allow packets to be propagated in a broadcast fashion to devices connected to the

    medium. A connection medium is logically connected to a node but physically connected to the nodes

    connection adaptor. It receives and sends packets to connection adaptors. All connection mediums are

  • 8/9/2019 Master's Project Report HVNS RC3

    26/77

    P a g e | 26

    currently reliable and will never drop packets or introduce errors into sent packets. The addition of

    unreliable media would necessitate the implementation of a reliable transport protocol.

    5.4. Protocol StackThe protocol stack is an abstraction that describes the collection of protocols installed on a node. It is alayered series of protocols where the lower level protocol (i.e. the protocol below) provides a service

    the higher level protocol (i.e. the protocol above). Figure 5-4 provides a visual depiction of the protocol

    stack in the internet protocol suite called TCP/IP as well as the protocol stack employed by node.

    Nodes Protocol

    StackTCP/IP

    Transport API

    Application Layer

    ConnectionAdaptor

    NetworkProtocolHandler

    TransportProtocolHandler

    Algorithm

    Network Layer

    Transport Layer

    Data Link Layer

    Physical Layer

    Comparison of ProtocolStacks

    Transport API

    FIGURE 5-4 MAPPING BETWEEN TCP/IP'S PROTOCOL STACK AND NODE'S PROTOCOL STACK

    The traditional TCP/IP protocol suite has 5 layers: Application, Transport, Network, Data Link, and

    Physical and a transport layer API that serves as the glue between an application and the services

    provided by the rest of the stack. Nodes protocol stack combines several of these layers. The

    connection adaptor provides general media access service by handling both the physical and link layer

    services. A NetworkProtocolHandler provides networking services by handling routing operations.

    TransportProtocolHandler provides unreliable transportservices and also provides the transport API to

    algorithms. The distribution algorithms sit at the top on the protocol stack and access the services of

    the stack via the Transport API (i.e. the IProtocolHandler interface).

    A protocol in this model implements the IProtocolHandler interface. IProtocolHandlers handle the

    following operations:

    Associate a protocol handler with a protocol name

  • 8/9/2019 Master's Project Report HVNS RC3

    27/77

    P a g e | 27

    Handle packets from the higher level protocol (implementation dependant) Handle packets from the lower level protocol (implementation dependant)

    All packets implement the IPacket interface. They have a protocol type, source, destination, and

    payload. The term used to refer to a packet is dependent upon the layer of the protocol stack. The

    transport layer deals with datagrams. The network layer deals with packets. The physical layer dealswith frames. Each layer is responsible for dealing with one type of packet which contains information

    pertinent to that layer. The generic termpacketwill be used when referring to a non-specific level of

    the protocol stack.

    Each layer of the protocol stack encapsulates the packet received from the higher level protocol before

    contacting the lower level protocol. Encapsulation adds header (and/or footer) information that is

    pertinent to the lower level protocol. The original packet from the higher level protocol is also added as

    apayloadto the new lower level packet. This is important since it allows the original information to be

    retrieved by the corresponding protocol handler on the protocol stack of the destination device.

    The typical operation for a protocol handler is as follows. The protocol handler processes a packet. If

    packet was received from a higher protocol then the next step of the chain is a lower protocol. The

    handler encapsulates the originalpacket as payload inside of a newpacket and sends it to the lower

    protocol. If packet was received from a lower protocol then next step of the chain is a higher protocol.

    The handler removes the payload from the packet and hands this off to the higher level protocol.

    Node A

    ConnectionAdaptor

    Network

    Algorithm Sending a Message to another Algorithm

    Transport

    Node B

    ConnectionAdaptor

    Network

    TransportMedium

    FIGURE 5-5 THE ENCAPSULATION PROCESS OF A PACKET AS IT MOVES DOWN THE PROTOCOL STACK ON NODE A AND UP THE PROTOCOL

    STACK ON NODE B.

    Protocol handlers handle one or more protocol types. Protocol handlers maintain references to the next

    higher and next lower protocol handlers in the stack. These references are setup up during theinitialization of the protocol handlers employed by a node.

  • 8/9/2019 Master's Project Report HVNS RC3

    28/77

    P a g e | 28

    Generalization is necessary to the advancement of knowledge; but particularly is indispensable

    to the creations of the imagination. In proportion as men know more and think more they look

    less at individuals and more at classes. They therefore make better theories and worse poems.

    - Thomas B. Macaulay

    6. Hardware ModelThe hardware model represents is an abstraction of a computer, its operating system, and several of the

    components that impact its performance. This model intersects the network model as it shares a view

    of the node and use of the connection adaptor. Distribution algorithms interface with the hardware

    models API to retrieve information, store information, and to make use of the network for

    communication. Hardware components, like network components, use operation bound simulatables

    as their base which allows their performance to be altered.

    All hardware objects share the following performance-altering properties:

    Transit time Maximum allowed operations Refresh interval

    Some of these were discussed in the context of operation bound simulatable and how they affect its

    state-changing behavior. The discussion here focuses on their meaning and use in a hardware context.

    Transit time is the delay associated with sending information from one hardware component to

    another. It is used to represent both the time it takes to process a request as well as the time delay

    associated with sending a response along the channel between the hardware source and the intended

    recipient. Implementation wise, transit times value is used when scheduling an event to be received by

    another device. The current simulator time plus the transit time is the soonest possible time that a

    simulatable may use for scheduling a new event.

    Maximum allowed operations affects how many operations can be performed by a piece of hardware

    within a given activity interval. Individual hardware components are responsible for implementing a

    coherent and reasonable view on what activities qualify as operations (i.e. quantify the value in

    operations of every activity/method). Operations in the context of an adaptor or harddrive correspond

    most closely to bandwidth. A harddrive that can perform 10 operations per one unit refresh interval can

    possibly send or receive 20 data per unit time. If data is assumed to be a byte and unit time a second,

    this corresponds to a bandwidth of 80 megabytes per second. Hardware components keep track of how

    many operations they have performed during a time interval and subtract operations as activities are

    performed.

    The refresh interval value can be thought of as a hardware components internal clock; the frequency at

    which a hardware component can operate. The time between refreshes is a hardware components

  • 8/9/2019 Master's Project Report HVNS RC3

    29/77

    P a g e | 29

    activity interval. The underlying operation bound simulatable uses the refresh interval to reset a

    hardware components operation count which signifies the beginning of a new activity which allows a

    hardware component to perform operations once more.

    6.1. Hardware Computer NodeThe Computer interface provides for the installation of algorithms, harddrives, and caches. It is the

    gateway that an algorithm uses to obtain access to hardware services performed by harddrives, caches,

    and adaptors.

    6.2. HarddriveHarddrives are large, long-term storage devices. They will generally have slow access times relative to

    caches and even connection adaptors. Harddrives store data for distribution algorithms. Harddrive size

    will generally be homogenous across all computer nodes in the network. The important exception is

    that the client node must have a harddrive large enough to store the entirety of the data that is to be

    distributed to servers on the network so that local-disk read time can be measured.

    Harddrives store IData objects which are associated with indices. Harddrives can fetch and store data

    from specified indices for a computer node.

    6.3. CacheCaches are small, short-term storage devices. Caches are much smaller than harddrives, but also much

    faster. They are optionally installed upon a computer node. Caches represent the use of main memory

    to store data to speed up data retrieval. A cache may be used by an algorithm to store information that

    it predicts it will need access to in the future. This allows an algorithm to pre-pay the cost of a slowharddrive access for a piece of data on the behalf of a future requester of that data. The algorithm can

    retrieve the data from the cache when a request for it is finally made allowing the requester to

    experience the fast cache access time and not the slow harddrive access time. Ideally, every request for

    data could be met with a cache hitwhere the prediction for cache storage/data usage was correct and

    the cache has the data necessary to field the request. A cache miss occurs when the prediction is

    incorrect and the cache does not have the data requested. Cache misses penalize the requester to pay

    the cost of the cache access time as well as the harddrive access which must be made.

    Caches, like harddrives, store IData objects which are associated with indices. The index associated with

    a piece of data is the same whether that data is stored in the cache or on the harddrive.

    6.4. Connection AdaptorConnection adaptors represent network interface cards and are used for communication across the

    network. Connection Adaptors operate on a packet level as does the medium to which they are

    connected. They transfer full objects without transformation to a separate physical representation (i.e.

  • 8/9/2019 Master's Project Report HVNS RC3

    30/77

    P a g e | 30

    bits) across connection media. Connection adaptor speeds are approaching and in some cases

    exceeding the transfer rates of local storage like harddrives [15].

  • 8/9/2019 Master's Project Report HVNS RC3

    31/77

    P a g e | 31

    A distributed system is one in which the failure of a computer you didnt even know existed can

    render your own computer unusable.

    - Leslie Lamport

    7. Distribution and Retrieval AlgorithmsA distribution and retrieval algorithm (DRA) deals with the logistics of distributing data from a local

    client node to a series of nodes remote to it on the network for storage, as well as retrieving this data at

    some future point in time. A DRA has the following high-level functions:

    Retrieve data from local storage. Set up and manage storage network of available nodes on a network. Retrieve data from remote storage nodes.

    Distribution algorithms perform different roles as part of the hardware model, network model, andsimulation environment.

    DRA implement the IAlgorithm interface and access the hardware model through installation onto an

    entity which implements the IComputerinterface like HardwareComputerNode objects. This interface

    grants a DRA the ability to reference storage devices like the computers harddrive and cache so that

    IData objects can be stored and retrieved locally. In order to use a storage device, DRAs must

    understand the interfaces presented by storage devices as made available through the storage devices

    supported messages.

    DRAs sit on top of the network protocol stack in the network model. DRAs implement the

    IProtocolHandlerinterface which connects them to the transport layer protocol handler. This

    connection provides DRAs with the ability to transmit and receive messages which contain control and

    data values stored as a packets payload to and from DRAs remote to them in the network.

    DRAs are operation bound simulatables in the simulation. As simulation agents they can schedule

    events containing messages for other simulatables like harddrives, caches, and the transport protocol

    handler.

    7.1. Operation ModelDRAs use a client-server approach to communication. One DRA is selected to be a client in theconfiguration file which builds the simulation. The other DRAs remain in a passive role until they elect

    and are thereafter selected to perform server duties.

    The simulation begins when the simulator schedules a SET_CLIENTevent to the client node which

    indicates it has been selected to perform the client role. The client then generates a specified quantity

    of data which will be used for both the local and remote read tests. Once these are complete, the client

  • 8/9/2019 Master's Project Report HVNS RC3

    32/77

    P a g e | 32

    sends itself a series of bootstrap events to allow it to continue processing and moving through its

    behavioral states.

    The clients general operation is roughly as follows:

    1. Select volunteer(s) and acknowledge their role as server(s).2. Read data from local storage.3. Distribute data to server(s).4. Await server(s) ready signal.5. Read data from server(s).

    The manner in which this is accomplished is implementation specific. Overviews of the two algorithms

    designed for this project are detailed in Sections 7.3 and 7.4.

    7.2. ImplementationDRAs are implemented using the state design pattern [20]. The state design pattern abstracts out statevalues and the unique behaviors associated with those state values into the form of a state object. The

    DRA itself is a state-context or state-holder object. The DRA holds a state object to which the it

    delegates method invocations. The DRAs behavior thusly depends upon the behavior/implementation

    of the state object it contains. The DRAs behavior can be altered by replacing the state object it holds.

    State-transitions occur when the state object performs this replacement on its state-holder.

    Algorithm

    IState _state

    delegateEvent( Event e )

    Init

    delegateEvent( Event e )

    Distribute

    delegateEvent( Event e )

    Read

    delegateEvent( Event e )

    IState

    delegateEvent( Event e )

    setState( IState s )

    IStateHolder

    delegateEvent( Event e )

    setState( IState s )

    IState _state

    FIGURE 7-1. UML OF STATE DESIGN PATTERN.

  • 8/9/2019 Master's Project Report HVNS RC3

    33/77

    P a g e | 33

    The state design pattern simplifies the creation of distribution algorithms which may have several roles

    and several states per role to fulfill. It successfully isolates responsibilities, which creates simpler and

    more easily tested code.

    7.3. Client-Managed Distribution AlgorithmThe Client-Managed Distribution and Retrieval Algorithm (CMDRA) is a client-server approach that

    requires a client which is active in the selection of servers, the fair distribution of data, and which

    maintains an index table mapping data indices to server addresses. CMDRA features the use of cache on

    servers to speed up retrieval operations. This section examines of the clients operation followed by an

    examination of the server(s)s operation. FSMs are provided for both the client operation and the server

    operation. The discussion text shares terminology with these diagrams and discusses the highlights of

    each state of the FSM.

    The operation of CMDRAs client is depicted as an FSM that is split into two parts, beginning in Figure

    7-2 and ending in Figure 7-3.

    AwaitVolunteersNullRole

    [receive SetClient

    broadcast VolunteerRequest]

    [receive ServerVolunteers

    and needMoreVolunteers()

    store volunteer address

    send VolunteerAccepted]

    [receive ServerAcknowledgesserveracknowledgements++

    and !needMoreAcknowledgements()

    send DoWork]

    Distribute

    [Receive DoWork&& needToDistribute()

    Send HD Request

    distributed++

    send DoWork]

    Client FSM

    [receive ServerAcknowledges

    acknowledgements++

    and haveAllAcknowledgements()]

    [Receive HD Response

    Send data next server round robin style]

    [receive ServerAcknowledges

    acknowledgements++

    and !haveAllAcknowledgements()]

    [receive ServerVolunteers

    send ClientRejectsVolunteer]

    [Receive ServerAcknowledges

    serveracknowledgements++

    needMoreAcknowledgements()]

    FIGURE 7-2. CMDRA CLIENT FSM (PART 1). FEATURES INCLUDE VOLUNTEER SELECTION AND THE DATA DISTRIBUTION PROCESSES.

    Potential clients begin in the NullRole stage. They transition into client status once they receive the

    SetClientmessage from the simulation. A client first attempts to locate some arbitrary number of

    servers which will each store some fractional slice of the total data to be stored. The client broadcasts a

    VolunteerRequestmessage to obtain these servers. The request message contains a count of the total

    amount of data that a server is expected to store. This volunteer request message is disseminated

    across all nodes in the network. Servers which are willing and able to store the specified amount of data

  • 8/9/2019 Master's Project Report HVNS RC3

    34/77

    P a g e | 34

    send back a ServerVolunteers message. The client accepts a user defined number of these servers and

    rejects the rest.

    Once all volunteers have been acknowledged the client begins data distribution. The client sends data

    requests to its harddrive for all indices which are to be sent. The harddrive in turn responds with the

    data stored in the index for each request it receives. The client sends the data from each of theseresponses to a server which is selected in a round-robin fashion. The client keeps track of index

    ownership for future retrieval by maintaining a mapping of servers and the indices they hold. Every data

    sent is acknowledged by the server as it is successfully stored. Once all data has been sent and all server

    acknowledgements have been received, the client proceeds to the server confirmation stage called

    ConfirmServerReady.

    ConfirmServerReady Read

    [Receive DoWorkand haveMoreDataStoreComplete()

    dataStoreComplete++send DataStoreComplete to server]

    [receive DoWorkand haveDataToRequest

    send ServerDataRequest

    dataRequests++send DoWork]

    [receive ServerReadyand !needMoreServerReady

    send DoWork]

    [receive DataResponseand dataResponse is valid

    dataresponses++and needMoreDataResponses]

    Done

    [receive DataResponsedataresponses++

    and !needMoreDataResponses

    send SimulationComplete]

    [receive ServerVolunteers

    send ClientRejectsVolunteer]

    [send DoWork]

    [receive ServerReadyserverready++

    and needMoreServerReady

    send DoWork]

    [receive ServerVolunteers

    send ClientRejectsVolunteer]

    Client FSM

    FIGURE 7-3. CMDRA CLIENT FSM (PART 2). FEATURES INCLUDE CONFIRMATION OF SERVER READYNESS AND THE REMOTE READING

    PROCESS.

    Inside ConfirmServerReady, the client indicates to each of the servers that it has finished sending it data.

    The client then waits for the servers to confirm that they are ready to respond to read requests for that

    data. Servers that have finished storing and processing data respond in time. The client enters the Read

    state once this has occurred.

    Inside Read, the client proceeds to send data requests to each server for the indices it knows that serverpossesses. The data received from these requests is compared against the data stored locally to ensure

    that it is correct. Once all valid data have been received the client ends the simulation.

  • 8/9/2019 Master's Project Report HVNS RC3

    35/77

    P a g e | 35

    The operation of CMDRAs server is depicted as an FSM inFigure 7-4 and Figure 7-5. Like the client, the

    server begins in a passive NullRole state without a specifically defined client or server role.

    [Receive DataStoreComplete

    send ServerReadysend DoWork]

    NullRole

    [VolunteerRequest && hasSpace()

    Mark request id.Send ServerVolunteers

    Broadcast VolunteerRequest.]

    Volunteered

    [VolunteerRequest&& !hasSpace()]

    AwaitStorage

    [receive ClientAcceptsVolunteer

    send ServerAcknowledges]

    [Receive DataStorage

    send ServerAcknowledges]

    Server FSM

    [receive ClientRejectsVolunteer]

    FIGURE 7-4. CMDRA SERVER FSM (PART 1). FEATURES INCLUDE THE VOLUNTEER AND DATA STORAGE PROCESSES.

    This changes when a would-be server receives a volunteer request from the client. Servers which have

    sufficient capacity and are willing to perform a storage role send a ServerVolunteers message to the

    client and proceed into the Volunteeredstate. They also rebroadcast the volunteer message to other

    nodes on the network (up until the time to live limit is reached) so that additional servers not directly

    connected to the client may also receive the message. Volunteered CMDRAs can either be accepted or

    rejected as a volunteer. Would-be servers that are rejected return to NullRole. Would-be servers that

    are accepted enter a storage request acceptance state calledAwaitStorage and acknowledge this

    transition with the client.

    InsideAwaitStorage, servers who are awaiting storage requests field these requests from the client and

    place data received into long-term storage on their harddrive. Servers acknowledge every piece of data

    received from the client. Eventually, the client will indicate that all data has been sent. The server then

    has time to process the data in some way (e.g. place data into cache, etc.). Once operations of this

    nature complete, the server proceeds to the Service state and confirms that it is now ready to field read

    requests.

  • 8/9/2019 Master's Project Report HVNS RC3

    36/77

    P a g e | 36

    Server FSM

    Service

    [Receive ClientDataRequest(index)

    send CacheRequest(index)]

    [Receive DoWork(cacheFreespace)

    Send HDRequest(index) from Cache to HD

    Send DoWork(cacheFreespace--)]

    [Receive CacheResponse(data, address)

    send DoWork(1)

    send DataResponse(data, address)]

    [Receive CacheResponse(null, index)

    Send HDRequest(index)]

    [Receive HDResponse(data, address)

    send DataResponse(data, address)]

    []

    FIGURE 7-5. CMDRA SERVER FSM (PART 2). FEATURES INCLUDE THE SERVICE PROCESS WHICH DEMONSTRATE CACHE AND HD STORAGEAND RETRIEVAL AND THE SERVICING OF CLIENT DATA REQUESTS.

    Inside Service, the server is responsible for fielding read requests from the client. The server receives

    requests for the data stored at an index. The server first attempts to retrieve this information from the

    cache. The cache can respond with data or a null response. Cache data responses are shipped off to the

    client. A null response forces the server to request the data from the harddrive. The hardrives data

    response always contains data and can never be null. This data is similarly shipped off to the client.

    During this time the server attempts to keep the cache filled with data. Cache hits result in new

    requests by the server to the harddrive indicating that the cache needs to be filled with more data (i.e.

    data which has not yet been requested).

  • 8/9/2019 Master's Project Report HVNS RC3

    37/77

    P a g e | 37

    7.4. Server-Managed Distribution AlgorithmThe Server-Managed Distribution and Retrieval Algorithm (SMDRA) is a client-server approach which

    offloads most the server selection, data distribution, and data mapping to a single primary server which

    is responsible for a collection of secondary storage servers. SMDRA features the use of cache as well as

    the use of data redundancy to speed up data requests. There are three roles present in this algorithm:

    client, primary server, and secondary server(s). This section examines the operation of each in turn.

    FSMs are provided depicting the operations of the client both types of servers. The discussion text

    shares terminology with these diagrams and discusses the highlights of each state of the FSM.

    The operation of SMDRAs client is depicted in Figure 7-6 and Figure 7-7.

    AwaitVolunteersNullRole

    [receive SetClient

    broadcast VolunteerRequest]

    [receive ServerReady

    send DoWork]

    Distribute

    [Receive DoWork

    && needToDistribute()

    Send HD Request

    distributed++send DoWork]

    Client FSM

    [receive ServerAcknowledgesacknowledgements++

    and haveAllAcknowledgements()]

    [Receive HD Response

    Send data to primary server]

    [receive ServerAcknowledges

    acknowledgements++

    and !haveAllAcknowledgements()]

    [receive ServerVolunteers

    send ClientRejectsVolunteer]

    AwaitFirstVolunteer

    [receive ServerVolunteers

    send AcceptedAsPrimary]

    [Receive ServerVolunteers

    relay to primary server]

    FIGURE 7-6. SMDRA CLIENT FSM (PART 1). FEATURES INCLUDE PRIMARY SERVER SELECTION, VOLUNTEER RELAYING, AND THE DATA

    DISTRIBUTION PROCESSES.

    Here again, would-be clients begin in the NullRole state until it receives a SetClientmessage from the

    simulation. The client broadcasts a VolunteerRequestmessage and entersAwaitFirstVolunteer. The

    volunteer request message is disseminated across all nodes in the network. Servers which are willing

    and able to store the amount of data in a slice send back a ServerVolunteers message.

    InsideAwaitFirstVolunteer, the server awaits the first of these messages. The sender of this first

    message is selected by the client to be the primary server and is sent an acknowledgement of this role.

    The acknowledgement includes information pertinent to the selection of additional volunteers including

    the number of base servers and the amount of data redundancy required.

    InsideAwaitVolunteers, the client relays all subsequent server volunteers to the primary server. It does

    this until the primary server indicates that it is ready to receive storage requests.

  • 8/9/2019 Master's Project Report HVNS RC3

    38/77

    P a g e | 38

    Inside Distribute, the client proceeds to enter the distribute stage where it sends all data to the primary

    server. It continues sending data until it exhausts its supply and receives an acknowledgement from the

    primary server that the data has been received. The client then proceeds to ConfirmServerReady.

    Client FSM

    ConfirmServerReady Read

    [receive DoWork

    and haveDataToRequest

    send ServerDataRequest

    dataRequests++

    send DoWork]

    [receive ServerReady

    and !needMoreServerReady

    send DoWork]

    [receive DataResponse

    and dataResponse is valid

    dataresponses++

    and needMoreDataResponses]

    Done

    [receive DataResponsedataresponses++

    and !needMoreDataResponses

    send SimulationComplete]

    [receive ServerVolunteers

    send ClientRejectsVolunteer]

    [receive ServerVolunteers

    send ClientRejectsVolunteer]

    [send DataStoreComplete]

    FIGURE 7-7. SMDRA CLIENT FSM (PART 2). FEATURES INCLUDE CONFIRMATION OF SERVER READYNESS AND THE REMOTE READING

    PROCESS.

    The client remains in the ConfirmServerReadyuntil the primary server indicates that it has completed its

    dissemination of the data. The client can then enter the Readstate.

    Inside Read, the client sends a data request for every piece of data sent to the primary server. Data

    responses will be received from the server (primary or secondary) that has the data and was selected by

    the primary server to field the request. The data received is compared against the data stored locally to

    ensure that it is correct. Once all valid data have been received the client ends the simulation.

  • 8/9/2019 Master's Project Report HVNS RC3

    39/77

    P a g e | 39

    The operation of SMDRAs primary server is depicted inFigure 7-8 and Figure 7-9.

    NullRole

    [VolunteerRequest && hasSpace()

    Mark request id.

    Send ServerVolunteers

    Broadcast VolunteerRequest.]

    Volunteered

    [VolunteerRequest

    && !hasSpace()]

    [receive AcceptedAsPrimary

    send ServerAcknowledges]

    Primary Server FSM

    [receive ClientRejectsVolunteer]

    AwaitVolunteers

    [receive ServerVolunteers

    and needMoreVolunteers()

    store volunteer address

    send AcceptedA


Recommended