Technische universiteit eindhoven PROGRESS 11 December 2002btheelen1 Performance Modelling of...

11 December 2002 www.ics.ele.tue.nl/~btheelen 1

technische universiteit eindhovenPROGRPROGREESSSS

Performance Modelling of Complex Hardware/Software Systems

B.D. Theelen



Overview• Introduction• System-Level Design• Software/Hardware Engineering• Case Studies• Modelling and Validation• Performance Evaluation• Accuracy Analysis• Summary



Introduction• Designing a complex system within limited time involves taking

important decisions in an early phase of the design process, which may have a deep impact on the performance of the system

• Early assessment of the impact of design decisions involves using system-level design methods and tools

Design

Tim

e

Concepts Requirements

Implementation

System-Level Design

Component-Level Design

(Automatic) System Implementation



Concepts & Requirements

ConceptModel

Questions

ExecutableModel

FormalProperties

ConceptsSatisfy Requirements

Formulation

Evaluation

FomalisationValidation Fomalisation

Component-Level Design

Milestone A

Milestone C

Milestone B

Sys

tem

-Le

vel

Des

ign

Fo

rmu

lati

on

Fo

rma

lis

ati

on

Eva

lua

tio

n

Yes

No

Three Phases• FormulationFormulation

– Informal identification and specification of system concepts and design issues

• FormalisationFormalisation– Formal specification of

behaviour and architecture– Validation of adequacy of

formal system specification– Formal specification of the

properties to evaluate• EvaluationEvaluation

– Property analysis and design decisions



Software/Hardware Engineering (SHE)• Formulation using (stereotyped) UMLUML diagrams

– Performed case studies: documents with plain texts and various diagrams• Formalisation based on system-level modelling language POOSLPOOSL

– Intended for complex real-time distributed hardware/software systems– Small set of very expressive language primitives– Formal semantics

• Defines unambiguously how to handle concurrencyconcurrency, timetime and probabilismprobabilism

• Evaluation– Performance evaluation founded on Markov chain analysisMarkov chain analysis

• Analytic computation or estimation by simulation– Formal verification of correctness properties based on model checkingmodel checking

• Tools– SHESim: Interactive creationcreation and validationvalidation of POOSL models– Rotalumis: Fast execution of POOSL models for analysisanalysis



Case Study: Network Processor

• (Deep) packet processing in telecommunication systems

• Utilisation of shared busses• Identify possible bottlenecks• Comparison POOSL - SystemC



Case Study: Dataflow System• Part of Alcatel’s Internet Router• Prepares/processes packets for correct routing• Arbitration of memory accesses from 3 components• Evaluate alternative arbitration mechanisms



Case Study: Internet Router

Is the specified flow-control mechanismsuitable for various product variants?

Inputs Outputs

SourcesInput Buffers Output Buffers

SinksSwitch Core

1 1

N N

1

N

N

1

N

1

1

N



Executable Model

• Modelling guidelinesModelling guidelines (validation)– Readability (aggregate data objects), postpone optimising for execution speed– Log considerations on the adequacy of abstractions

• Modelling patternsModelling patterns– Point abstractions, abstraction from scheduling mechanisms, stream-based

modelling, scalability of model, model refinement



Validation

• Validation guidelinesValidation guidelines– Perform step-by-step inspections– Analyse easy-to-check properties– Investigate impact of rare events– Consider symmetries in system– Recall considerations on adequacy of abstractions



Property Formalisation• Two options for expressing performance metrics

– Performance analysis based on external observersexternal observers: temporal rewards– ReflexiveReflexive performance analysis: POOSL

• Extend model with additional behaviour to evaluate performance metrics

• Guidelines for reflexive performance analysis– Define extensions in subclasses– Use method overriding capabilities of POOSL

Formal SemanticsFormal Semantics

Understandable POOSL Model

Markov Decision Process

+ Extensions for Performance Evaluation

+ Reward Structure

External SchedulerExternal Scheduler

Discrete-Time Markov Chain + Reward Structure



Performance Evaluation• Analytical computationcomputation: Ergodic Theorem• Estimation by simulationsimulation: Central Limit Theorem

– Example: packet-loss probability

– Enables accuracy analysis with confidence intervalsconfidence intervals– Enables automatic terminationautomatic termination of simulation when results are accurate

N(0,1) σ

μ)(1

μ then )(1

if d.1a.s.

1

n

iin

ii

XPacketLostn

nXPacketLostn



Accuracy Analysis• For Internet Router example: a 99% confidence interval

[0.001046, 0.001052] is obtained for the average latency– Estimation of average latency is 0.001049 seconds– We are 99% sure that the real average latency lies in [0.001046, 0.001052]– An upper bound for the relative error is 1%– Takes 0.0072 seconds of simulated real-time– Takes 11.9 hours of simulation time

• Central Limit Theorem only applicable for simple averages– Latency, packet loss probability

• Complex averages and variances: algebra of confidence intervalsalgebra of confidence intervals– Defines set operations on confidence intervals– Enables accuracy analysis for range of different averages and variances– Jitter, buffer occupancy, processor utilisation, throughput, burstiness

• POOSL library classes for analysing different long-run averages



Summary• Three large case studies in corporation with industrial partners• SHE method distinguishes three phases

– Formulation based on informal UML diagrams– Formalisation with formal modelling language POOSL

• Guidelines for modelling and validation• Modelling patterns• Guidelines for reflexive performance analysis

– Performance evaluation based on Markov chain analysis• POOSL model implicitly defines Markov chain• Extensions for performance analysis define reward structure• Analytical computation or estimation by simulation

• Techniques for performance analysis– Confidence intervals allow automatic termination of simulation– Algebra of confidence intervals for accuracy analysis of complex metrics

• Library classes for accuracy analysis

Date post:	31-Mar-2015
Category:	Documents
Upload:	shamar-atkinson
View:	213 times
Download:	0 times

Technische universiteit eindhoven PROGRESS 11 December 2002btheelen1 Performance Modelling of...

Documents