SENG521 (Fall 2002) [email protected] 1
SENG 521SENG 521Software Reliability & Software Reliability & TestingTesting
Software Reliability ModelsSoftware Reliability Models
(Part 2b)(Part 2b)
Department of Electrical & Computer Engineering, University of Calgary
B.H. Far ( [email protected] )http://www.enel.ucalgary.ca/~far/Lectures/SENG521/02b/
SENG521 (Fall 2002) [email protected] 2
ContentsContents Basic Features of the Software Reliability ModelsBasic Features of the Software Reliability Models Single Failure ModelSingle Failure Model Reliability Growth ModelReliability Growth Model Exponential Failure Class ModelsExponential Failure Class Models Weibull and Gamma Failure Class ModelsWeibull and Gamma Failure Class Models Infinite Failure Category ModelsInfinite Failure Category Models Bayesian ModelsBayesian Models Early Life-Cycle Prediction ModelsEarly Life-Cycle Prediction Models
SENG521 (Fall 2002) [email protected] 3
GoalGoal It is important to be able to
Predict probability of failure of a component or system
Estimate the mean time to the next failure Predict number of (remaining) failures
during the development. Such tasks are the target of the reliability
management models.
SENG521 (Fall 2002) [email protected] 4
BackgroundBackgroundRandomness & Randomness & ProbabilityProbability
SENG521 (Fall 2002) [email protected] 5
Randomness /1Randomness /1 Random actions:
Introduction of defects into the code Execution of the test-cases
We should define some random processes random processes to represent the randomness
How to handle randomness? Collect failure data through testing Find a distribution function that is a best-fit for the
collected data Make assumptions about the presence of errors and
reliability
SENG521 (Fall 2002) [email protected] 6
Randomness /2Randomness /2 What is a random variable?
A random variable x on a sample space S is a rule that assigns a numerical value to each outcome of S (a function of S into a set of real numbers)
In reliability modeling what can be represented by random variable? Time of failure Number of failures
SENG521 (Fall 2002) [email protected] 7
Probability Distribution /1Probability Distribution /1 Suppose that a random variable X assigns a
finite number of values to a sample space S Then X induces a distribution function f that
assigns probabilities to the points in Rx
Rx ={x1, x2, x3, …, xn}
f(xk) = P( X=xk)
SENG521 (Fall 2002) [email protected] 8
Probability Distribution /2Probability Distribution /2 The set of ordered pairs [xk, f(xk)] is usually
represented by a table or a graph (histogram)
The expected value of X, denoted by E(X) is defined by
E(X) = x1 f(x1) + x2 f(x2) + …+ xn f(xn)
0
5
10
15
20
1 1.5 2 2.5 3 3.5 4 4.5 5
Time (days)
Fre
qu
en
cy
(Fa
ilu
re/t
ime
un
it)
SENG521 (Fall 2002) [email protected] 9
Probability Distribution /3Probability Distribution /3 Let M(t) be a random process representing the
number of failures at time t The mean function μ(t) represents the expected
number of failures at time t μ(t) = E(M(t))
Failure intensity is the rate of change of the expected number of failures with respect to time
λ(t) = d μ(t) / dt (t) is the number of failures per unit time (t) is an instantaneous value
SENG521 (Fall 2002) [email protected] 10
Probability Distribution /4Probability Distribution /4 Discrete distributions:
Binomial distribution Poisson distribution
Continuous distributions: Normal / Gaussian distribution Lognormal distribution Weibull distribution Rayleigh distribution Exponential distribution Gamma distribution 2 distribution
SENG521 (Fall 2002) [email protected] 11
Binomial Distribution /1Binomial Distribution /1 Gives probability of exact number
of successes in n independent trials, when probability of success p on single trial is a constant.
Situations with only 2 outcomes (success or failure)
Probability remains the same for all independent trials (Bernoulli trials)
Jakob Bernoulli (1645-1705)
SENG521 (Fall 2002) [email protected] 12
Binomial Distribution /1Binomial Distribution /1 Probability of exactly x successes:
n : number of trials f(x) : probability of x successes in n trials p : probability of success q : probability of failure
p + q =1
xnxxnx qpx
nqp
xnx
nxf
!!
!
SENG521 (Fall 2002) [email protected] 13
Binomial Distribution /2Binomial Distribution /2 Probability of at least r successes:
n : number of trials f(x) : probability of x successes in n trials p : probability of success q : probability of failure p + q =1 F(r) : probability of obtaining r or fewer successes in n trials
xnxr
x
qpx
nrF
0
Calculating F(r) becomes increasingly difficult as n (sample set) gets larger
It is possible to find an approximate solution by means of a normal distribution
SENG521 (Fall 2002) [email protected] 14
Example: Binomial Example: Binomial DistributionDistribution
Acceptance sampling:Acceptance sampling: A lot is accepted if not more than 2 defectives are
found in a sample of 6. The defect probability is 25%. Probability of having exactly 2 defects in the lot is:
Probability of having more than 4 defects in the lot is:
0046.025.075.025.05
6654
297.075.025.02
62
615
42
FFrF
f
SENG521 (Fall 2002) [email protected] 15
Poisson Distribution /1Poisson Distribution /1 Some events are rather rare, they don’t happen
that often. Still, over a period of time, we want to say something about the nature of rare events.
Poisson distribution is special case of binomial distribution (either p or q is very small and n very large)
Conditions under which a Poisson distribution holds
counts of rare events all events are independent average rate does not change over the period of
interest
Simeon Poisson (1781-1840)
SENG521 (Fall 2002) [email protected] 16
Poisson Distribution /2Poisson Distribution /2 Poisson distribution is a special case of
binomial distribution (either p or q is very small and n very large): =np
: mean rate of occurrence (in statistics literature is usually denoted by )
x : observed number of failures
0,1,2,... xex!μ
f(x) μx
SENG521 (Fall 2002) [email protected] 17
Example: Poisson Example: Poisson DistributionDistribution
Suppose that the defect rate is only 2%
find the probability that there are 3 defective items in a sample of 100 items.
18.03
23
!
23
e
xf
SENG521 (Fall 2002) [email protected] 19
Exponential Failure Exponential Failure Time ModelsTime Models
SENG521 (Fall 2002) [email protected] 20
Exponential Failure Time Exponential Failure Time ModelsModels
Assumes finite failure models Functional form of failure intensity function
is exponential Binominal type:
All faults have a constant hazard rate z(t) = Hazard rate for the i-th fault is a function of the
remaining number of faults Failure intensity is teNt
SENG521 (Fall 2002) [email protected] 21
Various Reliability ModelsVarious Reliability Models Exponential Failure Time ModelsExponential Failure Time Models
Jelinski-Moranda model (JM) Nonhomogeneous Poisson Process model
(NHPP) Schneidewind model Musa’s Basic Execution Time model (BET) Hyperexponential model (HE) Others
SENG521 (Fall 2002) [email protected] 22
Jelinski-Moranda Model /1Jelinski-Moranda Model /1 Jelinski-Moranda model follows exponential
distribution but differs from the Geometric model in that the parameter used is proportional to the remaining number of faults rather than a constant.
All faults equally contribute to the reliability of the system
Hazard Rate z(t): Probability density function divided by reliability function.
is a constant
SENG521 (Fall 2002) [email protected] 23
Jelinski-Moranda Model /1Jelinski-Moranda Model /1 Assumptions:Assumptions:
The rate of fault detection is proportional to the current fault content of the software.
The fault detection rate remains constant over the intervals between faults.
A fault is corrected instantaneously without introducing new faults into the software.
Every fault has the same chance of being encountered within a severity class as any other fault in that class.
The failures are independent. Data requirement:Data requirement:
Either actual failure times: t1, t2, …, tn Or elapsed time between failures: xi = ti – ti-1
SENG521 (Fall 2002) [email protected] 24
Jelinski-Moranda Model /2Jelinski-Moranda Model /2 Meantime between failures at time t
MTTF = 1/ (N-(i-1)) Mean failures experienced
Failure intensity function
t
t
eNt
eNt
1 N total number of faults (constant)
Parameters to estimate using collected data are: N and
N total number of faults (constant)
Parameters to estimate using collected data are: N and
SENG521 (Fall 2002) [email protected] 25
Non Homogeneous Non Homogeneous Poisson Process Model /1Poisson Process Model /1
Assumes that the cumulative number of failures detected at any time follows a Poisson distribution.
Time periods can be unequal Data requirement:Data requirement:
Fault counts on each testing interval: f1, f2, …, fn
Completion time of each period: t1, t2, …, tn
SENG521 (Fall 2002) [email protected] 26
Non Homogeneous Poisson Non Homogeneous Poisson Process Model /2Process Model /2
Assumptions:Assumptions: Every failure has the same chance of being detected as
any other failure. The cumulative number of failures detected at any time
follows a Poisson distribution with mean (t). That mean is such that the expected number of failures in
any small time interval about t is proportional to the number of undetected failures at time t.
The mean is assumed to be a bounded non-decreasing function with (t) approaching in the limit, as the length of testing goes to infinity.
Perfect debugging is assumed.
SENG521 (Fall 2002) [email protected] 27
Non Homogeneous Poisson Non Homogeneous Poisson Process Model /3Process Model /3
Mean failures experienced
Failure intensity function
btNbet
bteNt 1
Parameters to estimate using collected data are: N and b
Parameters to estimate using collected data are: N and b
SENG521 (Fall 2002) [email protected] 28
Schneidewind ModelSchneidewind Model Assumes that the current fault rate might be a better
predictor of the future behaviour than the observed rate in the distant past
Suppose that there are n units of time all of fixed length
Has 3 variations: Utilize all individual fault counts from all n time periods Ignore fault counts from 1 to s-1 periods (forget the
distant past) Use cumulative fault count for 1 to s-1 periods and
individual fault counts for s to n
SENG521 (Fall 2002) [email protected] 29
Musa Basic Model /1Musa Basic Model /1 One of the most widely used software reliability
models. Uses execution time rather than calendar time.
0 and 1 are parameters. 0 is equal to the number of faults in the system and 1 is a fault reduction factor.
SENG521 (Fall 2002) [email protected] 30
Musa Basic Model /2Musa Basic Model /2 Assumptions:Assumptions:
The detections of failures are independent of one another.
The software failures are observed (i.e. the total number of failures has an upper bound).
The execution times (measured in CPU time) between failures are piecewise exponentially distributed.
The hazard rate is proportional to the number of faults remaining in the program.
The fault correction rate is proportional to the failure occurrence rate.
Perfect debugging is assumed.
SENG521 (Fall 2002) [email protected] 31
Musa Basic Model /3Musa Basic Model /3 Meantime between failures at time t
Mean failures experienced
Failure intensity function
tet 110
tet 110
t
MTTF nC
exp MTTFMTTF(t)00
0
n0 = total number of errors in the program
MTTF0=MTTF at the start of testing
C= test compression factor
SENG521 (Fall 2002) [email protected] 32
Musa Basic Model /4Musa Basic Model /4 Number of errors at time t
Reliabilityn0 = total number of errors in the program
MTTF0=MTTF at the start of testing
C= test compression factor
MTTFt
expR(t)
SENG521 (Fall 2002) [email protected] 33
Hyper-exponential Hyper-exponential Model /1Model /1
The basic idea is that the different sections (or classes) of the software experience an exponential failure rate; however, the rates vary over these sections to reflect their different natures. This could be due to different programming groups doing the different parts, old versus new code, sections written in different languages, etc.
We thus reflect the sum of these different exponential growth curves, not by another exponential, but by a hyper-exponential growth curve.
SENG521 (Fall 2002) [email protected] 34
Hyper-exponential Hyper-exponential Model /2Model /2
Data requirement:Data requirement: Fault counts on each testing interval: f1, f2, …, fn
Completion time of each period: t1, t2, …, tn
Failure intensity function
101101
1
i
K
iii
K
i
tii
pp
epNt i
SENG521 (Fall 2002) [email protected] 35