Examples Volume 2 Dugongs: a nonconjugate, nonlinear model Orange trees: a hierarchical, nonlinear model Multivariate Orange trees: a hierarchical, nonlinear model Biopsies: latent class model Eyes: normal mixture model Hearts: a mixture model for count data Air: covariate measurement error Cervix: case-control study with errors in covariates Jaw: repeated measures analysis of variance Birats: a bivariate Normal hierarchical model Schools: muliivariate hierarchical model of examination results Ice: non-parametric smoothing in an age-cohort model Beetles: logistic, probit and extreme value models Alli: multinomial logistic model Endo: conditional inference in case-control studies Stagnant: a change point problem Asia: an expert system References: Sorry - an on-line version of the references is currently unavailable. Please refer to the existing Examples documentation available from http://www.mrc-bsu.cam.ac.uk/bugs. Contents [1]
Transcript
Examples Volume 2
Dugongs: a nonconjugate, nonlinear model
Orange trees: a hierarchical, nonlinear model
Multivariate Orange trees: a hierarchical, nonlinear model
Biopsies: latent class model
Eyes: normal mixture model
Hearts: a mixture model for count data
Air: covariate measurement error
Cervix: case-control study with errors in covariates
Jaw: repeated measures analysis of variance
Birats: a bivariate Normal hierarchical model
Schools: muliivariate hierarchical model of examination results
Ice: non-parametric smoothing in an age-cohort model
Beetles: logistic, probit and extreme value models
Alli: multinomial logistic model
Endo: conditional inference in case-control studies
Stagnant: a change point problem
Asia: an expert system
References:Sorry - an on-line version of the references is currently unavailable.Please refer to the existing Examples documentation available fromhttp://www.mrc-bsu.cam.ac.uk/bugs.
Contents
[1]
Dugongs: nonlinear growth curve
Carlin and Gelfand (1991) present a nonconjugate Bayesian analysis of the following data setfrom Ratkowsky (1983):
The data are length and age measurements for 27 captured dugongs (sea cows). Carlin andGelfand (1991) model this data using a nonlinear growth curve with no inflection point and anasymptote as Xi tends to infinity:
Yi ~ Normal(µi, τ), i = 1,...,27
µi = α - βγXi α, β > 1; 0 < γ < 1
Standard noninformative priors are adopted for α, β and τ, and a uniform prior on (0,1) isassumed for γ. However, this specification leads to a non conjugate full conditional distributionfor γ which is also non log-concave. The graph and corresponding BUGS code is given below
This dataset was originally presented by Draper and Smith (1981) and reanalysed by Lindstromand Bates (1990). The data Yij consist of trunk circumference measurements recorded at time xj,j=1,...,7 for each of i = 1,..., 5 orange trees. We consider a logistic growth curve as follows:
The hybrid Metropolis algorithm is used to sample the theta parameters in this model. The steplength used for this algorithm adapts for the first 4000 iterations and these samples arediscarded from the summary statistics. A further 1000 update burn-in followed by 10000updates gave the following parameter estimates:
The current point Metropolis algorithm is used to sample the theta parameters in this model. TheGaussian proposal distribution used for this algorithm adapts for the first 4000 iterations andthese samples are discarded from the summary statistics. A further 1000 update burn-infollowed by 10000 updates gave the following parameter estimates:
We repeat the Otrees example, replacing the 3 independent univariate Normal priors for each φik, k=1,2,3 by a multivariate Normal prior φφφφ i ~ MNV(µµµµ, ΤΤΤΤ)
Spiegelhalter and Stovin (1983) presented data on repeated biopsies of transplanted hearts, inwhich a total of 414 biopsies had been taken at 157 sessions. Each biopsy was graded onevidence of rejection using a 4 category scale of none (O), minimal (M), mild (+) and moderate-severe (++). Part of the data is shown below.
The sampling procedure may not detect the area of maximum rejection, which is considered thetrue underlying state at the time of the session and denoted ti --- the underlying probabilitydistribution of the four true states is denoted by the vector p. It is then assumed that each of theobserved biopsies are conditionally independent given this truestate with the restriction that thereare no`false positives': i.e. one cannot observe a biopsy worse than the true state. We then havethe sampling model
bi ~ Multinomial(eti, ni)
ti ~ Categorical(p)
where bi denotes the multinomial response at session i where ni biopsies have been taken, andejk is the probability that a true state ti = j generates a biopsy in state k.The no-false-positiverestriction means that e12 = e13 = e14 = e23 = e24 = e34 = 0. Spiegelhalter and Stovin (1983)estimated the parameters ej and p using the EM algorithm, with some smoothing to avoid zeroestimates.
The appropriate graph is shown below, where the role of the true state ti is simply to pick theappropriate row from the 4 x 4 error matrix e. Here the probability vectors ej (j = 1,...,4) and p areassumed to have uniform priors on the unit simplex, which correspond to Dirichlet priors with allparameters being 1.
The BUGS code for this model is given below. No initial values are provided for the latent states,since the forward sampling procedure will find a configuration of starting values that iscompatible with the expressed constraints. We also note the apparent ``cycle'' in the graphcreated by the expression nbiops[i] <- sum(biopsies[i,]). This will lead Such ``cycles'' are
Bowmaker et al (1985) analyse data on the peak sensitivity wavelengths for individualmicrospectrophotometric records on a small set of monkey's eyes. Data for one monkey (S14 inthe paper) are given below (500 has been subtracted from each of the 48 measurements).
Part of the analysis involves fitting a mixture of two normal distributions with common variance tothis distribution, so that each observation yi is assumed drawn from one of two groups. Ti = 1, 2be the true group of the i th observation, where group j has a normal distribution with mean λj andprecision τ. We assume an unknown fraction P of observations are in group 2, 1 - P in group 1.The model is thus
yi ~ Normal(λTi, τ)
Ti ~ Categorical(P).
We note that this formulation easily generalises to additional components to the mixture, althoughfor identifiability an order constraint must be put onto the group means.
Robert (1994) points out that when using this model, there is a danger that at some iteration, allthe data will go into one component of themixture, and this state will be difficult to escape from ---this matches our experience. obert suggests a re-parameterisation, a simplified version ofwhich is to assume
λ2 = λ1 + θ, θ > 0.
λ1, θ, τ, P, are given independent ``noninformative" priors, including a uniform prior for P on (0,1).The appropriate graph and the BUGS code are given below.
The table below presents data given by Berry (1987) on the effect of a drug used to treat patientswith frequent premature ventricular contractions (PVCs) of the heart.
Farewell and Sprott (1988) model these data as a mixture distribution of Poisson counts in whichsome patients are "cured" by the drug, whilst others experience varying levels of response butremain abnormal. A zero count for the post-drug PVC may indicate a "cure", or may represent asampling zero from a patient with a mildly abnormal PVC count. The following model thus isassumed:
xi ~ Poisson(λi) for all patientsyi ~ Poisson(βλi) for all uncured patients
P(cure) = θ
To eliminate nuisance parameters li, Farewell and Sprott use the conditional distribution of yigiven ti = xi + yi. This is equivalent to a binomial likelihood for yi with denominator ti andprobability p = b /(1+b) (see Cox and Hinkley, 1974 pp. 136-137 for further details of theconditional distribution for Poisson variables). Hence the final mixture model may be expressedas follows:
Whittemore and Keller (1988) use an approximate maximum likelihood approach to analyse thedata shown below on reported respiratory illness versus exposure to nitrogen dioxide (NO2) in103 children. Stephens and Dellaportas (1992) later use Bayesian methods to analyse the samedata.
A discrete covariate zj (j = 1,2,3) representing NO2 concentration in the child's bedroomclassified into 3 categories is used as a surrogate for true exposure. The nature of themeasurement error relationship associated with this covariate is known precisely via acalibration study, and is given by
xj = α + β zj + ε j
where α = 4.48, β = 0.76 and ε j is a random element having normal distribution with zero meanand variance σ2 (= 1/τ) = 81.14. Note that this is a Berkson (1950) model of measurement error,in which the true values of the covariate are expressed as a function of the observed values.Hence the measurement error is independent of the latter, but is correlated with the trueunderlying covariate values. In the present example, the observed covariate zj takes values 10,30 or 50 for j = 1, 2, or 3 respectively (i.e. the mid-point of each category), whilst xj is interpretedas the "true average value" of NO2 in group j. The response variable is binary, reflectingpresence/absence of respiratory illness, and a logistic regression model is assumed. That is
yj ~ Binomial(pj, nj)logit(pj) = θ1 + θ2 xj
where pj is the probability of respiratory illness for children in the jth exposure group. Theregression coefficients θ1 and θ2 are given vague independent normal priors. The graphicalmodel is shown below:
Cervix: case - control study with errors incovariates
Carroll, Gail and Lubin (1993) consider the problem of estimating the odds ratio of a disease din a case-control study where the binary exposure variable is measured with error. Their exampleconcerns exposure to herpes simplex virus (HSV) in women with invasive cervical cancer (d=1)and in controls (d=0). Exposure to HSV is measured by a relatively inaccurate western blotprocedure w for 1929 of the 2044 women, whilst for 115 women, it is also measured by a refinedor "gold standard'' method x. The data are given in the table below. They show a substantialamount of misclassification, as indicated by low sensitivity and specificity of w in the "complete''data, and Carroll, Gail and Lubin also found that the degree of misclassification was significantlyhigher for the controls than for the cases (p=0.049 by Fisher's exact test).
They fitted a prospective logistic model to the case-control data as follows
di ~ Bernoulli(pi) i = 1,...,2044logit(pi) = β0C + βxi i = 1,...,2044
where β is the log odds ratio of disease. Since the relationship between d and x is only directlyobservable in the 115 women with "complete'' data, and because there is evidence of differentialmeasurement error, the following parameters are required in order to estimate the logistic model
The differential probability of being exposed to HSV (x=1) for cases and controls is calculated asfollows
The BUGS code is given below. The role of the variables x1 and d1 is to pick the appropriatevalue of φ (the incidence of w) for any given true exposure status x and disease status d. Since xand d take the values 0 or 1, and the subscripts for φ take values 1 or 2, we must first add 1 toeach x[i] and d[i] in the BUGS code before using them as index values for φ. BUGS does notallow subscripts to be functions of variable quantities --- hence the need to create x1and d1 foruse as subscripts. In addition, note that γ1 and γ2 were not simulated directly in BUGS, but werecalculated as functions of other parameters. This is because the dependence of γ1 and γ2 on dwould have led to a cycle in the graphical model which would no longer define a probabilitydistribution.
model
for (i in 1 : N) x[i] ~ dbern(q) # incidence of HSVlogit(p[i]) <- beta0C + beta * x[i] # logistic modeld[i] ~ dbern(p[i]) # incidence of cancerx1[i] <- x[i] + 1d1[i] <- d[i] + 1
We return to the Rats example, and illustrate the use of a multivariate Normal (MVN) populationdistribution for the regression coefficients of the growth curve for each rat. This is the modeladopted by Gelfand etal (1990) for these data, and assumes a priori that the intercept and slopeparameters for each rat are correlated. For example, positive correlation would imply that initiallyheavy rats (high intercept) tend to gain weight more rapidly (steeper slope) than lighter rats. Themodel is as follows
where Yij is the weight of the ith rat measured at age xj, and ββββ i denotes the vector (β1i, β2i). Weassume 'non-informative' independent univariate Normal priors for the separate componentsµβ1 and µβ2. A Wishart(R, ρ) prior was specified for ΩΩΩΩ , the population precision matrix of the
regression coefficients. To represent vague prior knowledge, we chose the the degrees offreedom ρ for this distribution to be as small as possible (i.e. 2, the rank of ΩΩΩΩ ). The scale matrixwas specified as
This represents our prior guess at the order of magnitude of the covariance matrix ΩΩΩΩ -1 for ββββ i(see Classic BUGS manual (version 0.5) section on Multivariate normal models), and isequivalent to the prior specification used by Gelfand et al. Finally, a non-informativeGamma(0.001, 0.001) prior was assumed for the measurement precision τc.
R = | 200, 0 || 0, 0.2 |
Birats Examples Volume II
[23]
model
for( i in 1 : N ) beta[i , 1 : 2] ~ dmnorm(mu.beta[], R[ , ])for( j in 1 : T )
Schools: ranking school examination resultsusing multivariate hierarcical models
Goldstein et al. (1993) present an analysis of examination results from inner London schools.They use hierarchical or multilevel models to study the between-school variation, and calculateschool-level residuals in an attempt to differentiate between `good' and `bad' schools. Here weanalyse a subset of this data and show how to calculate a rank ordering of schools and obtaincredible intervals on each rank.
Data
Standardized mean examination scores (Y) were available for 1978 pupils from 38 differentschools. The median number of pupils per school was 48, with a range of 1--198. Pupil-levelcovariates included gender plus a standardized London Reading Test (LRT) score and a verbalreasoning (VR) test category (1, 2 or 3, where 1 represents the highest ability group) measuredwhen each child was aged 11. Each school was classified by gender intake (all girls, all boys ormixed) and denomination (Church of England, Roman Catholic, State school or other); thesewere used as categorical school-level covariates.
Model
We consider the following model, which essentially corresponds to Goldstein et al.'s model 1.
+ β4 Girls' schoolj + β5 Boys' schoolj + β6 CE schoolj+ β7 RC schoolj + β8 other schoolj
log τij = θ + φ LRTij
where i refers to pupil and j indexes school. We wish to specify a regression model for thevariance components, and here we model the logarithm of τij (the inverse of the between-pupilvariance) as a linear function of each pupil's LRT score. This differs from Goldstein et al.'s modelwhich allows the variance σ2ij to depend linearly on LRT. However, such a parameterization maylead to negative estimates of σ2ij.
Prior distributions
The fixed effects βk (k=1,...,8), θ and φ were assumed to follow vague independent Normaldistributions with zero mean and low precision = 0.0001. The random school-level coefficients
Examples Volume II Schools
[26]
αkj (k = 1,2,3) were assumed to arise from a multivariate normal population distribution withunknown mean γγγγ and covariance matrix ΣΣΣΣ . A non-informative multivariate normal prior was thenspecified for the population mean γγγγ , whilst the inverse covariance matrix ΤΤΤΤ = ΣΣΣΣ -1 was assumedto follow a Wishart distribution. To represent vague prior knowledge, we chose the the degreesof freedom for this distribution to be as small as possible (i.e. 3, the rank of ΤΤΤΤ). The scale matrixR was specified as
which represents our prior guess at the order of magnitude of ΣΣΣΣ .
Note that school is a 1978 x 3 matrix taking value 1 for all pupils in school 1, 2 for all pupils inschool 2 and so on. For computational convenience, Y, mu and tau are indexed over a singledimension p = 1,...,1978 rather than as pupil i within school j as used in equations above. Theappropriate school-level coefficients for pupil p are then selected using the school indicator inrow p of the data array --- for example alpha[school[p],1].
Inits ( click to open )
Results
A 1000 update burn in followed by a further 10000 updates gave the parameter estimates
Estimating the ranks
The school-specific intercept αj1 measures the 'residual effect' for school j after adjusting forpupil- and school-level covariates. This might represent an appropriate quantity by which to rankschools' performance. We compute the ranks in BUGS using the "rank" option of the "Statistics"menu, which we set for the variable alpha at the same time as we set the "sample monitor"option. Since the rank is a function of stochastic nodes, its value will change at every iteration.Hence we may obtain a posterior distribution for the rank of alpha[, k] which may be summarizedby posterior histograms as shown below:
Ice: non-parametric smoothing in an age-cohortmodel
Breslow and Clayton (1993) analyse breast cancer rates in Iceland by year of birth (K = 11cohorts from 1840-1849 to 1940-1949) and by age (J =13 groups from 20-24 to 80-84 years).Due to the number of empty cells we consider a single indexing over I = 77 observed number ofcases, giving data of the following form.
In order to pull in the extreme risks associated with small birth cohorts, Breslow andClayton first consider the exchangeable model
They then consider the alternative approach of smoothing the rates for the cohorts by assumingan auto-regressive model on the β 's, assuming the second differences are independent normalvariates. This is equivalent to a model and prior distribution
We note that β1 and β2 are given "non-informative" priors, but retain a τ term in order to providethe appropriate likelihood for τ.
For computational reasons Breslow and Clayton impose constraints on their random effects βkin order that their mean and linear trend are zero, and counter these constraints by introducing a
i agei yeari casesi person-yearsi_____ ________________________________________
1 1 6 2 413802 1 7 0 43650
... ...77 13 5 31 13600
Ice Examples Volume II
[29]
linear term b x yeari and allowing unrestrained estimation of αj. Since we allow free movementof the β 's we dispense with the linear term, and impose a "corner" constraint α1 =0 .
model
for (i in 1:I) cases[i] ~ dpois(mu[i])log(mu[i]) <- log(pyr[i]) + alpha[age[i]] + beta[year[i]]
Dobson (1983) analyses binary dose-response data published by Bliss (1935), in which thenumbers of beetles killed after 5 hour exposure to carbon disulphide at N = 8 differentconcentrations are recorded:
We assume that the observed number of deaths ri at each concentration xi is binomial withsample size ni and true rate pi. Plausible models for pi include the logistic, probit and extremevalue (complimentary log-log) models, as follows
pi = exp(α + βxi) / (1 + exp(α + βxi)
pi = Phi(α + βxi)
pi = 1 - exp(-exp(α + βxi))
The corresponding graph is shown below:
Concentration (xi) Number of beetles (ni) Number killed (ri)______________________________________________________
Endo: conditional inference in case-control studies
Breslow and Day (1980) analyse a set of data from a case-control study relating endometrialcancer with exposure to estrogens. 183 pairs of cases and controls were studied, and the fulldata is shown below.
We denote estrogen exposure as xij for the ith case-control pair, where j=1 for a case and j=2 fora control. The conditional likelihood for the log (odds ratio) β is then given by Πi exp βxi1 / (expβxi1 + exp βxi2)
We shall illustrate three methods of fitting this model. It is convenient to denote the fixed diseasestatus as a variable Yi1 = 1, Yi2 = 0.
First, Breslow and Day point out that for case-control studies with a single control per case, wemay obtain this likelihood by using unconditional logistic regression for each case-control pair.That is
Yi1 ~ Binomial(pi,2)logit pi = β (xi1 − xi2)
Second, the Classic BUGS manual (version 0.5) section on Conditional likelihoods in case-control studies discusses fitting this likelihood directly by assuming the model
Yi. ~ Multinomial(pi., 1)pij = eij / Σj eij
log eij = β xij
Finally, the Classic BUGS manual (version 0.5) shows how the multinomial-Poissontransformation can be used. In general, this will be more efficient than using the multinomial-logistic parameterisation above, since it avoids the time-consuming evaluation of Σj eij. However,in the present example this summation is only over J=2 elements, whilst the multinomial-Poissonparameterisation involves estimation of an additional intercept parameter for each of the 183strata. Consequently the latter is less efficient than the multinomial-logistic in this case.
Status of controlStatus of case Not exposed Exposed____________________________________Not exposed n00 = 121 n01 = 7Exposed n10 = 43 n11 = 12
Endo Examples Volume II
[35]
We note that all these formulations may be easily extended to include additional subject-specificcovariates, and that the second and third methods can handle arbitrary numbers of controls percase. In addition, the Bayesian approach allows the incorporation of hierarchical structure,measurement error, missing data and so on.
All these techniques are illustrated in the code given below, which includes a transformation ofthe original summary statistics into full data. In this example, all but the second conditional-likelihood approach are commented out.
model# transform collapsed data into full
for (i in 1 : I)Y[i,1] <- 1Y[i,2] <- 0
# loop around strata with case exposed, control not exposed (n10)
for (i in 1 : n10)est[i,1] <- 1est[i,2] <- 0
# loop around strata with case not exposed, control exposed (n01)
for (i in (n10+1) : (n10+n01))est[i,1] <- 0est[i,2] <- 1
# loop around strata with case exposed, control exposed (n11)
for (i in (n10+n01+1) : (n10+n01+n11))est[i,1] <- 1est[i,2] <- 1
# loop around strata with case not exposed, control not exposed (n00)
for (i in (n10+n01+n11+1) :I )est[i,1] <- 0est[i,2] <- 0
# METHOD 3 fit standard Poisson regressions relative to baseline#for (j in 1:J) # Y[i, j] ~ dpois(mu[i, j]);# log(mu[i, j]) <- beta0[i] + beta*est[i, j];
#beta0[i] ~ dnorm(0, 1.0E-6)
Data ( cklick to open )
Inits ( cklick to open )
Results
A 5000 update burn in followed by a further 10000 updates gave the parameter estimates
mean sd MC_error val2.5pc median val97.5pc start samplebeta 1.871 0.4123 0.009414 1.111 1.844 2.761 5001 10000
Endo Examples Volume II
[37]
Stagnant: a changepoint problem(and an illustration of how NOT to do MCMC!)
Carlin, Gelfand and Smith (1992) analyse data from Bacon and Watts (1971) concerning achangepoint in a linear regression.
Note the repeated x's.
We assume a model with two straight lines that meet at a certain changepoint xk --- this is slightlydifferent from the model of Carlin, Gelfand and Smith (1992) who do not constrain the twostraight lines to cross at the changepoint. We assume
Yi ~ Normal(µi, τ)µi = α + βJ[i] (xi - xk) J[i]=1 if i <= k J[i]=2 if i > k
giving E(Y) = α at the changepoint, with gradient β1 before, and gradient β2 after thechangepoint. We give independent "noninformative'' priors to α, β1, β2 and τ.
Note: alpha is E(Y) at the changepoint, so will be highly correlated with k. This may be avery poor parameterisation.
Note way of constructing a uniform prior on the integer k, and making the regressionparameter depend on a random changepoint.
model
i xi Yi i xi Yi i xi Yi______________________________________________________
Lauritzen and Spiegelhalter (1988) introduce a fictitious "expert system" representing thediagnosis of a patient presenting to a chest clinic, having just come back from a trip to Asia andshowing dyspnoea (shortness-of-breath). The BUGS code is shown below and the conditionalprobabilities used are given in Lauritzen and Spiegelhalter (1988). Note the use of max to dothe logical-or. The dcat distribution is used to sample values with domain (1,2) with probabilitydistribution given by the relevant entries in the conditional probabilitytables.
