1
Use of Asymmetric Loss Functions in Use of Asymmetric Loss Functions in Sequential Estimation Problem for the Sequential Estimation Problem for the
Multiple Linear RegressionMultiple Linear Regression
Raghu Nandan SenguptaDepartment of Industrial and Management Engineering
Indian Institute of Technology Kanpur
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 2
What is this talk all about?
Use of Asymmetric Loss Functions in Sequential Estimation Problem for the Multiple Linear Regression
Raghu Nandan SenguptaJOURNAL OF APPLIED STATISTICS (2008) , 35, 245-261
Abstract: While estimating in a practical situation, asymmetric loss functions are preferred over squared error loss functions, as the former is more appropriate than the latter in many estimation problems. We consider here the problem of fixed precision point estimation of a linear parametric function in beta for the multiple linear regression model using asymmetric loss functions. Due to the presence of nuissance parameters, the sample size for the estimation problem is not known before hand and hence we take the recourse of adaptive multistage sampling methodologies. We discuss here some multistage sampling techniques and compare the performances of these methodologies using simulation runs. The implementation of the codes for our proposed models is accomplished utilizing MATLAB 7.0.1 program run on a Pentium IV machine. Finally we highlight the significance of such asymmetric loss functions with few practical examples.
Key words and phrases: loss function; risk; bounded risk; asymmetric loss function; LINEX loss function; relative LINEX loss function; stopping rule; multistage sampling procedure; purely sequential sampling procedure; batch sequential sampling procedure
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 3
Background
Any population is characterized by X (random variable) which has a particular distribution given by its cumulative distribution function (cdf), where the cdf is given by
P[X ≤ x] = F(x ; θ)Note:
1) In general we select a sample {X1, X2,….., Xn} of random observations to estimate θ
2) The statistics is given by Tn = T(X1, X2,….., Xn) which is an estimator of θ
3) If we consider ∆ as the error in our estimation process, then ∆ = (Tn - θ)
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 4
Loss functions
∆, the error results in different types of LOSS FUNCTIONS, denoted by L(∆) and few examples are
1) Absolute error loss function
L(∆) = |∆|2) Squared error loss (SEL) function
L(∆) = ∆2
3) Linear exponential (LINEX) loss function, [Zellner (1986)], a type of asymmetric loss function
L(∆) = b{ea∆ - a∆ -1}
where ′a′ (≠ 0) is the shape parameter and ′b′ (>0) is the scale parameter4) Balanced loss function (BLF), [Zellner (1994)]
L(∆) = w{g(θ) – g(Tn)}/{g(θ) – g(Tn)} + (1-w)(Tn - θ)2
where 0≤w≤1
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 5
LINEX loss function
L(∆) = b{ea∆ - a∆ -1}
L(∆) L(∆)
∆ ∆a > 0 a < 0
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 6
Few examples where asymmetric loss functions can be used
Marketing strategyExponential survival model, where X is the life time of a component with a pdf
f(x;µ,σ) = (1/σ)exp{-(x - µ)/σ}whereµ = minimum guarantee/warranty time/period1/σ = failure rate
Note: Use a LINEX loss function with an appropriatevalue for ′a′
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 7
Few examples where asymmetric loss functions can be used
Construction of a damUnderestimation of height of dam is more serious than overestimation
Note: Use a LINEX loss function with ′a′ (< 0)
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 8
Few examples where asymmetric loss functions can be used
Reliability of equipmentsExponential life time of equipments, where X is the life of an equipment with a pdf
f(x;θ) = (1/θ)exp{-x/θ}such that the reliability function R(t) is given by
R(t) = P[X > t] = exp{-t/θ}Over estimation of the reliability function can havemarked consequence than under estimation
Note: Use a LINEX loss function with ′a′ (> 0)
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 9
Risk
RISK = EXPECTED LOSS
Let us consider a simple example1) Consider we choose X1, X2,….., Xn (i.i.d) from X ~
N(µ,σ2), but with both µ and σ2 unknown.2) We are interested in estimating µ using3) The loss function for the LINEX loss is of the form
4) The risk is given by
( ) ( ) ( ) ( ), , 1na Xn n nR X E L X E e a Xµµ µ µ− = = − − −
1
1 n
n ii
X Xn =
= ∑
( ) ( ) ( ), 1na Xn nL X e a Xµµ µ−= − − −
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 10
Concept of Bounded Risk
RISK ≤≤≤≤ w (given or a known quantity)
5) The risk is given by
6) For bounded risk we must have
7) As both µ and σ2 are unknown we take the recourse of Sequential Sampling Techniques
( )2 2
, exp 12n
aR X
n
σµ
= −
( )2 2
2log 1e
an
w
σ≥+
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 11
Different Sequential Sampling Methodologies
1) Two stage sampling procedure [Stein (1945)]2) Purely sequential sampling procedure [Ray (1957)]3) Three stage sampling procedure [Hall (1981),
Mukhopadhyay (1980)]4) Accelerated sequential sampling procedure [Hall
(1983), Mukhopadhyay and Solanky (1991)]5) Batch sequential sampling procedure [Liu (1997)]
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 12
Estimation problem for the multiple linear regression
In the context of the multiple linear regression problem formulation we have just discussed for the first paper we now deal with the problem of estimation considering LINEX loss function ~~
/ βθ l=
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 13
Estimation problem for the multiple linear regression
Given ′n′ data points the usual least square error (LSE) estimator of β and the forecasted value of θ are
� βn = (X′′′′n Xn )–1X′′′′nYn
� /
~ ~n nlθ β=
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 14
Estimation problem for the multiple linear regression
However, Zellner(1986) has shown that when σ2 is known, under LINEX loss, the estimator thus found for SEL is inadmissible, being dominated (in terms of risk) by the estimator
( )2 1* / / /
~ ~ ~~ 2n n n na
l l X X lσθ β
−= −
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 15
Estimation problem for the multiple linear regression
Theorem: Under LINEX loss, the estimator
is dominated by the estimator of the form
even when σ2 is unknown
Here we replace σ2 by it usual predictor σ2n
nθ
( )2 1** / /
~ ~2n
n n n na
l X X lσθ θ
−= −
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 16
Estimation problem for the multiple linear regression
Thus under asymmetric loss function the shrinkage estimators and dominates
The corresponding risk of the shrinkage estimatoris given by
when σ2 is known
*nθ nθ
*nθ
( )2 2 1/ /
~ ~2 n na
l X X lσ −
**nθ
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 17
Estimation problem for the multiple linear regression
Now if the corresponding risk is bounded then we have
But if σ2 is unknown we have to solve the problem of finding the optimal sample size by taking the recourse of some adaptive sampling methodologies about which we have discussed before
( )2 2 1/ /
~ ~2 n na
l X X l wσ −
≤
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 18
Estimation problem for the multiple linear regression
Purely sequential sampling procedure
m m+1 N-1 N
( )2 2 1/ /
~ ~inf :
2n
n na
N n m l X X l wσ − = ≥ ≤
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 19
Estimation problem for the multiple linear regression
Purely sequential sampling procedureOne sampling stops we consider the two forecasted values
( )2 1** / /
~ ~2N
N N NNa
l X X lσ
θ θ−
= −
/
~ ~N Nlθ β=
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 20
Estimation problem for the multiple linear regression
Purely sequential sampling procedureThe corresponding risks are
( ) ( ) ( ), 1NaN NR E e aθ θθ θ θ θ− = − − −
( ) ( ) ( )**
** **, 1NaN NR E e a
θ θθ θ θ θ− = − − −
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 21
Estimation problem for the multiple linear regression
Batch sequential sampling procedure1) Choose a positive integer ′k′ and consider 0 < ρ1 < ρ2 < …< ρk <
1, thus the objective is to estimate ′k′ fractions of the sample size using sequential type sampling, but taking batches of observations at each stage.
2) We specify decreasing batch sizes for these ′k′ sampling stages as r1 > r2 >…> rk > 1.
3) In the final stage, sampling is done purely sequentially4) We start with an initial sample of size m and then, for t = 1,2,…,
define
m (m+r1*t1) (N-1*tk) N
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 22
Estimation problem for the multiple linear regression
Batch sequential sampling procedure
1)
2)..
k-1)
k)
( )2 2 1/ /1 1 1
~ ~inf :
2n
n na
R n m r t m l X X l wσρ
− = ≥ + > ≤
( )2 2 1/ /2 1 2 1 2
~ ~inf :
2n
n na
R n R r t R l X X l wσρ
− = ≥ + > ≤
( )2 2 1/ /1 1
~ ~inf :
2n
k k k k k n na
R n R r t R l X X l wσρ
−− −
= ≥ + > ≤
( )2 2 1/ /1
~ ~inf :
2n
k k k n na
N n R r t R l X X l wσ −
+ = ≥ + > ≤
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 23
Estimation problem for the multiple linear regression
Batch sequential sampling procedure
One sampling stops we consider the two forecasted values
( )2 1** / /
~ ~2N
N N NNa
l X X lσ
θ θ−
= −
/
~ ~N Nlθ β=
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 24
Estimation problem for the multiple linear regression
Batch sequential sampling procedure
The corresponding risks are
( ) ( ) ( ), 1NaN NR E e aθ θθ θ θ θ− = − − −
( ) ( ) ( )**
** **, 1NaN NR E e a
θ θθ θ θ θ− = − − −
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 25
Estimation problem for the multiple linear regression
If the parameter value is very small then considering a relative LINEX loss function would be more practical and advisable than a LINEX loss function. The relative LINEX loss function and its corresponding risk is
L(θ,T) = ea(T/θ -1) - a(T/θ - 1) -1
R(θ,T) = E[L(θ,T)] = E[ea(T/θ - 1) - a(T/θ - 1) -1]
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 26
Estimation problem for the multiple linear regression
Similar bounded risk problem formulation for the estimated value was undertaken and corresponding sequential sampling methodologies were considered for the case of relative LINEX loss function
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 27
Simulation for the estimation problem for the multiple linear regression
Data set used for simulationThe manuscript describing the data can be found at www.spatial-statistics.com. One can refer Kelley and Barry (1997) for further detailsThe MLR is of the form
)()()()(log 33
2210 MIMIMIMHVe ββββ +++=
)(log)(log)(log 654 PB
PTRMA eee βββ +++
εββ +++ )(log)(log 87 HHP
ee
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 28
Simulation for the estimation problem for the multiple linear regression
Data set used for simulationWhere1) MHV = Median house value2) MI = Median income3) MA = Housing median age4) TR = Total rooms5) B = Total bedrooms6) P = Population7) H = Households
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 29
Simulation for the estimation problem for the multiple linear regression
For the LINEX loss function the following sampling methodologies were considered with the starting sample size m=10
� PSL: Purely Sequential (m = 10)
� BSL(1): Batch sequential (m = 10, k+1 = 3; ρ1 = 0.80, ρ2 = 0.90, r1 = 24, r2 = 16, r3 = 8)
� BSL(2): Batch sequential (m = 10, k+1 = 3; ρ1 = 0.75, ρ2 = 0.85, r1 = 15, r2 = 10, r3 = 5)
For the relative LINEX loss function the following sampling methodologies were considered with the starting sample size m=10
� PSRL: Purely Sequential (m = 10)
� BSRL(1): Bath sequential (m = 10, k+1 = 3; ρ1 = 0.60, ρ2 = 0.90, r1 = 17, r2
= 13, r3 = 9)
� BSRL(2): Batch sequential (m = 10, k+1 = 3; ρ1 = 0.70, ρ2 = 0.80, r1 = 10, r2 = 7, r3 = 4)
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 30
Simulation for the estimation problem for the multiple linear regression
Consider a = -0.6 and w=0.03 for the relative LINEX loss function
%saveFor BSRL(1)
42 4 0.020655 0.020421 -0.009788 -0.010013 84.60
For BSRL(2)39 6 0.022570 0.022298 -0.007696 -0.007964 76.90
For PSRL
35 26 0.024785 0.024460 -0.005298 -0.005626 ----
N SO ),( θθ NR **( , )NR θ θ )},({2 θθ NRnd **2 { ( , )}ndNR θ θ
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 31
Acknowledgements for my visit� IUSSTF Fellowship Foundation
� Princeton University, USA
� Prof. Jianqing Fan, ORFE Department, Princeton University, USA
� Prof. Lawrence M. Seiford, IOE Department, University of Michigan, Ann Arbor
� Prof. Katta G. Murty, IOE Department, University of Michigan, Ann Arbor
� Prof. Romesh Saigal, IOE Department, University of Michigan, Ann Arbor
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 32
Contact Detail
Raghu Nandan Sengupta
Assistant ProfessorIndustrial & Management Engineering Department
Indian Institute of Technology KanpurKanpur 208 016, UP, INDIA
Ph: +91-512-259-6607; Fax: +91-512-259-7553Email: [email protected]
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 33
Research Areas
� Sequential Analysis� Financial Optimization� Statistical Reliability� Use of different Meta Heuristics
Techniques for Optimization
34
List of Publications1. LINEX Loss Function and its Statistical Application – A Review ; (co-
authors Saiobal Chattopadhyay and Ajit. Chaturvedi), DECISION , Jan-Dec, 1999, 26 , 1-4, 51-76.
2. Sequential Estimation of a Linear Function of Normal Means UnderAsymmetric Loss Function ; (co-authors Saibal Chattopadhyay and AjitChaturvedi), METRIKA , 2000, 52 , 3, 225-235.
3. Asymmetric Penalized Prediction Using Adaptive Sampling Procedures; (co-authors Saibal Chattopadhyay and Sujay Datta), SEQUENTIAL ANALYSIS , 2005, 24 , 1, 23-43.
4. Three-Stage and Accelerated Sequential Point Estimation of the Normal Mean Using LINEX Loss Function; (co-author Saibal Chattopadhyay), STATISTICS , 2006, 40 , 1, 39-49.
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 35
List of Publications5. Use of Asymmetric Loss Functions in Sequential Estimation Problem for
the Multiple Linear Regression, JOURNAL OF APPLIED STATISTICS , 2008, 35 , 8, 245-261.
6. "Impact of information sharing and lead time on bullwhip effect and on-hand inventory" ; (co-authors, Sunil Agrawal and Kripa Shanker), EUROPEAN JOURNAL OF OPERATIONAL RESEARCH , (Accepted and forthcoming).
7. Bankruptcy Prediction using Artificial Immune Systems" , (co-author Rohit Singh), LECTURE NOTES IN COMPUTER SCIENCE (LNCS), L.N.de Castro, F.J.Zuben and H.Knidel (Eds.), 2007, 4628 , 131-141.
R.N.Sengupta, IME Dept., IIT Kanpur, INDIA 36
Thank you all