Nonlinear Models with Spatial Data

William GreeneStern School of Business, New York

UniversityWashington D.C.

July 12, 2013

Binary Outcome: Y=1[New Plant Located in County]

Klier and McMillen: Clustering of Auto Supplier Plants in the United States. JBES, 2008

On Our Program School District Open Enrollment: A Spatial Multinomial Logit Approach; David

Brasington, University of Cincinnati, USA, Alfonso Flores-Lagunes, State University of New York at Binghamton, USA, Ledia Guci, U.S. Bureau of Economic Analysis, USA

Smoothed Spatial Maximum Score Estimation of Spatial Autoregressive Binary Choice Panel Models; Jinghua Lei, Tilburg University, The Netherlands

Application of Eigenvector-based Spatial Filtering Approach to a Multinomial Logit Model for Land Use Data; Takahiro Yoshida & Morito Tsutsumi, University of Tsukuba, Japan

Estimation of Urban Accessibility Indifference Curves by Generalized Ordered Models and Kriging; Abel Brasil, Office of Statistical and Criminal Analysis, Brazil, & Jose Raimundo Carvalho, Universidade Federal do Cear´a, Brazil

Choice Set Formation: A Comparative Analysis, Mehran Fasihozaman Langerudi,Mahmoud Javanmardi, Kouros Mohammadian, P.S Sriraj, University of Illinois atChicago, USA, & Behnam Amini, Imam Khomeini International University, Iran

Not including semiparametric and quantile based linear specifications

Also On Our Program Ecological fiscal incentives and spatial strategic interactions: the José Gustavo Féres Institute of Applied Economic Research (IPEA) Sébastien Marchand_ CERDI, University of Auvergne Alexandre Sauquet_ CERDI, University of Auvergne (Tobit) The Impact of Spatial Planning on Crime Incidence : Evidence from Koreai Hyun Joong Kimii Ph.D. Candidate, Program in Regional Information, Seoul National University & Hyung Baek Lim Professor, Dept. of Community Development SungKyul University (Spatially Autoregressive Probit) Spatial interactions in location decisions: Empirical evidence from a Bayesian Spatial Probit model Adriana Nikolic, Christoph Weiss, Department of Economics, Vienna University of Economics and Business A geographically weighted approach to measuring efficiency in panel data: The case of US saving banks Benjamin Tabak, Banco Central do Brasil, Brazil, Rogerio B. Miranda, Universidade Catolica de Brasılia, Brazil, & Dimas M Fazio, Universidade de Sao Paulo (Stochastic Frontier)

Spatial Autoregression in a Linear Model

2

1

1 1

1

2 -1

+ . E[ | ] Var[ | ]=

[ ] ( ) [ ] [ ]E[ | ] [ ]Var[ | ] [( ) ( )]Estimators: Various f

y = Wy Xβ εε X =0, ε X I

y = I W Xβ ε= I W Xβ I W ε

y X = I W Xβy X = I W I W

orms of generalized least squares. Maximum likelihood | ~ Normal[ , ]ε 0

Spatial Autocorrelation in Regression

2

2

11 1

12

( ) .E[ | ]= Var[ | ]=E[ | ]=Var[ | ] ( )( )

ˆ ( )( ) ( )( )1 ˆ ˆ( )( )ˆ N

ˆ The subject of much

y Xβ I - W εε X 0, ε X I y X Xβ

y X = I - W I - WA Generalized Regression Model

β X I - W I - W X X I - W I - W y

y- Xβ I - W I - W y- Xβ

research

Panel Data Applications (many at this meeting)

it it i it

t t t

E.g., N countries, T periodsy c

= N observations at time t.Similar assumptions Candidate for SUR or Spatial Autocorrelation model.

x βε Wε v

Analytical Environment

Generalized linear regression Complicated disturbance covariance matrix Estimation platform: Generalized least squares, GMM or maximum likelihood. Central problem, estimation of

Practical Obstacles Problem: Maximize logL involving sparse (I-W) Inaccuracies in determinant and inverse Appropriate asymptotic covariance matrices for estimators Estimation of . There is no natural residual based estimator Potentially very large N – GIS data on agriculture plots Complicated covariance structures – no simple transformations

Outcomes in Nonlinear Settings Land use intensity in Austin, Texas – Discrete Ordered Intensity = ‘1’ < ‘2’ < ‘3’ < ‘4’ Land Usage Types, 1,2,3 … – Discrete Unordered Oak Tree Regeneration in Pennsylvania – Count Number = 0,1,2,… (Excess (vs. Poisson) zeros) Teenagers in the Bay area: physically active = 1 or physically inactive = 0 – Binary Pedestrian Injury Counts in Manhattan – Count Efficiency of Farms in West-Central Brazil – Stochastic Frontier Catch by Alaska trawlers in a nonrandom sample

Nonlinear Outcomes Discrete revelation of choice indicates latent underlying preferences Binary choice between two alternatives Unordered choice among multiple choices Ordered choice revealing underlying strength of preferences

Counts of events Stochastic frontier and efficiency Nonrandom sample selection

Modeling Discrete Outcomes “Dependent Variable” typically labels an outcome

No quantitative meaning Conditional relationship to covariates

No “regression” relationship in most cases. Models are often not conditional means. The “model” is usually a probability Nonlinear models – usually not estimated by any type of linear least squares Objective of estimation is usually partial effects, not coefficients.

Nonlinear Spatial Modeling

Discrete outcome yit = 0, 1, …, J for some finite or infinite (count case) J. i = 1,…,n t = 1,…,T

Covariates xit

Conditional Probability (yit = j) = a function of xit.

Issues in Spatial Discete Choice A series of Issues

Spatial dependence between alternatives: Nested logit Spatial dependence in the LPM: Solves some practical problems. A bad

model Spatial probit and logit: Probit is generally more amenable to modeling Statistical mechanics: Social interactions – not practical Autologistic model: Spatial dependency between outcomes vs. utilities. Variants of autologistic: The model based on observed outcomes is

incoherent (“self contradictory”) Endogenous spatial weights Spatial heterogeneity: Fixed and random effects. Not practical?

The models discussed below

Two Platforms

Random Utility for Preference Models Outcome reveals underlying utility Binary: u* = ’x y = 1 if u* > 0 Ordered: u* = ’x y = j if j-1 < u* < j Unordered: u*(j) = ’xj , y = j if u*(j) > u*(k)

Nonlinear Regression for Count Models Outcome is governed by a nonlinear regression E[y|x] = g(,x)

Maximum Likelihood EstimationCross Section Case: Binary Outcome

Random Utility: y* = + Observed Outcome: y = 1 if y* > 0,

0 if y* 0. Probabilities: P(y=1|x) = Prob(y* > 0| )

x

x

ni ii=1

= Prob( > - ) P(y=0|x) = 1 - P(y=1|x) Likelihood for the sample = joint probability

= Prob(y=y| ) Log Likelihoo

x

x

ni ii=1d = logProb(y=y| )x

Cross Section Case: n Observations

1 1 1 1 1 1

2 2 2 2 2 2

n n n n n n

y =j | or > Prob( or > )y =j | or > Prob( or > )Prob Prob = ... ... ...y =j | or > Prob( or > )

Operate on the margin

x x xx x x

x x x

ni ii=1

t t2

al probabilities of n observations LogL( | )= logF 2y 1

1 Probit F(t) = (t) exp( t / 2)dt (t)dt2

exp(t) Logit F(t) = (t) = 1 exp(t)

X,y x

Spatially Correlated ObservationsCorrelation Based on Unobservables

1 1 1 1 1

2 2 2 2 2

n n n n n

y u u 0y u u 0 ~ f ,... ... ... ...y u u 0

In the cross section case, = . = the usual spatial weight matrix .

xx I I I

x

W W W

WW 0 Now, it is a full matrix.

The joint probably is a single n fold integral.

Spatially Correlated ObservationsBased on Correlated Utilities

* *1 1 1 11 1

* * 12 2 2 22 2

* *n n n nn n

y yy y

... ...... ...y y

In the cross section case= the usual spatial weight matrix .

x xx x

x x

W I W

W, = . Now, it is a full

matrix. The joint probably is a single n fold integral.W 0

LogL for an Unrestricted BC Model

n 1

1 1 1 2 12 1 n 1n 1

2 2 1 2 21 2 n 2n 2n

n n n 1 n1 n 2 n2 n

i i

LogL( | )=q 1 q q w ... q q wq q q w 1 ... q q wlog ... d... ... ... ... ... ...q q q w q q w ... 1

q 1 if y = 0 and +1 if

x x

X,y

i i y = 1 = 2y 1 One huge observation - n dimensional normal integral. Not feasible for any reasonable sample size. Even if computable, provides no device for estimating

sampling standard errors.

*1 11 1

*2 22 2

*n n

*i i

1 12 2 13 3 1 1

2 21 1 23 3 2 2

y yy y

...... ...y y

y 1[y 0]y 1[ (w y w y ...) 0]y 1[ (w y w y ...) 0] etc.

The model based on observa

n n

xx

x

xx

W

bles is more reasonable.There is no reduced form unless is lower triangular.This model is not identified. (It is "incoherent.")

W

Spatial Autoregression Based on Observed Outcomes

See, also, Maddala (1983)

From Klier and McMillen (2012)

Solution Approaches for Binary Choice

Approximate the marginal density and use GMM (possibly with the EM algorithm) Distinguish between private and social shocks and use pseudo-ML

Parameterize the spatial correlation and use copula methods

Define neighborhoods – make W a sparse matrix and use pseudo-ML Others …

*i i i ij jj i

* ii i i

i ij jj i

*i i i

* i ii i i

ii

2 2 2i ijj i

Spatial autocorrelation in the heterogeneity

y w

y 1 [y 0], Prob y 1Var w

or

y u

y 1 [y 0]Prob y 1Var u

1 w

x

x

x

x x

GMM in the Base Case with = 0Pinske, J. and Slade, M., (1998) “Contracting in Space: An Application of Spatial Statistics to Discrete Choice Models,” Journal of Econometrics, 85, 1, 125-154.Pinkse, J. , Slade, M. and Shen, L (2006) “Dynamic Spatial Discrete Choice Using One Step GMM: An Application to Mine Operating Decisions”, Spatial Economic Analysis, 1: 1, 53 — 99.

1

i

*= + , = + = [ - ] = uCross section case: =0Probit Model: FOC for estimation of is based on the

ˆ generalized residuals u

y W uI W u

A

X ε

x xx x

i i i

n i i iii=1

i i

= y E[ | y ](y ( )) ( ) = ( )[1 ( )]x 0

See, also, Bertschuk, I., and M. Lechner, 1998. “Convenient Estimators for the Panel Probit Model.” Journal of Econometrics, 87, 2, pp. 329–372

GMM in the Spatial Autocorrelation Model

1*= + , = +

= [ - ] = uAutocorrelated Case: 0Moment equations are still valid. Complication is computing the variance of the moment equations fo

y W uI W u

A

X ε

xi i i

ii

n iiii=1

r the weightingmatrix, which requires some approximations.Probit Model: FOC for estimation of is based on the

ˆ generalized residuals u = y E[ | y ]

y a ( ) z

x

x x

i

ii

i i

ii ii

a ( ) = 1a ( ) a ( )

0

Requires at least K+1 instrumental variables.

Using the GMM Approach

Spatial autocorrelation induces heteroscedasticity that is a function of Moment equations include the heteroscedasticity and an additional instrumental variable for identifying . LM test of = 0 is carried out under the null hypothesis that = 0. Application: Contract type in pricing for 118 Vancouver service stations.

1

1* 2ii i

*ii i i

i

* = + , = + = ( - )

[ ], Var[ ] = ( - ) ( - ) ,

Prob(y 1)

A Spatial Logit Model

y X e e We I W

d = 1y 0 e I W I W

x x

i i i

*i i i

i 1 1*i i

i i i ii2i

1 2 n

Generalized residual u d , Instruments

(1 )u /

, =( - ) ( - )u / (1 ) A

[ , ,... ]

Algorithm

Iterated 2SLS (GMM) q = ( , ) ( )

Zx

g A I W W I Wx

G g g g

u Z Z Z

0

1k k k k

1

k k k k k

k k

k 1 k

1

1. Logit estimation of =0, ˆˆ2. = ( - ), ( )

ˆ ˆ ˆ3.

ˆ ˆ4. until is sufficiently small.

ˆ ˆ

( , )

G

u d G Z Z Z Z G

G G G u

Z u

β|

An LM Type of Test? If = 0, g = 0 because Aii = 0 At the initial logit values, g = 0 If = 0, g = 0. Under the null hypothesis the entire score vector is identically zero. How to test = 0 using an LM test? Same problem shows up in RE models But, here, is in the interior of the parameter space!

Pseudo Maximum Likelihood Maximize a likelihood function that approximates the true one Produces consistent estimators of parameters How to obtain standard errors? Asymptotic normality? Conditions for CLT are more difficult to establish.

Pseudo MLE

1

*i i i ij jj i

* * 2 2i i i ij iij i

*= + , = + = [ - ] = uAutocorrelated Case: 0y Wy 1[y 0]. Var[y ] 1 W a ( )Implies a heteroscedastic probit.Pse

y X W uI W u

A

x

θ ε

udo MLE is based on the marginal densities.How to obtain the asymptotic covariance matrix?[See Wang, Iglesias, Wooldridge (2013)]

n i ii 1

i

1MLE

Estimation and Inference

(2y -1)MLE: logL = log

ˆ n ( ) ( ) =Score vector

implies the algorithm, Newton's Meth

Heteroscedastic Probit Approac

S

h

x

H S

od.

EM algorithm essentially replaces with during iterations.(Slightly more involved for the heteroscedasticity. LHS variablein the EM iterations is the score vector.)To compute the asymptotic c

H X X

ovariance, we need Var[ ( )]Observations are (spatially) correlated! How to compute it?

S

ˆ ˆ ˆ(data, ) (data, ) (data, )ˆ(data, ) Negative inverse of Hessianˆ(data, ) Covariance matrix of scores.

ˆHow to compute (data, )Terms are not independent in a spatial setting.

V A B AAB

B

Covariance Matrix for Pseudo-MLE

‘Pseudo’ Maximum LikelihoodSmirnov, A., “Modeling Spatial Discrete Choice,” Regional Science and Urban Economics, 40,

2010.

1 1

1 tt 0

* * , 1( * ) for all n individuals* ( ) ( )

( ) ( ) assumed convergent = = + where

Spatial Autoregressioy Wy X y y 0y I W X I WI W W

AD

n in Utili

A-D

ties

nj 1 ij j

i ii

= diagonal elements*

Private Social Then

aProb[y 1 or 0| ] F (2y 1) , pd

Dy AX D A-D

Suppose individuals ignore the social "shocks."xX

robit or logit.

Pseudo Maximum Likelihood Bases correlation on underlying utilities Assumes away the correlation in the reduced form Makes a behavioral assumption Still requires inversion of (I-W) Computation of (I-W) is part of the optimization process - is estimated with . Does not require multidimensional integration (for a logit model, requires no integration)

Other Approaches

Beron and Vijverberg (2003): Brute force integration using GHK simulator in a probit model. Impractical. Case (1992): Define “regions” or neighborhoods. No correlation across regions. Produces essentially a panel data probit model. (Also Wang et al. (2013)) LeSage: Bayesian - MCMC Copula method. Closed form. See Bhat and Sener, 2009.

Case A (1992) Neighborhood influence and technological change. Economics 22:491–508

Beron KJ, Vijverberg WPM (2004) Probit in a spatial context: a monte carlo analysis. In: Anselin L, Florax RJGM, Rey SJ (eds) Advances in spatial econometrics: methodology, tools and applications. Springer, Berlin

See also Arbia, G., “Pairwise Likelihood Inference for Spatial Regressions Estimated on Very Large Data Sets” Manuscript, Catholic University del Sacro Cuore, Rome, 2012.

Partial MLE (Looks Like Case, 1992)

*1 1 1 1j jj 1

* * 2 21 1 1 1j 11j 1

*2 2 2 2j jj 2

* * 2 22 2 2 2j 22j 2

* *1 2

Observation 1y Wy 1[y 0] Var[y ] 1 W a ( )

Observation 2y Wy 1[y 0] Var[y ] 1 W a ( )

Covariance of y and y = a

x

x

12( )

Bivariate Probit Pseudo MLE Consistent Asymptotically normal? Resembles time series case Correlation need not fade with ‘distance’

Better than Pinske/Slade Univariate Probit? How to choose the pairings?

/2

,

]

| |)

SAR

SEM

2

2

2

Core Model= ρ + + Spatial autoregression or

= ρ + , Spatial error model (only one at a time)

~ N[ ,σCensoring : Probit (0,1), Tobit (0,+)

-ρLikelihood : L(ρ, ,σ2πσ

y* W y * Xβ uu W u εε 0 I

I Wβ n

)

2exp2σ

( - ρ ) - for SAR( - ρ )( - for SEM

ε ε

ε I W y * Xβε I W y * Xβ

LeSage Methods - MCMC

• Bayesian MCMC for all unknown parameters

• Data augmentation for unobserved y*

• Quirks about sampler for rho.

An Ordered Choice Model (OCM)

1

1 2

2 3

J -1 J

j-1

y* , we assume contains a constant termy 0 if y* 0y = 1 if 0 < y* y = 2 if < y* y = 3 if < y* ...y = J if < y* In general: y = j if < y*

βx x

j

-1 o J j-1 j,

, j = 0,1,...,J, 0, , j = 1,...,J

OCM for Land Use Intensity

A Dynamic Spatial Ordered Choice ModelWang, C. and Kockelman, K., (2009) Bayesian Inference for Ordered Response Data with a Dynamic Spatial Ordered Probit Model, Working Paper, Department of Civil and Environmental Engineering, Bucknell University.

* *i i i i j-1 i j i

* *ir ir i ir ir j-1 ir j

Core Model: Cross Section y , y = j if y , Var[ ] 1Spatial Formulation: There are R regions. Within a region y u , y = j if y Spatial he

βx

βx

2ir r

2v

1 2 1v

teroscedasticity: Var[ ]Spatial Autocorrelation Across Regions = + , ~ N[ , ] = ( - ) ~ N[ , {( - ) ( - )} ] The error distribution depends on 2 para

u Wu v v 0 Iu I W v 0 I W I W

2vmeters, and

Estimation Approach: Gibbs Sampling; Markov Chain Monte CarloDynamics in latent utilities added as a final step: y*(t)=f[y*(t-1)].

Data Augmentation

Unordered Multinomial Choice

j ij ij

Underlying Random Utility for Each Alternative U(i,j) = , i = individual, j = alternative Preference Revelation

Y(i) = j if and only if U(i,j) > U(i,k

Core Random Utility Model

x

1 J

1 J

) for all k j Model Frameworks

Multinomial Probit: [ ,..., ] ~N[0, ] Multinomial Logit: [ ,..., ] ~iid type 1 extreme value

Spatial Multinomial ProbitChakir, R. and Parent, O. (2009) “Determinants of land use changes: A spatial multinomial probit approach, Papers in Regional Science, 88, 2, 328-346.

jt ijt ik ijt

nij il lkl 1

Utility Functions, land parcel i, usage type j, date t U(i,j,t)=(In France) Spatial Correlation at Time t wModeling Framework: Normal / Multinomial ProbitEsti

x

mation: MCMC - Gibbs Sampling

Mixed Logit Models for Type of Residential Unit

First Law of Geography: [Tobler (1970)] ‘‘Everything is related to everything else, but near things are more related than distant things”.

* http://www.openloc.eu/cms/storage/openloc/workshops/UNITN/20110324-26/Giuliani/Giuliani_slides.pdf* Arbia, G., R. Benedetti, and G. Espa. 1996. Effects of the MAUP on image classification. Geographical Systems 3:123–41.* http://urizen-geography.nsm.du.edu/~psutton/AAA_Sutton_WebPage/Sutton/Courses/ Geog_4020_Geographic_Research_Methodology/SeminalGeographyPapers/TOBLER.pdf

Is there a second law of geography?

Heisenberg?

Location Choice Model Common omitted geographic features embedded in the random utility functions Cross – nested multinomial logit model

Does the model extension matter?

Does the model extension matter?

Canonical Model for Counts

j

Poisson Regression y = 0,1,...

exp( ) Prob[y = j| ] = j! Conditional Mean = exp( )Signature Feature: EquidispersionUsual Alternative: Negative BinomialSpatial Effect: Filtered

x

x

i i i

ni im m im 1

through the mean = exp( + ) = w

x

Rathbun, S and Fei, L (2006) “A Spatial Zero-Inflated Poisson Regression Model for Oak Regeneration,” Environmental Ecology Statistics, 13, 2006, 409-426

Zero Inflation

There are two states Always zero Zero is one possible value, or 1,2,…

Prob(0) = Prob(state 1) + Prob(state 2) P(0|state 2) Used here as a functional form issue – too many zeros.

Numbers of firms locating in Texas counties: Count data (Poisson)

Bicycle and pedestrian injuries in census tracts in Manhattan. (Count data and ordered outcomes)

A Blend of Ordered Choice and Count Data Models

Kriging

Spatial Autocorrelation in a Sample Selection Model

Alaska Department of Fish and Game.

Pacific cod fishing eastern Bering Sea – grid of locations

Observation = ‘catch per unit effort’ in grid square

Data reported only if 4+ similar vessels fish in the region

1997 sample = 320 observations with 207 reported full data

Flores-Lagunes, A. and Schnier, K., “Sample Selection and Spatial Dependence,” Journal of Applied Econometrics, 27, 2, 2012, pp. 173-204.

Spatial Autocorrelation in a Sample Selection Model

LHS is catch per unit effort = CPUE Site characteristics: MaxDepth, MinDepth, Biomass Fleet characteristics:

Catcher vessel (CV = 0/1) Hook and line (HAL = 0/1) Nonpelagic trawl gear (NPT = 0/1) Large (at least 125 feet) (Large = 0/1)

Spatial Autocorrelation in a Sample Selection Model*1 0 1 1 1 1 1

*2 0 2 2 2 2 2

21 1 12

122 12 2

*1 1

2

0~ , , (?? 1??)

0

Observation Mechanism

1 > 0 Probit Model

i i i i ij j ij i

i i i i ij j ij i

i

i

i i

i

y u u c u

y u u c u

N

y y

y

x

x

*2 1 if = 1, unobserved otherwise.i iy y

Spatial Autocorrelation in a Sample Selection Model

1 1 1

1 (1)1 1 1 2

2* (1) 2 (1)1 0 1 1 1 1 1 1

* (2) 2 (2 0 2 2 2 1 1

= Spatial weight matrix, 0.

[ ] = , likewise for

( ) , Var[ ] ( )

( ) , Var[ ] ( )

ii

N Ni i ij i i ijj j

Ni i ij i i ijj

y u

y u

u CuC C

u I C u

x

x

22)1

(1) (2)1 2 12 1

Cov[ , ] ( ) ( )

N

j

Ni i ij ijju u

Spatial Weights

2

1 ,

Euclidean distance

Band of 7 neighbors is usedRow standardized.

ijij

ij

cd

d

Two Step Estimation

0 122 (1)(1) (2)

1 11

2(1) 1 0 1

22 (1)1 1

Probit estimated by Pinske/Slade GMM

( )( ) ( )

( )

( )

Spatial regression with included IMR i

i

NNijij ij jj

iN

ijj i

Nijj

x

x

n second step(*) GMM procedure combines the two steps in one large estimation.

Spatial Stochastic FrontierProduction function modely = + εy = + v - uv unexplained noise = N[0,1]u = inefficiency > 0; efficiency = exp(-u)Object of estimation is u, not Not a linear regression. Fit by MLE or MCMC.

β x β x

247 Spanish Dairy Farms, 6 Years

A True Random Effects Model*

ij

i ij ij ij

1 n

i k

y Output of farm j in municipality i in Center-West Brazil

y α + +v - u

(α ,...,α ) conditionally autoregressive based on neighborsα -α is smaller when municipalities i and k are closer tog

ij

β x

ether

Spatial Stochastic Frontier Models; Accounting for Unobserved Local Determinants of Inefficiency.Schmidt, Moriera, Helfand, Fonseca; Journal of Productivity Analysis, 2009.

* Greene, W., (2005) "Reconsidering Heterogeneity in Panel Data Estimators of the Stochastic Frontier Model", Journal of Econometrics, 126(2), 269-303

Spatial Frontier Models

Estimation by Maximum Likelihood

Cost Model

Production Model

LeSage (2000) on Timing“The Bayesian probit and tobit spatial autoregressive models described here have been applied to samples of 506 and 3,107 observations. The time required to produce estimates was around 350 seconds for the 506 observations sample and 900 seconds for the case involving 3,107 observations. … (inexpensive Apple G3 computer running at 266 Mhz.)”

Time and Space (In Your Computer)

Thank you!

…………………………………………….