1/25

Investigating Covariate Selection for a BayesianCrop Yield Forecasting Model

Habtamu Benechahabtamu.benecha@nass.usda.gov

Nathan B. Cruzenathan.cruze@nass.usda.gov

United States Department of AgricultureNational Agricultural Statistics Service (NASS)

Federal Committee on Statistical MethodologyResearch and Policy Conference

March 9, 2018

“. . . providing timely, accurate, and useful statistics in service to U.S. agriculture.” 1

2/25

Background and Research Questions

I By mandate, NASS produces monthly crop yield forecasts

I Official forecasts are consensus estimates of the AgriculturalStatistics Board (ASB)

I Recent research in support of the forecasting program

I Bayesian hierarchical models

I Combine data from multiple surveys and covariates

Goal: Which observable covariates are most relevant?

FCSM 2018–Investigating Covariate Selection for a Bayesian Crop Yield Forecasting Model 2

3/25

Motivating Example: Forecasting in a Drought Year, 2012Corn

FCSM 2018–Investigating Covariate Selection for a Bayesian Crop Yield Forecasting Model 3

Speculative Region for Corn

Non−Spec

Speculative

USDA NASS Corn for Grain Estimation Program

4/25

NASS Survey Data and Reporting TimelineObjective Yield Survey (OYS)

I Field measurements at sampled plots (Aug.- Dec.)

Agricultural Yield Survey (AYS)

I Interview conducted monthly (Aug.-Nov.)

December Crops Acreage, Production, and Stocks Survey (APS)

I Interview conducted post-harvest

5/25

Survey Estimates for 2004-2016

6/25

7/25

Bayesian Hierarchical Model for Speculative Region

NotationI µt–true yield

I yktm–observed yield

I k ∈ {O,A,Q}–survey index

I t ∈ {1, ...,T}–year index

I m ∈ {8, 9, 10, 11, 12}

Stage 1

yktm|µt ∼ indep N(µt + bkm, s

2ktm + σ2

km

), (1)

k = O,A; m = 8, 9, 10, 11, 12

yQt |µt ∼ indep N(µt , s

2Qt

)(2)

Stage 2µt ∼ indep N

(z′tβ, σ

2η

)(3)

FCSM 2018–Investigating Covariate Selection for a Bayesian Crop Yield Forecasting Model 7

8/25

Bayesian Hierarchical Model for Speculative Region

Diffuse prior distributions on data and process model parameters

I Θd ≡(bkm, σ

2km)

I Θp ≡(β, σ2

η

)Likelihood function–assuming conditional independence

[yO , yA, yQ |µt ,Θd ] =∏

k∈{O,A,Q}

[yk |µt ,Θd ] (4)

Posterior distribution

[µt ,Θd ,Θp|yO , yA, yQ ] ∝∏

k∈{O,A,Q}

[yk |µt ,Θd ][µ|Θp][Θd ][Θp] (5)

FCSM 2018–Investigating Covariate Selection for a Bayesian Crop Yield Forecasting Model 8

9/25

Bayesian Hierarchical Model–State Level Yield

State-level counterparts indexed by j ∈ {1, 2, ..., J}

Unconstrained State Model–Define µt· ≡ (µt1, µt2, . . . , µtJ),

µt·|y ,Θd ,Θp ∼ indep MVN

(vec

(∆2j

∆1j

), diag

(1

∆1j

))(6)

Constrained State Model–Enforce constraint by conditioningstate vector µt· on µt =

∑j wjµtj(

µt1, µt2, . . . , µt(J−1))∼ MVN(µ̄, Σ̄) (7)

µtJ = µt −1

wtJ

J−1∑j=1

wtjµtj (8)

FCSM 2018–Investigating Covariate Selection for a Bayesian Crop Yield Forecasting Model 9

10/25

Covariates for the j th State

µtj ∼ N(x′tjβj , σ

2η

)Current model for corn includes covariates:

I T: Trend

I P: Average July precipitation (NOAA)

I M: Average July temperature (NOAA)

I C: Crop condition rating, % rated excellent + good, Week 30(NASS)

For the Speculative Region: covariate values are defined asweighted averages of state-level covariate values

FCSM 2018–Investigating Covariate Selection for a Bayesian Crop Yield Forecasting Model 10

Additional Covariate

I Early season model-forecasts

I Drought severity index

I D = %D3 + %D4

1

I Pool of available covariates: {T, P, M, C, D}I Potential interactions

I Optimal set of covariates, parsimony

1http://droughtmonitor.unl.edu 11/25

Challenges in Selecting Appropriate Covariates

I Repeated measure of yield over five months

I Defining a pool of potential covariatesI Crop-specific knowledgeI Standard variable selection methods often point to different

sets of covariatesI Step-wise regressionI LASSOI Spike-and-slab regression (Ishwaran and Rao, 2005; Kou and

Mallick, 1998), etc

I Example: {P,M}, {P,M,C,D}, {P,M,D} and {T,P,M,C,D} -’best’ for the Spec-region in 2016

I ’Best’ sets of covariates depend on state, year and month

12/25

Proposed Approach

1. Start with alternative sets of covariates that are selected mostfrequently by traditional variable selection methods

2. Fit models for months from August to December and for years2012, 2013, 2014, 2015 and 2016.

3. Criteria for decision: percent relative difference from Dec.estimates

J =(Aug. forecast - Dec. estimate)

Dec. estimate× 100

13/25

Model ComparisonI A total of 17 covariate combinations

I Subsets of {T,P,M,C,D,TD}I Comparisons of models

14/25

Model ComparisonI A total of 17 covariate combinations

I Subsets of {T,P,M,C,D,TD}I Comparisons of models

15/25

Variables with smallest percent relative differences

Covariate-sets

State Without Drought With Drought asMain Effect

With Interaction(D*T)

A - ’13, ’15 ’12, ’14, ’16B ’13, ’14, ’15 ’16 ’12C ’13 ’12, ’15, ’16 ’14D ’12 ’13, ’14, ’15 ’16E ’14 ’13, ’15, ’16 ’12F ’12, ’14 ’13, ’15 ’16G ’12, ’13 ’14, ’15, ’16 -H ’14 ’13 ’12, ’15, ’16I ’12, ’15 ’13, ’14, ’16 -J ’15 ’13 ’12, ’14, ’16

16/25

Model Comparison for State B

17/25

Model Comparison for State I

18/25

Model Comparison for the Spec-region

I DIC values for December 2016 corresponding to covariate-sets{T,P,M,C}, {T,M,D} , {T,P,D} , {P,C, T*D} ,{T,P,C,D,T*D},{T,P,C,D} are 162.92, 163.06, 163.07, 162.93, 163.15 and 163.02respectively. 19/25

20/25

Conclusions

Investigated sensitivity of model forecasts to linear modelspecification

I Inclusion of a ‘drought’ covariate improved early yield forecasts

I No one-size fits all set of covariates

I State-specific covariates may be considered

Contact:habtamu.benecha@nass.usda.gov

FCSM 2018–Investigating Covariate Selection for a Bayesian Crop Yield Forecasting Model 20

21/25

Select ReferencesAdrian, D. (2012). A model-based approach to forecasting corn and soybean yields. Fourth International

Conference on Establishment Surveys.

Cruze, N. B. (2015). Integrating Survey Data with Auxiliary Sources of Information to Estimate Crop Yields. InJSM Proceedings, Survey Research Methods Section. Alexandria, VA: American Statistical Association.

Cruze, N. B. (2016). A Bayesian Hierarchical Model for Combining Several Crop Yield Indications. In JSMProceedings, Survey Research Methods Section. Alexandria, VA: American Statistical Association.

Cruze, N. B. and Benecha, H. (2017). A Model-Based Approach to Crop Yield Forecasting. In JSM Proceedings,Section on Bayesian Statistical Science. Alexandria, VA: American Statistical Association.

Nandram, B., Berg, E., and Barboza, W. (2014). A hierarchical Bayesian model for forecasting state-level cornyield. Environmental and Ecological Statistics, 21(3):507–530.

Nandram, B. and Sayit, H. (2011). A Bayesian analysis of small area probabilities under a constraint. SurveyMethodology, 37:137–152.

Wang, J. C., Holan, S. H., Nandram, B., Barboza, W., Toto, C., and Anderson, E. (2012). A Bayesian approachto estimating agricultural yield based on multiple repeated surveys. Journal of Agricultural, Biological, andEnvironmental Statistics, 17(1):84–106.

FCSM 2018–Investigating Covariate Selection for a Bayesian Crop Yield Forecasting Model 21

Supplementary material 1

22/25

Supplementary material 2

23/25

Supplementary material 3

24/25

Supplementary material 4

25/25