Integrating Judgmental and Quantitative Forecasts

Stephen MacDonald, ERS/USDA

Research Center on Forecasting Seminar

January 17, 2007

Introduction

• Futures markets often find USDA’s forecasts crucial

• Resource constraints have reduced the staff-years USDA has for forecasting

• Quantitative methods can be used to supplement USDA’s traditionally judgmental forecasts– Example: international commodity trade

USDA forecasts: Two perspectives

1 2

Overview of USDA Forecasts

• National Agricultural Statistics Service (NASS) estimates U.S. production of more than 100 commodities

• 7 of these commodities have been legislatively deemed “market sensitive”– Wheat, corn, soybeans, cotton, citrus, cattle, and

hogs

• Since 1973, USDA has published demand forecasts as well– Interagency Commodity Estimates Committees

Interagency Commodity Estimates Committee (ICEC)

• ICEC comprised of: Economic Research Service (ERS)

Foreign Agricultural Service (FAS) Farm Service Agency (FSA) Agricultural Marketing Service (AMS)

World Agricultural Outlook Board (WAOB) ,chair

• Methodology of the ICEC:“A consensus…approach is used to arrive at supply and

demand estimates. Consensus forecasts employ ‘models’ of all types, formal and informal.”

February 2007 example: India 2006/07 cotton exports

• Forecasts available from several sources:• 4.2 million bales (mb): U.S. embassy (Delhi)• 3.9 mb: India Cotton Advisory Board• 4.1 mb: International Cotton Advisory Committee• “USDA forecast too high”: personal

communication from industry analysts

• No actual data was available—Indian official trade data is significantly lagged

• USDA’s forecast: 5 mb

January 2008: India exports

• 10 months of marketing year data published

• Averaged 437,000 bales per month

• Compared to 2006, Aug-May trade is:

• 1.3 m. bales higher• 44 % higher

• During previous 4 years:• Aug-May was 84% of year

0

200

400

600

800

1000

1200

Aug Oct Dec Feb Apr Jun

Thousand bales

20052006

Data available through May 2007

Declining resources: FAS & ERS

0200400600800

100012001400

1995 1998 2001 2004 2007

U.S. Embassy reports: foreign grains,

oilseeds, and cotton

0100200300400500600700

1995 1998 2001 2004 2007

ERS staff-years

Changing Forecasting Environment

• A consensus (Delphi) approach is resource-intensive: expertise– or labor–intensive

• Falling cost of data-processing and acquisition can offset reduced staffing

• Timely international commodity trade data commercially available– replacing embassy reports

Import forecasts based on data through month x

cxm

mmy

xm

mmy

YY

b

xm

m

xm

mmymyYY

a

xm

mmy

Y

I

III

IIII

x

II

,1

,

1

,1,1

,

:C

:B

12:A

d

i

i iY

xm

mmiy

xm

mmy

Y

I

I

IID

3

:

3 ,

,

Empirical confidence intervals

• Assume future errors distributed same as past

• Assume errors are normally distributed, with mean of zero

• Calculate a 90 percent confidence interval for each forecast using estimated variance and t distribution– Variance estimated with past forecasts errors

Alternative forecasts for India exports

• Weight forecasts by inverse of confidence interval

• Analysis of trade data corroborates USDA

• But: international organization has forecast outside of 90% confidence interval

Forecast LevelConfidence Limit

Million bales

USDA 5.1 --

ICAC 4.4 --

A 5.2 0.1

B 4.8 0.1

C 5.0 0.4

D 5.3 0.1

Average 5.0 0.1

Forecasting after structural change

0

1000

2000

3000

4000

5000

6000

1995 1997 1999 2001 2003 2005 2007

Thousand bales• Past error variances may be poor guide

• Genetically modified cotton increases exports

• Convert confidence limits to percentages of past Indian exports:

Forecasts

• Example– 100,000 / 837,000 = 12 %– 0.12 * 5.0 mb = 0.6 mb, alternative confidence limit

837,000 bales = 99-05 average

Forecasts: adjusted confidence limits

• Proportional confidence limit suggests ICAC forecast is not incompatible with published trade data

• However, actual exports totaled 4.4 mb already

• Alternative adjustments may be more appropriate

Forecast LevelConfidence Limit

Million bales

USDA 5.1 --

ICAC 4.4 --

A 5.2 0.5

B 4.8 0.5

C 5.0 2.8

D 5.3 0.5

Average 5.0 0.7

Integration with judgmental forecasts

• Confidence intervals expand compatibility of quantitative estimates with market intelligence from embassies and industry

• Also provide weights for combining forecasts—add intuitive appeal

• Can be incorporated into rules of thumb to guide judgmental decision-making

Integration with judgmental forecasts12/31/2007 Argentina Australia Belgium Brazil Canada

Data ends: Oct-07 Oct-07 Sep-07 Nov-07 Oct-07*(See note)

Imports

USDA (December 07) 225 0 130 400 120

Change USDA1-- -- -- -- --

Proposed forecast2 165 0 145 225 145

Advisability of change3-- -- 103% 294% 220%

Forecast range4

Minimum 80 0 130 165 135 Maximimum 255 0 155 285 160

Exports

USDA (December 07) 150 1,400 30 2,800 0

Change USDA1Reduce -- -- Reduce --

Proposed forecast2 40 1,400 45 1,950 0

Advisability of change3339% -- -- 209% --

Forecast range4

Minimum 11 1,250 29 1,500 0 Maximimum 75 1,550 60 2,350 0Source: ERS calculations using data from Global Trade Information Service (GTIS).

Conclusion

• USDA forecasting is increasingly substituting “capital” for labor

• We are exploring how to most efficiently exploit the growing availability of data

• We are determining how best to integrate these quantitative forecasts into USDA’s judgment-based system