The Evolution of FRASER and AlFRED: St. Louis Fed Real Time Data Services Robert H. Rasche Federal...

The Evolution of FRASER and AlFRED:

St. Louis Fed Real Time Data Services

Robert H. RascheFederal Reserve Bank of St. Louis

Views expressed are mine and do not necessarily reflect official positions of

the Federal Reserve System.

Real-Time Data Initiatives

• FRASER – Federal Reserve Archival System for Economic Research– Electronic Image Archive of Economic

Data and Source Materials

• AlFRED – Archival FRED– Real-time Data Base System for

Macroeconomic Time Series

FRASER• Electronic image archive of historic economic statistical

publications (.pdf files)– Organized chronologically and by subject

• Completed: – All Bank Statistics, Annual Statistical Digests, Banking and

Monetary Statistics – Economic Indicators – Productivity and Costs Statistical Releases– Capacity Utilization Statistical Releases; – Business Conditions Digest– Economic Report of the President– Reference materials from A. Meltzer, Volume 1

• Short-term Goals:– Survey of Current Business; H.6 Money Releases– Construction Review– McChesney Martin Papers

• Future Additions:– Federal Reserve Bulletin (joint effort with BoG)– Congressional Hearings and Testimony of FR Officials

http://fraser.stlouisfed.org/

Publication: Economic Indicatorsiss. pgs.

N/A (no publication available) 140Needs to be scanned 280 Posted since 1/06 471Scanned (needs to be cleaned) 40Needs to be rescanned 36Cleaning to be checkedScanned & CleanedOCR Run

Metadata entered & files prep'd for posting 5/1948 - locate complete copies of pages 1 & 2Posted to FRASER 25111 6/1948 - rescan with new pages 10-11, 18-19PDF downloaded from Internet site

1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977January 40

February 40 40

MarchApril 40

May 36

June 35 40 40

July 40

AugustSeptember

OctoberNovemberDecember

1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

January 40 40

February 40 40

March 40 40

April 40 40

May 40 40 40

June 40 40 40

July 40 40 40

August 40 40 40

September 40 40

October 40 40

November 40 40

December 40 40

Publication: Business Cycle Developments / Business Conditions Digestiss. pgs.

N/A (no publication available) 9 1116 total pages: 37743

Needs to be scanned 3 372 total pages scanned: 37179

Scanned (needs to be cleaned) total pages cleaned: 37055

Needs to be rescanned 4 440

Cleaning to be checked 3 366

Scanned & Cleaned 8 996

OCR Run 1 132

Transferred to FRASER 3 384

Metadata entered & files prep'd for postingPosted to FRASER 311 35433 3 issues of 1990 available in library

S T U V W X Y Z AA AB AC AD AE1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990

January 68 72 76 92 84 84 92 124 124 124 126 132 132 132 136 124 124 124 124 124 124 124 124 124 124 124 124 124

February 68 76 76 84 84 84 92 124 124 124 128 132 132 132 136 124 124 124 124 124 124 124 122 124 124 124 124 124

March 68 76 76 84 84 81 92 124 124 124 128 132 132 132 136 124 124 124 124 124 124 124 124 124 124 124 124 124

April 72 76 78 84 84 90 92 124 124 124 132 132 132 132 136 124 124 124 124 124 124 124 124 124 124 124 124

May 72 76 76 84 84 92 92 124 124 124 132 132 132 136 136 124 124 124 124 124 124 124 124 124 124 124 124 124

June 72 76 76 82 84 90 92 124 124 124 132 132 132 136 136 124 124 124 124 124 124 124 124 102 124 124 124 124

July 72 76 84 84 84 92 92 124 128 124 132 132 132 136 124 124 124 124 124 124 124 126 124 124 124 124 124

August 72 76 80 84 84 92 92 124 126 124 132 132 132 136 124 124 124 124 124 124 124 124 124 124 124 124

September 72 90 80 84 84 92 92 124 124 124 132 132 132 136 124 124 124 124 124 124 124 112 124 122 124 124 124

October 72 72 76 80 84 84 96 92 124 124 124 132 132 132 136 124 124 124 124 124 124 126 122 124 124 120 124

November 68 72 76 80 84 86 92 124 126 124 124 132 132 132 120 124 124 124 124 124 124 124 128 124 124 124 124 124

December 68 72 76 80 84 84 92 124 124 124 124 132 132 132 124 124 124 124 124 124 124 124 124 124 124 124 124 124

Publication: Survey of Current Business todo: 107 17409iss. pages

N/A (no publication available) 11 572 total pages: 84532

Needs to be scanned 107 17409 total pages scanned: 80566 95%

Scanned (needs to be cleaned) total pages cleaned: 80566 95%

Needs to be rescanned 1 108

Cleaning to be checked 33 5538


OCR Run 10/1940 - 4,5File transferred to FRASER 41 6146 11/1941 - TOC-2 12/1957 - locate new 17,18Metadata entered & files prep'd for posting 11/1943 - TOC, darken S5-S12 2/1959 - locate new copyPosted to FRASER 966 65802 12/1943 - darken entire issuePDF downloaded from Internet site

T U V W X Y Z AA AB AC AD AE1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949

January 52 52 53 53 52 52 52 60 60 60 60 60 60 60 60 60 60 64 72 60 60 60 68 59 60

February 172 164 148 148 152 152 52 60 60 60 60 64 60 60 100 100 108 108 64 60 64 72 76 76 76

March 52 52 60 60 52 52 52 60 60 60 64 60 100 60 60 60 60 68 60 60 60 60 67 64

April 52 52 56 56 52 52 52 60 60 76 64 60 60 60 60 60 64 70 60 60 60 64 67 68

May 52 56 54 54 52 52 56 60 60 60 64 60 60 60 60 60 68 68 60 60 66 68 67 68

June 52 52 52 52 52 52 60 60 60 60 64 60 60 60 60 60 68 72 64 64 64 68 67 68

July 52 52 52 52 52 52 60 60 60 68 64 60 60 60 68 62 68 67 60 64 68 68 75 76

August 164 168 148 152 152 152 60 60 60 64 59 60 60 60 60 64 68 68 60 60 68 68 67 68

September 48 56 52 52 56 52 60 60 57 76 64 64 60 60 60 60 68 72 68 60 60 68 68 67 68

October 52 52 52 52 52 52 60 60 60 60 64 64 60 60 60 60 68 72 60 60 60 76 68 67 68

November 52 56 60 56 52 52 60 60 60 60 64 64 60 60 60 60 68 68 60 60 64 60 68 67 68

December 52 52 52 52 52 52 60 60 60 60 64 64 60 60 60 60 68 64 60 60 64 68 68 67 68

1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979January 64 68 64 60 68 68 64 68 64 68 64 64 60 68 72 68 72 76 76 84 84 84 80 84 92 84 248 68 88 80

February 76 76 76 76 68 72 76 76 72 64 72 76 72 72 60 64 68 68 60 72 84 64 72 92 100 80 76 84 80 116

March 64 64 68 64 68 72 64 64 64 72 64 68 68 68 68 76 72 76 80 96 92 100 100 96 100 92 100 104 108 108

April 68 68 68 68 68 64 60 64 68 68 68 64 68 68 76 72 64 68 72 76 80 68 92 88 92 104 96 92 76 116

May 68 68 68 68 68 68 68 68 60 72 68 76 64 68 68 68 84 88 72 164 80 78 202 88 164 68 96 63 92 100

June 68 68 68 68 68 68 76 72 64 68 68 64 68 76 72 76 84 88 96 108 104 108 100 100 103 99 104 116 164 112

July 80 72 76 76 80 72 76 76 64 88 80 80 80 84 80 64 84 88 96 96 100 100 100 96 108 96 112 108 116 116

August 68 68 68 68 68 64 68 76 68 76 68 76 76 72 68 100 80 84 96 68 88 88 84 96 140 84 108 108 116 176

September 68 68 64 68 68 76 64 64 68 64 68 68 76 68 72 100 80 96 88 92 98 94 88 100 100 104 96 100 100 92

October 68 68 64 64 64 68 64 64 64 64 68 60 64 68 64 72 68 96 96 80 84 78 72 88 84 108 72 88 100 82

November 68 68 64 64 68 68 64 68 64 68 68 68 72 68 76 70 72 64 68 92 62 72 80 68 84 72 68 72 76 84

December 68 66 64 72 72 68 80 68 68 68 64 68 72 68 72 72 84 96 80 92 84 99 100 100 92 80 96 92 104 108

M N O P Q R S T U V W X1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

January 88 84 88 86 76 88 116 100 72 80 96 124 208 140 128 116 112 117 124 200 152 136

February 80 112 104 92 72 88 92 84 68 76 96 116 127 108 160 116 116 124 116 160 223 124

March 116 112 108 112 108 116 164 104 120 128 108 132 159 172 160 132 100 104 152 108 156 120 136

April 112 116 112 100 100 112 108 76 116 107 128 128 171 132 196 128 124 148 180 148 232 148 156

May 80 100 116 92 124 104 112 92 116 84 120 116 143 188 120 116 108 200 136 160 142 156 172

June 112 116 112 108 120 116 116 128 140 136 176 140 179 168 165 156 108 160 150 128 202 128 160

July 100 116 175 140 160 100 148 176 176 184 212 164 188 218 188 108 136 184 180 196 218 148 156

August 116 116 112 108 108 108 128 144 128 132 168 172 207 196 196 152 168 208 224 160 214 228 224

September 96 108 112 100 100 96 116 108 107 104 176 164 198 172 172 140 136 228 198 168 172 188 180

October 110 88 84 88 76 88 83 80 76 120 148 167 120 120 132 128 216 232 180 224 192 228

November 116 92 95 76 76 92 76 80 92 84 112 116 119 124 124 120 200 166 136 132 196 156

December 116 104 104 108 108 132 92 88 88 92 124 152 131 114 108 120 132 140 186 120 160 157128

160

Publication: Economic Report of the President todo: 0 0iss. pages

N/A (no publication available) total pages: 20604

Needs to be scanned total pages scanned: 20604 100%


Needs to be rescannedCleaning to be checkedScanned & CleanedOCR RunTransferred to FRASER Metadata entered & files prep'd for postingPosted to FRASER 66 20604PDF downloaded from Internet site

* * * * * * * *1947 1947M 1948 1948M 1949 1949M 1950 1950M 1951 1951M 1952 1952M 1953 1954 1955 1956 1957 1958 1959

69 92 126 148 136 136 205 172 252 288 232 200 228 238 220 252 216 210 239

* * * * * * * * * * * * *1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989260 232 312 303 314 296 312 322 322 332 345 312 294 312 364 368 274 310 392 316 340 368 296 296 354 364 388 380 356 444

* * * * * * * * * * * *1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006432 420 432 478 408 401 406 432 402 449 424 392 441 397 417 335 401

Publication: NIPA Tables from SCBs todo: 0 1176iss. pages

N/A (no publication available) total pages: 984

Needs to be scanned 1176 total pages scanned: 984 100%


Needs to be rescannedCleaning to be checkedScanned & CleanedOCR Run 90 274

File transferred to FRASER 36 112

Metadata entered & files prep'd for posting 80 254

Posted to FRASER 169 344

PDF downloaded from Internet siteT U V W X Y Z AA AB AC AD AE

1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976January 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 3 4 3 3 3 3 3 3 3 3 3 11

February 2 2 2 2 2 2 2 2 2 2 2 3 4 3 5 3 3 3 3 3 3 3 3 3 3 3

March 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3

April 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3

May 2 2 2 2 2 2 2 2 2 2 3 2 3 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3

June 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 4 2 3 3 3 3 3 3 3 3 3 3

July 5 5 3 6 9 6 7 11 11 10 4 5 8 9 6 6 3 6 5 5 5 5 4 4 4 4 3 3

August 1 2 2 2 2 2 2 2 2 2 2 2 2 3 4 4 4 6 3 3 3 3 3 3 3 3 3 3 3

September 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3

October 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3

November 2 2 2 2 2 2 2 2 2 2 2 4 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3

December 1 1 1 1 1 1 1 1 1 1 7 1 1 2 3 2 2 2 3 3 3 3 3 3 3 3 3 3 3

1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006January 3 3 3 3

February 3 3 3 3March 3 3 3 3

April 3 3 3

May 3 3 3 3

June 3 3 3

July 3 4

August 3 3 3

September 3 3 3

October 3 3

November 3 3 3

December 3 3 3

Publication: Federal Reserve Bulletin todo: 251 39049iss. pages

N/A (no publication available) 70560 total pages: 52172 61855Needs to be scanned 9660 total pages scanned: 52172

Scanned (needs to be cleaned) 1 112 total pages cleaned: 52060

Needs to be rescannedCleaning to be checked 251 29389


OCR Run 3/1921 - 264File transferred to FRASER 10/1922: 1169Metadata entered & files prep'd for posting 3/1925 - cover - ii, 171Posted to FRASER 11/1924 - last 2 pagesPDF downloaded from Internet site

T U V W X Y Z AA AB AC AD AE1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949

January 118 136 132 144 80 76 80 80 56 98 68 80 76 96 92 98 100 116 102 104 110 120 140 116

February 106 140 146 80 80 84 76 64 56 88 76 80 112 80 110 108 102 110 114 128 144 116

March 132 116 124 136 104 84 80 104 68 68 64 68 112 72 78 106 102 86 96 96 146 124 108 120

April 112 136 124 136 90 72 88 68 80 88 72 60 96 96 112 110 114 88 116 122 108 140 130 140

May 148 152 124 136 76 76 84 72 64 56 80 100 123 84 102 122 132 122 116 116 112 156 136 160

June 116 128 132 120 80 84 96 68 80 72 114 88 108 128 136 96 120 106 108 114 122 148 148 148 152

July 116 144 226 92 84 112 104 100 68 84 108 76 84 80 120 126 116 92 132 172 148 132

August 136 140 128 116 72 80 60 76 76 92 64 72 116 84 110 136 106 102 134 134 140 156 144

September 120 128 136 108 88 112 88 68 68 80 88 80 148 76 132 146 118 106 114 148 136 164 152

October 120 120 124 96 72 72 56 64 60 64 88 68 88 100 116 112 122 90 112 90 110 132 148 132 136

November 136 124 128 84 76 64 68 52 96 56 64 92 84 122 96 92 142 114 94 114 100 108 120 132 140

December 136 148 132 114 116 104 160 92 96 104 130 128 128 101 144 134 120 102 138 120 144 156 136 136

1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979January 134 132 120 104 124 118 108 124 132 136 120 148 121

February 144 136 120 128 140 134 148 144 144 146 192 144 155

March 140 116 120 108 124 152 108 108 132 124 128 156 166

April 112 124 140 116 144 144 129 116 126 130 168 132 136

May 128 140 124 132 120 124 115 144 140 174 176 128 146

June 152 132 144 134 148 116 112 128 112 114 140 148 140

July 172 168 128 116 112 140 140 120 148 146 164 130 140

August 172 160 120 140 152 140 276 140 144 178 160 180 156

September 184 180 128 104 124 140 151 128 108 164 162 128 176

October 158 128 104 140 108 108 118 128 146 132 152 148 146

November 144 112 92 128 100 132 110 108 116 170 140 144 162

December 180 192 124 148 160 136 144 136 140 197 144 132 166

AlFRED• Extension of FRED

– http://alfred.stlouisfed.org

• Since FRED became Web-based in 1996, we have preserved real-time archive files (generally at weekly frequency.)

• Over that time period FRED has expanded from roughly 381 to 1175 U.S. macroeconomic time series (and 1881 regional series)

• Additional real-time archival data from NET and MT since 1967.

Measuring Valid Time

• Ideally want the release dates for each update of the series:– Have collected all NIPA release dates from

1948– Have Employment release dates from

1950s– Have IP release dates from 1927– Have Productivity & Costs, ECI release

dates from beginning– Have Money Stock (H.6) release dates

from 1960– Have Housing Starts release dates from

1950

Measuring Valid Time

• Protocol when release dates are not available from source agency:– Searchable Historical archive of NY

Times and Wall Street Journal– Has proved an extensive source of

exact release dates.• e.g. have confirmed exact date for all

but one NIPA release date (5/18/62) • Were able to pin down an interval of

about a week for that release – guessed based on the typical release timing.

Development Strategy

• Have now uploaded the FRED archive– Real time histories of all the series

contained in FRED since Dec 1996. – Have not included daily data

• Typically Interest rates and exchange rates last since data generally don’t revise (although publication errors are not unknown.)


• Extended FRED archives back using NET and MT histories

• Series we have available– Civilian and Payroll Employment

(monthly) since Jan-66– Industrial Production (monthly)

since May-27– Retail Sales (monthly) since 66– Nominal Personal Income (Monthly)

since Jan-66

A Tabular Guide to the NIPA Accounts in AlFRED

August 2006

Blue = complete history; brown = history in process; orange = in FRED archive only























– M1 (monthly data– weekly valid time dates) since 60

– M2 (monthly data – weekly valid time dates) since 62

– Real Personal Income (monthly) since Nov-76

– Real Manufacturing and Trade Sales since Dec-76 with some gaps

– Output per hour private business sector since Mar-71 with some gaps


– Output per hour nonfarm private business sector since Jun-68 with some gaps

– Compensation per hour nonfarm private business sector since Dec-72 with some gaps

– Unit labor Costs private business sector since Mar-71 with some gaps


– Nominal Disposable Personal Income (monthly) since Dec-79

– Real Disposable Personal Income (monthly) since Dec-79

• Add additional series and/or extend scope of current FRED series– As users, your Priorities???


• Evolve the User Access and Download System– Implement selection of a range of valid

dates for a series.– Loop over vintages for a specified list

of series (i.e. recursive data sets)– Unique source material citation string

(replication)– As users, what are your desired

features??

879 National Series

Excel Output Files

• WP-2005-021A Marcelle Chauvet, and Jeremy M. Piger, "A Comparison of the Real-Time Performance of Business Cycle Dating Methods,"

March 2005

http://research.stlouisfed.org/econ/piger/

http://research.stlouisfed.org/econ/piger/

http://research.stlouisfed.org/wp/2005/2005-021.pdf



Questions and Comments

Contact:

[email protected]

Date post:	29-Mar-2015
Category:	Documents
Upload:	quintin-pillsbury
View:	217 times
Download:	0 times

The Evolution of FRASER and AlFRED: St. Louis Fed Real Time Data Services Robert H. Rasche Federal...

Documents