LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
DMIN 2013, LAS VEGAS
Neural Network Forecasting with the S&P 500 Index Across Decades
M.E. Malliaris & A.G. MalliarisJuly 23, 2013
The 9th International Conference on Data Mining, Las Vegas
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
QUESTIONS• Are there stable patterns of directional
movement in the S&P 500?• If so, can we use those patterns to
forecast?• Do these patterns and their
importance change over time?
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
DATA SET• Closing values of the S&P 500 from 1950
through 2010 [6 decades]• Derived variables:
– % change in closing– Moving averages– Patterns of Up and Down movement – Number of Ups
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
DATA SET DIVISIONS• Divided by Decade for analysis and
training• 10 Years used for training,
– e.g., 1950 - 1959• The Year immediately following a
Decade was used as a validation set– e.g., 1960
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Two-Day Strings
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Two-Day Strings
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Two-Day Strings
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Up Movement Only
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Up Movement Only
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Up Movement Only
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Inputs
Input ExampleToday’s Closing Value 1132.99Percent Change in the Closing Value 1.60Today’s Closing Direction U4 Day Moving Average of Closing 1137.08Percent Change in the 4-day Mov. Avg. -0.0593Number of Up Closings in last day 0 or 1Number of Up Closings in last 2 days 1Number of Up Closings in last 3 days 2Number of Up Closings in last 4 days 2Number of Up Closings in last 5 days 3
2-day Up and Down pattern DU
3-day Up and Down pattern UDU
4-day Up and Down pattern DUDU
5-day Up and Down pattern UDUDU
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Data Sets
Training SetValidation Sets
Jan 1 through Dec 31Jan 1, 1950 -- Dec 31, 1959 1960Jan 1, 1960 -- Dec 31, 1969 1970Jan 1, 1970 -- Dec 31, 1979 1980Jan 1, 1980 -- Dec 31, 1989 1990Jan 1, 1990 -- Dec 31, 1999 2000Jan 1, 2000 -- Dec 31, 2009 2010
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
METHODOLOGY• Neural Network
• Structure: Identical for all networks– 14 Inputs– One Hidden Layer with 9 nodes– 1 Output [tomorrow’s direction]– Random Seed: 229176228– 30% used to prevent over-fitting
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Software: IBM’s SPSS Modeler 14
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
ResultsTraining Decade
Tr Percent Correct
Validation Year
Val Percent Correct
1950-59 59.00% 1960 59.92%
1960-69 59.90% 1970 62.99%
1970-79 59.94% 1980 58.10%
1980-89 55.18% 1990 54.94%
1990-99 56.25% 2000 52.78%
2000-09 52.96% 2010 47.22%
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Percent Correct DirectionsTraining Set Validation Set
Down Up Down Up50s 53.90% 60.95% 61.68% 58.62%60s 57.94 61.00 67.02 60.6270s 58.88 60.94 53.25 60.2380s 53.21 56.16 63.16 54.2790s 53.43 57.98 56.70 50.3200s 50.67 55.01 40.74 54.70
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Top Five InputsFirst PercChgClose PercChg4MA PercChgClose PercChg4MA PercChg4MA PercChg4MA
Second
Third
Fourth
Fifth
Decade 50s 60s 70s 80s 90s 00s
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Top Five InputsFirst PercChgClose PercChg4MA PercChgClose PercChg4MA PercChg4MA PercChg4MA
Second PercChg4MA NumUps4 PercChg4MA Close NumUps5 NumUps4
Third
Fourth
Fifth
Decade 50s 60s 70s 80s 90s 00s
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Top Five InputsFirst PercChgClose PercChg4MA PercChgClose PercChg4MA PercChg4MA PercChg4MA
Second PercChg4MA NumUps4 PercChg4MA Close NumUps5 NumUps4
Third String5Days PercChgClose Close String5Days PercChgClose MA4day
Fourth
Fifth
Decade 50s 60s 70s 80s 90s 00s
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Top Five InputsFirst PercChgClose PercChg4MA PercChgClose PercChg4MA PercChg4MA PercChg4MA
Second PercChg4MA NumUps4 PercChg4MA Close NumUps5 NumUps4
Third String5Days PercChgClose Close String5Days PercChgClose MA4day
Fourth String4Days String4Days MA4day NumUps3 MA4day String5Days
Fifth
Decade 50s 60s 70s 80s 90s 00s
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Top Five InputsFirst PercChgClose PercChg4MA PercChgClose PercChg4MA PercChg4MA PercChg4MA
Second PercChg4MA NumUps4 PercChg4MA Close NumUps5 NumUps4
Third String5Days PercChgClose Close String5Days PercChgClose MA4day
Fourth String4Days String4Days MA4day NumUps3 MA4day String5Days
Fifth NumUps1 String5Days NumUps2 PercChgClose String3Days Close
Decade 50s 60s 70s 80s 90s 00s
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Summary• Neural Networks with the same structure
were trained for six decades• This identical structure,using the same
inputs, was useful for over six decades. • All variables were generated from the S&P
500 closing price• Variable importance shifted slightly over time• Successful forecasting was possible
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESSLOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS
Future Research• Future research might investigate a smaller
training time, say a rolling window of one or two years.
• This might enable us to see the importance of specific variables gradually shifting over time.
LOYOLA UNIVERSITY CHICAGO QUINLAN SCHOOL OF BUSINESS