Pinder An Excel Solver Exercise to Introduce Nonlinear Regression 671808- 1 AN EXCEL SOLVER EXERCISE TO INTRODUCE NONLINEAR REGRESSION Jonathan P. Pinder, PhD, Wake Forest University, 336 Farrell Hall, Wake Forest University, Winston-Salem 27109 [email protected], (336) 758-5036 ABSTRACT Business students taking business analytics courses that have significant predictive modeling components, such as marketing research, data mining, forecasting, and advanced financial modeling, are introduced to nonlinear regression using application software that is a ‘black box’ to the students. Thus, although correct models are estimated, students often do not obtain a thorough understanding of the nonlinear estimation process. The exercise presented in this paper was created to demonstrate to students the need for nonlinear regression estimation – rather than using linear transformations and Ordinary Least Squares (OLS) and subsequently demonstrate the nonlinear optimization process to estimate nonlinear regression models. Using the spreadsheet exercise, students can see effects on the fit of the model by changing the model parameters as they change the values of the decision variables. After applying the spreadsheet to further exercises, students have expressed a deep understanding of the linear regression software. This exercise is innovative because the active learning exercise requires the students to make the logical connections between the structure of the model, the model’s parameters, and the objective function. Keywords: Analytics, Regression, Nonlinear Optimization INTRODUCTION The current job market for business graduates in business analytics is drawing business students to analytical courses. Specifically, advanced students are interested in learning nonlinear regression modeling because it is a leading-edge business analytics predictive modeling tool. Davenport and Harris (2007) and Fisher and Raman (2010) present numerous predictive modeling applications of nonlinear regression models for financial, operations, marketing, and human resource functions. Because nonlinear regression is not constrained to the assumption of linearity, it offers a deeper core paradigm for modeling complex data. This paper presents a method for introducing the concepts of nonlinear regression modeling to students in business analytics. When first learning regression modeling, students are confronted with the decision of model structure; specifically, a linear versus nonlinear fit. This article presents an active learning classroom exercise for students in business analytics that introduces nonlinear regression by using Excel’s Solver optimization add-in for nonlinear optimization. The exercise presented in this paper was designed to validate for students the need for nonlinear regression estimation and then demonstrate the nonlinear optimization process to estimate nonlinear regression models. The exercise first requires students to compare the prediction accuracies of a linear versus a logarithmic (loglinear) model. Subsequently, students use Excel’s Solver to obtain model parameters that minimize the Sum of Squared Errors (SSE); this provides an appropriate
Transcript
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 1
AN EXCEL SOLVER EXERCISE TO INTRODUCE NONLINEAR REGRESSION
Jonathan P. Pinder, PhD, Wake Forest University, 336 Farrell Hall, Wake Forest University, Winston-Salem 27109
Business students taking business analytics courses that have significant predictive modeling components, such as marketing research, data mining, forecasting, and advanced financial modeling, are introduced to nonlinear regression using application software that is a ‘black box’ to the students. Thus, although correct models are estimated, students often do not obtain a thorough understanding of the nonlinear estimation process. The exercise presented in this paper was created to demonstrate to students the need for nonlinear regression estimation – rather than using linear transformations and Ordinary Least Squares (OLS) and subsequently demonstrate the nonlinear optimization process to estimate nonlinear regression models. Using the spreadsheet exercise, students can see effects on the fit of the model by changing the model parameters as they change the values of the decision variables. After applying the spreadsheet to further exercises, students have expressed a deep understanding of the linear regression software. This exercise is innovative because the active learning exercise requires the students to make the logical connections between the structure of the model, the model’s parameters, and the objective function.
The current job market for business graduates in business analytics is drawing business students to analytical courses. Specifically, advanced students are interested in learning nonlinear regression modeling because it is a leading-edge business analytics predictive modeling tool. Davenport and Harris (2007) and Fisher and Raman (2010) present numerous predictive modeling applications of nonlinear regression models for financial, operations, marketing, and human resource functions. Because nonlinear regression is not constrained to the assumption of linearity, it offers a deeper core paradigm for modeling complex data. This paper presents a method for introducing the concepts of nonlinear regression modeling to students in business analytics.
When first learning regression modeling, students are confronted with the decision of model structure; specifically, a linear versus nonlinear fit. This article presents an active learning classroom exercise for students in business analytics that introduces nonlinear regression by using Excel’s Solver optimization add-in for nonlinear optimization. The exercise presented in this paper was designed to validate for students the need for nonlinear regression estimation and then demonstrate the nonlinear optimization process to estimate nonlinear regression models. The exercise first requires students to compare the prediction accuracies of a linear versus a logarithmic (loglinear) model. Subsequently, students use Excel’s Solver to obtain model parameters that minimize the Sum of Squared Errors (SSE); this provides an appropriate
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 2
comparison of the linear and loglinear models. These methods are then implemented in a spreadsheet model of Expenditures per Consumer Unit for Entertainment data. This exercise is innovative because the active learning exercise requires the students to link the structure of the model and the objective function by selecting model parameter values to improve the objective function value.
LITERATURE REVIEW
Seber and Wild (2003) present a rigorous and detailed discussion of the merits of nonlinear regression estimation versus linearized transformation. In particular they state:
“ the parameters of a linearized model are often not as interesting or as important as the original parameters. In physical and chemical models the original parameters usually have a physical meaning, e.g. rate constants, … . Therefore, given the availability of efficient nonlinear algorithms, the usefulness of linearization is somewhat diminished.”
Similarly, the original parameters of various business and economic models have a specific meaning, such as the rate of return in a financial model. Thus, Seber and Wild’s treatment for testing of nonlinear versus linearized transformed models provides the theoretical foundation for the beginning of the exercise in which a linear fit is compared to an exponential (LN(Y)) fit.
Evans (2013) describes three phases of business analytics: 1) descriptive analytics, 2) predictive analytics, and 3) prescriptive analytics. The first phase is characterized by the use of descriptive statistics to explore data. The second phase is characterized by the use of statistical models, such as nonlinear regression, to predict or forecast. The third phase is characterized by the use of optimization to determine optimum decisions. The integration between the optimization of the third phase and the model-building of the second phase provided a starting point for the design of the optimization/model estimation portion of the exercise.
With the elements of nonlinear regression and the modeling problem context established, a pedagogical methodology that conveys the principles of nonlinear estimation is needed. Calling for the students to undertake the role of manually changing the model parameters to reduce SSE was a clear means of having them identify the structure of the model, model parameters, and the objective function of the optimization problem.
Bonwell and Eison (1991) state that active learning requires students to have an active engagement in the acquisition and synthesis of the learning material. Active learning is often contrasted to the traditional lecture format in which students passively receive information from the instructor. Prince (2004) defined common forms of active learning and examined the effectiveness of active learning. Prince (2004) defines the core elements of active learning as student activity and engagement in the learning process and shows that considerable support exists for the efficacy of the core elements of active learning. Specifically, Prince (2004) states that introducing activity into lectures can significantly improve recall of information and presents evidence that supports the benefits of student engagement. For the setting of business analytics, Riddle (2010) gives an example of an active learning exercise for introducing the formulation of linear programming models.
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 3
The active learning aspect of the exercise strengthens students’ acquisition and retention of the intellectual links between the structure of the model, the model’s parameters, and the objective function. The combination of an effective pedagogy with an innovative nonlinear regression methodology in a compelling context provided the basis for the active learning exercise.
THE EXERCISE
The exercise is designed to accomplish three learning objectives: (1) assess the fit of nonlinear regression models; (2) estimation of nonlinear models by changing model parameters until a specified objective function is optimized; and (3) apply nonlinear optimization to estimate nonlinear statistical models for actual data in which the students may not be aware of the current specific statistical estimation procedure. The objectives are presented in three stages of approximately 20 to 25 minutes each.
The first stage of the exercise is designed to teach assessment of regression model fits and demonstrate that transforming nonlinear models to OLS models is not as effective as nonlinear estimation; thus establishing the necessity for nonlinear regression. This is accomplished by using a randomly generated example set of data. The data in the example was created by generating X from a continuous uniform distribution over the interval [10, 100) and generating Y using the equation:
Y = β0eβ1x+ (1)
where β0 = 10, β1 = 0.025, and is generated from a Normal distribution with an average of 0 and a standard deviation of 5. Thus, the true model parameters are known to the students. Next, the regression results for linear and logarithmic models from a set of simulated data (Exhibit 1) are presented. Students spontaneously compare r2
Linear to r2LN to ascertain the best model
structure and arrive at the incorrect conclusion that because r2Linear > r2
LN (0.9023 > 0.8873), then the linear model is a better fit than the logarithmic model.
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 4
Exhibit 1: Regression Results for Linear and Logarithmic Models for Example Data
Linear Model: Y = β0 + β1X r2 0.9023 Std Error of Est 9.6033
After discussing these numerical regression results, the graph of the simulated data with the true model, linear model, and logarithmic model (Exhibit 2) is presented to the students. From Exhibit 2, students easily detect that the logarithmic model provides a better fit than the linear model. Thus, they recognize that r2
LN (0.8873) represents the fit of the transformed estimates to the transformed data rather than the fit of the estimates to the actual data. It follows logically that the Standard Error of the Estimate for the logarithmic model (SEELN) is not in the original units. Thus, SEELN is not comparable to SEELinear and r2
LN is not comparable to r2Linear. This lack
of comparability between the r2s creates an immediate concern for the students because they cannot readily determine the appropriate structure using r2 nor SEE (Seber and Wild (2003)).
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 5
Exhibit 2: Linear and Logarithmic Models fitted to Example Data
At this point, Y is regressed against estimated Y values from the logarithmic model:
Y = β0 + β1Yest (2)
This regression is known as the Mincer-Zarnowitz (MZ) regression (Diebold (2006)). The MZ regression provides three very useful results: 1) the r2 represents the percentage of variation in the data explained by the model in the original units of the data, 2) the SEE is also in the original units of the data, and 3) a joint test of the coefficients: β0 = 0 and β1 = 1 tests the model for bias. The first two results are useful as measures of fit that can be used to compare various models regardless of their origin. The third result can also be used to un-bias estimates if needed. Exhibit 3 shows the MZ regression for the fit of the logarithmic model on the simulated data:
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 6
Exhibit 3: Mincer-Zarnowitz Regression Results for Logarithmic Model of Example Data
Mincer-Zarnowitz Regression: Y = β0 + β1 Est.Y r2 0.9694Std Error of Est 5.3752
ANOVA: Mincer-Zarnowitz Regression Source SS df MS F p-value Model 226,927.04 1 226,927 7,854 0.0000 Residual 7,165.42 248 29Total 234,092.45 249
Because the MZ results are in the same units as the original data, the SEEMZLN and r2LN are now
comparable to SEELinear and r2Linear respectively. Note that t-stat* for β1 is |1- β1|/SEβ1, thus
testing β1 = 1. The MZ results indicate that, while the estimates (Yest) produced by the logarithmic model are statistically better than those produced by the linear model, they are biased. Once corrected, such biases would not change the resulting r2 or the SEE. At this point, the students are directed towards the Sum of Squared Errors (also called Residual Sum of Squares), SSE, and reminded:
r2 = 1 – SSE/ SST (3)
and
SEE = MSE1/2 = (SSE/(n-k))1/2 (4)
where n is the number of observations and k is the number of model parameters. Thus, minimizing SSE will minimize SEE and maximize r2. Albritton and McMullen (2006) present an example of minimizing SSE for a forecasting problem to integrate management science with statistics for the classroom.
The second stage of the demonstration is the estimation of nonlinear models by changing model parameters until a specified objective function is optimized. For this stage, students are directed to a spreadsheet (shown Exhibit 4) in which Yest is to be estimated directly by referencing the decision variable cells for β0 and β1 ($C$27 and $C$28 respectively):
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 7
Exhibit 4: Starting Point for Exponential Model (min SSE) of Example Data
The formula for Yest (Est Y) in cell E43 is: =$D$27*EXP($D$28*C43). After this formula has been copied to the appropriate range, the formulae for squared errors (e^2) are created in column F and the sum is computed, thus calculating SSE. The students are given starting values of β0 = 10 and
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 8
β1 = 0 and asked for interpretations; thus reminding the students that β0 is an initial amount and β1 is a growth rate.
Next, students are asked how to improve the model. Students readily recommend ‘bending the curve up’ by increasing the growth rate (β1). Students accomplish this by changing cell C28 (refer to Exhibit 3). The decrease in SSE from 588,397.19 to 95,945.08 is emphasized. Growth rates of 2% to 4% are entered into the spreadsheet and the students note the effects on the fit. Exhibit 5 shows the results of a 2% growth rate while retaining β0= 10. A quick poll is made of the students to determine who has achieved the lowest SSE.
Exhibit 5: 2% Growth Rate for Exponential Model (min SSE) of Example Data
Students are then asked how to obtain the most accurate growth rate. Common answers from students include “use Goal Seek to make the SSE smaller”, “use a one-way Data/Table to enumerate possible answers”, and finally “use the Solver to optimize Sum of Squared Errors”. Given the students’ required course in introductory business analytics (management science), it is natural for the students to use Excel’s Solver to optimize SSE by changing both β0 and β1. The results of the optimization are shown in Exhibit 6:
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 9
Exhibit 6: Optimization Results for βo and β1 of Exponential Model (min SSE) for Example Data
It is emphasized to the students that the results are unbiased (β0 = 0 and β1 = 1 for the MZ regression) and that SSEminSSE, SEEminSSE, and r2
minSSE are improved from SSEMZLN, SEEMZLN, and r2
LN (see Exhibit 3).
With the concepts explained and demonstrated in the second stage, the students are prepared for the third stage of the exercise: applying nonlinear optimization to actual data. This is accomplished by repeating the process presented in stage two for Expenditures per Consumer Unit for Entertainment for the years 1985-2009 (http://www.census.gov/compendia/statab/2012/tables/12s1232.pdf). This data exemplifies data of the type encountered by practitioners. Other economic data, such as housing starts, and revenues for various corporations (e.g., Amazon, Walmart, Pepsi, Ford) are treated in subsequent homework exercises.
Exhibit 7 shows a graph of the data, the logarithmic transform model, and the minimized SSE model. The corresponding numerical results of the logarithmic regression and optimized (min SSE) exponential model are shown in Exhibit 8.
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 10
Exhibit 7: Logarithmic Regression versus Optimized Exponential Model for Entertainment Data
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 11
Exhibit 8: Results of Logarithmic Regression versus Optimized Exponential Model for Entertainment Data
Mincer-Zarnowitz Regression: Y = β0 + β1 Est.Y r2 0.9688Std Error of Est 87.8642Parameter Estimate Std Error t-stat* p-value β0 -26.6172 71.5902 -0.37 0.7134 β1 1.0157 0.0378 0.41 0.6788
ANOVA TABLE: Mincer-Zarnowitz Regression Source SS df MS F p-valueModel 5,514,647 1 5,514,647 714 0.0000Residual 177,563 23 7,720.12 Total 5,692,210 24
Exponential Model: Y = β0 e β1 X
Decision Variables β0 1149.7020β1 0.0363
Results r2 0.9692Std Error of Est 87.3466
ANOVA TABLE: Equivalent to Mincer-Zarnowitz Regression Source SS df MS F p-valueModel 5,516,733 1 5,516,733 723 0.0000Residual 175,477 23 7,629.43 Total 5,692,210 24
Students conclude that the results of both models are unbiased and that the optimized exponential model is better than the model of the logarithmic transformed data. This leads to the conclusion that using optimization will be no worse than transformed OLS and can be significantly better.
Subsequent class sessions involve the development of nonlinear models for forecasting (Diebold (2008)) and maximum likelihood estimation models for multinomial logistic regression (Hosmer and Lemeshow (2000)). These models are then applied to models for insurance risk management, revenue management, customer retention, and supply chain management.
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 12
CONCLUSIONS
This exercise is intended to teach students how to (1) assess the fits of various models; (2) estimate nonlinear models using nonlinear optimization software (Excel’s Solver); and (3) apply nonlinear optimization for actual nonlinear models. Unfortunately, because the course is a new advanced business analytics ‘topics’ elective, no effects on student learning, relative to other teaching methods, were measured. This is certainly an area for future research and validation.
The fact that none of the students had any prior exposure to nonlinear regression creates a common baseline so that the students’ qualitative assessments of the exercise provide a qualitative measure of the efficacy of this exercise as a pilot pedagogy. Exhibit 9 summarizes student assessments of the exercise’s effectiveness (based upon similar university course evaluation instruments) by two classes of MBA students enrolled in an advanced business analytics course with agent-based modeling as a significant curricular component. The assessment process was voluntary and anonymous; the response rate was 100% of enrollment (66). The student responses offer evidence that the lecture promotes student learning as intended.
APPENDIX
The spreadsheet shown in Exhibit 10 was constructed prior to class. The data consists of 250 observations located in cells C43:C292 and D43:D292, generated as previously described in the paper. The formula for Yest (Est Y) in cell E43 is: $D$27*EXP($D$28*C43). This formula is then copied from cell E43 to the range E43:E296. The squared errors (e^2) are created by entering the formula =(D43-E43)*(D43-E43) in cell F43 and then copying that formula from cell F43 to the range F43:F296. Next, the sum of squared errors (SSE) is computed in cell F41 by the formula =SUM(F43:F292). The ANOVA table (as shown in Exhibit 10) is created as a tool to compare the results of the optimization to other regressions. The ANOVA table is created by referencing the SSE from cell F41 in cell D38 and then completing the remainder of the table using the standard ANOVA table formulae.
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 13
Exhibit 10: Starting Point for Exponential Model (min SSE) of Example Data
The optimization nonlinear regression parameters (β0 and β1) begins with the initial values of β0 = 10 and β1 = 0, located in cells D27 and D28 respectively. Other values, such as β0 = 0 and β1 = 0, would have served as well for the initial values. Next, the Solver add-in is a started by selecting it from the Analysis portion of the Data tab in Excel’s Ribbon. Exhibit 11 shows the Solver’s dialog box with the appropriate inputs for this exercise.
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 14
Exhibit 11: Inputs for Excel’s Solver
Cell D38 (SSE in the ANOVA table) is selected as the Objective cell. Select the Min option for the objective function. Cells D27 and D28 are selected as the “Changing Variable Cells” (decision variables). Be sure that the Make Unconstrained Variables Non-negative checkbox is not checked and that the GRG Nonlinear “Solving Method” is selected. After these inputs have been set, then click on the Solve button and the Solver will change the decision variables until the objective cell is optimized.
Exhibit 12 shows the results after the Solver has minimized the SSE. Again note that minimizing SSE maximizes r2, as indicated by the formula in cell D33: =1-D38/D39; that is r2=(1-SSE/SST).
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 15
Exhibit 12: Spreadsheet Results After Excel’s Solver Optimization
REFERENCES
Albritton M. D. & McMullen, P. R. (2006). Classroom integration of statistics and management science via forecasting. Decision Sciences Journal of Innovative Education,4(2), 331-336.
Pinder An Excel Solver Exercise to Introduce Nonlinear Regression
671808- 16
Bonwell, C. C., & Eison, J. A. (1991). Active learning: Creating excitement in the classroom. ASHE-ERIC Higher education report no. 1. Washington, DC: School of Education and Human Development, George Washington University.
Davenport, T. H., & Harris, J. G. (2007). Competing On Analytics, The New Science of Winning. Boston, MA: Harvard Business School Press.
Dielbold, F. X. (2008). Elements of Forecasting (4th ed.). Mason, OH: South-Western, Cengage Learning.
Evans, J. R. (2013). Business Analytics; Methods, Models, and Decisions. New York, NY: Pearson.
Fisher, M. L., & Raman, A. (2010). The New Science of Retailing, How analytics are transforming the supply chain and improving performance. Boston, MA: Harvard Business School Press.
Hosmer D. W. & Lemeshow, S. (2000). Applied Logistic Regression (2nd ed.). New York, NY: John Wiley & Sons.
Manning, W. G. & Mullahy, J. (2001). Estimating log models: to transform or not to transform? Journal of Health Economics,20, 461-494.
Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Educaion, 93(3), 223-231.
Riddle, E. J. (2010). An active learning exercise for introducing the formulation of linear programming models. Decision Sciences Journal of Innovative Education, 8(2), 367-372.
Seber, G. A. F. & Wild, C. J. (2003). Nonlinear Regression. New York, NY: John Wiley & Sons.
