Math 385/585 Applied Regression Analysis Fall 2017 Section 001 1:50 to 2:50 M W F Instructor: Dr. Chris Edwards Phone: 948-3969 Office: Swart 123 Classroom: Swart 3 Text: Applied Linear Statistical Models,5 th edition, by Kutner, Nachtsheim, Neter, and Li. Earlier editions of the text will likely be adequate, but you will have to allow for different page numbers and homework problem numbers. Catalog Description: A practical introduction to regression emphasizing applications rather than theory. Simple and multiple regression analysis, basic components of experimental design, and elementary model building. Both conventional and computer techniques will be used in performing the analyses. Prerequisite: Math 201 or Math 301 and Math 256 each with a grade of C or better. Course Objectives: Linear models in statistics are the backbone of many applications, including regression and ANOVA techniques. Math 385 focuses students on the regression aspect of modeling while Math 386 focuses students on the ANOVA aspect. In Math 385, students will learn how to calculate and interpret regression estimates, including parameter estimates, fits, and residuals, and will be able to perform statistical inference. In addition to simple linear regression, successful students will understand the issues introduced in multiple linear regression, including polynomial regression and non-linear regression. Finally, the student will be able to assess model adequacy and know methods to update and improve the model. Upon successful completion of the course, students are expected to have the ability to complete the following: • Identify and understand the components and assumptions for the standard linear regression model • Use statistical inference on regression model coefficients, including confidence intervals and hypothesis tests • Construct and interpret the ANOVA table for describing a linear regression model • Calculate and analyze residuals from a regression model • Perform diagnostics on a regression model, including assessing lack of fit • Perform remedial measures such as transformations to improve a regression model • Understand how linear algebra can be used to describe a multiple regression model • Perform inference in multiple regression and understand how the increased number of dimensions adds complexity to the interpretations due to collinearity • Understand how to fit polynomial regression models • Know how to use indicator variables in regression models • Be able to build a model from a pool of variables, using techniques such as Best Subsets and Stepwise Regression • Identify outliers, in both the X and Y dimensions, in multiple regression models • Understand the basics of non-linear regression, including Logistic Regression
Catalog Description: A practical introduction to regression emphasizing applications rather thantheory. Simple and multiple regression analysis, basic components of experimental design, andelementarymodelbuilding.Bothconventionalandcomputertechniqueswillbeusedinperformingtheanalyses.Prerequisite:Math201orMath301andMath256eachwithagradeofCorbetter.
Course Objectives: Linear models in statistics are the backbone of many applications, includingregressionandANOVAtechniques.Math385 focusesstudentsontheregressionaspectofmodelingwhile Math 386 focuses students on the ANOVA aspect. In Math 385, students will learn how tocalculateandinterpretregressionestimates,includingparameterestimates,fits,andresiduals,andwillbeabletoperformstatisticalinference.Inadditiontosimplelinearregression,successfulstudentswillunderstand the issues introduced inmultiple linear regression, including polynomial regression andnon-linearregression.Finally,thestudentwillbeabletoassessmodeladequacyandknowmethodstoupdateandimprovethemodel.
Homework: I will collect (around) 5 homework problemsapproximatelyonceeveryotherweek.Theduedatesare listedonthecourseoutlinebelow.Isuggestthatyouworktogetherinsmallgroupsonthehomeworkifyoulike;don’tforgetthatIamaresourceforyoutouse.Oftenwewillusecomputersoftwareto perform our analyses; include printouts where appropriate,but pleasemake your papers readable. In otherwords, I don’twant25pagesofprintouthanded in ifyoucansummarize it intwopages.
Mypersonalbelief is thatone learnsbestbydoing. Ibelieve thatyoumustbe trulyengaged in thelearningprocesstolearnwell.Therefore,Idonotthinkthatmyroleasyourteacheristotellyoutheanswerstotheproblemswewillencounter;ratherIbelieveIshouldpointyouinadirectionthatwillallow you to see the solutions yourselves. To accomplish that goal, I will find different interactiveactivitiesforustoworkon.Yourjobistouseme,yourtext,yourfriends,andanyotherresourcestobecomeadeptatthematerial.TheDayByDaynotesalsoincludeSkillsthatIexpectyoutoattain.
Math 585 Expectations: Expectations for the graduate students are understandably more rigorousthanfortheundergraduatestudent.StudentstakingMath585willhaveanextratheoreticalproblemadded to eachhomework, tobe assignedduring the semester. In addition, a final projectworth50pointswillbedueattheendofthesemester.Thisprojectwillinvolveacompleteanalysisofadataset,includingmodelestimation,development,andvalidation.
GradePointAverage. Thedirector of admissions of a small college selected 120 students atrandomfromthenewfreshmanclassinastudytodeterminewhetherastudent’sgradepointaverage(GPA)attheendofthefreshmanyear(𝑌)canbepredictedfromtheACTtestscore(𝑋). The results of the study follow. Assume that first-order regression model (1.1) is
appropriate.
a.) Obtain the least squares estimates of 𝛽! and 𝛽!, and state the estimated regressionfunction.
b.) Plottheestimatedregressionfunctionandthedata.Doestheestimatedregressionfunctionappeartofitthedatawell?
a.) Obtaina99percentconfidence interval for𝛽!. Interpretyourconfidence interval.Does itinclude zero? Why might the director of admissions be interested in whether theconfidenceintervalincludeszero?
b.) Test,usingtheteststatistic𝑡∗,whetherornotalinearassociationexistsbetweenstudent’sACTscore(𝑋)andGPAattheendofthefreshmanyear(𝑌).Usealevelofsignificanceof0.01.Statethealternatives,decisionrule,andconclusion.
b.) What isestimatedbyMSR inyourANOVAtable?ByMSE?UnderwhatconditiondoMSRandMSEestimatethesamequantity?
c.) Conduct and 𝐹 test of whether or not 𝛽! = 0. Control the 𝛼 risk at 0.01. State thealternatives,decisionrule,andconclusion.
d.) Whatistheabsolutemagnitudeofthereductioninthevariationof𝑌when𝑋isintroducedintotheregressionmodel?What istherelativereduction?What isthenameofthelattermeasure?
a.) Plot the data, with the least squares regression line for ACT scores between 20 and 30superimposed?
b.) Ontheplotfrompart(a),superimposeaplotofthe95percentconfidencebandforthetrueregressionlineforACTscoresbetween20and30.Doestheconfidencebandsuggestthatthetrueregressionrelationhasbeenpreciselyestimated?Discuss.
3.3,p.146-147
RefertoGradePointAverageProblem1.19.
a.) PrepareaboxplotfortheACTscores𝑋!.Arethereanynoteworthyfeaturesinthisplot?
b.) Prepareadotplotoftheresiduals.Whatinformationdoesthisplotprovide?
c.) Plot the residuals𝑒! against the fitted values𝑌!.What departures from regressionmodel(2.1)canbestudiedfromthisplot?Whatareyourfindings?
d.) Prepareanormalprobabilityplotoftheresiduals.Alsoobtainthecoefficientofcorrelationbetween the ordered residuals and their expected values under normality. Test thereasonablenessof thenormalityassumptionhereusingTableB.6and𝛼 = 0.05.Whatdoyouconclude?
e.) Conclude theBrown-Forsythe test todeterminewhetherornot theerror variance varieswith the level of𝑋. Divide the data into the two groups,𝑋 > 26 and𝑋 ≥ 26, and use𝛼 = 0.01. State the decision rule and conclusion. Does your conclusion support yourpreliminaryfindingsinpart(c)?
f.) Information is given below for each student on two variables not included in themodel,namely,intelligencetestscore 𝑋! .
3.21,p.151
Derivetheresultin(3.29):
𝑌!" − 𝑌!"!
!!
!!!
!
!!!
= 𝑌!" − 𝑌!!
!!
!!!
!
!!!
+ 𝑌! − 𝑌!"!
!!
!!!
!
!!!
SSE=SSPE+SSLF
Homework3 DueOctober233.17,p.150-151
Sales growth. A marketing researcher studied annual sales of a product that had beenintroduced10yearsago.Thedataareasfollows,where𝑋istheyear(coded)and𝑌issalesinthousandsofunits:
𝑖: 1 2 3 4 5 6 7 8 9 10
𝑋!: 0 1 2 3 4 5 6 7 8 9
𝑌!: 98 135 162 178 221 232 283 300 374 395
a.) Prepareascatterplotofthedata.Doesalinearrelationappearadequatehere?
b.) Use the Box-Cox procedure and standardization (3.36) to find an appropriate powertransformationof𝑌.EvaluateSSEfor𝜆 = 0.3, 0.4, 0.5, 0.6, 0.7.Whattransformationof𝑌issuggested?
a) Fit regressionmodel (6.5) to the data for three predictor variables. State the estimatedregressionfunction.Howare𝑏!,𝑏!,and𝑏!interpretedhere?
b) Obtain the residuals andprepare aboxplotof the residuals.What informationdoes thisplotprovide?
c) Plottheresidualsagainst𝑌,𝑋!,𝑋!,𝑋!,and𝑋!𝑋!onseparategraphs.Alsoprepareanormalprobabilityplot.Interprettheplotsandsummarizeyourfindings.
d) Prepare a time plot of the residuals. Is there any indication that the error terms arecorrelated?Discuss.
e) Dividethe52casesintotwogroups,placingthe26caseswiththesmallestfittedvalues𝑌!into group 1 and the other 26 cases into group 2. Conduct the Brown-Forsythe test forconstancyoftheerrorvariance,using𝛼 = 0.01.Statethedecisionruleandconclusion.
7.4,p.289
RefertoGroceryretailerProblem6.9.
a) Obtaintheanalysisofvariancetablethatdecomposestheregressionsumofsquares intoextrasumsofsquaresassociatedwith𝑋!;withX3,given𝑋!;andwith𝑋!,given𝑋!andX3.
b) Test whether 𝑋! can be dropped from the regression model given that 𝑋! and X3 areretained.Use the𝐹∗ test statisticand𝛼 = 0.05. State thealternatives,decision rule,andconclusion.WhatistheP-valueofthetest?
c) Does SSR(𝑋!)+ SSR(𝑋!|𝑋!) equal SSR(𝑋!)+ SSR(𝑋!|𝑋!) here? Must this always be thecase?
7.17,p.290
RefertoGroceryretailerProblem6.9.
a) Transform the variables by means of the correlation transformation (7.44) and fit thestandardizedregressionmodel(7.45).
b) Calculate the coefficients of determination between all pairs of predictor variables. Is itmeaningfulheretoconsiderthestandardizedregressioncoefficientstoreflecttheeffectofonepredictorvariablewhentheothersareheldconstant?
c) Transform the estimated standardized regression coefficients bymeans of (7.53) back totheones for the fitted regressionmodel in theoriginalvariables.Verify that theyare thesameastheonesobtainedinProblem6.10a.
8.16,p.337-338
Refer to Grade point average Problem 1.19. An assistant to the director of admissionconjecturedthatthepredictivepowerofthemodelcouldbeimprovedbyaddinginformationonwhetherthestudenthadchosenamajorfieldofconcentrationatthetimetheapplicationwassubmitted.Assumethatregressionmodel(8.33)isappropriate,where𝑋! isentrancetestscore and 𝑋! = 1 if student had indicated a major field of concentration at the time ofapplicationand0ifthemajorfieldwasundecided.DataforX2wereasfollows:
𝑖: 1 2 3 … 118 119 120
𝑋!!: 0 1 0 … 1 1 0
a) Explainhoweachregressioncoefficientinmodel(8.33)isinterpretedhere.
b) Fittheregressionmodelandstatetheestimatedregressionfunction.
c) Testwhether the𝑋! variable can be dropped from the regressionmodel; use𝛼 = 0.01.Statethealternatives,decisionrule,andconclusion.
d) Obtaintheresiduals for regressionmodel (8.33)andplot themagainst𝑋!𝑋!. Is thereanyevidenceinyourplotthatitwouldbehelpfultoincludeaninteractionterminthemodel?
a) Developafirst-orderlinearregressionmodelforrelatinglastyear’sprofitorloss(𝑌)tosizeofbank(𝑋!)andtypeofbank(𝑋!,𝑋!).
b) Statetheresponsefunctionsforthethreetypesofbanks.
c) Interpreteachofthefollowingquantities;
1) 𝛽!
2) 𝛽!
3) −𝛽! − 𝛽!
Homework5 DueNovember299.15,p.378-379
Kidney function. Creatinineclearance(𝑌) is an importantmeasureof kidney function,but isdifficult to obtain in a clinical office setting because it requires 24-hour urine collection. Todeterminewhetherthismeasurecanbepredictedfromsomedatathatareeasilyavailable,akidneyspecialistobtainedthedatathatfollowfor33malesubjects.Thepredictorvariablesareserumcreatinineconcentration(𝑋!),age(𝑋!),andweight(𝑋!).
a) Prepare separate dot plots for each of the three predictor variables. Are there anynoteworthyfeaturesintheseplots?Comment.
b) Obtainthescatterplotmatrix.Alsoobtainthecorrelationmatrixofthe𝑋variables.Whatdo the scatter plots suggest about the nature of the functional relationship between theresponsevariable𝑌andeachpredictorvariable?Discuss.Areanyseriousmulticollinearityproblemsevident?Explain.
c) Fit themultiple regression function containing the threepredictor variables as first-orderterms.Doesitappearthatallpredictorvariablesshouldberetained?
9.16,p.379
RefertoKidneyfunctionProblem9.15.
a) Usingfirst-orderandsecond-ordertermsforeachofthethreepredictorvariables(centeredaroundthemean)inthepoolofpotential𝑋variables(includingcrossproductsofthefirst-orderterms),findthethreebesthierarchicalsubsetregressionmodelsaccordingtothe𝐶!criterion.
b) Istheremuchdifferencein𝐶!forthethreebestsubsetmodels?
9.19,p.379
RefertoKidneyfunctionProblem9.15.
a) Using thesamepoolofpotential𝑋 variablesas inProblem9.16a, find thebest subsetofvariablesaccordingtoforwardstepwiseregressionwith𝛼limitsof0.10and0.15toaddordeleteavariable,respectively.
b) Howdoesthebestsubsetaccordingtoforwardstepwiseregressioncomparewiththebestsubsetaccordingtothe𝑅!,!! criterionobtainedinProblem9.16a?
10.10a,p415
RefertoGroceryretailerProblems6.9and6.10.
a) Obtainthestudentizeddeletedresidualsandidentifyanyoutlying𝑌observations.UsetheBonferronioutliertestprocedurewith𝛼 = 0.05.Statethedecisionruleandconclusion.
Homework6 DueDecember1110.10b-f,p415
RefertoGroceryretailerProblems6.9and6.10.
b) Obtainthediagonalelementsofthehatmatrix.Identifyanyoutlying𝑋observationsusingtheruleofthumbpresentedinthechapter.
c) Managementwishestopredictthetotallaborhoursrequiredtohandlethenextshipmentcontaining 𝑋! = 300,000 cases whose indirect costs of the total hours is 𝑋! = 7.2 and𝑋! = 0 (no holiday in week). Construct a scatter plot of 𝑋! against 𝑋! and determinevisually whether this prediction involves an extrapolation beyond the range of the data.Also,use (10.29) todeterminewhether anextrapolation is involved.Doyour conclusionsfromthetwomethodsagree?
d) Cases16,22,43,and48appeartobeoutlying𝑋observations,andcases10,32,38,and40appear to be outlying𝑌 observations. Obtain the DFFITS,DFBETAS, and Cook’s distancevaluesforeachofthesecasestoassesstheirinfluence.Whatdoyouconclude?
f) Calculate Cook’s distance 𝐷! for each case and prepare an index plot. Are any casesinfluentialaccordingtothismeasure?
11.29,p.479
RefertoMuscleMassProblem1.27.
a) Fitatwo-regionregressiontree.Whatisthefirstsplitpointbasedonage?WhatisSSEforthistwo-regiontree?
b) Find the second split point given the two-region tree in part (a). What is SSE for theresultingthree-regiontree?
c) Findthethirdsplitpointgiventhethree-regiontreeinpart(b).WhatisSSEfortheresultingfour-regiontree?
d) Prepareascatterplotofthedatawiththefour-regiontreeinpart(c)superimposed.Howwell does the tree fit the data?What does the tree suggest about the change inmusclemasswithage?
