+ All Categories
Home > Documents > Statistical Models - UniTrento

Statistical Models - UniTrento

Date post: 19-Mar-2022
Category:
Upload: others
View: 6 times
Download: 0 times
Share this document with a friend
12
Statistical Models University of Trento - FBK 16 February, 2015 1 / 12
Transcript
Page 1: Statistical Models - UniTrento

Statistical Models

University of Trento - FBK

16 February, 2015

1 / 12

Page 2: Statistical Models - UniTrento

Information about the course

Website: http://science.unitn.it/~pugliese/

Timetable: Monday 13.00 - 15.00Thursday 13.00 - 16.00

e-mail: [email protected], [email protected]

Office Hours:Pugliese Monday 9-10.30

Filosi Monday 17-18.30 (sending and e-mail before would be better)

Exam:Written exam: March, 27thScript Submission: March, 30thOral Presentation: April, 1st to 8th

Reference book:Julian J. Faraway, Practical Regression and Anova using Rhttp://cran.r-project.org/doc/contrib/Faraway-PRA.pdf

2 / 12

Page 3: Statistical Models - UniTrento

Course Subjects

Intro to statistical models (general ideas)

Intro to R a statistic environment

Intro to linear models

C.I., tests, linear models, residuals and PRESS

Regression in R

ANOVA and polynomial regression

Generalize Linear Models (GLM)

Model choice

3 / 12

Page 4: Statistical Models - UniTrento

What is Statistical Learning?Examples

Med Science: Predict whether a patient, hospitalized due to a heart attack, will have a secondheart attack. The prediction is to be based on demo- graphic, diet and clinicalmeasurements for that patient.

Economics: Predict the price of a stock in 6 months from now, on the basis of companyperformance measures and economic data.

Automation Identify the numbers in a handwritten ZIP code, from a digitized image.

Med Science: Estimate the amount of glucose in the blood of a diabetic person, from theinfrared absorption spectrum of that person’s blood.

Demography: Identify the risk factors for prostate cancer, based on clinical and demographicvariables.

Common features?Keywords: Data Mining, Statistics, Artificial Intelligence

4 / 12

Page 5: Statistical Models - UniTrento

Learning from data

A typical scenario:

Given an outcome measurement, quantitative (such as stock price) orcategorical (such as heart attack/no heart attack), that we wish to predictbased on a set of features (such as diet and clinical measurements).

We have a training set of data, in which we observe the outcome and featuremeasurements for a set of objects (such as people).

5 / 12

Page 6: Statistical Models - UniTrento

Learning from dataDifferent Approaches

SupervisedPresence of the outcome variable

Examples:

Regression

Machine Learning

Decision Tree

Kernel Estimators

UnSupervisedAbsence of the outcome variable

Examples:

Clustering

Hidden Markov Models

Pricipal Component Analysis,Dimesionality Reduction

Neural Network

Describe how the data are organized and clustered in case of unsupervised learning.Build a statistical model for predicting or estimating an output based on one or more inputs.

6 / 12

Page 7: Statistical Models - UniTrento

A brief of history of Statistical Learning

1805 - 1809

Lege

ndre

and Gau

ss: pu

blish

edthe

first p

aper

onlea

stsq

uare

met

hods.

1936

- 1940

Fisher : invented the linear

discriminant analysis method. A

newmethod called logistic

regressionhas been proposed.

1980 - 1986

Breim

an, F

riedm

an, O

lshen

and

Stone:

Classifi

catio

nan

d

Regre

ssio

nTr

ees

Hastie

, Tibs

hiran

i :Gen

erali

ze

Additive

Models

7 / 12

Page 8: Statistical Models - UniTrento

Examples of Statistical ModelsGraphically: Income vs Years of Education

● ●

●●

●●

● ●

10 12 14 16 18 20 22

2030

4050

6070

80

Education

Inco

me

● ●

●●

●●

● ●

10 12 14 16 18 20 22

2030

4050

6070

80

Education

Inco

me

8 / 12

Page 9: Statistical Models - UniTrento

General Definition of Statistical Models

Input or independent Variables X: In the example the Years of Education

Output or dependent Variables Y: In the example the Income

Definition:

Given X = (X1,X2, . . . ,Xp), we assume there is a relationship between Y and X such as we canwrite:

Y = f(X) + ε

where ε is a random error term andf represent the information that X provide about Y .

9 / 12

Page 10: Statistical Models - UniTrento

Why to estimate f?Prediction

Prediction Settings:

X is known

Goal Y cannot be obtained (easily)

Model:

Y = f (X ) for n → ∞, ε → 0

where f is an estimate of f and Y represents the resulting prediction of Y . In this case we are notinterested in the exact form of f , as soon as the prediction Y of Y are accurate.

10 / 12

Page 11: Statistical Models - UniTrento

Prediction Errors

Given the model Y = f (X ) which is an estimate generated fromY = f (X ) + ε:

Error and Variance

Define the expected value of the square difference between thepredicted and the true values as:

E(Y − Y )2 = [f (X )− f (X )]2︸ ︷︷ ︸Reducible error

+ Var (ε)︸ ︷︷ ︸Irreducible error

11 / 12

Page 12: Statistical Models - UniTrento

Why to estimate f?Inference

Inference Settings:

X is known

Goal Understand relationship between X and Y → exact form of f

Typical Questions:

Which predictors are associated with the response?

What is the relationship of each predictor with the response?

Can we use a linear equation to capture the relation between Yand X?

12 / 12


Recommended