23
SMOOTHING AGE PROFILES: WEIGHTING ISSUE Austrian case Jože Sambt University of Ljubljana, Faculty of Economics, Slovenia Berkeley, California January 23, 2007
SMOOTHING AGE PROFILES: WEIGHTING ISSUE
Austrian case
Jože Sambt University of Ljubljana, Faculty of Economics, Slovenia
Berkeley, CaliforniaJanuary 23, 2007
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
+
Age
Eu
ros
Age profile of “other private consumption”
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
+
Age
Eu
ros
Without expandcl, 0.1
Without expandcl, 0.2
Without expandcl, 0.3
Nonsmoothed, weighted
Age profile of “other private consumption”
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
+
Age
Eu
ros
Without expandcl, 0.1
Without expandcl, 0.2
Without expandcl, 0.3
With xpandcl, 0.1
Nonsmoothed, weighted
Age profile of “other private consumption”
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
+
Age
Eu
ros
Without expandcl, 0.1
Without expandcl, 0.2
Without expandcl, 0.3
With xpandcl, 0.1
With expandcl, 0.2
Nonsmoothed, weighted
Age profile of “other private consumption”
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
+
Age
Eu
ros
Without expandcl, 0.1
Without expandcl, 0.2
Without expandcl, 0.3
With xpandcl, 0.1
With expandcl, 0.2
With expandcl, 0.3
Nonsmoothed, weighted
Age profile of “other private consumption”
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
2200015 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Age
Eu
ros
Nonsmoothed, weighted
Age profile of gross wage and salary earnings
Age profile of gross wage and salary earnings
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
2200015 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Age
Eu
ros
Without expandcl, 0.1
Without expandcl, 0.2
Nonsmoothed, weighted
Age profile of gross wage and salary earnings
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
2200015 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Age
Eu
ros
With expandcl, 0.1
With expandcl, 0.2
Without expandcl, 0.1
Without expandcl, 0.2
Nonsmoothed, weighted
Age profile of private health expenditures
0
100
200
300
400
500
600
700
800
900
10000 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
0
100
200
300
400
500
600
700
800
900
10000 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Unsmoothed, weighted
Without expandcl, 0.1
Without expandcl, 0.2
Without expandcl, 0.3
Age profile of private health expenditures
0
100
200
300
400
500
600
700
800
900
10000 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Unsmoothed, weighted
With expandcl, 0.1
With expandcl, 0.2
With expandcl, 0.3
Without expandcl, 0.1
Without expandcl, 0.2
Without expandcl, 0.3
Age profile of private health expenditures
Loss of accuracy in original data because of preparing them (with expandcl) for weighted
lowess smoothing
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Age
Eu
ros
Original - weighted
Unweighted
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Age
Eu
ros
Original - weighted
Unweighted
Average weight=1
Loss of accuracy in original data because of preparing them (with expandcl) for weighted
lowess smoothing
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Age
Eu
ros
Original - weightedUnweightedAverage weight=1Average weight=2Average weight=3Average weight=4Average weight=5Average weight=6Average weight=7Average weight=8Average weight=9Average weight=10
Loss of accuracy in original data because of preparing them (with expandcl) for weighted
lowess smoothing
Lost information and increased number of observations at different average weights
Average absolute
difference (%)
Number of observations
Unweighted 6.18 20,028
Average weight=1 2.24 26,593
Average weight=2 0.77 42,751
Average weight=3 0.35 61,313
Average weight=4 0.24 80,583
Average weight=5 0.18 100,318
Average weight=6 0.15 120,055
Average weight=7 0.12 140,057
Average weight=8 0.12 160,414
Average weight=9 0.11 180,224
Average weight=10 0.08 200,338
Sensitivity of final results to different mutliplier values; lowess factor 0.1
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Unsmoothed
Lowess without expandcl
Sensitivity of final results to different mutliplier values; lowess factor 0.1
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Unsmoothed
Lowess without expandcl
Average weight: 1
Sensitivity of final results to different mutliplier values; lowess factor 0.1
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Unsmoothed
Lowess without expandcl
Average weight: 1
Average weight: 2
Sensitivity of final results to different mutliplier values; lowess factor 0.1
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90
Unsmoothed
Lowess without expandcl
Average weight: 1
Average weight: 2
Average weight: 3
Average weight: 10
Sensitivity of final results to different mutliplier values; lowess factor 0.1
Friedman's SuperSmoother method (Supsmu)
0
2000
4000
6000
8000
10000
12000
14000
16000
0 5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
+
Age
EU
R p
er
ca
pit
a
Unsmoothed profile
Lowess (expandcl) 0.1
Lowess (expandcl) 0.2
Lowess (expandcl) 0.3
R - Supsmu
R - Supsmu (base=5)
Conclusions
1. Using STATA lowess function without using expandcl (i.e. ignoring weights at smoothing) produces profiles which can be heavily biased. Ignoring weights is not acceptable for the Austrian case.
2. During the workshop some countries presented twin peak (consumption) profile. If they used STATA lowess smoothing without expandcl function, it would be desirable to check if in some of them it is maybe just a smoothing problem.
3. Proper implementation of expandcl STATA function before using STATA lowess smoothing method seems to be adequate general and robust approach with acceptable calculation time.
