College Rankings and What You Need to Know
Smith and US News Fall 2014
US News & World Report America’s Best Colleges:
History and Methodology
US News Rankings: History • Began in 1983 as a reputation survey; expanded to further data
collection • Institutions ranked based on Carnegie Classifications; Smith is ranked
among National Liberal Arts Colleges • Methodology changes since 2007
– Reduction in weight of selectivity, increase in grad rates – Addition of Pell Grants into graduation rate model – Addition of high school counselor reputation ratings – Increase in weight of graduation rate performance – Decrease in weight of reputation survey
• In recent years, multiple institutions have admitted publicly to providing false data to US News.
US News: Where Smith Ranks and Why
19 20 18 19
14
18 18 17 19 19
13 15
13 14 12 13
11
14 12
10
13
9 10 10 9 7 7
14
0
5
10
15
20
25
30
35
40
45
50
2015201420132012201120102009200820072006200520042003200220012000199919981997199619951994199319921991199019891988
Smith US News Ranking
US News: Top Liberal Arts Colleges 2015 (Published September 2014)
• Williams (1) • Amherst (2) • Swarthmore (3) • Wellesley (4) • Bowdoin (5) • Pomona (5) • Middlebury (7) • Carleton (8) • Claremont McKenna (8) • Haverford (8) • Davidson (11) • Vassar (11) • USNA “Annapolis” (13)
• Washington and Lee (14) • Colby (15) • Hamilton (15) • Harvey Mudd (15) • Wesleyan (15) • Bates (19) • Grinnell (19) • Smith (19)
Also: • Bryn Mawr (27) • Barnard (32) • Mount Holyoke (41)
4
30.0%
22.5%
20.0%
12.5%
10.0%
5.0%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
Graduation andRetention
Reputation Score Faculty Resources Selectivity FinancialResources
Alumni Giving
Weight in Ranking
US News: What Matters? • Reputation Survey
– Peer ratings and guidance counselor ratings. • Graduation and Retention
– The percentage of first-year students who graduate in 6 years or less. – The percentage of first-year students who are still enrolled the following fall.
• Financial Resources – Educational Expenditures per Student: Funds spent on research, public service, instruction, academic
support, student services, institutional support, and operations and maintenance, adjusted for the percent of students that are undergraduate.
• Faculty resources: – Faculty Salaries: Salary plus fringe benefits for all faculty, adjusted for cost of living in area. (Another
spending measure). – Faculty composition: The percentage that are full-time, with top terminal degree. – Class size: Percent of fall classes under 20 students and 50+.
• Selectivity – Admit rate, SAT scores, class rank of entering students.
• Alumnae Giving – Percent of alumnae of record who donated in any amount in a given year.
US News: What Matters?
“Spend money.
As much as possible. Mostly on faculty.”
Alumni Giving (36)
Graduation and Retention (35)
Selectivity (31)
Financial Resources (19)
Faculty Resources (29)
Reputation (10)
-25
-20
-15
-10
-5
0
5
10
15
Sub-rank Relative to Overall Rank 2015
What Goes Into the Rankings
US News Rankings and Sub-Rankings
Smith Overall , 19
Peer Assessment, 10
Grad/Retention, 35
Faculty Resources, 29 Selectivity, 31
Financial Resources, 19
Alumni Giving, 36
0
10
20
30
40
50
60
70
80
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Reputation 22.5%
US News Rankings
Smith Overall , 19
Peer Assessment, 10
0
10
20
30
40
50
60
70
80
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
US News Reputation Survey
*Peer assessment score changed in 2011 to 100-point scale; data for 11-13 are converted to 5-point scale. HS Counselor scores have only been released since 2014.
4.3 4.3 4.3 4.2 4.2 4.4 4.2 4.3 4.2 4.2
4.4 4.5
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
2006 2007 2008 2009 2010 2011* 2012* 2013* 2014 2015
Peer Assessment HS Counselor Assessment
Graduation and Retention 30%
US News Rankings
Smith Overall , 19
Peer Assessment, 10
Grad/Retention, 35
0
10
20
30
40
50
60
70
80
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Six-Year Graduation Rates (Four-year Averages)
84% 84%
86% 86% 87%
86% 85% 85% 85% 85%
75%
80%
85%
90%
95%
100%
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Note: Scale 75-100%
First-Year Retention ( Four-year Average)
91% 91% 91% 90% 90% 91% 91%
92% 92% 93%
75%
80%
85%
90%
95%
100%
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Note: Scale 75-100%
Predicted vs. Actual Graduation Rate
85%
83%
81% 82%
83% 82% 82% 82%
81% 82% 84%
86% 85%
86%
88%
84% 85% 85% 85%
86%
75%
80%
85%
90%
95%
100%
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Predicted Graduation Rate Actual Graduation Rate
Note: Scale 75-100%
Faculty Resources 20%
US News Rankings
Smith Overall , 19
Peer Assessment, 10
Grad/Retention, 35
Faculty Resources, 29
0
10
20
30
40
50
60
70
80
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Classes With <20 Students
68%
71% 70%
68% 69%
67% 68%
66%
68% 68%
50%
55%
60%
65%
70%
75%
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Note: Scale 50-75%
Additional Faculty Resource Inputs
• Faculty salaries, including benefits (up 2.6%) • Percent of faculty with terminal degree (99%) • Classes with 50+ students (5%) • Student/faculty ratio 9:1 • Percent of faculty that are full-time (97%)
Selectivity 12%
US News Rankings
Smith Overall , 19
Peer Assessment, 10
Grad/Retention, 35
Faculty Resources, 29 Selectivity, 31
0
10
20
30
40
50
60
70
80
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Average SAT Scores
1270 1263 1250 1270 1274
1300 1306 1309 1309 1333
1000
1100
1200
1300
1400
1500
1600
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Admit Rate
57%
48%
53% 52%
48% 47% 47%
45% 42% 43%
25%
30%
35%
40%
45%
50%
55%
60%
65%
70%
75%
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Note: Scale 25-75%
Entering Students in Top 10%
59% 61% 61%
63% 64% 66%
60% 61%
68%
62%
25%
30%
35%
40%
45%
50%
55%
60%
65%
70%
75%
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Financial Resources 10%
US News Rankings
Smith Overall , 19
Peer Assessment, 10
Grad/Retention, 35
Faculty Resources, 29 Selectivity, 31
Financial Resources, 19
0
10
20
30
40
50
60
70
80
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Educational Expenditures per Student
$42,758 $42,284 $42,710 $44,132 $47,378
$50,008 $50,719 $51,422 $51,922 $55,300
$-
$10,000
$20,000
$30,000
$40,000
$50,000
$60,000
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Alumnae Giving 5%
US News Rankings
Smith Overall , 19
Peer Assessment, 10
Grad/Retention, 35
Faculty Resources, 29 Selectivity, 31
Financial Resources, 19
Alumni Giving, 36
0
10
20
30
40
50
60
70
80
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Alumnae Giving Rate (Two-Year Avg)
41%
35% 36% 38%
40% 40% 38%
36% 35% 35%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Note: Scale 0-50%
The Rankings in Context
Smith vs. Sisters 2015
Smith
Wellesley
Bryn Mawr
Mount Holyoke
Overall Rank 19 4 27 41 Overall Score 84 93 80 73 Reputation Score 4.2 4.4 4.0 4.0
Graduation and Retention Rank 35 12 50 50 Average Freshman Retention Rate 93% 96% 91% 91% Predicted Graduation Rate 82% 91% 85% 83% Actual Average Graduation Rate 86% 91% 84% 82% Over-performance/ Under-performance 4 0 -1 -1
Faculty Resources Rank 29 9 25 76 % of Classes with fewer than 20 students 68% 73% 75% 66% % of Classes with 50 or more Students 5% 0.4% 3% 4% Student/Faculty Ratio 9/1 7/1 8/1 10/1 Selectivity Rank 31 8 28 60 Standardized Test Score 1220-1450 1310-1510 1210-1470 1220-1430 First-years in Top 10% of HS Class 62% 83% 65% 57% Acceptance Rate 43% 29% 40% 47% Financial Resources Rank 19 8 19 38 Alumni Giving Rank 36 11 25 48 Average Alumni Giving Rate 35% 47% 38% 31%
Smith and the Rankings
• There is widespread frustration with US News, but it can’t be ignored.
• Top weighted inputs: – Graduation/retention rates – Reputation scores – Faculty resources
• Smith is strong in reputation and financial resources. • Smith is weaker on alumnae giving and graduation
rates.
Reality of Rankings
• Peers are tightly clustered; many multi-way ties. – In 2015, ties for 5th, 8th (3-way tie), 11th, 15th (4-way tie),
19th, 24th, and 27th (all 3-way ties). – In 2014, ties for 4th (3-way tie), 7th, 9th (3-way tie), 14th,
17th (3-way tie), 20th, 22nd, 25th (5-way tie). • Measures are highly constrained, largely by economic
factors. • Movement in rankings has been accomplished by a handful
of lower-ranked schools, but requires focused, costly efforts; top 10 is highly stable.
Efforts to Affect Rankings
• Most movement within 1-2 slots up or down is noise.
• Significant effort is required for small moves. • Movement gets harder nearer the top. • Any of these efforts entail significant financial
expenditures.
Talking about Rankings • Rankings largely measure inputs, not what happens in the
classroom or the graduates we produce. • US News strongly rewards institutional wealth and expenditures. • Small changes can have what appear to be large effects on an
individual school’s ranking. • All the top schools work on the same measures to stay on top. • Managing rankings metrics to promote improvement has been
demonstrated to be expensive and sometimes unethical (Emory, Claremont McKenna, George Washington, Bucknell, Tulane).
• Rankings don’t measure success in achieving the mission.
27%
47%
53%
56%
85%
0% 20% 40% 60% 80% 100%
Rankings in national magazines
I was offered finanical assistance
This college's graduates get good jobs
Graduates admitted to top grad/prof schools
Has a very good academic repuation
Percent of Entering Students Marking Reason for Choosing Smith as Very Important: 2013