Rankled by the RankingsAuthor(s): TERRY CARTERSource: ABA Journal, Vol. 84, No. 3 (MARCH 1998), pp. 46-50, 52Published by: American Bar AssociationStable URL: http://www.jstor.org/stable/27839879 .

Accessed: 14/06/2014 06:04

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

American Bar Association is collaborating with JSTOR to digitize, preserve and extend access to ABA Journal.

http://www.jstor.org

This content downloaded from 195.34.79.223 on Sat, 14 Jun 2014 06:04:57 AMAll use subject to JSTOR Terms and Conditions

COVER STORY / LAW SCHOOLS

Rankled by

the

Rankings BY TERRY CARTER

During his recently completed year as president of the Association of American Law Schools, John E. Sexton was on a passionate mission to tear

down the rankings of law schools published every year by U.S. News & World Report

It was on Sexton's New York University School of Law letterhead last April that 149 other deans from aals member schools joined him in complaining to the publisher of U.S. News about law school rankings.

And by Sexton's hand, the aals has asked a re searcher at the Rand Corp., a research institute and think-tank, to critique U.S. News9 methodology.

Sexton also was a mover and shaker behind an un precedented mailing last month from the Law School Admission Council to more than 70,000 law school ap plicants, criticizing rankings in general and U.S. News in particular.

But at the same time, Sexton has pursued another mission with equal or greater passion?one that many colleagues believe is contrary to his aals efforts.

While leading the charge against U.S. News, the NYU law school dean also sends his alumni magazine to every law professor at every one of the 180 ABA-ac credited law schools in the nation. The motivation seems obvious to Sexton's peers. They believe he is elbowing for a better place among the Top 10 in the U.S. News rankings, which give a lot of weight to the reputation survey conducted by the magazine.

There is the rub. Law school deans fear the rank ings, and they hate them. At the same time, they play to the rankings and use them to their advantage.

As U.S. News & World Report publishes its eighth annual ranking of U.S. law schools this month, the legal education community's long-simmering campaign against the popular moneymaker suddenly has taken on the force of a jihad. But at the same time, the deans

Terry Carter is a reporter for the ABA Journal.

46 ABA JOURNAL / MARCH 1998

- - - 4 - ---.--.---.-......

& --f

OR'&

-.- -r

This content downloaded from 195.34.79.223 on Sat, 14 Jun 2014 06:04:57 AMAll use subject to JSTOR Terms and Conditions

Many legal educators have fumed for years about comparing the

'quality' of law schools. But now, even as schools

jockey for recognition, an effort to undercut the

'rankings chase' is

picking up steam.

have never worked harder to land as high on the list as possible.

Scratch a dean and get an earful: Rankings ran kle, and rankings rule.

Sexton says that in the next decade the negative impact of what he believes is the flawed methodol ogy of the U.S. News rankings will bring about the demise of some law schools that otherwise would have remained viable. That means, he continues, that some law schools that probably should fail will not.

And that, he says, is tragic. But the story of Sex ton's own school, while significant because of NYU's prominence, might lend itself more to melodrama? many colleagues envy his problems. Because of his comfortable position in the rankings, he says, he can afford to take the lead in fighting U.S. News.

Just the same, Sexton's peers believe he is very sensitive to his school's U.S. News ranking?a steady No. 6?and is particularly interested in changing places with uptown rival Columbia University School of Law in the fiercely competitive New York market. Columbia peers down from just a single slot above NYU.

But in that elite tippy-top tier, one or two rungs can make a big difference in who gets the very best stu

dents, and, to a lesser but still important ex tent, whose students get the very best jobs? law firms tend to have their own ranking systems based on previous hires.

Not taking any chances, Columbia has followed Sexton's lead and for the first time blanketed the nation last month with its own alumni magazine.

"John Sexton is a fabulous dean and has done a lot both to improve NYU law school and make the world know about it," says Dean Paul Brest of Stanford Law School in California, discussing the increased promo

, I tional mailings from schools piling up on W??? his desk on their way to the trash can. "But ???? Tve always thought it a waste of money. It's W?? mainly our alumni who care what we do."

HI That faint praise might be expected, jflH considering that Stanford's perennial perch

~H in US. News is No. 3, behind only Yale and Harvard.

Brest, incidentally, declined to sign Sex ton's letter to the editor because it asked that, if the rankings must stay, they at least be limited to the Top 20. Brest believes that some lower-ranked schools may want the compar isons. And it is not for him, from above the

fray, he explains, to deny them that. Sexton maintains that he is, simply, "mar

keting conscious. Sending out the magazine has nothing to do with rankings. It's seizing the leadership position in legal education.

The rankings game shows signs of turning into an 'arms race.' ?Dean Ronald Cass

ABA JOURNAL / MARCH 1998 47

This content downloaded from 195.34.79.223 on Sat, 14 Jun 2014 06:04:57 AMAll use subject to JSTOR Terms and Conditions

It's got a very different purpose." He points to the University of Michigan

Law School in Ann Arbor, which was sending its alumni magazine to one and all long before U.S.

News began ranking schools. "And it was per ceived to be a leadership school," he says.

Sexton may have been sending mixed sig nals along with his magazine, and he's far from

being alone in doing so, but there are few lead ers in legal education who do not join him in

saying that rankings harm legal education. They complain, for example, that U.S. News9

emphasis on Law School Admissions Test scores and undergraduate grade-point averages has moved schools to greater concern about

them, and in turn hurt minority enrollment. "The U.S. News rankings have a clear im

pact on minority candidates because of the em

phasis on LSATs," says Rudolph C. Hasl, dean of St. John's University School of Law in New York City and immediate-past chair of the ABA Section of Legal Education and Admissions to the Bar. "It has a pernicious effect on morale in the schools."

Assessing the Fallout Rankings are changing the law schools. "The rankings are influencing admissions

decisions, and maybe some curricular deci

sions," says Philip D. Shelton, president and executive director of the Law School Admission Council in Newton, Pa. "And it's not all that clear whether those are positive influences and

improving educational programs, or rather that

they're just designed to impact rankings." Rankings bring pressure from chancellors,

regents, university presidents and alumni to move their schools up, or at least recover from slides. For some in admissions offices and ca reer services, the word has been, "Get the num bers up, whatever it takes?or else."

That pressure has helped fuel the 30 per cent turnover in recent years among law school career counselors and placement directors, says Pamela S. Malone, assistant dean for ca reer services at Vanderbilt University School of Law in Nashville and a past president of the National Association for Law Placement. "I am aware of a substantial number of colleagues who have left under the pressure or been forced

out," she says. Law school applications have declined consider

ably since US. News launched its annual rankings in 1990. The slide may have been coincidental, but it

made the impact of the rankings all the more powerful. Applications peaked at 99,300 in 1990-91 and have

dropped steadily to an estimated 72,300 in 1996-97. That has intensified competition in recent years for the best students.

At the same time, the number of first-year seats in law schools have remained roughly constant at about

42,000. When nearly 100,000 applicants sought to fill them eight years ago, all schools could stay above the

median in quality when deciding whom to accept. Now, with just 72,000 applicants after that same number of seats, some schools are forced to dip below the median.

And now the students are more choosy. "I wish I

.. . - - .

- - - -'-6

; -l

v 'A

had a nickel for every student who's come up to me and asked where we rank in U.S. News," says Vice Dean Laurence M. Rose at the University of Miami Law School in Coral Gables, Fla. "I could stop teaching and retire."

Some schools are quick to tout a jump in the rank

ings, including one that bragged about moving up a

tier, though that was due only to US. News9 decision to

change its groupings, says Dean Judith W. Wegner of the University of North Carolina School of Law in

Chapel Hill and a past aals president. Perhaps all the jockeying for position is under

standable, considering the impact of the US. News

rankings. The year after the magazine named Chicago Kent College of Law as first among five "up-and-com ers" among law schools in 1991, applications to the school increased 30 percent, despite the fact that ap

48 ABA JOURNAL / MARCH 1998 ABAJ/DAVID WEINTRAUB

This content downloaded from 195.34.79.223 on Sat, 14 Jun 2014 06:04:57 AMAll use subject to JSTOR Terms and Conditions

||wBBB plications nationwide had begun their long, pre

m???f Jfl cipitous slide. ^IBS'^-j?j?^B

^ews soon shopped the "up-and-com ^ |^ ers" listing, and has tinkered with other cate ^^Hw^H gories and methodologies over the years, i^^^^^^^fl Since 1994, the magazine has ranked the f^^^^^^H top 50 schools and ranks the others in three ad I^^HH^H ditional tiers of descending quality. Within each j^^HHSB tier, the schools are Usted alphabetically, though

numerical listings for the tiers could be deter

I^^HhBI mined by doing the math with the various sur ^^^ p vey scores detailed in the rankings book U.S. ^^^HHH?? News began publishing in 1994.

j^^^^^^l In 1990 and 1991, the magazine ranked only HHB^^

the top 15; the next two years it ranked the top Wfr ^^^J 25 plus four lower tiers. "*T||||I The U.S. News rankings give a weight of 25

^^[^^^ percent to the yearly academic reputation sur

]BP vey; 25 percent for selectivity, based on LSATs, * GPAs and the proportion of applicants accepted;

Ib^ 'j???r 20 percent for employment success, factoring in Mr jobs both at graduation and nine months out,

jjpF along with bar passage rates; 15 percent for a

|F reputation survey sent to lawyers and judges; and 15 percent for faculty resources, including expenditures per student, financial aid and re search materials.

Changes in emphasis and methodology over athe years have been a source of tremendous frustration for the deans, says Dean Hasl of St. John's, who serves on Sexton's AALS steering committee coordinating the attack on U.S.

News. "You don't know what new factors might be in the next rankings, and if you adjust prior ities in terms of budget or something like that, you can't be sure the rules won't be different."

Many critics believe the reputation surveys should be dropped from any ranking formula be cause too many of those queried offer opinions on schools about which they know nothing. But even some who gripe about it admit privately that they do it themselves.

This aspect of the rankings is seen as self propelling. Reputations build slowly and die even more slowly, but students still flock to a school with a good reputation it no longer de serves. To highlight this arbitrariness, one critic reportedly sent out similar surveys that listed the Massa chusetts Institute

of Technology and Perm State University, which both did well despite the fact that they do not have law schools.

The reputation sur veys are the most sub jective aspect of the U.S.

News rankings, and the most criticized. They are heavily weighted, with the combined academic survey and the survey of lawyers and judges counting 40 percent in the overall rankings. The surveys go to four people at each law school?the dean, academic dean, chair of the faculty hiring committee, and the most recently tenured professor. They also go to 1,310 non academics?senior judges and lawyers involved in hiring recent law school graduates.

"And that produces NYtTs [alumni magazine] mailing," says Shelton of the Law School Admission Council. "There are hundreds of mailings in the law school community. They know people are going to get these surveys."

It has become, says Dean Ronald A. Cass of Bos ton University School of Law, "like an arms race. If you can't get an enforceable agreement to stop, then what's in your interest? It's also the classic 'prisoners dilemma' game theory where if one talks and the other does not, the one who talks gets a reduced sentence."

An Error Brings on the Heat Many of the prisoners are making a lot more noise

now. Suddenly in the past year, U.S. News9 critics have become more like pit bulls in stridency and effort, which escalated last March, when they tasted a bit of the magazine's blood.

That was when US. News accidentally transposed some figures in its methodology, skewing 33 of the top 50 rankings. The magazine had to pull its paperbacks from the newsstands, reportedly at a cost of more than $500,000. (The book, which comes out at the same time as the magazine issue containing the rankings, offers much more detail and analysis. The book also ranks graduate schools, as well as business, medical, ost?o pathie and engineering schools.)

Within a few weeks of that snafu, Sexton and his fellow deans sent their letter to U.S. News editor James Fallows, asking him to stop ranking law schools. The letter included detailed criticism of the

methodology along with a pointed reference to the re call for corrections.

"We hope to convince you to abandon an enter prise which we believe you should regard as an embar rassment to your magazine?especially in light of re cent events," the deans wrote.

Fallows responded in a letter saying he intends to continue the rankings but that they had raised "some points worthy of discussion." Though his letter was brief, Fallows responded directly to one criticism: the complaint that the reputation surveys are unreliable because respondents can't possibly know enough about all the schools.

"We do not expect educators to be able to rank all schools in their fields; we expect most respondents to evaluate only schools with which they are familiar," Fallows wrote. "Each of the law schools in our survey

amass at least 400 responses."

'Perhaps we've been too honest' in reporting our numbers.

?Dean Mary Kay Kane

Fallows set in motion a series of meetings between his editors and Sexton's group that neither side feels has accomplished much.

Sexton terms the deans' letter the "mini malist agenda" because it asks that the mag azine at least limit the rankings to 20 schools.

Last month's mass mailing from the LSAC to all law school applicants is the so-called "maximalist agenda," questioning any and all

validity of the U.S. News rankings. It was put togeth er by Peter M. Shane, dean at the University of Pitts burgh School of Law, and another member of Sexton's AALS steering committee on rankings.

The next move likely will come soon, when Rand completes its critique of the magazine's methodology. "Their only instruction was to take a hard look," says Carl C. Monk, AALS executive director. "Well wait to get the report and see what we do."

ABA JOURNAL / MARCH 1998 49

This content downloaded from 195.34.79.223 on Sat, 14 Jun 2014 06:04:57 AMAll use subject to JSTOR Terms and Conditions

U.S. News editors say the magazine has no inten tion of killing off its popular rankings. "We think it's the best current snapshot of where things stand in

legal education," says national news editor Ted Gest. "We're serving consumers, the law school applicants."

Cooking the Books to Get Ahead So the reality is that law school rankings are not

likely to go away. And while law school leaders eschew them, many also embrace and kowtow to the rankings, which leaves them sometimes dripping crocodile tears over sour grapes, and worse.

There is, for instance, the little-noted scandal in which a number of law schools gave inflated figures to U.S. News that differed from those they submitted to the ABA's accrediting offices. Calculatedly or not, num bers submitted to the magazine helped the schools in the rankings.

At the 1995 ABA Midyear Meeting, recalls Arthur R. Gaudio, deputy consultant on legal education for the association, the gathering of law deans heard an an nouncement that 12-15 schools had played games be tween the ABA and U.S. News with differing data.

"They were squirming and looking at each other, won

dering, 'Are you one of them? Are you?' " Gaudio says.

The ABA information had been confidential, given only to schools accredited by the association. But some

disgruntled deans slipped the data to U.S. News to prove that others were cheating.

Many expect such discrepancies to decline con

siderably since the ABA in 1997 began publishing Ap proved Law Schools Statistical Information, which con tains definitive statistics on much information in the

objective categories of the U.S. News rankings. "There are rumors that some schools cheated in

the early days to get higher rankings and don't have to cheat anymore, now that they get better students be cause of the higher ranking," says Richard O. Lempert, a critic of rankings who teaches at the University of

Michigan Law School and chairs the university's soci

ology department. And that leaves some others bemused. "Perhaps

we've been too honest," says Dean Mary Kay Kane of the University of California's Hastings College of the Law in San Francisco. U.S. News ranked Hastings a

very respectable No. 20 among all law schools in 1994, then 45th in 1995. Last year, the school plummeted fur

ther, out of the Top 50. Some of Hastings' decline was attributed to the

sagging San Francisco legal market, which the big public law school feeds. The school also is said to be a victim of shifting methodologies as U.S. News con tinues to tinker with its formula. But the school al so had itself to blame.

Growing Ranks John Wehrli is one of those driven people who don't

sleep much. So while he is doing graduate work in molecular and cellular biology, he also is in ajoint JD/MBA study program and works at a law firm in San Francisco.

Wehrli also has put up a popular, detailed World Wide Web site that ranks law schools. Not only that, but he analyzes and critiques other rankings on the Web site, and averages their results.

"I've been amazed at the number of hits on the Web," says Wehrli, who also ranks business schools and doctoral programs. "I was getting over 700 e-mails a week and had to

stop trying to answer them all." Wehrli wants his ranking system and survey of

rankings to encourage interested students and others "to look behind the rankings and analyze them for themselves."

Commercial content providers have asked him to sign on with them, Wehrli says, but he wants to stay away from advertising and maintain a bias-free neutrality.

Wehrlfs Web site at www.wehrli.ilrg.com/ has become well-known to leaders in legal education, but it also is just another example of the recent spate of rankings, rankings and more rankings. In that field, U.S. News & World Report is far from alone, l?ese are some of the better-known law school rankings:

The Gourman Report is an academic ranking rich in statistics that is published every two years by National Education Standards. It is named for Jack Gourman, who has been preparing the report since 1980. A listing of its law school rankings is available with analysis from John Wehrli

on the Net at http://wehrli.ilrg.eom/jdgourman.html.

The Legal Gourmet Report., 1997-98, Ranking of Law Schools by Educational Quality, by Brian Ixiter at the University of Texas School of Law, is published on the university's Web site at www.dla.utexas.edu/depts/ philosophy/faculty/leiter/LGO.

The Princeton Review offers rankings in categories such as teaching quality (based on student surveys), student/faculty ratios, quality of life, admissions and demand, and women student and faculty numbers. Contact Princeton University at www.princetonreview.com/legal/rank.

Judging the Law Schools was published in 1996 by Thomas E. Brennan, a former Michigan Supreme Court justice who is the founder and president of the Thomas M. Cooley Law School in Lansing, Mich. Relying heavily on data in the 1994 Review of Legal Education published by the ABA Section of Legal Education and Admissions to the Bar, Brennan says he "does the math" that few other people bother with. Brennan offers seven different Top 20 rankings, using different indexes, such as quality, institutional, faculty and competition, it is available for $25 at the Cooley Law School bookstore, (517) 371-5140, ext. 308, and it is also in libraries at A A-approved law schools.

Approved Law Schools, published in 1997 by the American Bar Assocation, does not rank the schools but gives definitive statistics on most of the categories included in the U.S. News rankings. It is available from the ABA Service Center, 1-800-285-2221, for $19.95, or at major bookstores. ?Terry Carter

50 ABA JOURNAL / MARCH 1998

This content downloaded from 195.34.79.223 on Sat, 14 Jun 2014 06:04:57 AMAll use subject to JSTOR Terms and Conditions

; Ifi fl ^j^^ ^^^h^h

^^^^^ ̂

On the heels of its rankings' slide, Hastings launched new promotional materials touting its strengths and uniqueness, and recently started an in tensified job placement program with one-on-one coun

seling. The school also hired someone to work strict ly on placement in the summer and fall, cold-calling small and mid-sized law firms to open doors for its graduates.

"That's a good re sult from all this con

troversy," says Dean

Kane. "It's forced us to do some things perhaps faster than we other wise would have."

Rankings get a lot of attention for a number of rea sons, and it appears there is no stopping them.

At the 1997 aals meeting, University of Virginia law professor Alex M. Johnson Jr. suggested a boycott of U.S. News, asking schools not to cooperate in surveys or in providing data. His comments drew applause? and nothing else.

Yet whatever the Rand Corp. finds in its study of US. News' methodology, and whatever Sexton's steer

ing committee on rankings subsequently recommends, the genie is out of the bottle. Perhaps the most signifi cant response to the controversy over U.S. News rank ings is a cottage industry of other rankings.

Some critics have created their own listings, vary ing criteria and the weights given to them. These rankings tend to bump their own schools upward. One who did so, Brian Leiter at the University of Texas Law School in Austin, says the aals is wrong to oppose

rankings, though Leiter disagrees with U.S. News' methodology.

"The reality is that students are interested, and for reasons that are not entirely silly," says Leiter. "It's reasonable for them to be concerned with prestige and reputation, and rankings speak to that."

Leiter's rankings emphasize faculty quality,

Visit 'The Ranking Game' Web site and create your own listing.

?Professor Jeffrey Stake

primarily through scholarly writings. He elimi nates factors in U.S. News rankings that he be lieves go against large public schools like his own, such as financial aid. While scholarships are not as plentiful at state schools, tuition usu

ally is much lower. His methodology brings 11 new schools into the top 25, and his own moves from 18th up to 9th.

Another critic may have come up with the best way to get around the inherent subjectivity in

rankings. Professor Jeffrey E. Stake at the Indiana Uni versity School of Law-Bloomington recently launched a

World Wide Web site, called "The Ranking Game" (http: //monoborg.law.inmana.edu/LawRarik/ranLkgame.html), at which visitors can shop ? la carte and create their own rankings according to their own wants and needs.

They can choose among a number of criteria simi lar to those in US. News, as well as some others, and apply their own subjective weights and emphases. Stake's Web site then ranks the law schools for them.

A mention of this do-it-yourself system catches Sexton's attention as he expounds on the destructive effects of U.S. News rankings. The facts are there, and all the subjectivity comes from the person trying to de cide which school to attend.

"That's very close to the point I'm making," he says, then returns to his complaints about U.S. News.

52 ABA JOURNAL / MARCH 1998 abaj/dave repp

This content downloaded from 195.34.79.223 on Sat, 14 Jun 2014 06:04:57 AMAll use subject to JSTOR Terms and Conditions