19

Fourth Northumbria

IntroductionThis paper examines aspects of strategic planning andmanagement and explores ways in which evaluationcan be integrated with the wider context.

The terms policy and strategy are closely related,and in some cases are used interchangeably. I followAllen & Wilson (1997) and distinguish between them.

Policies provide the principles for courses ofaction.

Strategies define the means of implementing cours-es of action.

A more formal definition of strategy can be found inthe literature of management.

Strategy is the direction and scope of an organisa-tion over the long term: which achieves advantagefor the organisation through its configuration ofresources within a changing environment, to meetthe needs of markets and to fulfil stakeholderexpectations. (Johnson & Scholes, 1999, p.10).

This definition sounds as if it were designed forcommercial organisations, but the same basic conceptsapply in the kind of public sector and not-for-profitorganisations for which many librarians work.We allhave stakeholders, and like it or not, libraries and infor-mation services of all kinds are competing with otherunits and organisations to meet the needs of markets,which can be defined as potential users.We can readilysee that this definition of strategy adapts to ‘organisa-tions’ such as local and national governments.

Note the words highlighted in italics. It is some-thing of a paradox that strategy is generally defined as“long term” when the political reality is all about theshort term.

Strategy operates at several levels, and we can distin-guish at least three of these.

Organisational strategy is about the overall pur-pose and scope of the organisation.

Business strategy is about competing successfullyin a specific market. A business unit is part of theorganisation for which there is a distinct external mar-ket for goods or services. In the public sector, theequivalent of a business unit might be a part of theorganisation or service for which there is a distinctclient group.

Operational strategy is all about the deploymentof people and other resources to deliver the higher-level strategies.

Most librarians operate at the level of the operationalstrategy, and so it is not surprising that most of the liter-ature of performance measurement is at this level. Allthe manuals and standards are directed at defining andusing evaluation at the operational level. There areexamples of ways of taking a more holistic view. In1995 we read:

‘Performance indicators are designed to give a fullyrounded picture.Therefore any judgement [of alibrary] must take indicators from each aspect ofperformance [since] these aspects are interrelated.’(Joint Funding Councils, 1995)

More recently the balanced scorecard technique hasbecome recognised as a more universal statement ofthe same concept and has had some application inlibraries (Pienaar and Penzhorn, 2000). This paper isnot a review of methods, but the added value approachalso has some potential for evaluating achievement ofstrategies. (Karunaratne, 1978; Ford, 1989; Hudson,2000; MacEachern, 2001).

As has been pointed out (Willemse, 1995; Cullen1998), operational indicators are usually defined bylibrarians, and their acceptance by funding bodies athigher levels has been difficult to achieve. I believethat the reason for this is that we have not been verygood at relating library activities to the higher-levelstrategies - business or organisational. I want to exploreways in which we might do that.

What then are the strategic uses of evaluation andperformance measurement? It is clear that they can beused in two ways: firstly to assess the achievement ofstrategy and secondly to influence the development ofstrategy.

Assessing strategic outcomesFirst, then, some examples of strategy to illustrate howevaluation and performance measurement is, or mightbe, used to assess the achievement of strategies.Thesecome from a number of levels. In decreasing order ofgeneralisation, they are• Supranational• National – government• Library sectors• National

Strategic Uses of Evaluation andPerformance Measurement

Geoffrey FordDirector of Information Services and University Librarian, University of Bristol, UK

• Public • Educational • SpecialistI will take them in reverse order.

Specialist librariesSpecialist libraries, for example those serving a busi-ness, or a health care facility, are usually well integratedwith their parent organisations, and a lot of work hasbeen done on the value of the library’s contribution.There is a useful compilation of review papers thatgive some pointers (Feeney and Grieves, 1994).

Educational librariesAn excellent example of a well thought out perform-ance measurement structure integrated with librarygoals and mission was described at the firstNorthumbria Conference (Willemse, 1995). However,that paper did not explicitly describe the linksbetween the library's activity and the mission of theuniversity it served.

Typically a university librarian is presented with anoutward facing mission statement like the following:

The University is committed to excellence in teach-ing and learning within an environment of interna-tionally recognised research. (University of Bristol,1998, p. 1)

This is translated into a series of strategic goals orobjectives, which give some clues as to how we mightgo about fulfilling the mission.What are the indicatorsof performance that enable the librarians to determinewhether they are contributing effectively to the univer-sity strategy?

‘Unless we can show how a library contributes ordoes not contribute…to the educational life of itsmarket, the library manager is inevitably in a weakposition vis à vis his academic colleagues’ (Xavier,2000, p. 27).

My suggestions for evaluating the library’s contribu-tion to the university strategy are shown in Table #1.

20

Fourth Northumbria

University StrategicObjectivesEnhance its status as aninternationally recognisedresearch university

Provide excellent teaching atall levels

Produce graduates who areadaptable and alert to thebenefits of lifelonglearning

Give greater emphasis togrowth in postgraduate stu-dent numbers, particularlyresearch students

Maintain a balance of basic,strategic and contractresearch

Promote interdisciplinaryresearch both within itselfand with other institutions

Recruit and retain excellentstaff and improve their effec-tiveness through training anddevelopment

Optimise the use ofresources to improve theworking environment andrange of services for stu-dents and staff

Improve the quality of theenvironment for the peoplewho live and work in theUniversity and for the widercommunity

Achieve a level of income,which will allow for bal-anced growth, adequate capi-tal investment and provide asound financial base

Library Objectives

Maintain and improve statusas a research library

Meet expanding demands forprovision of support ofteaching and learning

Improve services to post-graduate students

Improve capability to sup-port research

Co-operate with otherlibraries to support research

Recruit and retain excellentstaff and improve theireffectiveness

Seek ways to improve effi-ciency and co-operate withother sections of theUniversity to develop infor-mation services

Improve workingenvironment

Increase ability to generateincome

Activities (examples)

Build extension to mainlibrary.Adequate funding to supportacademic researchprogramme

Electronic reserve collectionof teaching materials.Targeted services for part-time and distant learners.

Students trained in informa-tion skills

Extended opening hours invacations

Adequate budget to maintainaccess to journals

Free access for externalresearchers. Co-operativeacquisitions with other localuniversities

Programme of transferableskills training for all librarystaffIncreased involvement indevelopment and research

Close links with ComputingService and EducationTechnology Service to devel-op managed learning envi-ronment and hybrid library

Ergonomic workstations for allAir conditioning fixed

Level of income achieved

Outcomes (???)

Research collections attractstaff and students. Externalfunding received

Happy students Alumni donate money forlibrary

Alumni reporting value ofinformation skills training

Happy graduate students

Happy academic staff

Happy researchers

Competent and adaptablelibrary staff

Services perceived as ‘joinedup’ by users

Happy library staff and users

Happy administrators

21

Fourth Northumbria

Table 1: Library performance related to university objectives

One of the strategies of the university is to maintain itsprofile as a research-intensive university. Can wedemonstrate at all that anything to do with library serv-ices has any impact on the research quality? In theUnited Kingdom every 4 or 5 years there is a ResearchAssessment Exercise, in which publicly funded univer-sities are required to take part.The outcome of thisexercise leads to a series of scores: each university isassessed on the research quality in each of its subjectsresearched, and for a period of 4 to 5 years after theexercise the government funding to those universitiesis determined in part by the results of the exercise. Byadding the scores on each subject together for eachuniversity, we can get an overall score for the researchquality of the university.We can then plot this againstalmost any input or output relating to library services.Whether by chance or not it turns out that there is aclear correlation between use of the ISI databases andthe research assessment scores (East, 1997).There isthen perhaps an argument to use this indicator strate-gically to justify the continued subscription to ISI data-bases or to suggest, if they don’t already subscribe, that

funds should be made available in order to boost thechances of a university in gaining better researchscores.

Public librariesA survey of UK public libraries has shown that a major-ity were engaged in community development work,but in only a minority of cases was there a formal strat-egy (McKrell and others, 1997).The majority weremonitoring their effectiveness in this area, but there isnot enough detail in the survey to show how this isbeing done: a list of comments gives pointers to placeswhere further enquiry would be fruitful.A more com-plete example was described at the secondNorthumbria Conference (Giappiconi, 1998). I haveextracted some information to illustrate my generalpoint (Table 2).Those who have visited the publiclibrary at Fresnes (France) will appreciate that ThierryGiappiconi has certainly been very effective in attract-ing funds to develop library services there.

22

Fourth Northumbria

Community StrategicObjectiveEncourage reading anddevelopment of the bookand literature

Library Goal(example)Support, stimulate andextend school work in read-ing training

Performance Indicator(example) Number of books borrowedper user aged 6-14 years

Outcome(???)Literate population

Table 2: Public policy and the public library (adapted from Giappiconi, 1998)

National LibrariesSome national libraries are examples of a rare breed: libraries which are not part of larger organizations, butwhose purpose in life is just to exist. It is not quite so simple, because they do have to get funding from govern-ment sources.The Library of Congress has become a national library by stealth: it was designed as a library forthe United States Congress, but its mission reveals the truth:

The Library’s mission is to make its resources available and useful to the Congress and the American peopleand to sustain and preserve a universal collection of knowledge and creativity for future preservation (quotedin Davies, 2000, p. 167).

The Library adopted a strategic plan in 1997, but the Annual Report of the Librarian for 1998 does not givethe main headings of the strategy, nor does it specifically identify achievements related to the plan.There are afew numbers that can indicate how well the overall mission is being met (Table 3).

MissionMake resources available toCongress

Make resources available toAmerican people

Make resources useful

Sustain & preserve universalcollection

Measure>560,000 requests

> 1.1 million enquiries

>2 million items used

Ca 3.8 million itemsacquired, discarded, pre-served, etc.

Staff714

765

1189

Transactions/staffApprox. 800

>1440

>2625

Ca. 3200

Table 3: Achieving the Library of Congress mission (compiled from Library of Congress, 1999)

One strategic aim that is mentioned in the AnnualReport is that determining costs of processes is a highpriority, so we might expect future reports to showsome scrutiny of the numbers in the final column.

There is no question that the British Library ismeant to be a national library. It is now in the middleof a period of rapid change and strategic development.It has long been known as an international researchlibrary of world renown with major collections avail-able to anyone who needs to use them. It is perhapsinteresting to look back at an earlier strategy.TheNational Lending Library (NLL), now part of the BritishLibrary, was originally established to increase the avail-ability of scientific and technical literature in the UK.The strategy to achieve this had two dimensions: cov-erage, and speed of supply.There is some independentevidence of the impact that the Library had on thespeed of supply. In 1958-59, the median supply time forinterlibrary loans to UK university libraries was 10 days(Mackenzie, 1960).This was before the NLL had gotinto it stride. Fifteen years later, by which time the NLLhad become the major supplier of interlibrary loans,

the median supply time had come down to 6 days(Barker, 1974).

The British Library has recently issued a documentoutlining its new strategic directions, defining its mainand enabling strategies with illustrative outcomes.Table 4 gives some examples of headings in the strategy and my suggestions as to how these might bemeasured.

I should stress that these are only illustrations.TheLibrary commissioned a study to clarify relationshipswith university libraries, which made some recommen-dations about the kind of performance regime thatwould be required (Office for Public Management,2001).The strategy to increase co-operation with otherinstitutions has been made operational, in part, throughthe Co-operation and Partnership Programme andfunding is available to develop co-operative pro-grammes with public libraries and academic libraries.The resonance with government policies (see below)is shown through the initial call for proposals, whichwas to work with public libraries to widen access tocollections and to support lifelong learning.

23

Fourth Northumbria

Table 4: British Library strategic directions (adapted from British Library, 2001)

Strategy Possible Performance Indicators (examples)Collection building strategyEnsure improved coverage of the UK national

published archive % of UK publications availableIncrease collecting of digital materials % of UK digital publications availableDevelop greater collaboration with other libraries % of non-UK publications available nationallyAccess strategyMake the library’s collections more accessible % of searches of OPAC that find items that are

to users in the catalogueExtend opportunities for enjoyment % of visitors who enjoy

Policy values (Overman and Cahill, 1990, Government information principles reported in Rowlands, 1997b) (from Carbo, 1997)

Access and freedom: people need to be Convenient access to all government well informed information;

Privacy Services accessible to all persons Openness: the right to know about decision making Safeguard privacy

processes Education and training in rights and Usefulness: fit for purpose responsibilitiesCost and benefit: reconciling commercial interests Ensure quality, integrity and appropriate

with the public interest preservation and archiving Secrecy and security No charge on the superhighway widest Ownership: intellectual property rights, and possible cost-effective dissemination

reconciling commercial interest with the needs Ensure securityof individuals Private sector to provide value-added information

and servicesConsultation with all interest groups

Table 5: Information policy values in action

National strategiesNow, to turn to the context within which librariesoperate: national and supra-national information poli-cies and strategies.We must ask the question - What arestrategies at this level for? Are libraries and informationservices self-evidently Good Things, so that the pur-pose of a national strategy is to nurture and sustainthem? After all,“...survival…is itself an index to major-ity opinion” (Orwell, 1947), and libraries have certainlysurvived. Perhaps we can get some clues from lookingat how things work now, whatever the situation mayhave been in the past.

It is my impression, looking from outside, that therehave always been visions of national information poli-cies in the USA. Perhaps this is because there has beenfor a long time now a government agency, the NationalCommission on Libraries and Information Science,responsible for advising the President and Congress onpolicy. Overman and Cahill (1990) described the valuesthat underpinned US Federal activities in the informa-tion policy area over several decades.We can illustratethe application of these values by reference to theClinton-Gore administration’s vision of the “informationsuperhighway”. Carbo (1998) has described the devel-opment of the policy on the National InformationInfrastructure, giving the strategic overview.The‘Government information principles’ she describesmap so neatly on to the values (Table 5), that it looksas if the original analysis by Overman and Cahill didindeed identify the fundamental values.

So, the strategy for developing the US NationalInformation Infrastructure is underpinned by somefundamental values.These values can be considered toreflect the interaction between two dimensions ofinformation policy. One dimension is about the flow of information, and the other is about value.These

dimensions can be illustrated in a 2 x 2 matrix (Table 6).

Returning to the strategy for developing the USNational Information Infrastructure, the five fundamen-tal goals of the strategy can be summarised as to:

1. Make information technology work to advanceAmerican values: this covers the top half of thematrix - mainly Citizenship, but also cultural identi-ty in the Protectionism box.

2. Use information technology to build stronger com-munities: Citizenship.

3. Enable everyone to participate: the left hand side,Citizenship and Consumer Choice.

4. Ensure that everyone takes responsibility:Citizenship and Protectionism

5. Maintain world leadership in developing theInformation Superhighway: CompetitiveAdvantage.

Supra-national The European Community has had a number of strate-gies related to libraries and information services overthe years, and Oppenheim (1998) has provided a use-ful summary, albeit now outdated.The latest initiative isto promote the development and use of European digi-tal content on the global networks - E-content.This willfocus on the market implementation of Europe’s con-tent potential and not on the technological aspects ofthe global networks.The strategic objectives are to:

• improve access to and use of public sectorinformation

• enhance content production in a multilingual andmulticultural environment

• increase dynamism in the digital content market

24

Fourth Northumbria

Information as a public goodINFORMATION FOR CITIZENSHIP INFORMATION PROTECTIONISMopen, unrestricted information flows closed, restricted information flows• Free public libraries • data privacy• Internet • censorship• Freedom of information • national security• Access to democracy • commercial secrecy• Advice services • cultural identity• Legal deposit • national champions

Information as a tradable commodityINFORMATION FOR CONSUMER CHOICE INFORMATION FOR COMPETITIVE ADVANTAGE• freedom of the press • Intranet• the mass media • competitive intelligence• universal service • patents• public-private synergy • intellectual property rights• charged library services • market research• infotainment • consultancy

Table 6: Mapping the information policy construct (from Rowlands, 1997a)

Overall, the strategy seems to fit into the matrixalready described above (Table 6) but the difficulties inevaluating it are clear.Taking the first of the objectives,how do we measure access to public sector informa-tion now? Can we operationalise that concept todefine a metric that will still be valid after that informa-tion has been made available electronically? The con-cept of “improving access” has more than one dimen-sion. One dimension means,“increasing the range ofinformation that is available on the Internet” and that isrelatively easy to quantify. Another dimension is “thetime taken to get information”. Do we have a bench-mark now for this metric, and when we have digitisedeverything, how will we normalise the metric to allowfor the fact that not everyone has access to theInternet from his or her home?

Social qualityThe analysis so far has still failed to answer the question,“What are high level strategies for”? The con-cept of social quality may be useful.This has beendefined as:

‘The extent to which citizens are able to participatein the social and economic life of their communitiesunder conditions which enhance their well-beingand individual potential.’ (Beck and others, 1997,page 3)

The dimensions of social quality show how a gov-ernment might adopt policy aims that were intendedto increase social quality.

Socio-economic security: the way in which essentialneeds of citizens are fulfilled by systems and structuresfor welfare provision.

Social inclusion – the principles of quality and equi-ty and their structural causes.

Social cohesion – the processes surrounding socialnetworks and infrastructures.

Empowerment – enabling citizens to develop theirfull potential.

Since some of these dimensions are potentiallymutually in opposition in regard to outcomes, it is nec-essary to use them as a sort of balanced scorecardinvolving all processes and outcomes to get an overallview. Berman & Phillips (2001) have applied thismodel to the information context and have illustratedthe concept with some information indicators at vari-ous levels.Table 7 gives the framework for theEmpowerment element.

The UK government and its agencies have beenquite active in supporting libraries recently and theirpolicy agenda can be interpreted in terms of thedimensions of social quality (Table 8).

The initiatives for public libraries are steamingahead, and keeping track of all these initiatives is amajor task for a non-specialist. One of the more

25

Fourth Northumbria

Element of SocialQualityEmpowerment

InputInformation resourcesavailable: networks,public access points,web sites, etc.

ProcessAccessibility of infor-mation sources andparticipation in net-working.

OutcomesAchievement of infor-mational competenciesand capabilities.

Use of information indaily life.

ImpactSelf-reported, subjec-tive evaluations of per-sonal empowermentand quality of lifeachieved through useof informationresources.

Table 7: Information indicators for the nation-state (extracted from Berman & Phillips, 2001)

Table 8: Social quality and UK library and information strategies

Dimension Strategies relevant to librariesSocio-economic security • National Grid for Learning (networking schools)

• People’s Network (networking public libraries)• SuperJANET (networking higher education)• New Opportunities Fund (digitisation of content)• Distributed National Electronic Resource

Social inclusion • National curriculum - key information skills for children• IT training for teachers• IT training for public library staffs• eLib - Netskills training materials (for librarians)

Social cohesion • Freedom of information legislation• Electronic delivery of government services

Empowerment • most of the above

unusual features of this group of strategies is that eval-uation of achievement is ongoing.There is a databaseof indicators on the web, NETbase, where we havemeasures and indicators of performance relating tonetwork infrastructure, training and staffing, contentand service developments, networked services policyand collaborative working.Table 9 illustrates some ofthe relevant indicators.

It is perhaps significant that the director of the net-work project is Chris Batt, who was a member of thePublic Libraries Research Group, one of the groupsthat had a lot of influence on the development of aperformance measurement culture in UK libraries.

Public library standards In carrying these policies through to realisation, thepresent UK government, while giving the kind of polit-ical support to public libraries that has been lackingfor many years, has at the same time increased thedegree of accountability to central government.The lat-est expression of this has been the publication of a setof standards, examples of which are given in table 10.These are explicitly related to the social inclusionagenda.

26

Fourth Northumbria

Indicator Value at 16 May 2001Network infrastructureLibraries with internet access 62%Training and staffingLibraries which provide some form of internet training to customers 93%Content and service developmentLibrary catalogue remotely accessible to customers 33%Enquiry services available remotely to customers 55%Libraries subscribing to e-journals or e-data services 53%Networked services policyLibraries with electronic collections development policy 20%Libraries with Internet filtering software 64%Internet charging policy employed 86%Partnerships and collaborative working Libraries with commercial

partnerships to deliver ICT access 24%Libraries involved in collaborative digital content creation 78%

Table 9: Achieving the People’s Network – UK indicators (Peoples’ Network, 2001)

Table 10: Standards for UK public libraries (extracted from Great Britain. Department for Culture, Mediaand Sport, 2001)

StrategicobjectivesConvenient andsuitable access

Electronic access

Ensure user satisfaction

Provide choice inbooks etc

Measure or indicator

% of households living within n miles of a static library

% of libraries with access to on-line catalogues

Electronic work stations per thousand population% of users reporting success in obtaining specific book

% of users reporting success in gaining information from asearch or enquiryQuality index

Items added per year per 1000 populationTime taken to replenish lending stock

Standard to be achieved by200485% within 2miles (rural)100 % within 1 mile (inner London) 100% of libraries open >10hours per week0.665%

75%

To be devised

2168.5 years

27

Fourth Northumbria

Influencing the strategic processThis analysis of the higher-level strategies gives us theframework to help in the second strategic use of evalu-ation and performance measurement: to influence thedevelopment of strategy.There is a spectrum of strate-gy development, which can be described as rangingfrom systematic to chaotic (Johnson & Scholes, 1999).

• Logical incremental• Rational command• Muddling through• Externally dependent

Two of these are relevant to most librarians (Table 11).

Libraries are typically professional services embed-ded within public sector organisations, so it is easy tosee how in this context there may be conflictsbetween the different levels of strategy.

For example:• Government policy: commitment to lifelong

learning and social inclusion• University strategy: increase number of students

on part time and distance learning masters’ levelcourses

• Library services: conflicts arise because part timestudents need longer loan periods than full timestudents

Policy makers and strategists In order to influence strategy we need to identify thepolicy makers and the strategists.

A useful typology was set out by Strachan &Rowlands (1997), which can be adapted to allow for adistinction between policy and strategy (Table 12).

Type of organisationProfessional service

“Muddling through”

Public sector

“Externally dependent”

Dominant dimensionsCulturalPoliticalIncremental

PoliticalEnforced choice

Characteristics of strategyNegotiation and compromise to accommodate conflict-ing interestsStrongly influenced by groups with control over criticalresources Routines embedded in historyImposed by external forcesRestricted choiceGroups dealing with external environment have mostinfluencePolitical activity within organisation and betweenexternal agencies

Table 11: Organisational strategies (adapted from Johnson & Scholes, 1999)

Policy-maker/StrategistLegislator

Bureaucrat

Interest group

Role

• make decisions• set policy direction• monitor issues

• develop strategic options• implement strategies • review strategies

• develop issues• advocate policy and

strategy• lobbying/agenda setting

Information sources

• public opinion, constituents, press, lobby groups• political parties, system outputs

(e.g. from Commissions, bureaucracy, etc)• debates in legislature, policy papers, expert opinion,

etc• system outputs• commissioned work• policy papers, expert opinion, analysis of

indicators, data and trends, research data etc• published data and indicators• research results in the literature• own/commissioned research

Table 12: Policy makers and strategists (adapted from Strachan & Rowlands, 1997)

28

Fourth Northumbria

The same authors have identified factors affecting useof information by policy-makers, summarised as:

BARRIERS TO INFORMATION TRANSFER AND USE:

situational - operational data not directly relevant orin suitable format; too much; time and costconstraints;cognitive - technical language; failure to recogniseimportance; complexity of issues;scientific - probabilistic nature of information makespolicy making difficult; no research in area; difficultto perform experiments to produce informationthat can reliably be generalised to policy problems.

FACTORS PROMOTING INFORMATION TRANSFER AND USE:

policy makers have a preference for:information from research funded, conducted orcommissioned by themselves;information which accords for their intuitive feel fora problem;interpersonal sources over print-based;

clearly presented materials that avoid technical jar-gon and make limited use of statistical quantifica-tion (Lindblom and Woodhouse, 1993);synthesised and evaluated research which includes adiscussion of the policy implications of the results(Nelson et al., 1987).

There are some examples that can be cited of librar-ians seeking to influence the strategic process: onesuccessful (Carmel, 1995) and one that has somepotential (SCONUL and HCLRG, 2000).

Why bother?The importance of getting involved can be demonstrat-ed by an illustration of the policy making process(after Mintzberg, 1994).

It is clear that if the outcomes are evaluated only interms of the intended strategy and deliberate actions,then the services being evaluated may be in danger ofbeing perceived to be performing badly. In the organi-sational context, a library service is clearly an interestgroup. It is desirable therefore that both are recog-nised by the library managers in order to influence thestrategy and to ensure that the evaluation relates to theemergent strategy.

Synthesis/way forwardSince most libraries and information services actuallyserve multiple constituencies, it is important thatstrategies address these multiple needs, and that evalu-ation should be both multifaceted and holistic, involv-ing as many of those affected by the strategies as possi-ble. This immediately shows that evaluation must beboth quantitative and qualitative; the balanced score-card offers a convenient tool that encompasses bothaspects.

It is in the interests of librarians to have as large agroup of potential users as possible, and as many capa-ble users as possible. When evaluating informationstrategies, we need to ask who will benefit, who hasaccess to services but is disadvantaged, and who isexcluded? How do these outcomes square with thoseintended originally? (Eisenschitz, 1997).

In order to achieve our goals there is no alternativeto getting involved in the development of organisation-al strategies. In attempting to influence strategy, usingwhatever evaluation and performance data is appropri-ate, it is important to recognise that there are threeinterpretations of the policy-making process (Rowlandsand Turner, 1997).

• Rational actor - based on perfect knowledge,explicit goals, cold analysis, objective, impartial,fair.

• Bureaucratic imperative - influenced by values andbeliefs, organisational context, uncertainty, shortterm, incremental.

• Garbage can - choices looking for problems, issuesand feelings looking for decision situations in

organisational learning

power relationsad hoc decisions

Emergent strategy

Realised strategy Outcomes

interest groups

unrealised strategy

Deliberate actions

Intended strategy

29

Fourth Northumbria

which they might be aired, solutions looking forissues to which they might be the answer, deci-sion-makers looking for work!

At the evaluation stage, it helps to assume that poli-cy-making is a rational process. In terms of writing his-tory, long after the event it becomes clear that thegarbage can model is always present, with elements ofrationality and bureaucracy thrown in, and that the bal-ance between them shifts with time and place.

Finally, the process of policy making has beendescribed in terms that could be used to describestrategic evaluation, and indeed the whole practice ofperformance measurement:

“…a primeval soup [in which] ideas float around,confront one another and combine.The soupchanges in a process of natural selection, survival,demise and recombination.” (Kingdon, 1984, p. 104)

ReferencesAllen, D. and Wilson,T. (1997) ‘Information systems

strategy formulation in higher education’ inRowlands, I. editor: Understanding informationpolicy: proceedings of a workshop held atCumberland Lodge, Windsor Great Park, 22-24July 1996. London: Bowker Saur 178-190.

Barker, E.J. (1974) The costs and supply-delays ofinterlibrary loans. University of Sheffield, M.A.dissertation.

Beck,W. and others (1997) ‘Introduction’ in Beck,W.and others editors: The social quality of Europe.The Hague, Netherlands: Kluwer Law International1-13.

Berman,Y. and Phillips, D. (2001) Information andsocial quality. Aslib Proceedings, 53(5), 179-188.

British Library (2001) New strategic directions.London: British Library.<http://www.bl.uk/shape.html>

Carbo,T. (1998) ‘Information policy and the USNational Information Infrastructure’ in Grieves, M.editor (1998) Information policy in the electronicage. London: Bowker Saur 71-94.

Carmel, M. (1995) ‘Using contracts to improve quality’in Wressell, P. editor: Proceedings of the 1stNorthumbria international conference on per-formance measurement in libraries and informa-tion services held at Longhirst…31 August to 4September 1995. Newcastle upon Tyne: InformationNorth, 33-41.

Cullen, R. (1998) ‘Does performance measurementimprove organisational effectiveness? A post-modernanalysis’ in Wressell, P. & Associates editors:Proceedings of the 2nd Northumbria international

conference on performance measurement inlibraries and information services held atLonghirst…7 to 11 September 1997. Newcastleupon Tyne: Information North, 3-20.

Davies, J.E. (2000) ‘What’s the use of a library?Assessing performance and outcomes.’ The LawLibrarian, 31 (3), 166-174.

East, H. (1997) ‘The limitations of ‘free’ access in acade-mia’ in Rowlands, I. editor: Understanding informa-tion policy: proceedings of a workshop held atCumberland Lodge, Windsor Great Park, 22-24July 1996. London: Bowker Saur 221-231.

Eisenchitz,T. S. (1997) ‘Legal and regulatory frameworksfor information policy’ in Rowlands, I. editor:Understanding information policy: proceedings ofa workshop held at Cumberland Lodge, WindsorGreat Park, 22-24 July 1996. London: Bowker Saur128-145.

Feeney, M. and Grieves, M. editors (1994). The valueand impact of information. London: Bowker Saur.

Ford, G. (1989) ‘A perspective on performance meas-urement.’ International Journal of Informationand Library Research, 1(1), 12-23.

Giappiconi,T. (1998) ‘Performance measurement andmanagement strategy: a public library perspectivefrom France’ in Wressell, P. & Associates editors:Proceedings of the 2nd Northumbria internation-al conference on performance measurement inlibraries and information services held atLonghirst…7 to 11 September 1997. Newcastleupon Tyne: Information North, 21-29.

Great Britain. Department for Culture, Media and Sport.(2001) Comprehensive, efficient and modern pub-lic libraries - standards and assessment. London:DCMS. <http://www.culture.gov.uk/PDF/libraries_pls_assess.pdf>

Hudson,A. (2000) ‘An added-value approach to per-formance statistics.’ Librarians’World, 9(1), 2-5.

Johnson, G. and Scholes, K. (1999) Exploring corporatestrategy. 5th edition. London: Prentice Hall Europe.ISBN: 0 13 080740 0

Joint Funding Councils’Ad-hoc Group on PerformanceIndicators for Libraries (1995.) The effective aca-demic library: a framework for evaluating the per-formance of UK academic libraries. Bristol: HEFCE.

Karunaratne, N.D. (1978) ‘Assessing performance inlibraries.’ Long Range Planning, 11 April 1978,66-70.

Kingdon, J.W. (1984) Agendas, alternatives and publicpolicies. Boston, MA: Little Brown.

Library of Congress (1999) Annual report of theLibrarian of Congress for the fiscal year ending30 September 1998.Washington, DC: Library ofCongress.

Lindblom, C.E. and Woodhouse, E.J. (1993) The policy-making process. 3rd edition. New Jersey: PrenticeHall.

MacEachern, R. (2001) ‘Measuring the added value oflibrary and information services: the New Zealandapproach.’ IFLA Journal, 27(4), 232-237.

Mackenzie,A.G. (1960) ‘Interlibrary loans in British uni-versities: a survey.’ Journal of Documentation,16(2), 41-64.

McKrell, L. and others (1997) Libraries and communitydevelopment: national survey. Liverpool: LibraryAssociation Community Services Group. (BritishLibrary Research and Innovation Centre Reports, 86).

Mintzberg, H. (1994) The rise and fall of strategicplanning. Hemel Hempstead: Prentice HallInternational.

Nelson, C.E. and others (1987) ‘The utilisation of socialscience information by policy-makers.’ AmericanBehavioural Scientist 30(6), 569-577.

Office for Public Management (2001) British Libraryand UK higher education: strategic objectives andperformance study report. London: Office for PublicManagement.

Oppenheim, C. (1998) ‘Current UK and EU informationpolicy’ in Grieves, M. editor (1998) Informationpolicy in the electronic age. London: Bowker Saur45-70.

Orwell, G. (1947) ‘Lear,Tolstoy and the fool’ reprintedin Shooting an elephant and other essays, London:Secker & Warburg, 1950, 33-56.

Overman, E.S. and Cahill,A.G. (1990) ‘Information poli-cy: a study of values in the policy process.’ PolicyStudies Review, 9(4), 803-818.

People’s Network (2001) NETbase. <http://www.peo-plesnetwork.gov.uk/netbase/index.html>

Pienaar, H. and Penzhorn, C. (2000) ‘Using the balancedscorecard to facilitate strategic management at anacademic information service.’ Libri, 50(3), 202-209.

Rowlands, I. (1997a) ‘General overview’ in Rowlands, I.editor: Understanding information policy: proceed-ings of a workshop held at Cumberland Lodge,Windsor Great Park, 22-24 July 1996. London:Bowker Saur 1-16.

Rowlands, I. (1997b) ‘National and international per-spectives: overview’ in Rowlands, I. editor:Understanding information policy: proceedings ofa workshop held at Cumberland Lodge, WindsorGreat Park, 22-24 July 1996. London: Bowker Saur83-88.

Rowlands, I. and Turner, P. (1997) ‘Models and frame-works for information policy research’ in Rowlands,I. editor: Understanding information policy: pro-ceedings of a workshop held at CumberlandLodge, Windsor Great Park, 22-24 July 1996.London: Bowker Saur 46-60.

SCONUL and HCLRG (2000) UK higher educationlibrary management statistics 1998-99. London:SCONUL/HCLRG.

Strachan, J. and Rowlands, I. (1997) ‘Information for pol-icy-making’ in Rowlands, I. editor: Understandinginformation policy: proceedings of a workshopheld at Cumberland Lodge, Windsor Great Park,22-24 July 1996. London: Bowker Saur 61-73.

University of Bristol (1998) University Plan1998-2007. <http://www.bris.ac.uk/Publications/Plan/index.htm>

Willemse, J. (1995) ‘The impact of performance meas-urement on library and information services: adecade of experience’ in Wressell, P. editor:Proceedings of the 1st Northumbria internationalconference on performance measurement inlibraries and information services held atLonghirst…31 August to 4 September 1995.Newcastle upon Tyne: Information North, 11-32.

Xavier, C. (2000) ‘The use of management informationin an FE college library’ in Spiller, D. editor:Performance measurement in further educationlibraries: proceedings of a seminar held atLoughborough University, 9-10 March 1999.Loughborough: Library & Information StatisticsUnit 24-29.

30

Fourth Northumbria