Document ofThe World Bank

Report No: 21766 ET

PROJECT APPRAISAL DOCUMENT

ONA

PROPOSED CREDIT

_N THE AMOUNT OF SDR 3.9 MILLION(US$ 4.9 MILLION EQUIVALENT)

TO THE

GOVERNMENT OF ETHIOPIA

FOR A

DISTANCE LEARNING LIL PROJECT

April 4, 2001

Operational Quality & Knowledge UnitAfrica Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS(Exchange Rate Effective December 1, 2000)

Currency Unit = BirrETBirr = US$0.12US$1 = ETBirr 8.3

FISCAL YEARJuly 1-June 30

ABBREVIATIONS AND ACRONYMS

LIL Learning and Innovation Loan

CSC Civil Service College

IDE Institute of Distance Education

GDLN Global Distance Learning Network

NGO Non-governmental Organization

WBI World Bank Institute

CAS Country Assistance Strategy

SOE Statement of Expenditures

TTL Task Team Leader

SA Special Account

IDA International Development Association

UNDP United Nations Development Program

VSAT Very Small Aperture Terminal

DL Distance Learning

Vice President Callisto MadavoCountry Director Oey Astra MeesookSector Director John RoomeTeam Leader Nicholas Bennett

ETHIOPIADistance Learning LIL Project

CONTENTS

A: PROJECT DEVELOPMENT OBJECTIVE

1. Project development objective ............................................................. 12. Key performance indicators ............................................................ 1I

B: STRATEGIC CONTEXT

1. Sector-related Country Assistance Strategy (CAS) goal supported by the project ..................... 12. Main sector issues and Government strategy ............................................................ 2

C: PROJECT DESCRIPTION SUMMARY

1. Project components ............................................................ 32. Benefits and target population ............................................................ 53. Institutional and implementation arrangements ............................................................ 54. Financial Management ............................................................ 55. Procurement ........................................................... 6

D: PROJECT RATIONALE

1. Project alternatives considered and reasons for rejection ............................................................62. Major related projects financed by the Bank and/or other development agencies ......................73. Lessons learned and reflected in the project design ....................................... . . 74. Indications of borrower commitment and ownership ....................................... 85. Value added of Bank support in this project ....................................... 8

E: SUMMARY PROJECT ANALYSIS

1. Economic ...................................... 82. Financial ....................................... 83. Technical ....................................... 94. Institutional ...................................... 95. Environmental ...................................... 106. Social ...................................... 107. Safeguard Policies ...................................... 108 Business Policies ...................................... 10

F: SUSTAINABILITY AND RISKS

1. Sustainability ...................................... I 12. Critical Risks ...................................... 113. Possible Controversial Aspects ....................................... 11

G: MAIN LOAN CONDITIONS

1. Effectiveness Conditions ................................. 122. Other ................................. 12

H. READINESS FOR IMPLEMENTATION ................................. 12

I. COMPLIANCE WITH BANK POLICIES ................................. 12

ANNEXES

Annex 1. ...... Project Design Summary

Annex 2. .... .. Detailed Project Description

Annex . ...... Estimated Project Costs

Annex 4 . ..... Economic Justification Summary

Annex 5 .... .. Financial Summary

Annex 6 ...... Procurement & Disbursement Arrangements

Annex 7 ...... Project Processing and Budget Schedule

Annex 8 ...... Documents in the Project File

Annex 9 .... .. Statement of Loans and Credits

Annex 10 ...... Country at a Glance

Annex 11 .... .. Organization, Staffing & Technical Capacity of IDE

Annex 12 ...... Relationships Between Regional Centers & CSC

Annex 13 .... .. Technical Assistance Requirements

Annex 14 .... .. Rationale for Use of Internet to Deliver Courses

Annex 15 ...... Monitoring, Evaluation and Performance Indicators

MAP ... ... IBRD 31158

ETHIOPIADistance Learning LIL Project

Project Appraisal Document

Africa Regional OfficeAFMET

Date: April 4, 2001 Team Leader: Nicholas BennettCountry Manager/Director: Oey Astra Meesook Sector Manager/Director: John RoomeProject ID: P069083 Sector: Other Education

Lending Instrument: LIL Theme(s): Distance LeamingPoverty Targeted No [XI Yes [ IIntervention:

IDA Others Total Commitment ClosingUS$ m % US $ m US$xm Date Date

Credit 4.9 69% 2.2 7.1 2002 2004 Govemment ofEthiopia

Project Financing Data[I Loan [x I Credit [ Grant [] Guarantee 1 Other [Specify]For Loans/Credits/Others:Amount (US$m): US$ 4.9 millionProposed terms: [ ] To be defined [XI Multicurrency [ Single currency

l [ ] Standard [ ] Fixed [ ] LIBOR-Variable based

Grace period (years): 10Years to maturity: 40Commitment fee: 0.5%Service charge: 0.75%Front-end fee on Bank loan:

Government 1.4 0.8 2.2IDA 0.5 4.4 4.9

Total: 1.9 5.2 7.1Borrower: Government of EthiopiaResponsible agency(ies): Civil Service College

Estimated disbursements (Bank FY/US$M): US$4.9 million

Annual 1.6 2.1 1.3Cumulative 1.6 3.7 4.9Project implementation period: FY2002 to FY2004Expected effectiveness date: June 1, 2001 Expected closing date: June 30, 2004Implementing agency: Civil Service CollegeContact person: Dr. Haile Michael AberraAddress: P.O. Box 5648 Addis Ababa, EthiopiaTel: (251-1) 60-17-63 Fax: (251-1) 60-17-64 E-mail: Hailem@exciet.com

A. Project Development Objective

1. Project Development Objective

As a Learning and Innovation Credit (LIL), the prime objective of this project is to test thecomparative learning outcomes, the comparative demand, the comparative costs and costeffectiveness, and the sustainability of several different distance learning approaches beingimplemented by the Institute of Distance Education (IDE) in the Civil Service College (CSC). Thefindings of these comparative analyses will be disseminated widely, both inside and outside thecountry. A secondary objective is to develop the capacity of the IDE at the CSC to manage; a) theGlobal Distance Learning Network (GDLN) link, both at the federal, and at five regional centers; b)the development and use of appropriate printed distance learning materials; and c) the establishmentof regional and sub regional distance learning outreach centers.

2. Key Performance Indicators

Performance indicators will be monitored during implementation along with periodic evaluation andwill include the following:

* Demand for the different types and topics of distance learning (DL) courses in Addis and theregions;

* Percentage of available time that the central and regional GDLN interactive video facilities areactually used (available time is Monday through Friday from 9 a.m. to 9 p.m.);

* Percentage of GDLN central and regional link costs met from other revenue sources such as a)renting video link services to other government agencies, the private sector and donors; b)selling internet access to the general public especially at regional centers; c) subscriptions fromprivate individuals participating in internet based courses; d) revenues from computer literacytraining; and e) donor funding of specific courses;

* Learning achievement by course, delivery mode, and tutoring support;

* Learner satisfaction by course, delivery mode, and tutoring support;

* Time to completion by course, delivery mode, and tutoring support; and

* Cost effectiveness of learner achievement by delivery mode.

B. Strategic Context

1. Sector Related Country Assistance Strategy (CAS) goal supported by the project

CAS Report No. 17009-ET & Interim Support Strategy Report No. IDA\R2000-198Date of latest discussion: August 19, 1997 (CAS); December 5,2000 (ISS)

In the existing (1997) CAS, the proposed strategy aims to reduce poverty both directly and bypromoting sustained economic growth notably by creating an environment conducive to rapidprivate sector and export development. This is to be achieved through four clusters of activities, thefirst of which deals with policy and capacity; and the fourth with policy and human development.One of two main aims of this first cluster is to have "improvements in capacity for implementing

-2 -

projects and policies". The proposed operation directly contributes to the achievement of this firstcluster through massive distance learning programs directed at civil service, NGO, and private sectorpersonnel; and also contributes to the fourth cluster, human development.

Interim Support Strategy for the Federal Democratic Republic of Ethiopia Report No 21189ET. November 9 2000

A new CAS was being prepared early in 1999, but preparation and discussion had to be delayed asthe conflict with Eritrea flared up in February of that year. Only when a cease fire was agreed inJune 2000 did work recommence at which time it was decided to prepare an Interim SupportStrategy, which was discussed and approved by the Board in November 2000. In this strategy,capacity issues occupy center stage, as both Government and the Bank have agreed that withoutconcerted efforts to improve capacity at all levels Government's ambitious development objectiveswill not be achieved. In this Interim Strategy this project is specifically mentioned.

2. Main Sector Issues and Government Strategy

With the change in Government in 1991, and the consequent opening up of the administration andthe private sector that followed, combined with the decentralization of real authority to nine regionalstates and two city administrations in 1994, human capacity in the public sector has been stretchedbeyond its limits. Of the 300,000 federal and regional civil servants 83% do not even have a postsecondary diploma. By far the largest number of civil servants are employed by the regions.

In September 1998 a high level Capacity Development Committee was established, chaired by thePrime Minister, and made up with a number of key ministers. In December 1998 the committeeissued its first draft capacity building strategy document, in which the Civil Service Collegeoccupied a central role. Six sub-committees were then established which reported to a nationalcapacity building seminar in early June 1999, again chaired by the Prime Minister. The key role ofthe CSC was discussed, and it was agreed that the existing five year plan of the college would needto be re-examined, taking account of the overall civil service reform program. At that time there wasno discussion of distance learning options.

Concurrently in 1998 the CSC held a "Visioning Conference" with the college stakeholders, whichforged agreement on a more up-to-date and precise mission statement for the College. Thisstatement was formally considered and adopted by a joint meeting of the Board of the College andkey staff on January 16, 2000. It emphasized the key role the College will play in using appropriatedistance education media, and in running appropriate programs to strengthen the capacity ofEthiopia's civil service, both at the federal and regional levels through:

* Training and producing competent civil servants;* Giving special assistance to civil services in the newly emerging regional states;* Contributing to the strengthening of public and private partnerships; and* Conducting research and providing consultancy services in key areas of civil service

development.

3. Learning and Development Issues to be Addressed by the Project:

As the modern world becomes increasingly complex with new knowledge and new technologybecoming more and more essential, there is a danger that Ethiopia will be left behind. Access to thisnew knowledge and technology is limited by poor internet services from the EthiopianTelecommunication Corporation as the only provider, and is costly (up to $25 an hour for those in

- 3 -

the regions), and thus those who use the internet are so thinly spread that it is difficult for Ethiopiansto remain connected to this modem world. This is why it is being proposed to upgrade andstrengthen the Global Distance Learning Network, internet and virtual learning facilities in the CSCin Addis, and to extend these to five regional centers where there is the highest concentration of civilservants. These links will only be used for education and training purposes.

The ravages of decades of war, combined with the strains on the existing stock of skilled manpowerresulting from the 1994 creation of regional states, has meant that no sector of the economy hasadequate numbers of qualified and trained manpower. In addition, no sector is sufficiently in touchwith relevant up-to-date global developments. It is as much the private and the NGO sectors thatneed knowledge of the latest ideas, as the public sector, and thus many of the offerings from theupgraded and extended GDLN will be open to all and actively marketed. More serious is the factthat the vast proportion of the manpower who need to have their knowledge upgraded, and couldbenefit from links with the outside world, are employed by regional states and currently are notcomputer literate and do not have any access to the internet.

In each of the regions there are tens of thousands of recently recruited public servants, who have hadno basic training that is related to their jobs. A detailed needs assessment was carried out by IDEstaff in October 2000, in four regions, and from this it was clear that the highest priority for suchtraining was in accountancy, closely followed by management, with many thousands of civilservants needing training in each. The only way to reach such cadres is through correspondence orelectronic distance learning systems. These correspondence courses will be based on printedmaterials supplemented with audio and videocassettes, and with twice monthly face to face tutorialsessions. These courses will be supported by live lectures from CSC staff in Addis Ababa directlybeamed at the regional centers where appropriate. In parallel with the printed media courses IDEwill select and subscribe to appropriate web based courses, and will develop one or its own.

Given the need to test some potentially high cost programs, combined with the constrainedbudgetary situation in Ethiopia, it will be necessary for the project to provide the CSC with somesupport to cover the initial recurrent costs (on a declining basis,78% in the first year,59% in thesecond, and 35% in the third) for both the GDLN and the printed media DL, over an initial three-year period.

C. Project Description Summary1. Project components

1. GDLN Facilities OhrEduc. 1.0 15%0.1%Upgrading & Extension2. Satellite Rental, Maint., & Other Educ. 1.4 19% 0.8 11%Other Oper. Costs for GDLNCenters3. Dev. Of Printed Media Other Educ. 1.3 18% 0.9 12%Distance Learning4. Training Other Educ. 0.8 11% 0.0 0%5. Capacity Development & Other Educ. 2.6 37% 2.4 34%ITA I_____Ioa 7.1 10%4.I_

I ~~~Totall 7.1 I100% 1 4.9 t 69%

- 4 -

A range of different Distance Learning approaches is being developed by the IDE to enable it toreach its huge potential audience of civil servants needing training in Addis Ababa and in sevenregional centers; including:

Component 1. GDLN Facilities Upgrading and Extension:

* upgrading the existing GDLN videoconferencing and internet link at the CSC to the highestinternational standards, and to use both for international training, and for beaming localcourses to regional centers;

* establishing modified GDLN links with five regional centers, (Bahir Dar, Makelle, Nazareth,Awassa, and Dire Dawa) beaming programs developed at the IDE, and also drawn from theinternational GDLN, to these centers. The centers can be considered as extension arms ofthe IDE/GDLN, offering regional students the same programs as those in the CSC, by July2002. These regional centers will be established in existing tertiary level institutions, andthe rooms provided will only need wiring, computer networking, and security modifications.They will be equipped with computers (on which learners will view the video programs),and audio visual equipment for participant use;

Component 2. Satellite Rental, Maintenance, and Other Operational Costs for GDLNCenters:

* satellite rental, facility maintenance, and other related operational cost activities resultingfrom the central GDLN from March 2001. 22% of these costs to be generated from learners(directly or via sponsorship) in 2001/2, 41% in 2002/3, and 65% in 2003/4;

Component 3. Development of Printed Media Distance Learning:

- developing printed distance learning materials, eventually in three disciplines, and launchingcourses in these (June 2002 diploma in accounting and diploma in management, and January2004 diploma in a third subject) to be administered through trained tutors at 7 selectedregional sites, with each enrolled learner taking three years to complete;

Component 4. Training:

* running large scale computer literacy, and remedial English courses for civil servants at theregional centers on a full cost basis, designed to bring them into the computer age, andprepare them to benefit from the internet;

* enrolling at least 1300 learners in the print based distance learning in 2002, 2600 in 2003,and 4700 in 2004. Participants and their employers will be expected to pay two thirds of thetotal running costs of their courses. On the basis of current estimates the participant share isexpected to be ETB 800 per year. On the basis of information collected during the needsassessment, ETB 600 could be paid by the employer per year;

Component 5. Capacity Development and TA:

* Establishing two other regional centers without the GDLN satellite link (Jinga, and AddisAbaba), equipped with computers and audio visual equipment;

providing technical assistance and training in effective use of interactive video and internet,the development and use of print based correspondence courses, the development and useof internet courses, and training in procurement and financial management.

In addition constant monitoring and reporting on all activities, combined with a detailed evaluation(Annex I 1) at the mid point, and the end of the project will be an integral part of the project.

2. Benefits and Target Population

The target population includes all civil servants, private sector and NGO employees in the AddisAbaba area, and all middle level public and some private sector staff employed in seven regionalstate and municipal headquarters. Through appropriate training, and video and internet contact withknowledge centers throughout the world it is expected that their productivity will be enhanced.

3. Institutional and Implementation Arrangements

The CSC is a semi-autonomous training institution established in 1995 by proclamation from theCouncil of Ministers (Regulation No 3/1996: Annex 12). Under this proclamation CSC wasestablished as a higher level institution falling directly under the Prime Minister, created to train civilservants both at the federal and the regional state levels. What programs are launched by the collegeis decided by a Board, made up largely of Government Ministers and leading public servants. Theproject, and its three inter-linking components has been considered and endorsed by the board.

IDE is an integral part of the CSC. It will be responsible for all distance education activitiesincluding the GDLN center, the printed media distance education, and the video and internet linkswith regional centers. Currently it is constrained with what it can do, not only because it isdependent on normal government administrative procedures, but also because of shortages of fulltime professional staff. Regional staff can be recruited as their salaries will be funded throughregional incomes. The organization and staffing of IDE is outlined in Annex 11, and itsrelationships with the regional centers in Annex 12.

Draft legislation has been prepared to provide the college with academic, administrative, andfinancial autonomy. This legislation is along the lines being proposed for universities.

The needs assessment, which was carried out in October 2000, was designed to find out what werethe priority learning needs as expressed by learners and their employers, what they would beprepared to pay, how much time they could spend a week in following the courses, where theregional centers could be established, from where the necessary tutors could be drawn, and howmuch they would expect to be paid.

The IDE is developing skills in marketing its Distance Learning programs, and already has a demandfor more programs and training slots than can currently be offered. It will need to spend more timeon analyzing the demand for different subjects that can be taught through distance education,respond to this demand and insist on increasingly wide use of the new technologies by the distancelearners.

4. Financial Management

The project accounting system will be an integral part of the Financial Management System of theCollege and will provide timely and reliable information to monitor the project's implementationtowards its agreed objectives. The College will be responsible for all expenditure accounting,

- 6 -

including all financial transactions of the project, and the preparation of disbursement request. Aproject finance officer will be designated to exclusively handle project accounting functions underthe supervision of the Head of the Finance Division. Supporting documentation for all expenditureswill be kept for at least one year after the audit report is submitted to the Bank. A manual system ofdouble entry accounting would be used by the project at least for the LIL period and would move toa computerized accounting system in the next phase of the project.

To this effect, the College will prepare a simple financial management manual before anydisbursement is made from the proceeds of the credit (Annex 4). As there will be seven regionalcenters, and the centers would generate revenue from fees, and effect payments for tutors'honorarium and operating costs, the manual should clearly describe the flow of funds from thecenters to the College, and how payments could be made for costs incurred at the regional centers.

5. Procurement

The Administration and Finance department of CSC will manage all procurement activities and theProcurement and Property Administration Division that is under the Department shall be responsiblefor the processing of contracts from preparation of bid documents until award of contract andcompletion of services as per the contract. In order to enhance the capacity of the Department, aprocurement officer with adequate qualification will be employed in the division prior toeffectiveness. The Procurement officer should have a minimum of I"5 degree and at least 2 to 3 yearsof experience in procurement. An additional Logistics officer to handle all activities related totransport, installation of Goods will be assigned to the division at the time the Goods start to arrive atthe Center for the use of the center and the regional centers (Annex 6).

Procurement will be carried out by the Civil Service College on the following:

* preparation of annual work plans and procurement plans;

* development of Terms of Reference (TOR), short-lists, and Requests for Proposals (RFP)for consulting assignments, preparation of bidding documents, including technicalspecifications for goods contracts; advertising contracts (where appropriate); and evaluatingbids and proposals; preparing no objection letters for onward transmittal to the Bank forapproval;

* setting up a good procurement filing system, and maintaining procurement documentation;and

* preparation of plans for annual training programs that set out the objective of the training,areas of training, and the number of trainees, cost estimate, timing and venue (e.g. local oroverseas) for each training activity.

D. Project Rationale

1. Project Alternatives Considered and Reasonsfor Rejection

Initially it was planned to go ahead with only upgrading the GDLN site in the CSC in Addis.However, as a central part of the mandate of the CSC is to serve the development of the civilservices in the Regional States, especially those in emerging regions, any system that only helpstrain people in Addis Ababa misses the large proportion of civil servants most needing training. Ifthere had not already been considerable sunk capital in the GDLN it would have been difficult to

- 7 -

justify the costs of the network link, given the difficulty, in the Ethiopian context, to chargesufficiently high user fees to cover satellite rental and other operational costs. Government thenlooked at the economics of extending the GDLN and web based training to the regions, in order tomake the whole program cost effective.

Two-way video conferencing systems (which allow interactivity) currently cost more than threetimes as much as unidirectional systems to establish, and require well trained technicians tomaintain. It is thus not cost effective or feasible to establish and operate such sophisticated systemsin regional centers. A unidirectional video link (with multidirectional voice and internetconnections) is a much more cost effective option, and is also simpler to maintain. This was thechoice that was finally made for the regional connections.

Finally, only by expanding the use of the regional centers to also cover print based distance learningleading towards recognized diplomas, could a fully justified range of services be implemented.

2. Major Related Projects Financed by the Bank and other Development Agencies (Completed orongoing)

RELAT PROJEC %PN 'RA T G _ _ _ _

Education Sector Development Program U IDASeventh IDA Education Project (closed) U IDABasic Education Project, involving retraining 20,000 USAIDteachers through radio, printed media and tutor baseddistance methodsEquipping radio and TV production studios at the JICAEducation Media Agency in the Ministry of Education,and radio production studios at 7 Regional EducationBureau for distance education programs directed at schoolteachers and studentsOpen University support for management training UNDP/British Councilactivities in Management Training Institutes in theregional statesCivil Service Reform Program (including support for the UrNDPCivil Service College) IIP Rating: HS (Highly Satisfactory); S (Satisfactory); U (Unsatisfactory): HU (HighlyUnsatisfactory)

3. Lessons Learned and Reflected in Proposed Project Design

All of the distance learning projects funded by the World Bank are at their early stage ofimplementation and it is, therefore, not possible to reflect in the design of this project appropriatelessons learned from these operations. However, in order to carry out implementation activitiesmore efficiently, the project design has been kept simple, with IDE and the regional centers havingsignificant autonomy, and flexibility in administrative and financial management arrangements.Nevertheless, it should also be noted that given resource constraints it will never be possible to haveeither the personnel or the recurrent budget necessary for effective implementation for any high costand high tech project provided from Government budgets alone, and thus high levels participant costsharing will be expected from those benefiting in all the programs, with any excess capacitymarketed to the private and NGO sector. UNDP has been supporting the regular face-to-faceprograms at CSC almost since its inception five years ago, and still its support is needed. Six other

- 8 -

donors have either been supporting particular regular instruction, or helping finance advancedstudies for CSC teaching staff.

4. Indications of Borrower Commitment and Ownership

A joint meeting of the Board and the management team in the CSC formally approved the threemajor components of this project in April 2000. In addition, the CSC has allocated space on theground floor of the main building for the GDLN facilities, and one entire floor in the teaching blockfor IDE staff offices and meeting rooms. The college has also agreed to use a part of their limitedbudget to finance all regular staff salaries, both at the center and core staff in the regions, and someother operational costs needed for distance learning activities. The CSC had no reduction in itsoperation budget in FY 2000, despite the constrained fiscal position due to the conflict with Eritrea,and an increase of only 8.8% in 2001. Currently its recurrent budget is running at about ETB 12million p.a.

5. Value Added of Bank Support in this Project

The Bank is very much in the forefront of the multi donor efforts to establish a Global DistanceLearning Network, with Ethiopia as one of the first group of 10 countries in the network. It is alsoactively helping the Government in programs to improve the administrative and managementcapacity of federal and regional governments, and developing partnerships between the public andthe private sector. It has, in addition, considerable experience in financing print media baseddistance learning programs throughout the world. ISG has specially developed for Government thebest technical solutions for extending at least the key GDLN services to five regions. In addition theBank will be able to help obtain a VSAT operating license for the GDLN site in the CSC Addis, andfor the five regional centers.

E. Summary Project Analysis

1. Economic

The comparative costs of the various forms of distance learning proposed compared with the costs ofalternative delivery modes will be evaluated during implementation.

2. Financial

As far as the GDLN is concerned, the high costs of the satellite link and of the equipmentmaintenance, and the relatively low levels of utilization mean that fees of the order of $US100 perstudent videoconference hour will eventually need to be charged to reach a break even point. Thismeans between $400 and $800 for participation in a normal short course. This is very much higherthan the local capacity to pay. Currently the maximum level that can be charged seems to be of theorder of $150 per course. In the initial period the operational costs resulting from the project will besupported from the Credit on a declining scale (78%, 59%, and 35% in Year 1, 2, &3 respectively).In the longer run, with aggressive marketing of available services, especially internet courses forprivate individuals, about half the running costs of GDLN will be able to be metdirectly. Trainingfunds from donor agencies (i.e. "scholarships") will thus have to be obtained for the GDLNcomponent to be sustainable. The operational costs at the regional centers will be substantiallylower than those at the center, and with there being little competition for internet services in theregions, it should be possible to market the excess capacity to private internet users for 50 Birr anhour, generating about three quarters of the operation costs. For the print media distance learning,by year 3, with a fee of 800 Birr p.a. ($100), two thirds of the operational costs (excluding start-up

-9-

costs) will be met by the learner and his/her employer. The remaining expenditures will be supportedthrough the regular budget from the CSC. During the initial years the tutor costs of the printed mediaprograms will be financed through the project on the same declining basis as with the GDLN.

In total it is estimated that approximately $2.1 million will be generated from users during the life ofthe project, or 56% of total operation costs ($0.8 million from public sector users and $1.3 millionfrom private sector users).

3. Technical

The basic equipment for the Ethiopian GDLN center is already in place, has been used, has nowbeen well protected against power surges and failures and is working well. It merely needs to beupgraded. The technology for a modified GDLN link at the regional centers has been designedjointly by ISG and a satellite vendor, has been specially developed for the Ethiopian situation, and ifsuccessful will be a model that can be copied elsewhere. The regional centers will receive theirprograms through the GDLN hub in Washington, (and later from other hubs too), in the same way asthe Addis GDLN center, and will connect with the Addis center through this hub. The British OpenUniversity has been advising the CSC on the start up of the print media based distance learning, andmight be recruited (possibly on a sole source basis) to advise during execution. Other high leveltechnical assistance on the most appropriate use of the new technology will also be sought.

4. Institutional

Though the CSC has professional staff who are qualified to evaluate the use made by the graduatesof the skills they gain from CSC courses, up until recently there has been no systematic evaluation.However CSC has launched an impact evaluation of the first phase of the UNDP support for theCollege, which has provided staff with appropriate evaluation experience. Under this LIL there willneed to be continuous data collection and analyses on all the different courses offered, and theircosts. In addition at the mid-point of the project (around October 2002), and towards the end(around May 2004) there will be more detailed evaluations, involving external consultants.

Project Management. The management structure of the CSC has recently been strengthened withthe addition of two Vice Presidents, one for academic affairs, and the other for administration. Inaddition ten staff have been employed in the IDE, six of them heading the different units. They havehad considerable experience in working with several donors, both bilateral and multilateral, but withprocurement and finance managed by the donors themselves. Only with the UNDP project havethey had a dedicated accountant, and have recently taken over procurement and project managementresponsibility. Neither does the CSC have any experience in establishing or managing outreach orregional centers, but has worked with regional governments and institutions.

Procurement Issues. Though the IDE will be the implementation organization responsible for theoverall management of this LIL, as much as possible existing administrative structures in CSC willbe used. Procurement will thus be carried out by the Administration and Finance department of thecollege. The Procurement and Property Administration Division that is under that Department willbe responsible for the processing of contracts from preparation of bid documents until award ofcontract, and completion of services as per the contract. Its staff currently have no knowledge orexperience of World Bank procurement procedures. They will thus participate in Africa widetraining in World Bank procurement processes. They will also need training in contractmanagement, and logistics. If necessary, local consultants will be hired to enhance theimplementation capacity.

- 10-

Financial Management Issues. The issue of the lack of financial autonomy for the CSC, andwithin the CSC for the IDE, will be difficult to resolve until the Charter is approved. The Boardof the CSC approved in its January 2001 meeting that all revenues earned by IDE will not go intothe general revenue part of CSC, but will be kept for financing IDE expenditure. However, feesearned will still only be used to offset approved government budgetary allocations, and anyunused fees will have to be returned to the Ministry of Finance (MOF) at the end of any fiscalyear. This will complicate the decentralized operations at the regional centers, which will bealmost entirely financed through the fee incomes earned at these centers. As an interimarrangement, CSC has already requested MOF that fee income earned should be retained fromone year to the next even if 100% of the fees collected are offset against their regular budgets.IDA will be advised by the CSC as soon as a resolution is reached on this issue.

5. Environmental

There are no environmental risks of any kind that can be identified. There is no new construction,but only some rehabilitation/rewiring of existing buildings both in the center and in the regions.This is a training project.

6. Social

This project is not designed to target the poor, nor females, nor those from any particular ethnicgroup, but only those who are already in Federal and Regional Civil Service employment, and thosefrom the private sector who can afford to pay.

7. Safeguard Policies

Environmental Assessment (OD 4.01) NANatural Habitats (OP/BP/GP 4.04) NAForestry (OP 4.36) NAPest Management (OP 4.09) NACultural Property (OPN 11.03) NAIndigenous Peoples (OD 4.20) NAInvoluntary Resettlements (OP 4.30) NASafety of Dams (OP 4.37) NAProjects on International Waterways (OP 7.50) NAProjects in Disputed Areas (OP 7.60) NA

8. Business PoliciesBUSINESS POLICY 'PPLICA'LEFinancing of recurrent costs (OMS 10.02) YESCost sharing above country three-year average (OP/GP/BP 12.10) NORetroactive financing above normal limits (OP/GB/BP 12. 10) NOFinancial Management (OP/B 10.02) YESInvolvement of NGOs (GP 14.70) NOOther NO

F. Sustainability and Risks

1. Sustainability: N/A

2. Critical Risks

From Outputs to ObjectiveParticipants in interactive GDLN video M Involve users in the selection ofprograms will gain knowledge and skills programs.which will enable them to enhance their workefficiency. _

Participants in the long print media Distance M Involve employers in the design ofLearning courses will gain knowledge and the programsskills which will enable them to enhance theirjob efficiencyParticipants in the long print media courses M Make the language level in programswill complete their course relatively simple. Limit the quantity

of printed materials. Make tutorialsessions interesting.

Electronic Distance Learning approaches are S Ensure that all proposed participantscost effective in the Ethiopian situation complete a computer literacy course

and have easy access to the internet.Program evaluation will take place regularly, M There will be supervision missionsand notice will be taken of the evaluation which will emphasize the importancefindings in future planning activities constant monitoring and evaluationFrom Components to OutputsThere will be a sufficient number of S It is expected that WBI will arrangeappropriate GDLN video programs available for program content from otherto Ethiopia for there to be efficient use of the providers to be beamed through thefacilities. link.The programs being offered by GDLN are S Potential users involved in selectionrelevant to the needs of the potential learners. of programsThere will be sufficient demand for the print M Salary increments are dependent onmedia distance learning programs from those formal qualifications of civil servants.already employed at the regional level, at thefee being proposed.There will be sufficient demand from those S Easy internet access, and bulkfrom the regional governments for subscriptions to key courses to lowerparticipation in Web Based courses costs.Overall Risk Rating M

Risk Rating - H(High Risk), S (Substantial Risk), M (Modest Risk), N (Negligible or Low Risk)

3. Possible Controversial Aspects

Currently the GDLN site at the CSC is operating without the formal approval of the EthiopianTelecommunications Agency. The Ethiopian Telecommunications Agency (ETA) will need toapprove this site, and the five additional sites, at least for education and training purposes, as isallowed according to the Council of Ministers Regulation no 47/1999. The law mandates the ETA to

- 12 -

respond within 90 days to requests to establish private internet links. It should be emphasized thatthese VSAT links will only be used for education and training purposed.

G. Main Credit Conditions

Effectiveness

1. Project Account and has deposited therein the initial deposit;2. PIM and the PIP adopted, both in form and substance acceptable to IDA; and3. Appointment of a procurement officer with qualifications and experience acceptable to IDA.

H. Readiness for Implementation

1. (a) The civil engineering design documents for the first year's activities are complete andready for the start of project implementation.

X 1. (b) Not applicable.

X 2. The procurement documents for the first year's activities are complete and ready for the startof project implementation.

X 3. The Project Implementation Plan has been appraised and found to be realistic and ofsatisfactory quality.

4. The following items are lacking and are discussed under loan conditions (Section G):

1. Compliance with Bank Policies

X 1. This project complies with all applicable Bank policies.

2. The following exceptions to Bank policies are recommended for approval. The project_complies with all other applicable Bank policies.

. Nicholas Bennett John Room 0 Astra MeesookTeam Leader Sector Director Country Director

- 13 -

Annex 1:Project Design Summary

ETHIOPIA: Distance Learning LIL Project

Project Development , Objective:

Project will help improve Number of employed Government high level That increasednational capacity to people trained by Capacity Development training, and openingimplement projects and discipline and media Committee Reports to outside influencesprograms will enhance capacityOutputs:

Enhance the efficiency of 800 trained in diploma Special monitoring and For critical risks, seethe public and private programs in each of evaluation studies and Section F.2sectors staff; accountancy and reports, both by CSC staff

management; and independentevaluators will be

Develop the capacity of the Learning achievements of commissioned at the midIDE in the CSC to manage those trained equivalent point of the project, andDistance Education; to those in the full time at the end of the project,

programs; to determine whichaspects will be extended

Test comparative Demand for places more in future.effectiveness of alternative than actual places in eachDL approaches. distance course;

Dropout rates less than10% p.a;

More than 80% of thoseunder training have fullsets of curriculummaterials on time;

DL unit costs of differentprograms lower thanthose of full timeprograms.

Project Components/Sub- Inputs: (budget for each Project reports: (from Componentscomponents: component): to Outputs)

GDLN Facilities Upgrading US$1.0 million Evaluation reports The centers provide& Extension; comparing the total per- the services expected.

student recurrent costs,Satellite Rental, US$1.4 million and the proportion met byMaintenance, & Other students for differentOperating Costs for GDLN; types of training.

- 14-

Development of Printed US$1.3 million Visits to the sevenMedia Distance Learning; regional centers.

Training; US $0.8 million

Capacity Dev. & TA US $2.6 million

- 15 -

Annex 2:

Detailed Project DescriptionETHIOPIA: Distance Learning LIL Project

The main target audience of the Civil Service College are the 300,000 Federal and Regional civilservants, especially the 83% who do not even have a diploma, and those working in the regions.

Project Activities

A range of different Distance Learning approaches is being developed by the IDE to enable it toreach its huge potential audience of civil servants needing training in Addis Ababa and in sevenregional centers; including:

1. GDLN Facilities Upgrading and Extension (US$1.0 million):

* upgrading the existing GDLN videoconferencing and internet link at the CSC to the highestinternational standards, and to use both for international training, and for beaming localcourses to regional centers;

- establishing modified GDLN links with five regional centers, (Bahir Dar, Makelle, Nazareth,Awassa, and Dire Dawa) beaming programs developed at the IDE, and also drawn from theinternational GDLN, to these centers. The centers can be considered as extension arms ofthe IDE/GDLN, offering regional students the same programs as those in the CSC, by July2002. These regional centers will be established in existing tertiary level institutions, andthe rooms provided will only need wiring, computer networking, and security modifications.They will be equipped with computers (on which learners will view the video programs),and audio visual equipment for participant use;

2. Satellite Rental, Maintenance, and Other Operational Costs for GDLN Centers (US$1.3million):

* satellite rental, facility maintenance, and other related operational cost activities resultingfrom the central GDLN from March 2001. 22% of these costs to be generated from learnersin 2001/2, 41% in 2002/3, and 65% in 2003/4;

3. Development of Printed Media Distance Learning (US$1.3 million):

* developing printed distance learning materials, eventually in three disciplines, and launchingcourses in these (June 2002 diploma in accounting and diploma in management, and January2004 diploma in a third subject) to be administered through trained tutors at 7 selectedregional sites, with each enrolled learner taking three years to complete;

4. Training (US$0.8 million):

* running large scale computer literacy, and remedial English courses for civil servants at theregional centers on a full cost basis, designed to bring them into the computer age, andprepare them to benefit from the internet;

- 16 -

enrolling at least 1300 learners in the print based distance learning in 2002, 2600 in 2003,and 4700 in 2004. Participants and their employers will be expected to pay two thirds of thetotal running costs of their courses. On the basis of current estimates the participant share isexpected to be ETB 800 per year. On the basis of information collected during the needsassessment, ETB 600 could be paid by the employer per year;

5. Capacity Development and TA (US$2.6 million):

* Establishing two other regional centers without the GDLN satellite link (Jinga, and AddisAbaba), equipped with computers and audio visual equipment;

* providing technical assistance and training in effective use of interactive video and internet,the development and use of pent based correspondence courses, the development and useof internet courses, and training in procurement and financial management.

In the design of the print based correspondence programs there has been great attention devoted todesigning a program that is both feasible for the majority of potential learners who have limited time,and even more limited fluency in the English language, and to keep the costs at an affordable level.Thus the length of the standard course has been fixed at three years, the pages of core textssubstantially lower than in comparative programs, and the tutor learner ratio at 30:1.

Both printed media correspondence courses, and the use of modern ICT are expensive, and couldimpose an unbearable burden on the limited budgets of the CSC. There will thus be significant usercharges for all participants or their employers. In the longer run, with aggressive marketing ofavailable services, especially internet courses for private individuals, about half the running costs ofGDLN will be able to be met. The operational costs at the regional centers will be substantiallylower than those at the center, and with there being little competition for internet services in theregions, it should be possible to market the excess capacity to private internet users for 50 Birr anhour, generating about three quarters of the operation costs . For the print media distance learning,by year 3, with a fee of 800 Birr p.a. ($100), two thirds of the operational costs (excluding start-upcosts) will be met by the learner and his employer. The remaining expenditures will be supportedthrough the regular budget from the CSC. During the initial years the tutor costs of the printed mediaprograms will be financed through the project on the same declining basis as with the GDLN.

In total it is estimated that approximately $2,094,000 will be generated from users during the life ofthe project, or 56% of total operation costs ($829,000 from public sector users and $1,265,000 fromprivate sector users)

GDLN Technical Details

The following table summarizes the technical specifications of the existing GDLN facilities at theCSC, the proposed specifications of the upgraded facilities, and the proposed specifications of eachof the five regional centers.

GDLN Facility Antenna Video In Video Out Data In Data Out VoiceExisting 3.8 M 128/256 kbps 56/56 kbps 16 kbpsUpgraded 3.8 M 256/256 kbps 512/64 kbps 2*16 kbpsRegional 3.8 M 256 kbps (in only) 256/64 kbps 2 Lines

- 17 -

The upgraded GDLN facilities will allow clear video images on a large screen, multiple voicechannels, and up to 40 internet users at any one time. Each regional center will have clear videoimages on the PC, but no video interactivity, two voice channels, and up to 20 internet users at anyone time. If the need arises in future, the regional facilities can be upgraded into full GDLNcenters.

Project Evaluation

As a Learning and Innovation Loan (LIL) summative evaluation is of central importance. A localinstitution or firm will be hired to carry out this evaluation in two stages. First, after two years ofimplementation and secondly at the end of the project. The major issues that will be looked at willinclude, for each major subject area being taught, the following:

* Learning achievement by course, delivery mode, and tutoring support;* Learner satisfaction by course, delivery mode, and tutoring support;* Time to completion by course, delivery mode, and tutoring support;* Cost effectiveness of learner achievement by delivery mode.

In addition to this summative evaluation which will be used to determine the future direction ofdistance and virtual learning activities, CSC/IDE will also carry out various formative studies(starting with a pilot testing of all materials developed) to help it improve the effectiveness oflearning during implementation.

Role of the Regional Centers and Their Relationship with CSC

The operation of the regional centers is relatively complicated. On average, by the end of theproject, each regional center will have around 700 learners and 130 tutors in the printed mediaprograms, and perhaps an equivalent number in computer literacy and miscellaneous internet-basedcourses. The centers will have to collect fees from all the regular users, and pay all the tutors etc.They will also select candidates for the courses; distribute materials; arrange for classrooms fortutorial sessions, organize examinations, and keep records of all students. In addition they will needto provide assistance and maintain equipment. On top of all this regional centers with video andinternet facilities will need to market and sell substantial underutilized time to the private sector andto individuals, operate the facilities 12 hours a day, seven days a week, and collect and deposit thefees. A review of the staffing requirements of the regional centers is needed as soon as possible, todetermine the real staffing requirements. In addition the financial control systems will be elaborated.Finally guidelines for the operation of the regional centers will be finalized as soon as possible, anddiscussed with the Bank and other stakeholders before any regional centers are formally operational .

Capacity Development of CSC/IDE Through Technical Assistance

It is extremely important that the capacity of the CSC/lDE is enhanced to enable it to design andmanage the various forms of distance learning being tested, so when a decision is made, at the end ofthis LIL, as to the most appropriate media to be used, it will be ready to move ahead in the chosenmedia. It is also important that the distance facilities already available at the CSC/IDE (internet andvideo conference) be used to provide much of the TA needed, thus obviating the need to pay fortravel and per diem of experts.

For the print media course development and implementation, an institutional contract will be givento an organization experienced in the provision of printed based correspondences. This organization

- 18 -

will be responsible for the training of the CSC/IDE staff in the design and running of printed mediaDL courses, the training and initial supervision of course material developers, the design and initialrunning of tutor training, the choice of appropriate pedagogic strategies, strategic planning, andlearner assessment

For the internet-based course development, training to a significant number of CSC/IDE staff shouldbe given through the internet. In addition, an internet-based technical assistance service will berecruited to provide advice and assistance in the design, organization and management of internet-based courses by the CSC. Finally advice will be sought on the most appropriate internet-basedcourses that could be used by Federal and Regional civil servants in upgrading their knowledge andskills.

One of the problems with the interactive VC facilities that will be upgraded through this project isthat they are currently not used frequently. The same will apply to the new facilities beingestablished at the regional centers. It is intended to recruit a short-term consultant on the cuttingedge of this technology to advise on some of the most appropriate program providers, and tonegotiate appropriate terms for using these courses on behalf of the CSC/IDE.

Project Administrative Arrangements

The project will be implemented mainly through existing CSC structures. There will be no projectmanagement or implementation unit. Responsibility will be divided between the IDE (for academicimplementation and marketing of services); and the Administration and Finance Department (forprocurement, budget, finance, disbursement, and auditing aspects of the project).

The IDE will need to expand its staff substantially so that it can work on the development of internetand IT courses, and market the spare IT capacity it has available both at the center and in the regions.The Administration and Finance Department will need to recruit a procurement specialist, and aliaison officer, and to provide training for the procurement specialist and his assistant, andorientation for the accounts staff in Bank accounting and disbursement requirements. Theprocurement specialist will have to be recruited as soon as possible so that procurement of essentialequipment can start. Other training will be given to administrative staff as necessary for effectiveimplementation

The draft Project Implementation Manual explains some of the procedures that will need to befollowed. This manual needs further elaboration. It is important to note that standard procurementcontracts for the upgrading of existing GDLN facilities, and the establishment and installation ofregional center equipment will be used, and thus no bidding will be necessary. Similarly standardmaintenance contracts for the GDLN will be used.

- 19-

Annex 3:

Estimated Project CostsETHIOPIA: Distance Learning LIL Project

(US$ million)

1. GDLN Facilities Upgrading & Extension 0.2 _ . 1.02. Satellite Rental, Maint., & Other Oper. Costs for 1.3 GDLN Centers3. Dev. Of Printed Media Distance Learning 0.6 0.6 1- 24. Training 0.7 - 0.

5. Capacity Development & TA 0.3 2.1 2.4

Total Baseline Cost 1.8 4.8 6.6

Physical & Price Contingency 0.1 0.4 0.5

Total Project Cost 1.9 5.2 7.1

Goods 0.1 1.1 1.2Works 0.1 0.2 0.3Services 0.3.3Training 0.2 0.2Operating Costs 0.8 1.3 2.1

Total Project Costs 1.9 5.2 7.1

- 20 -

Annex 4

ETHIOPIA: Distance Learning LIL ProjectEconomic Justification Summary

NOT APPLICABLE

-21 -

Annex 5Financial Summary

ETHIOPIA: Distance Learning LIL Project

Project Costs (US$ million)

Investment Costs 1.3 2.1 1.5

Recurrent Costs 0.8 0.8 0.5

Total 2.1 3.0 2.0

Financing Sources (% of Total)

IDA 62% 72% 74%Benefiaries/Government 38% 28% 26%

Total 100% 100% 100%

- 22 -

Annex 5a

Financial Management AssessmentETHIOPIA: Distance Learning LIL Project

FINANCIAL MANAGEMENT SYSTEM

Existing Financial Management System of the College

The implementing agency of the project is the Ethiopian Civil Services College (CSC) through theInstitute of Distance Learning (IDE). The College, like other ministries and commissions, obtains thelarge part of its annual budget from the Federal Government of Ethiopia, Ministry of Finance(MOF). The remaining part of the annual budget is from its own sources and donors. In fiscal year1999/2000, 65% of the total annual budget was from MOF, 24% from own sources and 11% fromUNDP. A cash basis of accounting is used by the College. The receipts of cash and withdrawals arerecorded in standardized journals issued by MOF. The CSC follows the financial regulations ofMOF. No annual accounts are prepared by the College. The Ministry of Finance is responsible forthe compilation of annual accounts.

Currently, the College is administering a UNDP Fund amounting to US$3.2 million. The Collegeregularly reports to UNDP on a standardized format in a very simplified way. Annual accounts of theproject have been audited by external auditors appointed by UNDP.

The Finance Division is under the Finance and Administration Department, and has two sections,Accounts and Budget. The Finance Division comprises 9 staff, one BA holder, 6 diploma holdersfrom Commercial Colleges and 2 with vocational training. The practical experience for most of themis on a cash basis of accounting, based on the MOF standard formats.

Internal Control

The College has an Internal Audit Division consisting of two staff, 1 BA and I diploma holder. Thedivision usually conducts pre and post audit activities. There is a strong push from the MOF toengage the internal auditors in the post audit activities. The MOF inspectors review the financialtransactions of the College on a regular basis and the Federal Auditor General also conducts annualaudits as and when they feel necessary.

LACI CapabilityThe capacity of the college to handle LACI/FMI procedures as outlined in the LACI handbook islimited as discussed above. Thus, the College is not ready for PMR based disbursements based onthe LACI guidelines of the Bank. Disbursements will begin using the traditional disbursementprocedures, which is currently in place in the Bank, and will continue up to approximately 18months after effectiveness. Under the existing disbursement procedures, payments from the proceedsof the credit would be made by direct payments, special commitments and SOE procedures. Detailedguidance could be obtained from the Disbursement Handbook issued by the Bank and training wouldbe given at the project launch workshop and other training to be held in Africa. The following stepsshould be undertaken before the College is able to meet PMR based disbursements:

Appointment of a Project Finance Officer .......................... Underway/Before effectiveness(To deal with all donor funds)

Development of a financial management manual ............... Before Disbursement starts

-23 -

Evaluation of the College's capacity by the Bankstaff to determine compliance withthe LA CI guidelines .................................... 12 months after effectiveness

Preparation of draft PMR for initial review .......................... 15 months after effectiveness

Production of PMRs .................................... 21 months after effectiveness

Provided these steps are taken, it is expected that the College might be LACI capable 18 monthsfrom the date of effectiveness.

Accounting, Financial reporting and Auditing:

Accounting:

The project accounting system will be an integral part of the Financial Management System of theCollege and will provide timely and reliable information to monitor the project's implementationtowards its agreed objectives and also give early warning of problems in project implementation.The College will be responsible for all expenditure accounting, including all financial transactionsof the project. A project finance officer will be designated to exclusively handle project accountingfunctions under the supervision of the Head of the Finance Division. A manual system of doubleentry accounting would be used by the project at least for the LIL period and would move to acomputerized accounting system in the next phase of the project. To this effect, the College wouldprepare a simple financial management manual before any disbursement is made from the proceedsof the credit. The manual will, among other things, include flow of funds, formnats to be used, chartof accounts, formats of annual financial statements, procedures in managing bank accounts,accounting policies and procedures, documentation and others. As there would be seven regionalcenters for distance learning purposes, and the centers would generate revenue from fees, and effectpayments for tutors' honorarium and operating costs, the manual should clearly describe the flow offunds from the centers to the College, and how payments could be made for costs incurred at theregional centers.

The college has prepared a Project Implementation Manual to be used as a guide by theimplementers of the project. The manual needs some revision in the areas of the financialmanagement sections.

Financial Reporting:

Under the existing disbursement procedures

The college will prepare quarterly and annual financial reports to be submitted to IDA for its review.The annual financial reports will include the following:

-Balance sheet-Income and Expenditures Statement-Special Account movement-Statements of Expenditures

Specific guidelines could be obtained from the Financial Accounting, Reporting and AuditingHandbook issued by the Bank.

- 24 -

Under PMR based disbursement procedures of LA CI

* Sources and uses of Funds* Balance Sheet* Uses of Funds by Project Activity - Showing the expenditure as per project activities

agreed with the Bank* Cash withdrawal (Disbursement) - Showing summary withdrawals* Special Account Statement - Showing reconciliation of IDA Project funds* Cash Forecast - Showing cash requirements in next 2 quarters

Specific guidelines could be obtained from the LACI Handbook issued by the Bank.

Auditing:

According to the Bank's policy, the required financial statements including the Special Account andSOEs for each year should be audited in accordance with standards on auditing that are acceptable tothe Bank. An audit of such financial statements includes:

* An assessment of the adequacy of accounting and internal control systems to monitorexpenditures and other financial transactions and ensure safe custody of project-financedassets;

* A determination as to whether the CSC have maintained adequate documentation on allrelevant transactions; and

* Verification of expenditures submitted to the Bank, which are eligible for financing, andidentification of any ineligible expenditures.

The audited financial statements should be submitted to the Bank six months after the close of eachfiscal year. The auditors are also required to issue a management letter (long form report), whichcontains weaknesses observed in the accounting and internal control systems, and accounting recordsand their recommendations for improvements.

The college will ensure that an independent auditor acceptable to the Bank is appointed in sufficienttime to carry out its responsibilities.

- 25 -

Annex 6Procurement & Disbursement

ETHIOPIA: Distance Learning LIL Project

The Administration and Finance department of CSC will manage all procurement activities and theProcurement and Property Administration Division that is under the Department will be responsiblefor the processing of contracts from preparation of bid documents until award of contract andcompletion of services as per the contract.

Capacity Development

In order to enhance the capacity of the Department a procurement officer with adequate qualificationwill be employed in the division prior to the date of credit effectiveness. The Procurement officershould have a minimum of 1st degree and at least 2 to 3 years experience in procurement. Anadditional Logistics officer to handle all activities related to transport, installation of Goods will beassigned to the division at the time the Goods start to arrive at the CSC for the use of the center andthe regional centers.

Procurement Records Management and Procurement Register

The Project will maintain an efficient procurement record management system. Documentsregarding individual procurement contracts will be kept in separate files and shall include alldocuments pertaining to the procurement process. The files will include Bid documents.

RFPs, Solicitation letters/ Invitation for Bid, Tender announcements, evaluation Reports, Contactagreements, Bank No Objection Letters, Inspection Reports, Goods Receipt Vouchers, CompletionReports and other relevant document in one location.

The Division will maintain a register of procurement activities carried out by method of procurementboth in hard copy and electronic format. These would include Name of Contract, Description,Quantity, Contract Value in currency of contract and USD, No Objection Date/s, Contract AwardDate and Contract Completion date. The Register will be updated on a regular basis and checked toinsure that thresholds and aggregate limits are not exceeded.

Tender Committee

A tender committee in which the procurement officer is a member will be established and will beresponsible for the opening and evaluation of tenders. The committee shall have a minimum of 4members. A technical committee will be formed for each tender under evaluation on ad-hoc basis. Atleast two members of the committee should have adequate technical knowledge and qualificationregarding the specific type of procurement package and should be able to conduct a thoroughtechnical evaluation.

Training

Staff involved in the procurement process should, as early as possible, participate in procurementtraining conducted at the regional level (ESAMI or GIMPA). Project funds are available to financethis training. The other staff of the department should participate in appropriate national training toinsure that they have adequate capacity to handle their individual duties and responsibilities.Training need shall be identified after reviewing the current capacity and qualification of each of thestaff.

-26 -

Procurement Plan

A preliminary general procurement plan identifying the methods of procurement and thresholds waspresented at negotiations. Prior to inviting tenders for each fiscal year a detailed procurement planwill be furnished to the association for its review and approval. The procurement plan for the firstproject year will be presented to the association satisfactory in form and substance prior to thebeginning of the 2001/02 fiscal year.

Procurement of Goods and Equipment

Goods will be procured through Internal Competitive Bidding and to the extent practicable, contractsfor goods shall be grouped in bid packages estimated to cost $100,000 equivalent or more each. Forthe procurement of Global Distance Learning Link equipment, the World Bank will provide thedetailed specifications of this equipment, and assistance in evaluating the offers in order to ensureconsistency with other GDLN sites.

Goods estimated to cost less than $50,000 equivalent per contract, up to an aggregate amount not toexceed $400,000 equivalent works estimated to cost less than $20,000 equivalent per contract, up toan aggregate amount not to exceed $100,000 equivalent may be procured through NationalCompetitive Bidding procedures.

Printing services may also be procured through national Competitive Bidding procedures.

Goods, including books, estimated to cost less than $30,000 equivalent per contract, up to anaggregate amount not to exceed $200,000 equivalent, may be procured under contracts awarded onthe basis of international or national shopping procedures.

Civil works estimated to cost less than $20,000 equivalent per contract, up to an aggregate amountnot to exceed $100,000 equivalent and furniture, may be procured under contracts awarded on thebasis of national shopping procedures.

Books which must be purchased from the original publisher, and equipment and spare parts whichneed to be compatible with existing equipment of a proprietary nature, subject to IDA's priorapproval, may be procured through Direct Contracting.

Procurement through UNAgencies

Vehicles, computers and office equipment estimated to cost $70,000 equivalent or less per contractup to an aggregate amount not to exceed $100,000 equivalent, may be procured from the Inter-Agency Procurement Services Office (IAPSO) of the United Nations Development Program(UNDP).

Procurement of Works

Civil Works to modify Regional Distance Learning Centers not exceeding USD 20,000 up to anaggregate amount not exceeding USD 100,000 may be procured using National Shopping. Contractsfor Civil works above USD 20,000 up to an aggregate amount not exceeding UD 100,000 may beprocured using National Competitive Procedures.

- 27 -

Consultants' Services

Quality- and Cost-based Selection

Consultant's services shall be procured by using Quality and Cost Based Selection (QCBS) Methodexcept as otherwise provided in the following paragraphs. The short list of consultants for servicessuch as Preparation of Evaluation reports and technical documents estimated to cost less thanUS$50,000 equivalent per contract, may entirely consist of national consultants in accordance withthe provisions of paragraph 2.7 of the Consultants Guidelines.

Other Procedures for the Selection of Consultants

Least-cost Selection. Services for audits estimated to cost less than US$ 50,000 equivalent percontract may be procured under contracts awarded using least-cost selection method in accordancewith the provisions of paragraphs 3.1 and 3.6 of the Consultant Guidelines.

Selection Based on Consultants' Qualifications: Services for technical assistance under the CapacityDevelopment Component of the Project estimated to cost less than $50,000 equivalent per contractmay be procured using procedures under contracts awarded through "Selection Based onConsultants' Qualifications" method

Single Source Selection. Services which are estimated to cost less than $200,000 equivalent percontract or more in exceptional circumstances, may, with IDA's prior agreement, be procured onSingle Source Selection methods.

Individual Consultants. Services for tasks that meet the requirements set forth in paragraph 5.1 ofthe Consultant Guidelines may be procured under contracts awarded to individual consultants inaccordance with the provisions of paragraphs 5.1 through 5.3 of the Consultant Guidelines.

Review by the Bank

Prior Review

Prior to commencing any procurement activity for Goods, Works and Services the proposedprocurement plan shall be presented to the Bank for its review and approval. The proposed plan forthe selection of consultants shall be reviewed in accordance with the provisions of paragraph 1 ofAppendix I to the Consultant Guidelines.

The first two procurement packages for goods, works or services irrespective of their value will bereviewed by the IDA in accordance with the Banks Prior Review Procedures.

Goods and Works

IDA's prior review procedures would apply to all purchases for which the contract value isequivalent or in excess of US$50,000 for goods and works.

With respect to each contract for goods to be procured through direct contracting and estimated tocost the equivalent of $20,000 or more, the following procedures will be followed:

* Prior to the execution of any contract under shopping procedures, IDA will be provided a reporton the comparison and evaluation of quotations received;

- 28 -

* Prior to the execution of any contract procured under direct contracting procedures, IDA will beprovided a copy of the specifications and the draft contract; and

* The procedures set forth in paragraphs 2(f), 2(g) and 3 of Appendix I to the Guidelines shallapply.

Consultant Services

The Bank shall review the Terms of Reference for all Consultancy services. Contracts for servicesabove USD 50,000 for firms and USD 20,000 for Individuals require the prior no objection by theBank.

Post Review

With respect to each contract not subject to prior review post review shall be conducted inaccordance with the procedures set forth in paragraph 4 of Appendix I to the Guidelines.

Table Al: Project Costs by Procurement Arrangements(US$ Million)

- ~ (0.1) (0. 1) (0.2)

Goods, Equipment & Vehicles 1.6 0.4 0.1 2.1(1.5) (0.4) (0.1) (2.0)

Consultant Services 1.7 1.7(1.6) (1.6)

Training 0.1 0.1

(0.1) (0.1)

Operating Costs 3.0 3.0

(1.0) (1.0)

Total Project Cost 1.6 0.6 4.9 7.1

of which IDA Funded by IDA (1.5) (0.5) (2.9) (4.9)

- 29 -

Table A2: Consultants Selection Arrangements(US$ Million)

. 8.Se ,, -Mtbod -otal

__ 7~~k7l,1& 7T

A. Firms 1.3 l 1.3

(1.3) (1.3)

B. Individuals 0.4 0.4

(0.4) (0.4)

Total 1.3 0.4 1.7

Of which Funded by IDA (1.3) (0.4) (1.7)

QCBS = Quality - and Cost-Based SelectionLC = Least Cost SectionIC = Individual ConsultantsOthers = Selection of individual consultants (per Section V of Consultants' Guidelines),

Commercial Practices, Educational Institutions, etc.

Table B: Thresholds for Procurement Methods and Prior Review(US$ Million)

~~~~~~~~~~4 ~ ~ ~ ~ ~ ~ ~ ~ ,

Any Category. First two Prior Reviewpackages.

1. Goods > US$50,000 ICB Prior reviewLess than US$50,000 NCB Post review

Less than US$30,000 NIS Post reviewPrinting Services All NCB Post review2. Civil Works Less than US$ 100,000 NCB Post review

Less than US$ 20,000.00 NS

3. ServicesFirns: > US$ 50,000 QCBS Prior review

Below US$ 50,000 QCBS/LC/Other Post Review

Individuals: >US$5 0,000 IC Prior review

Less than US$50,000 IC Post review

Overall Procurement Risk Assessment:

HighAverageLow

- 30 -

Risk Mitigation Measures

In order to enhance the procurement capacity of the Administration and Finance Department, aprocurement officer with adequate qualification will be employed in the division. The Procurementofficer should have a minimum of 1st degree and at least 2 to 3 years experience in procurement. Hewill be recruited prior to effectiveness

Staff involved in the procurement process should, as early as possible, participate in procurementtraining (financed with project funds) conducted at the regional level (ESAMI or GIMPA). Trainingneed shall be identified after reviewing the current capacity and qualification of each of the staff.

Prior to inviting tenders for each year a detailed procurement plan will be furnished to theassociation for its review and approval.

Frequency of procurement supervision missions proposed: Once every 3 months during the firstyear of implementation and once every six months thereafter (includes special procurementsupervision for post-review/audits).

-31 -

Disbursement

Allocation of credit proceeds (Table C)

It is estimated that the project would be implemented over a three and a half year period starting onApril 1, 2001 and the closing of the project is set for July 30 2004. At this stage, the project is notready for disbursement according to Project Management Reports (PMR) as discussed in the Bank'sLoan Administrative Change Initiative handbook (LACI). Therefore, disbursement would be madeagainst standard IDA documentation using traditional disbursement procedures as outlined in theBank's disbursement handbook, at least during the first 18 months of implementation. For theremaining period of the project, disbursement might be based on quarterly PMR after the assessmentof the financial management systems of the College by the Bank.

Table A below sets out categories of expenditures to be financed by the Bank, allocation of theamounts to each category and the percentage of expenditures to be financed by the Bank for eachcategory:

Table C: Allocation of Credit Proceeds

1. Goods, Equipment & Vehicles 1.8 100%

2. Civil Works 0.2 100% of foreign and 80% local3. Consultant Services & Training 1.5 100% of Foreign & 80% Local

4. Operating Costs GDLN Centers 0.7 78% until July 7, 2002;59% from July 8, 2002 to July7, 2003; and35% thereafter.

Operating Costs printed DL 0.3 78% until July 7, 2002;59% from July 8, 2002 to July 7, 2003; and35% thereafter.

5. Unallocated 0.4

TOTAL 4.9

Use of Statements of Expenditures (SOEs):

Disbursements would be fully documented except for contracts below the equivalent ofUS$20,000 for individual consultants and technical assistance and US$50,000 for civil works, goodsand consulting firms. For such contracts, the borrower would be allowed to submit withdrawalapplications based on Statement of Expenditures (SOEs). All supporting documents for suchapplications would be retained by the College and made readily available for review by periodic IDAsupervision missions and extemal auditors. All expenditures related to contracts above the equivalentof US20,000 for individual consultants and technical assistance and US$50,000 for civil works,goods and consulting firms would be fully documented, and would not be able to be paid fromSOEs.

- 32 -

Special Account (SA):

To facilitate payment of expenditures by the project, Special Account in US dollars would beestablished in the National Bank of Ethiopia or in a commercial bank acceptable to IDA. TheAuthorized Allocation to the Special Account is US$500,000. IDA would make initial estimateddeposit of US$250,000 from the Credit immediately after effectiveness until the aggregate amount ofwithdrawals from the Credit Account plus the total amount of all outstanding special commitmentsentered into by IDA pursuant to Section 5.02 of the General Conditions shall be equal to or exceedthe equivalent of SDR 800,000.

Flow offunds:

The Bank's share of eligible expenditures would be paid from the Special Account, except forexpenditures greater than 10% of the outstanding Special Account advance, which could be madedirectly by IDA.

Project Accounts/Counterpart Fund Accounts

A project accounts would be opened at the National Bank of Ethiopia or at a commercial bank underterms and conditions satisfactory to IDA. The project account would be used for deposits andwithdrawals related to fees collected by the College, and with government contributions. It has beenagreed during appraisal that Birr 1.5 million would be deposited in the project account for the firstyear of implementation, and subsequent deposits would be made on the basis of fee incomes andannual project budgets. It has also been agreed that the College will design a simple, but efficientsystem of fund flow for revenue generated internally by regional centers.

The College would ensure that disbursements are effected in accordance with Bank procedures andwould have financial monitoring responsibility.

- 33 -

Annex 7:

Project Processing and Budget ScheduleETHIOPIA: Distance Learning LIL Project

Project Schedule Plhaned Actual

Time taken to prepare the project (months) 4 Months 12 MonthsFirst Bank mission (identification) Not Planned August 1999Appraisal mission departure November 2000 November 2000Negotiations January 2001 March 21, 2001Planned Date of Effectiveness April 2001

Bank Staff who worked on the Project included:

Name Rank Borrnwer Spec Wty_ ,______ __ I_Unit Agenc LNicholas Bennett AFMET _Task Team LeaderJoanne Caper HDNED ______Distance Leamning AlternativesSivaramen Ramaswawmy ISGNC Upgrading GDLN Facilities, Regional

I__= =_____ Center ConnectivityAbebaw Alemayehu AFMET _ Evaluation & Monitoring SpecialistSamuel Haile Selassie AFMET | Procurement SpecialistEshetu Yimer AFMET Costing, Accounting, Disbursements, AuditRamesh Ramaswamy r | Regional Center ConnectivityRey Castro AFTI2 Operations SpecialistSolange Alliali LEGOP Sr. Legal CounselKafu Awunyo LEGOP Legal CounselSteve Gaginis LOAG2 _ Disbursement OfficerSubhash Dhingra AFTQK Sr. Procurement Specialist

- 34 -

Annex 8:

Documents in the Project File*ETHIOPIA: Distance Learning LIL Project

A. Project Implementation Documents

Project Implementation ManualFinancial Management Manual

B. Bank Staff Assessments

Procurement AssessmentFinancial Assessment

- 35 -

Annex 9: Statement of Loans and CreditsETHIOPIA: Distance Learning LIL Project

as of March, 2001

Original Amounts Difference between ExpectedIn US$ million and Acttual Disbursements

Project FY Purpose IDA Cancel Undisb Orig F|ormallyID_ Revised

P050342 2001 WOMEN'S DEVELOPMENT INITIATIVES 5.00 0.00 4.98 0.27 0.00P067084 2001 EMERGENCY RECOVERY AND REHAB. 230.00 0.00 234.72 0.00 0.00P073196 2001 DEMOBILIZATION & INTEGRATION 170.60 0.00 174.15 0.00 0.00P000756 1999 HEALTH SECTOR 100.00 0.00 74.33 30.88 0.00P000732 1998 EDUCATION SECTOR INVESTMENT 100.00 0.00 66.00 27.04 0.00P000755 1998 ROAD SEC. DEV. PROG. 309.20 0.00 229.04 146.95 0.00P000733 1998 AG: RESEARC & TRAIN 60.00 0.00 45.81 13.23 0.00P000771 1996 ESRF 1 120.00 11.48 42.98 62.49 40.87P000764 1996 WATER SUPPLY DEV&REH 35.70 0.00 15.12 19.12 0.00P000752 1995 NATIONAL SEEDS PROJECT 22.00 0.00 10.51 12.27 0.00P000753 1995 NAT. FERTILIZER PROJ 120.00 0.08 19.49 26.41 0.00P000734 1993 ROAD REHABILITATION 96.00 0.00 27.84 33.33 0.00

I F 1356 94497 372.00 40.87

STATEMENT OF IFC'sHeld and Disbursed Portfolio

Mar-2001In Millions US Dollars

Committed DisbursedIFC IFC

FY Approval Company Loan Equity Quasi Partic Loan Equity Quasi Partic

Total Portfolio: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

Approvals Pending Commitment

FY Approval Company Loan Equity Quasi Partic1999 Sheba Tannery 5,000.00 0.00 3,570.00 0.00

Total Pending Commitment: 5,000.00 0.00 3,570.00 0.00

- 36 -

Annex 10: Country at a GlanceETHIOPIA: Distance Learning LIL Project

Sub-POVERTY and SOCIAL Saharan Low-

Ethiopia Africa income Development diamond1999Population, mid-year (Jn&12±!1 62.8 642 2,417 LiFe expectancyGNp per capita (Adacmerteo( 1'S*? 100 500 410GNP (tAdasmncfho4 sMC?A 6.5 321 988

Aveerage annual growth. 1993-99

Population J.-,? 2.7 2.6 tS Labor Force {e-? 2.3 2.6 2.3 GNP / Gross

per I a= IprimaryMost recent estimate (latest gear available. 1993-88) capita nroliment

Poverty f.a dakbintwdowM 45Urban population i3ct&rdpcxnaMt,/ 17 34 31Life expectaney at birth iVeM-¶/ 43 s0 60Infant mortality (pn2x?SIeAW&c 107 92 77Child malnutrition {xcV4ens0o@r5 .. 32 43 Access to safe waterAccess to improved water source fx ffpcWerJ 27 43 64Illiteraey (?cfpcdwe 09 63 39 39Gross primary enrollmeni fdM ep afkvV 43 78 96 ---

Male 55 85 102 _____

Female 31 71 86

KEY ECONOMIC RATIOS and LONG-TERM TRENDS

1979 1089 1998 1999Economic ratioS'

GOP jtSSA&xc/.. 8.1 6.6 6.5Gross domestic investmentlGOP .. 13.5 17.2 17.8Exports of goods and serviceslGDP .. 9.2 15.8 13.7 TradeGross domestic savingstGDP .. 7.9 5.3 2.7Gross national savingsGfOP . S.8 9.3 7.1

Current account balancetGOP .. -3.6 -7.9 -10.4Interest paymentslGDP - 0.9 1.6 1.2 Oomestic rInvestmentTotal debttGDP .. 96.8 149.5 141.9 SavingsTotal debt servicete2ports 62 40.2 3.1 17.6 1Present value of debtIGOP .. .. 120.5 ..Present value of debtlexports .. .. 751.5

Indebtedness1979-83 1989-99 1998 1999 1999-03

GOP 0.8 4.2 -1.0 5.8 5.6 Eri*wGNP per capita -1.9 0.9 -4.2 3.3 3.1 -Exports of goods and services 2.8 5.6 -7.8 -9.3 7.0

STRUCTURE of the ECONOMY1979 l18S 19S8 1909 Growth of investment and GOP (X)

Agriculture .. 48.5 52.3 52.3 100Industry .. 14.2 11.1 11.1 sX

Manufacturing - 8.5 7.0 7.0Services .. 37.3 36.5 36.5 o -

Private consumption .. 73.5 80.4 81.3 .;0 94 95 " I$General government consumption .. 18.7 14.2 16.0Imports of goods and services .. 14.8 27.7 28.8 -GDI 0DP

faxwaoe 1979-89 1989-99 1998 1999 Growth of emports and imports (XI

AgricultuTe -0.4 2.5 -10.3 4.2 100 TIndustry 1.2 4.6 6.3 7.6 7sManufacturing -0.9 4.8 5.8 4.9

Services 2.8 5.8 10.4 8.2

Private consumption -0.3 3.4 -2.4 4.7General government consumption 4.4 2.7 30.6 18.5 oGross domestic investment 3.4 9.9 0.1 9.5 -9 1 99Imports of goods and services 2.5 4.0 11.7 6.2 --- Exports -O- ImportsGross national product 0.7 4.2 -1.8 5.9

Note: 1999 data are preliminary estimates.'The diamonds show four key indicators in the country (in bold) compared with its income-group average. If data are missing, the diamond will

be incomplete.

- 37 -

Ethiopia

PRICES and GOVERNMENT FINANCEDo ries1378 1989 1S98 1999 Inflation [X)

isy

Consumer prices 16.0 9.6 3.7 4.2 10

Implicit GDP deflator . 4.3 9.7 33 s J

CfoweerA baacf o aA0-e

Current revenue .. 23.2 18.0 17.5 -10

Current budget balance .. 0.6 2.3 -0.1Overall surplusideficit .. -10.9 -6.5 -7.3 GDP del l.tor CPI

TRADE1979 19889 S98 1999 Emport and import levels (USS mill-)

Total exports (Fob) 360 444 602 484Coffee .. 303 420 281Hides .. 60 51 57 1,500

ManufacturesTotal imports (ci) 589 1,020 1.519 1.570 *Food .. 186 172 172Fuel and energg 103 246 194Capital goods .. 397 569 651 o

Export price index (/SW!A.t?. 82 97 79 93 94 95 91 9? 0* 9Import price index 1qttirA%?j 117 104 101 * Export: * Imporst

Terms of trade (AtFtL-At .. 70 93 79

BALANCE oF PAYMENTS1979 1989 199S 1999 Cuorent account balance to GOP (x)

Exports oF goods and services 455 752 1.037 894 Imports of goods and services 660 1,201 1.815 1,866 2 rResource balance -205 -449 -778 -972

Net income -3 -83 41 -85-Net current transfers 83 238 349 374

Current account balance -124 -294 -520 -683 IFinancing items (net) 79 284 650 633 -

Changes in net reserves 45 10 -129 49 -12

A*Ar:Reserves including gold (i$tm&ac/ 319 123 412 434Conversion rate (LIFC, caWl .1 2.1 2.1 8.9 7.5

EXTERNAL DEBT and RESOURCE FLOWS1979 1989 1998 1999

fti cM |-"Af l Composition oF 1999 debt (US* milL)Total debt outstanding and disbursed 740 7.842 9.812 9,288

IBRD 59 31 0 0 G: 613

IDA 221 718 1,586 1.948 F: 333 tS48

Total debt service 29 304 33 160IBR0 9 13 0 0IDA 2 9 28 31 C: 127

Composition of net resource FlowsOfficial grants .. .. 418Official creditors 95 402 -209 -267 0:1;7 32

Private creditors 6 -70 -6 -10 0: 1.5'Foreign diect investment .. .. 4 .: 4,72?Portfolio equity .. .. 0

World Bank programCommitments 0 72 609 160 A - IBRD E - Bil4tenrl

Disbursements 47 70 72 148 B - IDA D - Other multilateral F - Private

Principal repagments 4 14 17 19 C - IMF G - Short-term

Net flows 43 56 56 129 1Interest payments 7 8 11 12Net transfers 36 48 44 117

Development Economics 8121100

- 38 -

Annex 11

Organization, Staffing and Technical Capacity of IDEETHIOPIA Distance Learning Project (EDLP)

Organizational Structure and staffing of EDLP

EDLP will operate through one central (Addis Ababa) and seven regional centers. The centraldistance learning center is located in Addis Ababa at the Civil Service College (CSC). The proposedregional centers are to be located in Amhara (Bahir Dar), Tigray (Makelle), SNNP(Awassa), Oromia(Nazareth), Somali (Jinga), Addis Ababa, and Dire Dawa/Harrari. The central distance learning (DL)center at the CSC has been organized under the IDE ( Institute of Distance Education), which is partof the CSC management structure. The central DL center has been designed to be the node in aglobal network of World Bank training centers, which have full multi-media (voice, video and data)connectivity to each other and centers of knowledge around the world.

The IDE is organized within the CSC, with its own offices and classrooms. The CSC has 3 facultiesand 2 institutes ( Institute of Distance Education [IDE], and Federal Regional Government ResearchInstitute). The CSC board is the highest policy level of the CSC structure.

CSC Board. The Board is chaired by a representative from the Prime Minister's Office. The CSCPresident is a member and acts as its secretary.

CSC and IDE. The IDE is headed by a Dean, who will keep the CSC management informed on keyissues relating to IDE's operations. The Civil Service College in general, and IDE in particular willbe responsible for: i) ensuring that EDLP operates according to the guidelines and regulationsspecified in the Implementation Manual; ii) Preparing annually for the Board and IDA a review ofEDLP performance (including a financial report), and a consolidated work program and budgetbased on the pipeline of training proposals received by the central and regional centers; iii)supervising and monitoring the activities of the regional centers; iv) marketing the activities of theDL, processing training requests, supervising implementation of DL courses; v) managing thedevelopment of printed learning and purchasing printing materials and course materials; vi)maintaining a project performance monitoring and information system, and cooperating with theinternal and external evaluators to ensure adequate and appropriate data collection for evaluationreports.

Regional Centers. Seven regional centers are responsible for the day-to-day EDLP operationincluding registering students, collecting fees and maintaining accounting records, hiring andsupervising tutors, monitoring, implementing, and submitting reports and documentation to thecentral DL center.

Staffing of the Central and Regional DL centers. IDE and the central DL center are alreadyoperational, and are staffed with 6 department heads, namely i) Academic Development, ii) Regionalservices, iii) Research, Evaluation and Training, iv) Central Support Services, v) DLC - VideoConferencing and Technology, and vi) Operations. (Responsibilities for each are detailed in Table1)

For the 3.5 years duration of the LIL, the IDE plans to have 28 staff, 17 professionals and 11support. It also needs a marketing specialist who can market the IDE training programs and servicesto the private sector.

- 39 -

Eighteen staff will be required in the regional centers, including 12 professionals and 6 support staff.There was some debate regarding whether the Addis office would require hiring an additionalregional coordinator or whether those responsibilities could be assumed by the head of the CentralSupport Unit. We believe that they can be absorbed by this Unit

Tablel: EDLP Professional Staff Requirements

Position/ Tasks Qualification and Skills Number Add'IDepartment (Existing) required

1. IDE - * Overall responsibility for A higher degree, with IDean the sustainable operation of experience in

the centers education, preferably* Manage the operation and in distance education

accounts of the central andregional centers

2. Marketing * Liaison with private sector A degree in business,companies to identify have a workingtraining needs and market knowledge ofcourses offered by IDE. communication

* Promote learning events technology. Have anwith general public. entrepreneurial spirit,

* Prepare an annual business marketing skills andplan for the centers business management

experience in privatesector.

3. Academic * Manage the development A higher degree in 1 2Development and/or adaptation of course education, with

material. emphasis incurriculumdevelopment.

4. Research * Undertake research on A higher degree in 1 2and Training distance education education researchEvaluation * Identify ongoing training and/or training

needs evaluation.* Identify problems of

distance learners and workto solve

* Evaluate training* Pilot test print materials* Coordinate training with

the course providers* Supervise collection of data

for ongoing monitoring andevaluation

-40 -

Position/ Tasks Qualification and Skills Number Add'IDepartment (Existing) required

5. Regional * Supervise registration of A higher degree in 1 2Services participants education.Support * Follow-up assignment

* Identify potential traineesin the civil service

* Coordinate contact withregions and civil serviceinstitutions

* General quality control ofregional operations

* Train tutors6. Central * Manage and maintain the A degree inSupport accounts of the centers Administration andServices * Publicity finance

* Procure and distributematerials and supplies

* Arrange contracts withprinters

7. DL center * Manage the operation of DL manager with a 3(CSC) the DL center higher degree inVideo * Coordinate course information technologyConference offerings of course and experience inand Internet providers education. One trainingbased * Ensure the proper coordinator/facilitator

distribution of all course with a higher degree inmaterials education and excellent

* 1 acilitate the smooth communication skills.delivery of courses One technologydelivered through technician with avideoconference and university degree ininternet computer science

* Moderate discussiongroups during localactivities

* Support the DistanceLearning Center'stechnology infrastructure

-41 -

Position/ Tasks Qualification and Skills Number Add'lDepartment (Existing) required

8. Regional * Manage the regional 3 people in each 18Center center regional center. OneManagers * Be the primary contact coordinator with a

persons with regional higher degree inparticipants in courses education, and/orand seminars business

* Coordinate course administration. Oneofferings with the IDE technologist ( assistant

* Supervise the registration technician) with aof participants diploma/degree in

* Ensure the proper computer science. Onedistribution of course secretary with amaterials diploma in secretarial

* Participate in pilot science and experiencematerial testing in accounting.

* Collect fees* Supervise maintenance

of records

Organizational Structure - Structure of EDLP Central and Regional centers

CSD/EDLP Board

Civil Service College I

Academic DLC Research, Regional Regional Central OperationsDevelopment Videoconf & Training Centers Services Support

E-leaming Evaluation (7) Services

Regional Centers:MakelleAwassaBahir DarAddis AbabaNazarethJingaDire Dawa/Harrari

-42 -

Strengthening technical capacity in the Institute of Distance Education (IDE)

A. Objective

Objective: The IDE capacity building program aims: i) to provide IDE staff (central and regional)with the capacity to manage the DL and carry out the responsibilities described in theImplementation manual, and ii) increase the knowledge base of IDE through supporting andencouraging studies that will increase the effectiveness of DL activities.

The IDE has the overall responsibility for the sustainable operation of the DL centers. The majortask of the IDE is to be able to generate sufficient activity in courses, seminars, and workshops thatwill justify an income sufficient to pay for network access, staff cost and maintenance anddepreciation of equipment. This requires training of project staff in i) DL management, ii)communication technology, iii) marketing and business management, iv) designing Web pages andmanaging Web content, vi) familiarization in E-learning (on line learning), vii) training evaluation,viii) pilot testing print materials, ix) basic computer literacy and internet use, and x) desk toppublishing.

B. Capacity Building Targets:

In view of the above objectives, two capacity building and training targets have been selected forfinancing under EDLP; i) Strengthen the IDE management and staff, enable IDE to put in placesufficient organizational capacity to carry out its responsibilities reflected in the implementationmanual, and ii) increase the knowledge base of IDE through studies, research and impact evaluationon beneficiaries.

C. Beneficiaries of the Capacity building:Two levels of beneficiaries have been identified:

The Center: The DL central coordinating unit and IDE which must be able to ensure that EWDLPoperates according to agreed procedures, establish and manage the DL system, manage thedevelopment and adaptation of Course material.

The Region: IDE regional centers' staff who must have adequate capacity to carry out the day to dayoperation of DL, to administer the center, process registration of participants, to maintain projectmonitoring system, and supervise and report implementation of DL activities in the region.

D. Review process:

An assessment will be made at the end of one year to ascertain success in achieving the capacitybuilding targets outlined above. The problems and constraints observed will be taken into account inplanning implementation strategies for subsequent years. The schedule for review of the contributionof the component will be arranged in line with the supervision of the overall project.

-43 -

Summary of capacity building targets and expected outputs with preliminary cost estimates

Capacity building targets Expected outputsI.Strengthen the IDE central management and * Increased capacities in Basic computer literacy forstaff, enabling IDE to put in place sufficient 17 professional central staff of IDE.capacity to carry out its duties and mandate. . Increased capacity in designing Web page and

managing Web content of one staff of IDE.* Increased capacities in desktop publishing of 4

secretaries of IDE.. Increased capacities in the management of DL of one

staff of IDE (Visit to South Africa).* Increased marketing and business management

capacity of 3 staff of IDE.. Increased capacities in E-leaming of 3 IDE staff.* Study tour to similar activities of other countries for

# regional coordinators conducted.* MIS for IDE designed and installed.* Vehicles and computers, procured for IDE central

Laboratory.2. Strengthen the IDE regional staff, enabling * Increased capacities in basic computer literacy of 6the staff to have sufficient capacity to carry out regional coordinators.its duties and mandate. * Increased capacities in the maintenance of equipment

and communication technology of 6 assistantregional technicians.

Computers and AV equipment procured for IDE regionalcenters.

3. Increase the knowledge base of IDE through * IDE DL activities evaluated annually.the support of studies and research that . Research undertaken to identify problems ofimprove the efficiency and effectiveness of the distance learners.DL program

- 44 -

Annex 12:

Relationships Between Regional Centers and CSCETHIOPIA Distance Learning Project

General

Each region will initially be staffed by a coordinator, a technician, and a secretary/accountant. As theneed arises the staffing could be expanded. The regional centers will recruit learners, collect theirfees, and manage the required number of tutors and tutorial sessions. The centers will be open 12hours a day seven days a week so that the private sector, and private individuals can benefit from thecomputer network and internet access

Before any regional center is opened the coordinator would have received a minimum of eight weeksof training. The Technicians, essential for the maintenance of equipment both at the DLC, and at theregional centers, would be provided with a fully paid course through the internet, designed toupgrade their skills.

Academic

The whole system needs of learner selection, tutor recruitment and academic control will not bebureaucratic, but will be based on a trusting relationship between the IDE and the regionalcoordinators. Most of the key processes, including learner admission will be almost entirely in thehands of the regional coordinators, and be based on the following three criteria:

* Overall grade in the ESLC, and the grade in English, and possibly maths;* Recommendation of his/her employer,* Score in a standardized aptitude test

Each region would have a certain quota in a certain subject each year, and would merely take thecandidates up to the quota on the basis of merit.

The main academic control from the IDE will take place during examinations at the end of eachcourse, which will be set by IDE, which will also be involved indirectly in its marking.

Financial

It is expected that learners and their employers will together pay around two thirds of the regularoperational expenditures of the specific course they are taking. In order to avoid unnecessary toingand froing of funds

* Funds from the learner and his employer would be deposited in a local account of the CBE, andthe 'pay in' slip presented to the regional coordinator, as a final admission requirement;

* Each regional center would present an annual budget to the IDE/CSC for approval;* The Bankers of the regional center would be instructed to transfer to the central account of the

CSC any amount deposited by learners in excess of the regional budget, and if possible, not tomake payments in any month in excess of an agreed amount;

* For each check written by the regional coordinator there will need to be a second signatory.Who this should be still needs to be decided;

* Accounts and bank statements will be sent to the CSC monthly. This will be able to be done byinternet.

-45 -

Administrative

The coordinator will report to the Dean of the IDE for all academic matters, and to the Head of theAdministrative and Finance Department for budget and financial management issues.

A manual for the different aspects of running a regional center is under preparation. Once it is readyit will be tested with a number of potential regional coordinators. Given the fact that the coordinatorwill have at least four different functions, namely: as an academic advisor and mentor; as a financialcontroller, as a human resource manager, and as a promoter of the program; methods will have to befound to top up his salary to make the post attractive. In addition great care have to be devoted to hisselection; and considerable resources devoted to providing him with at least two months of carefullydesigned pre service training.

- 46 -

Annex 13:

Technical Assistance RequirementsETHIOPIA: Distance Learning LIL Project

The following technical assistance will be needed under the project in order to ensure that the mostappropriate and effective technologies are being used:

* Print Media Course DevelopmentTraining of Staff in Ethiopia ................................. ................. $410,000External Staff Training .................................................. $ 62,000Foreign Experts and Coordination .................................................. $ 63,000

* Internet Course DevelopmentTraining of Staff on Web Course Availability ................................... $100,000Training of internet course developers ............................................... $200,000Subscriptions to internet courses .................................................. $200,000

* More Effective Use of Video Conference FacilitiesVideo conference use and staff training ............................................. $ 100,000

* Summative and Impact Evaluators of DL AlternativesContract with local research inst .................................................. $ 40,000

* Administrative training of involved CSC Staff ..................................... $ 40,000* Distance Education Course Developers (from outside the CSC/IDE)

Allowances to external developers .................................................. $200,000Typing, layout, editing, proofreading ................................................ $200,000

GRAND TOTAL .............................. .................... $1,615,000

For the print media course development TA, an institutional contract would be given to anorganization such as the British Open University. This organization will be responsible for thetraining of the CSC/IDE management staff in the design and running of printed media DL courses,the training and initial supervision of course material developers, the design and initial running oftutor training, the choice of appropriate pedagogic strategies, strategic planning, and learnerassessment.

For the internet course use assistance will be sought in identifying the most appropriate web basedcourses for upgrading the skills of Federal and Regional civil servants. In addition an internet basedtechnical assistance service will be recruited to provide advice and assistance in the design,organization and management of internet courses through the regional centers with internet facilities.

One of the problems with the interactive VC facilities that will be upgraded through the LIL is thatthey are not used frequently. It is intended to recruit a short tem consultant on the cutting edge ofthis technology to advise on some of the most appropriate program providers and to negotiateappropriate terms for using these courses on behalf of the CSC, and its regional centers.

Regular monitoring of the use that is being made of the various DL courses being offered throughDL media will be kept by the research and evaluation department of the IDE, however mid term andfinal summative, cost effectiveness and impact evaluation will be contracted to a local researchorganization.

- 47 -

The procurement, administration, and probable finance management staff involved in theimplementation of this project will need training in World Bank Procedures.

The academic staff of the CSC is already fully occupied in teaching the regular full time students atthe college. Thus for the development of new distance learning courses, staff from other academicinstitutions will have to be employed on a piece work contract basis to develop the course materials.Groups of professionals will be employed to work together to produce a certain course according toacceptable standards.

-48-

Annex 14:

Rationale for Use of Internet to Deliver CoursesETHIOPIA: Distance Learning LIL Project

The Internet has been selected as a mechanism to deliver coursework for several reasons:* There is substantial research evidence that this approach is as effective, and often more effective,

than in-person and/or print-based distance learning.* There is some evidence that learners can learn faster using the computer, and, when such

learning is well-designed, is up to 55% faster when that learning is interactive.* Learner support and feedback can be provided much more quickly and frequently with internet

based distance courses than with print media based distance courses, thereby increasing theefficiency of students' learning as well as their motivation to continue learning -- particularlyimportant for distance learners.

* Communities of learners across regions can communicate with one another.* There is an extensive array of resource materials available to learners on the World Wide Web;

materials, knowledge and information that may be useful in their coursework and for theirprofessional responsibilities.

* There is also an extensive array of coursework available through the Internet, which may beaccessed by IDE to deliver to learners.

* It is increasingly possible that learners, most of whom are civil service employees, will be usingcomputers and the Internet in their jobs within the next several years.

* This experience will prepare them for the dramatic changes that are occurring worldwide in useof information te chnology, particularly for knowledge access.

Shortcomings of using the Internet to deliver training include:* limited access to the Internet, particularly in Ethiopia and particularly in most regional centers

and all rural areas;* lack of electric'y in some rural areas;* costs of hardware, software and Internet connectivity;- need for learner to be trained in the use of computers and the Internet;* need to train IDE staff (including tutors) in developing, delivering and supporting training

delivered on the Internet;* likelihood that some learners will be anxious and/or uncomfortable using the computer;* additional time required to train learners in computer-literacy skills;* need to produce/purchase printed materials to complement online learning;* need for students to be at an Internet access site in order to engage in some of the Internet-based

learning;* need for students to develop keyboarding skills - which can take longer to learn than computer

literacy.

-49 -

Annex 15:

Monitoring, Evaluation, and Performance IndicatorsETHIOPIA: Distance Learning LIL Project

Project Monitoring and Reporting

Regular and systematic collection of information and relevant data to monitor progress/performanceof delivery mechanisms as well as the impact of the activities financed under the project will beundertaken. This is essential not only to be able to adjust the project mechanisms as necessary foreffective project implementation but also for designing the expanded phase, if such a decision ismade.

Regional centers will prepare and submit to IDE central Office, quarterly, and annual progressreports, which will provide details on implementation progress based on implementation goals andtime table. The central IDE will prepare for submission to CSC Board and IDA, a consolidated semi-annual report within six weeks of the end of each half-year summarizing overall implementation,highlighting any specific problems that require attention, and proposing actions to correct theseproblems. The annual report will cover results of the whole year. Progress will be measured againstprevious year's Annual Work Plan and Budget

Project and Component Evaluation

The evaluation will be designed to provide two levels of information:

I) summative results that show comparative effectiveness and cost-effectiveness of the mainapproaches used in the project: print-based, Internet-based, and videoconferencing, and possiblecombinations of the various delivery media used; and

2) Implementation features of this project to provide IDE staff with information needed torefine the project mid-stream, and to create knowledge - to provide others wishing to implementsimilar projects with guidance for making sound decisions.

Each level of evaluation will require different staffing and data-collection efforts. The summativeevaluation will be conducted by a local researcher, external to IDE. The contract for the evaluationwill be let when the LIL is initiated to ensure that all data-collection systems are in place to be ableto answer the major study questions listed below. A mid-project evaluation will be conducted at theend of two years, and an end-of-project report will be provided within two months of projectcompletion. The electronic learning management system described below should be adequate forcollecting much of the data needed to answer the summative evaluation study questions.

Summative Evaluation

Major questions to be answered in the summative evaluation include:

For each course and program,' which delivery mode(s) and tutoring approach(s) results in:

* greater learner achievement;

'Program refers to an extended course of study that is made up of a number of courses that lead to a certificateor diploma - such as a diploma in accounting. Course will refer to a single topic within the program, such asAccounting 101 or Biology 101.

- 50 -

* greater learner satisfaction;

* fastest course and program completion time;

* highest program completion rate; and

* greatest overall cost effectiveness.

Implementation Evaluation

The implementation evaluation will require ongoing data collection and is much more complex thanthe summative evaluation.

Implementation issues for which data should be collected and reported include the following:

* Tutor support

Questions of concern include:

1. What is the student-tutor ratio that most cost-effectively produces the most course completers?

2. What tutoring strategies and frequency of contact produce the most favorable results, such as e-mail communication; in-person meetings; on-the-job support by professional colleagues;written feedback, etc.

3. What is the amount and type of training that results in the most effective tutor support?

4. Is tutor support provided online by E-learning companies effective -if E-learning companiesare used?

5. Can online learning programs be designed in ways to reduce or obviate the need for tutors?

6. Which types of individuals make the best tutors and at what cost? (e.g., professionalcolleagues, faculty from the CSC, faculty from other institutions, graduate students, high schoolteachers, private sector employees?

7. What is the most cost-effective way to pay tutors? E.g., per hour, per student, per course, persuccessful student completer, per assignment, per quality of student feedback, per studentsatisfaction?

* Learning materials, delivery and process

There was considerable discussion during the project design regarding course materials. A numberof issues around materials should be documented and reported, including:

1. What is the effectiveness and cost-effectiveness of locally-developed materials vs. purchasingoff-the-shelf materials. (This may vary by course content and provider.)

2. Do these comparisons hold with regard to online learning?

- 51 -

3. How much written text material are fully-employed students able to absorb during a courseterm?

4. Do these comparisons vary by provider and type of arrangement - for example, collaboration,vs. licensing?

5. What level familiarity and comfort in computer literacy do individuals need before they canbegin taking courses online?

6. What level of keyboarding skills should the online students be required (or encouraged?) tohave before initiating online courses? How long does it take to get them to various levels ofproficiency?

7. Is there any difference in effectiveness of videoconferencing courses based on whether there islive interactivity? Or if they are provided with videotapes of the videoconferencing session vs.live videoconference?

8. How do individuals deal with the issue of using print materials within the context of onlinecourses? And how does the bandwidth (speed of access to the Internet) influence time spentonline engaged in learning and course completion time?

9. What is the amount of learning materials that distance students can successfully complete,given that they are employed full time and that the materials are not written in their mothertongue.

Pilot Testing of Learning Materials:

Pilot testing is probably the most important, but least implemented aspect of learning materialsdevelopment. The process used to pilot test IDE's learning materials (whether developed or adoptedand whether in print or online) will be documented in detail and the findings of the pilots and thechanges made will be analyzed and summarized weekly. Such analysis can be a valuable resource toothers developing similar materials for similar audiences.

- Student Assessment

Many forms of student assessment promote the memorization of discrete facts, formulas andinformation, without encouraging learners to integrate the various aspects of their learning into anintegrated whole that can be applied in real-life settings. More authentic forms of assessment havebeen developed over the past several years and it would be useful for IDE staff to have professionaldevelopment in this area. The staff who should participate in such training include faculty andconsultants who will develop learning materials, tutors who will evaluate students' assignments andexaminations, and R&E staff who may be able to provide technical assistance in developingauthentic learning assessments.

* Language and study support

Many of the students that will be participating in the IDE programs have limited English-languagefacility. The faculty have identified these learning needs and propose to address them in two ways:1) by providing two semesters of English-language instruction, and 2) writing course materials insimpler, straightforward language with numerous examples, graphs, exercises and conceptualmodels.

- 52 -

1. Are these strategies adequate?

2. Would it be more cost-effective to provide diagnostic testing of each entering student and toprovide remedial instruction tailored to each student's particular English-language needs? Canthis diagnosis and remediation be done at a distance?

Courses for the private sector and general public.

The cost of the videoconference and Internet infrastructure and hardware is high. And it is unlikelythat it will be in use full time by IDE students. Can the IDE generate additional income that cancontribute to sustainability by providing courses to both the private sector and the general public, byusing these facilities during underutilized hours? Other distance learning institutions have reportedsuccess in this and is an area that should be documented and reported.

An individual will be employed by IDE to "market" such courses to the private sector and to cementrelationships between the IDE, the regional centers, the private sector institutions and the communityin the various regions. Some of the specific questions that need answering include:

1. What sorts of courses are attractive to the private sector? To the general public? It is expectedthat there is a high demand for computer courses.

2. What fees can IDE charge each group without out-pricing themselves?

3. What strategies will the IDE use to provide computer-related courses, given the shortage ofskilled/trained computer personnel and the competitive demand for their services? For example,could they recruit someone to train IDE students at IDE salaries, but who would be allowed touse the IDE facilities to offer courses to the public after hours and receive a "commission" forsuch courses? What kinds of waivers in government regulations would need to be made topermit such flexibility?

4. How would the IDE arrange to identify and/or develop courses to meet the needs of the privatesector and/or general public? Will online learning be useful to them for this purpose? Wouldproviding services to the private sector conflict with their primary mission of serving theEthiopian Government Civil Service agencies and personnel?

- 53 -

Some Key Performance Indicators

Indicator Planned (To be Actual Differencespecified annually)

Input Indicators* IDE share of total CSC budget* % of requests received for DL learning by type

of course

Process Indicators* Staff evaluation of training they receive in DL.* Quarterly staff evaluations of TA provided to

IDE* Quality of learning materials developed and/or

adopted/adapted, as judged by students.* Pilot testing and appropriate revision conducted

of all learning materials (records of learnerfeedback and changes made based on feedback)

* MIS system in place and used to monitorstudent enrollments, grades, course completionand time-to-completion by course and program.

* Tutor reports of satisfaction with trainingreceived.

* Quarterly reports from students regardingsatisfaction with tutoring support

* Quarterly reports from tutors regarding needsfrom IDE staff to support their effectiveness astutors

* Quarterly reports from IDE regardingmonitoring of tutors and appropriateadjustments made.

* Reports of revenue from private sectorOutcome Indicators* IDE Student enrollments as compared with

CSC Regular Enrollment.* Number and percent of course completions by

delivery media and type of tutor support* Number and percent of program completions

by delivery media and type of tutor support* Student GPA by course, program, delivery

media and type of tutor support* Time-to-completion by course, program,

delivery media and type of tutor support.* Learner satisfaction, compared with CSC and

by course, program, delivery media and type oftutor support

MAP SECTION

IBRD 31158

I ~~~~~~~~~~~~~~~~ ~~~~R e d 5

is' ~~~~~~~~~~~~~~~~~~~~~~j' \ ~~~~~~~~~~~~~~~~~~~R EP U BL IC / ERITREA~

.1p *Kh.a.- ~~~~~~~~~~~'REP. OF YEMEN

SUD AN Adq S a O F

~~~~ ele -~~~~~~~~~~~~~~~~~~~~~~ A~~~~~FAR /DJIBGUTJ

(ii~ / oss Ad,-~ ~ OAAN..G. SOMALIA

ADer '_ I_ AFAR Addis AIx,ba

0 _ 1A~~~~~~~~L '~~~~~ /..DJ I BOUTI.-0GEBL

) 0 AI~~~~~~~~oI~~~~eI Rohir ~~~~~~~~~~~~M H~~~~~J~~~ G ~~0OROMIYA SOMALI /

GA Ad. D-1. ~~~~~~~~~~~~S.N.N.PR.S./

-10' K /

~~ bit 0 / 0 10 ~~~~~~~UGANDA ' EY ~ 0 s

Arl. ~ ~ ~ ~ R ETH IOPIA

diLl, ~~~~~~~~~~~~* ~~~~~iZ, ~~~~~~~~~ 0 - ~~~~~~7ASPHALT, ALL-WEATHER ROADSL CTI,~~~~~~~~~~~~~~~~or

/ ~~~~~~~GRAVEL, ALL-WEATHER ROADS

SOMAll ~ ~~~~~~~~~~~~~~RURAL AND! OR DRY-WEATHER ROADS

Wor.,d Dle

DA,,. SELECTED TOWNS

- STATE/ REGION (UNOFFICIAL 199A BOUNDARIES)

68mi~~~~~~~~~~~~~~~~~~~~~~~~~~ch~~~~~~--- INTERNATIONAL BOUHDARIES

S U A N Ar oh

KILOMETERS

Thi, .p ole pd-de by IrMAp .,n5- lTeWrdSrk- IOtA/14Thboni.,olredeorr,rler,OrdnehnZ frea.eIon

NOVEMBER 23105