R.S. GoyalProfessor, Institute of Health

Management Research, Jaipur, India(email: rsgoyal@iihmr.org)

Evaluation Capacity Building in South Asia: Experiences, Lessons Learned and Way Forward 

Backdrop…..Most South Asian countries now have monitoring

and evaluation systems in place, however, these are largely concerned with;

thethe progress of implementation, rather progress of implementation, rather than assessing the social,than assessing the social, economic and economic and environmental impacts of projects. environmental impacts of projects.

Also, there seem to be no systems that assess the impact of policy interventions emerging from recent macro-level measures, such as;

liberalization,liberalization, privatization, and the privatization, and the preservation of women's rights. preservation of women's rights.

Further, donor agencies have played a key role in planning, implementing and financing various socio-economic development programs and projects in these countries.

In many cases, the outcomes of these interventions do not match the intended objectives.

It has also been observed that due to the lack of ongoing evaluation many governments fail to learn, in time, the way a project is unfolding and the manner in which it is generating benefits.

There are also many who simply do not see the benefits of evaluation and, consider it to be a donor-driven activity of no management use.

Those donors who do see evaluation as an

important tool to improve investment quality are now seeking to build evaluation as mandatory management style- result based management.

Success of these initiatives seems to have been constrained, among other things, by;lack of a unified approach;inadequate appreciation and analysis of

governmental culture;confusion about concepts and methodologies; lack of long-term commitment; and lack of either interest or resources—or both—

from the recipient governments. Lack of professional evaluation capabilities

to commission, manage, implement and use evaluation has also been sited as one of the major constrains.

Strengthening evaluation capacity in the South Asia region has therefore becomes critical to achieve results of development projects.

Institutionalization of evaluation training in academic institutions is identified as one of the most viable strategy.

With support from UNICEF ROSA, a group of select institutions in South Asia, embarked upon a path to offer professional evaluation training in a sustainable manner.

This paper discusses the process, outcomes, lessons learned and future directions of this initiative.

The Process…..Assessment of demand for professional evaluation

training.A well published call for expression of interest

sent by UNICEF ROSA indicated a widespread need of professional evaluators among donors and desire to participate in the capacity building process among leading institutions in the region.The feasibility assessment led to identification

of six institutions with essential facilities, expertise and commitment to participate in the capacity building program.

A key strategy of this initiative was to seek support from centers of excellence in evaluation training in the North and foster a North-South collaboration.Among numerous Northern institutions

showing interest; Carlton University (Canada) Carlton University (Canada) and London Metropolitan University (UK) and London Metropolitan University (UK) were identified.

To promote ownership and cross fertilization of ideas among participating institutions, a consortium of institutions on evaluation training in South Asia was formed.

To broaden the scope this initiative, the policy makers, commissioners of evaluation and evaluation professionals in South Asia were also brought in the loop.

It was agreed that training program would commence with an executive level course of one week duration.

The training curriculum was developed jointly by North and South institutions.

A training of trainers program was held in Dec. 07 at the author’s institute to test the training module and orient the faculty from South Asian institutions on delivering the course.

This program was attended by institutions from Bangladesh, India, Pakistan and Sri Lanka.

The feedback of this program was critically examined with reference to contents, training pedagogy and reading material provided.

The analysis shows that mode of delivery was good. However, contents need modification to suite the expectations of programs and professionals in the region.

The next step was setting up procedures by participating institutions to offer executive course on regular basis.

Several institutions managed this process efficiently. These include; BARC University, Dhaka, Institute of

Health Management Research, Jaipur, Lahore University of Management Sciences, Lahore, Sri Jayawardanapura University, Colombo.

Other institutions are in the process of

making arrangements.

The participating institutions started offering evaluation trainings in their respective areas of specialization.

Author offered the first course on Evaluating Healthcare Interventions on Oct. 13-17, 08. It was attended by 18 participants from South Asian region.

The training focused on; evaluation design, ethics and standard of

evaluation, result based management, types of evaluation and links to monitoring, developing terms of reference, process of carrying out evaluation and utilization of evaluation findings for program management.

The feedback of this training shows;it met the expectations of participants to a large it met the expectations of participants to a large

extent,extent,helped in enhancing the skills in planning helped in enhancing the skills in planning

evaluations, writing TOR, carrying out and evaluations, writing TOR, carrying out and utilizing evaluation findings,utilizing evaluation findings,

further skill building was however, sought on further skill building was however, sought on research design and data analysis process.research design and data analysis process.

In the light this feedback the training module was modified and the next executive level training was offered on March 2-6, 09.

This training attracted a very positive response and attended by 59 evaluation professionals from South Asia.

The feedback of second program was even more encouraging:Over 95 percent participants observed that

course was well structured and practical oriented.

Ninety five percent participants opinioned that course learning would help them in their immediate and future assignments.

Over 96 percent participants observed that course material was very relevant to their work.

Almost all participants wanted to extend the training period.

Lessons Learned…..The regional approach adopted in this

initiative has been a strength in its implementation and in moving it further. It enabled adoption of common applicable and It enabled adoption of common applicable and

culturally acceptable curriculum and training culturally acceptable curriculum and training pedagogy.pedagogy.

Coming together of institutions as a consortium for Coming together of institutions as a consortium for evaluation training has provided a region-wide evaluation training has provided a region-wide forum and an authenticated body.forum and an authenticated body.

It facilitated the program to move forward as a It facilitated the program to move forward as a body beyond the tenure of assistance of original body beyond the tenure of assistance of original donor agency and developing a plan for diploma donor agency and developing a plan for diploma program. program.

The collaboration between North and South Asian institutions played a vital role in the success of this initiative.TheThe former brought the state of art knowledge former brought the state of art knowledge

and approaches which later adapted to South and approaches which later adapted to South Asian learning context and needs.Asian learning context and needs.

In developing diploma program in evaluation this experience will be taken further through a greater engagement of South Asian partners and advanced knowledge and experiences from the North.

The main success of this capacity building process was its

sustainabilitysustainability. The South Asian Institutions were able to The South Asian Institutions were able to

run the course on a cost recovery basis.run the course on a cost recovery basis.Thus with infusion of a small start up Thus with infusion of a small start up

funding together with catalytic actions, funding together with catalytic actions, this intervention presents a viable model this intervention presents a viable model to address the need of professional to address the need of professional education. education.

Future Directions…..Planning has been initiated to offer a on

campus and online diploma in evaluation.Consolidation and standardization of training

module of one week executive level course for wider replicability.

Evolution of a regional network of professional evaluators to support the evaluation activities.