Water Implementation Projects In Rural

North India: Evaluating

implementation “on the ground”

Katharine Owens, University of Hartford, Politics and Government

Marcia Hughes, University of Hartford, Center for Social Research

Overview of Presentation Project efforts to date Evaluation, implementation, and

collaboration Assumptions of this evaluation work and

inherent challenges of project efforts Evaluation model and methodology:

Contextual Interaction Theory Analyses, findings, and uses of the model Recommendations and conclusion

Projects in Abheypur, India University of Hartford student-faculty

service-learning teams

Professional and student chapters of Engineers without Borders

Interdisciplinary projects: engineering, social science, and education/outreach projects focusing on water.

AbheypurPathways

Abheypur

Gurgaon

Delhi

“On the ground” in Rural India

Abheypur, India Project Team

Abheypur, India

The Projects January 2008: Installation of a solar powered well

pump and water storage system August 2008: Extension of the water supply to a poor

section of the village January 2009: Implementation of a rainwater-

harvesting system October 2010: Installation of a potable water supply

well and water storage tank for poorest community January 2010: Installation of a grey water drainage improvement project for the “Potter’s Village”

January 2011: Installation of a grey water drainage improvement project on main road of the village

October 2009 tank location

Abheypur, Haryana

2011 Soak Pit

• Solar panels• 2 wells• 4 tanks• Rainwater harvesting

• 2 tanks• Piped to girl’s school

• Well• Tanks

2010 Soak Pit

Main Road Drainage Problem

Right-of-Way

Private Farm

Gutter: 6” wide by 6” Deep

N S

W

Soak Pit

DrivewayBrick Wall

Gate

Continue Drain Across Road Drain Covered by Grate to Allow Vehicles & Carts to Pass

E

New Drain

No Gutter – Grade so swale takes water along side of road

About 3% grade from west side of road to soak pit

New Drain

Design for Soak Pit Implemented in 2011

Potters’ Village Drainage Problem

BEFORE

Potters’ Village - Soak Pit Design Jan 2010

Forebay Soak Pit

Brick Weir (Width = 3”)

Greywater Inflow

Overflow to swale during for rain events4’

Forebay Depth = 1.5’Soak Pit Depth = 4’Forebay and Soak Pit covered with slate

4’ 4’

Stones and rocks to reduce water velocity

Two layers (6”) of bricks are used for soil structural support to keep soak pit open. Thus, excavated diameter is 1’ wider and longer than internal dimensions listed.

AFTER

Implementation

When political ‘actors’ desire change in society, they develop and implement programs to address a perceived problem.

Researchers focus on the implementation stage of the policy process as a point where intentions and outcomes diverge

Two different processes: Developing a program/policy versus implementing a project/program/policy

Merging evaluation and implementation Program evaluators seek to quantify and

catalog the successes and failures of programs and policies.

Implementation policy analysts evaluates this stage in the process to better understand participants’ roles and any impediments to change.

We seek to infuse evaluation with implementation analysis methodology.

Merging the two Evaluation researchers find it’s not enough

to know whether outcomes are achieved, it’s important to understand how.

Using an implementation analysis tool in evaluation can answer questions about how people come together and work towards achieving a project (or fail to).

Scholars call “for responsive, contextual, flexible, adaptive, multidisciplinary, and mixed-methods approaches,” particularly when considering environmental programs and policies (Preskill, 2009, p. 99).

Collaboration is an imperative

(Gadja, 2004)

Being solo does not work for most of the problems that need addressing

Need shared efforts to achieve goals that would not otherwise be attainable working independently.

More and more, collaboration is becoming the method for addressing complex social problems

Understanding collaboration

Wostl et al. (2007) –

In managing water resources in particular, a paradigm shift is needed, one that recognizes the importance of

stakeholder involvement and collaboration.

Water: communal and imperative

Understanding collaboration

One goal: To develop communities of practice.

Would we recognize if we were not working collaboratively with our Indian partners?

Should we trust our own conceptions, or evaluate the process to confirm our goals of collaboration?

What does collaboration look like?The Practice of Collaboration Gajda and Koliba (2007) describe key

characteristics of interpersonal collaboration to be:

a shared purpose dialogue decision-making action, and evaluation (p. 29).

Water Implementation Projects in India: Inherent Challenges Working across cultures Across languages Across disciplines (engineers and social scientists) Across class (American middle-class and Indian

impoverished villagers) Caste distinctions remain present in the village Gender distinctions (American women in positions

of leadership, which may be atypical) There is short period to build trust among

partners, implement new projects, and manage and evaluate previous projects.

This evaluation work We work to accomplish project

implementation as well as outcome evaluation in India.

We find it important to engender true collaboration with India partners.

We use an implementation analysis theory to evaluate these water projects.

Contextual Interaction Theory finds project participants and other

stakeholders can influence implementation in critical ways.

analyzes collaboration between participants based on motivation, knowledge, and power, to shed light on how projects are implemented.

produces a prediction about interaction based on these core participant characteristics

Contextual interaction Theory allows the emphasis on implementation

(the how) as called for by evaluation researchers.

can provide evidence to those “on the ground” about their effectiveness in implementation.

allows large-n studies that enable researchers to find patterns in large data sets and enables multi-year studies.

Contextual Interaction Theory The motivation, knowledge, and power of

the implementer and target are the focus of the assessment.

Implementer: the student-faculty team, and a loose coalition including Navjoti, and the professional Engineers without Borders chapter

Target: local villagers, including those residing near the project and those taking an interest in the implementation of the project.

Research Questions How does motivation, knowledge, and

power among the different players affect implementation of the project?

What is the relationship between these characteristics among the people ‘on the ground’ and the outcomes of the implementation process?

How is the project plan or design implemented through collaboration processes?

Methods Faculty-student trip to Abheypur, India in

January 2010.

14 interviews using a semi-structured interview instrument.

Questions investigate the core elements of the theory (motivation, knowledge, and power)

We asked slightly different questions of the American and Indian counterparts as reflected their roles in the process.

Independent variablesMotivation

Knowledge

Power

Dependent variables Active cooperation Passive cooperation Forced cooperation Opposition Obstruction Joint learning No interaction Policy learning

Analysis[1] M scale=((proportion)-.50) *2, scale of -1.0 to +1.0[2] I and P scales= (proportion), scale of 0.00 to 1.00[3] n/a : participant did not answer questions on this concept.

Role MotivationM scale[1]  Category Knowledge

K scale[2]

Power  P scale

Implementer: Faculty-student team members

(+) 5 of 5 1.00 positive (+) 4 of 4 1.00 (+) 2 of 4 0.50

(+) 2 of 4 0.00 neutral (+) 3 of 6 0.50 (+) 0 of 2 0.00

(+) 2 of 2 1.00 positive (+) 4 of 6 0.67 (+) 0 of 2 0.00

(+) 1 of 2 0.00 neutral (+) 4 of 5 0.80 (+) 1 of 2 0.50

Implementer aggregate (+) 10 of 13 0.538462 positive (+) 15 of 21

0.7142857 (+)3 of 10 0.30

Target: Villagers

(+) 3 of 3 1.00 positive (+) 2 of 2 1.00 (+) 0 of 3 0.00

(+) 2 of 3 0.33 positive (+) 1 of 1 1.00 (+) 0 of 2 0.00

(+) 2 of 2 1.00 positive (+) 2 of 2 1.00 (+) 0 of 2 0.00

(+) 3 of 3 1.00 positive (+) 2 of 2 1.00 (+) 0 of 1 0.00

(+) 1 of 2 0.00 neutral n/a[3] n/a (+) 0 of 1 0.00

(+) 1 of 2 0.00 neutral (+) 1 of 1 1.00 (+) 0 of 2 0.00

(+) 2 of 2 1.00 positive n/a n/a n/a n/a

 Target aggregate (+) 14 of 17 0.647059 positive (+) 8 of 8 1.00 (+) 0 of 11 0.00

Analysis

Qualitative assessment

Implementers have some doubts about the work, their role, and the role of the target participants in the process.

"…but you know sometimes I wonder maybe there might be actually better things we could be doing with our time."

"I think [with] this project we weren't communicating with them before we got there, they didn't know our plan.”

New understandings of caste subgroups and inter and intra group dynamics

Analysis What is the potential for other scenarios to

emerge?

The strong imbalance of power does not come into consideration when both actor groups are in favor of a project.

But how might this impact implementation if one actor was no longer in favor of implementation?

If the target lacked motivation

If the implementer lacked motivation

Evaluating our collaboration

Recommendations (February 2010) Community-driven projects based on a

formalized decision-making process. establish a shared purpose/role among EWB

interdisciplinary team Always consider the

caste/subcaste/gender/poverty issues and related power/resources, knowledge, and motivation among different groups

Recommendations (February 2010) Make collaboration a goal

Are all stakeholders involved? Is there a shared purpose and understanding of

the project? Is there equal decision-making? Has the

community been involved and/or surveyed for their feedback?

Is there local leadership for the project (i.e., in the village)?

Expect and plan for disagreement and conflict.

Recommendations (February 2010) Streamline the collaborative process

Develop project ideas with community leaders and Navjyoti.

Navjyoti can survey/scan community for input/feedback on project ideas.

Navjyoti can serve as link between EWB and village and district leadership (i.e., for communicating project design development, and planning).

Fine tune plan and develop budget-tasks and responsibilities (prior to starting project).

Recommendations (February 2010) To make previous projects sustainable

Monitoring Planning modifications

Monitor systems to ensure proper use and satisfaction with system(s), community leadership, and overall quality of water

Conclusions It is both a goal and an obligation to work

in a collaborative way with local groups in India.

Using the theory we can gain insight into not only the outcome of implementation (cooperation), but also provide context for how and why participants are cooperating.

Conclusions and Next Steps

Our analysis also sheds light on how the people involved in this project influence it.

It allowed us to set meaningful goals for collaboration and partnership

Next Steps: Analysis of 2011 data